From 709b67f1ce29cfe6ca230d2989844ff8a0e34861 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 9 Dec 2020 21:36:54 +0100 Subject: [PATCH 001/214] Introduce rust-toolchain Make use of the new format, introduced in https://blog.rust-lang.org/2020/11/27/Rustup-1.23.0.html#new-format-for-rust-toolchain --- README.CN.md | 15 ++++----------- README.md | 18 ++++++------------ rust-toolchain | 4 ++++ 3 files changed, 14 insertions(+), 23 deletions(-) create mode 100644 rust-toolchain diff --git a/README.CN.md b/README.CN.md index ed767da69..9d82c8ada 100644 --- a/README.CN.md +++ b/README.CN.md @@ -52,26 +52,19 @@ _带上我最诚挚的问候,
Andre ([@andre-richter])_ 3. 安装正确的`Rust`工具链: 1. 如果你已经安装了一个版本的Rust: ```bash - rustup toolchain add nightly-2020-11-20 - rustup default nightly-2020-11-20 - rustup component add llvm-tools-preview - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils rustfilt ``` - 2. 如果你想要全新安装: + 1. 如果你想要全新安装: ```bash - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- \ - --default-toolchain nightly-2020-11-20 \ - --component llvm-tools-preview + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils rustfilt ``` -4. 如果你使用 `Visual Studio Code`,我强烈推荐你安装[Rust Analyzer 扩展]。 -5. 如果你使用的**不是**Linux,那么你还需要安装一些`Ruby` gems。 +1. 如果你使用 `Visual Studio Code`,我强烈推荐你安装[Rust Analyzer 扩展]。 +1. 如果你使用的**不是**Linux,那么你还需要安装一些`Ruby` gems。 ```bash sudo gem install bundler diff --git a/README.md b/README.md index 5af62a962..845aa87d1 100644 --- a/README.md +++ b/README.md @@ -65,29 +65,23 @@ other Unix flavors such as **macOS**, but this is only _experimental_. 1. [Install Docker][install_docker]. 1. Ensure your user account is in the [docker group]. -1. Install a suitable `Rust` toolchain: +1. Prepare the `Rust` toolchain. Most of it will be handled on first use through the + [rust-toolchain](rust-toolchain) file. What's left for us to do is: 1. If you already have a version of Rust installed: ```bash - rustup toolchain add nightly-2020-11-20 - rustup default nightly-2020-11-20 - rustup component add llvm-tools-preview - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils rustfilt ``` - 2. If you need a fresh install: + 1. If you need to install Rust from scratch: ```bash - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- \ - --default-toolchain nightly-2020-11-20 \ - --component llvm-tools-preview + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils rustfilt ``` -3. In case you use `Visual Studio Code`, I strongly recommend installing the [Rust Analyzer extension]. -4. If you are **NOT** running Linux, some `Ruby` gems are needed as well: +1. In case you use `Visual Studio Code`, I strongly recommend installing the [Rust Analyzer extension]. +1. If you are **NOT** running Linux, some `Ruby` gems are needed as well: ```bash sudo gem install bundler diff --git a/rust-toolchain b/rust-toolchain new file mode 100644 index 000000000..9c4710ceb --- /dev/null +++ b/rust-toolchain @@ -0,0 +1,4 @@ +[toolchain] +channel = "nightly-2020-12-08" +components = ["llvm-tools-preview"] +targets = ["aarch64-unknown-none-softfloat"] From d2fd4913ae6866026fbc814f851702bba292ce59 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 9 Dec 2020 21:45:10 +0100 Subject: [PATCH 002/214] CI: Ignore rust-toolchain and use latest nightly --- .github/workflows/build_rpi3.yml | 1 + .github/workflows/build_rpi4.yml | 1 + .github/workflows/sanity.yml | 1 + .github/workflows/test_integration.yml | 1 + .github/workflows/test_unit.yml | 1 + 5 files changed, 5 insertions(+) diff --git a/.github/workflows/build_rpi3.yml b/.github/workflows/build_rpi3.yml index ec9a49e00..c3bc3992e 100644 --- a/.github/workflows/build_rpi3.yml +++ b/.github/workflows/build_rpi3.yml @@ -31,6 +31,7 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | + rm rust-toolchain rustup self update rustup toolchain install nightly --component llvm-tools-preview rustup default nightly diff --git a/.github/workflows/build_rpi4.yml b/.github/workflows/build_rpi4.yml index 12f8dc498..8d479e8d9 100644 --- a/.github/workflows/build_rpi4.yml +++ b/.github/workflows/build_rpi4.yml @@ -31,6 +31,7 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | + rm rust-toolchain rustup self update rustup toolchain install nightly --component llvm-tools-preview rustup default nightly diff --git a/.github/workflows/sanity.yml b/.github/workflows/sanity.yml index 3d4d85625..46c559a8e 100644 --- a/.github/workflows/sanity.yml +++ b/.github/workflows/sanity.yml @@ -33,6 +33,7 @@ jobs: npm install prettier - name: Set up Rust nightly run: | + rm rust-toolchain rustup self update rustup toolchain install nightly --component rustfmt clippy rustup default nightly diff --git a/.github/workflows/test_integration.yml b/.github/workflows/test_integration.yml index cf5d70082..fa2fda987 100644 --- a/.github/workflows/test_integration.yml +++ b/.github/workflows/test_integration.yml @@ -31,6 +31,7 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | + rm rust-toolchain rustup self update rustup toolchain install nightly --component llvm-tools-preview rustup default nightly diff --git a/.github/workflows/test_unit.yml b/.github/workflows/test_unit.yml index d090e8a4c..741ff352b 100644 --- a/.github/workflows/test_unit.yml +++ b/.github/workflows/test_unit.yml @@ -31,6 +31,7 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | + rm rust-toolchain rustup self update rustup toolchain install nightly --component llvm-tools-preview rustup default nightly From d212348931b08b2d37b0a120cc053e863b4910a2 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 9 Dec 2020 22:27:30 +0100 Subject: [PATCH 003/214] Bump compiler version. We have to remove the `#[naked]` attribute from `_start` for now, since it emits a warning now when used with non-asm statements in the function body. For now, just hope for the compiler's mercy to not emit code using the stack pointer before we've actually set up a stack. --- 04_zero_overhead_abstraction/README.md | 9 +++++---- .../src/_arch/aarch64/cpu.rs | 5 +++-- 05_safe_globals/src/_arch/aarch64/cpu.rs | 5 +++-- 06_drivers_gpio_uart/README.md | 2 +- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 5 +++-- 07_uart_chainloader/README.md | 6 +++--- 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 5 +++-- 08_timestamps/README.md | 8 ++++---- 08_timestamps/src/_arch/aarch64/cpu.rs | 5 +++-- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs | 5 +++-- 10_privilege_level/README.md | 12 ++++++++++-- 10_privilege_level/src/_arch/aarch64/cpu.rs | 7 +++++-- .../src/_arch/aarch64/cpu.rs | 7 +++++-- .../src/_arch/aarch64/cpu.rs | 7 +++++-- 13_integrated_testing/README.md | 2 +- .../src/_arch/aarch64/cpu.rs | 7 +++++-- .../src/_arch/aarch64/cpu.rs | 7 +++++-- 15_virtual_mem_part2_mmio_remap/README.md | 2 +- .../src/_arch/aarch64/cpu.rs | 7 +++++-- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8400 -> 8064 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 7088 -> 6800 bytes X1_JTAG_boot/src/_arch/aarch64/cpu.rs | 5 +++-- rust-toolchain | 2 +- 23 files changed, 77 insertions(+), 43 deletions(-) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index c7ff8a5f6..ca679c52a 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -53,7 +53,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abs diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs --- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs +++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs -@@ -4,8 +4,34 @@ +@@ -4,8 +4,35 @@ //! Architectural processor code. @@ -72,8 +72,9 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstrac +/// +/// # Safety +/// -+/// - Linker script must ensure to place this function at `0x80_000`. -+#[naked] ++/// - Linker script must ensure to place this function where it is expected by the target machine. ++/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is ++/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; @@ -90,7 +91,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstrac //-------------------------------------------------------------------------------------------------- // Public Code -@@ -14,13 +40,7 @@ +@@ -14,13 +41,7 @@ /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs index d49c8aba0..64c5397fb 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/05_safe_globals/src/_arch/aarch64/cpu.rs b/05_safe_globals/src/_arch/aarch64/cpu.rs index d49c8aba0..64c5397fb 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 1bd8b26ae..08568cf5b 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -183,7 +183,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 05_safe_globals/src/_arch/aarch64/cpu.rs +++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -@@ -37,6 +37,17 @@ +@@ -38,6 +38,17 @@ // Public Code //-------------------------------------------------------------------------------------------------- diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index 53f131f88..1d5190ee1 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 378b6eb5d..0b48ef90a 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -179,8 +179,8 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs --- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs -@@ -21,12 +21,12 @@ - #[naked] +@@ -22,12 +22,12 @@ + /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { - use crate::runtime_init; @@ -194,7 +194,7 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ } else { // If not core0, infinitely wait for events. wait_forever() -@@ -55,3 +55,19 @@ +@@ -56,3 +56,19 @@ asm::wfe() } } diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/07_uart_chainloader/src/_arch/aarch64/cpu.rs index 3121e75ee..bd974d4d5 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::relocate; diff --git a/08_timestamps/README.md b/08_timestamps/README.md index b1244dcec..3148aa3a9 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -115,8 +115,8 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ 08_timestamps/src/_arch/aarch64/cpu.rs -@@ -21,12 +21,12 @@ - #[naked] +@@ -22,12 +22,12 @@ + /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { - use crate::relocate; @@ -130,7 +130,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a } else { // If not core0, infinitely wait for events. wait_forever() -@@ -39,15 +39,6 @@ +@@ -40,15 +40,6 @@ pub use asm::nop; @@ -146,7 +146,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -55,19 +46,3 @@ +@@ -56,19 +47,3 @@ asm::wfe() } } diff --git a/08_timestamps/src/_arch/aarch64/cpu.rs b/08_timestamps/src/_arch/aarch64/cpu.rs index 3f502ee02..eac29d8d7 100644 --- a/08_timestamps/src/_arch/aarch64/cpu.rs +++ b/08_timestamps/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs index 3f502ee02..eac29d8d7 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index c1e162876..a9035cec9 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -225,8 +225,14 @@ Minipush 1.0 diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch/aarch64/cpu.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ 10_privilege_level/src/_arch/aarch64/cpu.rs -@@ -21,18 +21,59 @@ - #[naked] +@@ -18,22 +18,65 @@ + /// # Safety + /// + /// - Linker script must ensure to place this function where it is expected by the target machine. +-/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +-/// actually set (`SP.set()`). ++/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up ++/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { - use crate::runtime_init; @@ -252,6 +258,8 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. ++/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up ++/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; diff --git a/10_privilege_level/src/_arch/aarch64/cpu.rs b/10_privilege_level/src/_arch/aarch64/cpu.rs index 3e5e57530..e546dd606 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs index 3e5e57530..e546dd606 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs index 3e5e57530..e546dd606 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 755576f95..e9509fd7e 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -935,7 +935,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_testing/src/_arch/aarch64/cpu.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ 13_integrated_testing/src/_arch/aarch64/cpu.rs -@@ -87,3 +87,20 @@ +@@ -90,3 +90,20 @@ asm::wfe() } } diff --git a/13_integrated_testing/src/_arch/aarch64/cpu.rs b/13_integrated_testing/src/_arch/aarch64/cpu.rs index 7d146d230..3e3ac706f 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs index 7d146d230..3e3ac706f 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 44a9eebd7..6dd7d6301 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -311,7 +311,7 @@ Minipush 1.0 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs --- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs +++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs -@@ -68,7 +68,7 @@ +@@ -71,7 +71,7 @@ ELR_EL2.set(runtime_init::runtime_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs index c68f1817d..d8bcc895c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[no_mangle] pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. @@ -39,6 +40,8 @@ pub unsafe fn _start() -> ! { /// - The HW state of EL1 must be prepared in a sound way. /// - Exception return from EL2 must must continue execution in EL1 with /// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. #[inline(always)] unsafe fn el2_to_el1_transition() -> ! { use crate::runtime_init; diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 064b41a451738e818b8ee9fe4b4aed8939d8088f..6d23c37d397a23916acbf64b4df727d723bf9647 100755 GIT binary patch delta 2821 zcmcgudu&@*8UOCRwsUWuwd2@HocLbH6e9s~{@LS?Q7D_lCWO-o%}%zS0xqVfX`|zLz4Akn zF#pEq{w<>n2AE`qXtRno zD?tdL6OzW};l%0RnP5f+ZZ%y3gINI*zM93Z-bS!Z!CGfpkP_eEwcJj&IF4H#=3!7W zMYRBkK-z1-CRxEIpA<&>P{5qTurIZ1UHvFKR#WenQ^M#(QMQP^h%ymnc67KZ$iB3| zCANHso5o_a#;~x_tk01SB)K=Z5$-oLL(L}Cz^poERY*nGX^&YLV6ng7!%aZ*GGSZ z8Rh_fk$xLNTJ(^s0+6S8nAvn`IYZzG%lziRZ}kkWpJmHGQ zkHqqU{|!tAh;r zRdt50n|{3=wbWD-$+Z4PPnU3*86XD`+O48X_s~Bm-HDsSFJ8V0BWe1M`Ci@=pt7Z# zCmuRvS>jKzuJNK)vSH%qJv3PUn%xVz500AkwIPfC{(~j@-v>nfLF%dSMx7E|id1U3 zM-OYd%+;*EI6gW1ei>YvEY)an9Qz@eJ@-AmkAq7=32w|WPTIt2)r?04SG(%LvH0$Z=#|K^ z$ePG$IhB>E7xe78llm(FFR=2?3UXp=Zp)E&m!lM(H|Xf!{4nAdY!PJsp|xd%FoMCM zS<~5O(q`xkl;EOF4KSGY_UNJMsDAq@kS`wqB35WQK&9av{dVQW?Or7P0X|RqbV*W| zaQ_)}r=IE{W%$1wQ@ny!kBsair$<`oV^u2mIGwNR!i-*gt17^82WgG9-TGG!W>_Jm zh3dqzU{9>mFIc@*?`#-OMd{J8+~S^e3!ShYaipq9pY+Ns7I(aVem}Wx7=OEdVIRF? zb@L-_)MfMWp*{3|o10(s(4)3{xM%4Z+fM#FDt*b;#EhRiy)br6dUbY{8B&fsg2skT zQQ|(e+;z#T**&UE$+xhg#^G|L&x1R>9-M(cXd2Zw3D-}3Bd0uq&Z;V|z zKd%zDzLlepoWdmCe7Z!(V}bsly3u=d(>ObK+t`gnj|$jBG*k+Y(Y+3m5?fIWk`oOz&Sz z!K(vHv`{2@a!hboz6x1k$u6-JQmgq%8cC9AHR^ zUO}&3FRbnYJzvw?QcP%@^ztn8t|BAe!^FPZ4*IcQ{V?2Les1hFOt8!0=0m&bUPreh zw1@O#3bQWA^}kT*!fTD4;`Aquy67_-##yYNJwtdY7>42e5S$MabEF8)e@H3b2pp-$ z)H4v79>>nOg|0%8Aco7>^6vwI$M->r-io@K1!$|&#?LEsud})0$2cWoUL!w+zQ_wo}a^`HWRhv-EOC(*1nr@HytQ& zclf^iMs)dUCINYFcR_v9j$YUF>^smm+D?_?sXRic{}`(8Vh4&ZvOk!*x0wmXobtmuhr2G z*S#H2xC#nbbouX2(EtlI|K08Ibr+p=xqCjY*^|XE+lp?VbzS_x^*PQpEEmGpHWd0u zajZ3nw5+bN^y`ko{G$#!)##SKC=}J`g+{qFv#l_iYoKp7wlv);`fwCu+`acq18s75 z2iA%^{BL)>UexcPT*LFUSJwC&ZaO6&?$`>SE@Eyi#$~;R?Q(IxsyP1_H?=7b^O{V* Hs~r9Z<o5G*ypArZunCb*7|)h7504^pBY*nTw~A}0XaJrHV}@_~QA z1m|`E*y};;?_%(Nk9n&Wx{W0LDXY4D@e2J)%C1A~uq zPVWpSgdeqaX7K@LUAt5n_?In5*DpJy)4f+$fSCM z4J-RabVv=c$ajj|UtiCaB7asZs|Oi~(124-QlbYlPc=be0!!JzNd5&R;`h`Q7gaSo z7{vgv?y^>Yj?wBLDHR z5aJBf8`1T^N1$i2L82l7MykLWWsJ8baywu?gBA1CyXAPMJ zTNe@C?TD@}^O8k_cLs%e5Z*zAcN>%9?Lc@#*tJw>0p9Hh@1GFfZ3r(7GzH;pL3mRT z-p)C-ogzt51Vn3+3AF)~kd54qZ+F(o6woP-4XZ&$iIsvf6J(VgpbuU&`GO(<49DCc zdt6pI71d`@zNeu?UX8sSD@{^j2z0b*!3L|$l$w;d4{iB4a-Cbk3Z-Y$liHXHbmf^W zkz=L5Riep^Lfx)S`eME(-u&^0;^XT&-~eq>s~cGi(UnCe(-F}Z9mB*o@m zuuSpU%jP^2>nbMKQ{T7uB`Nn`HYsx*X64`COjbU7j#s`#-m|!apWu*nFOm|w)=7$! zam3&G!GWXiSYZ4>illffQeukZ!y;4vF{DBDch)p2xtL_myC5Iy0@$v^Y@s{d<*50OvBW`Z3NKUbtH@NDE61UKPG%Fg}Sh^0+-(eBB140Qb@y z)w`HX^`3-5a7oG(VhosN)CAQpB*X2IBxsMi807C&O{_LKXnw45#4B^?M*Eh$*eIFdW%KG`x|O5J5xqIki)YG5n`78Q8w9QP9Iumy z_LY(Ai#*J$#F}1%Z9vkK-XPKRpEE1S4>QWF*BLlO2R4c(r#O4~2ARxor~iFk%@)N* z1;={W1?A*!MpqU#UbFDtQCu}+EmN#~Q&n%>yj@3ntVQhcWn|FmW%t#RUm+zGlfPM) zF>ex+t%{9QkZRjvs(gCrd|;dK$D?toNEo>XmHTX>IOg?O@@L&3-*ma<8>5^uT?q-= z?yY;z9Lu=Ml2Kcc*HtB-p5JmM4X+QJ;qzPO16R&WRme2HnU^7Q5bJv9RI-BWhupIj zy4OEc50BhapTt2MM&O2%@PQ3=^2X>2xziMCdv!xKsm@%&9^psI&89vo>CehQJN44U$sNI!gT z@|SeX>BA#jn9La8z{wtb>KQ&RV0Y0KK+6ddBX^>&#wa6Sc%YY5027R%FxhB#1!)Q( zeobaVApZ*;-No%0WFZdNMSQWO*d?sHRZ2vSS;AHhN3oExhw6AN8I;iCpt8_i+7j8% z;1Y4mbb&;Z&3cJLHQMr^i)0l-NtXwe^)NMgvWlB%+RZ4_xM8Mo-E*JoQH~OaW7RW8 zVV4S*j?y$e*cmslz@L_ZvhBh^TZ`pH;QiZm@^n4cbiVnDXYb`>H?M(w>n{FusE7I1 zD?R)CG^kwDdm*Bl*hH-)j(Cjn+c`Bf-T*ggKW|gE3 zA)m*RcmD(;17slENj}T6(r!=A7WuK6YDjn(AeGq;rjZ&K65T9>o!`>Xu)%l{KJZ8`~ zgZ_xTos(VriAk5*x?|Jk-D2M>yTraNVprcQJ2vgC6?b)S7PoKSA#U9%ie1Q}v@VD@ z*0g&J1y4h9vGzgwQd>|Y zbaOhnn{*a9Q+8W*>Q*5+Sg>SqlaX|m;p1@-YKq8*1$F*aMuz`u#;Xna<>c+WQ8HI3 qu?-@rEn2sH!F`nxGc6Q7^ey2d8u#h=O;4prE*5QJhehIbuKN#$->aVh diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 44e2fb7942454363644d51bf123834042de58a42..7517a699601e9f8a33042bf149d397324b7e364e 100755 GIT binary patch delta 1380 zcmah|Uuaup6hGg+NprJijY;#rtxa}qvvknPrpB%(=@?NSe16|^QeqMTwixA>tL;Sd}B2ooy z6`m{Bazs|6Ag|^))@SMFh@ahee#(BgeAA0wu?Zhlqu&$44=pPF z?8k7j_;~ov_m5#jVLw}6;VL3?Hl>J{oou>W|96yLwCz}M+vj-T5kY%$&MH@jn&ges zHhKM|AfINgy<*O9huP5W84bGV`%~g1~Ib4u4M*7EO--vQii0r0HwoyCT&kx1H=Xddc` zp2@@g7mEyYZ&fSE-zb7Zmn7@WC{~CkY*>_~9R?J!UrtZv z3@oyaxd*M8`4HQ5JugUYR6*FkSciLnn|HGF?gQQ;1yxSZ9XTYr%vzDNw%dWsEf)n0n0(dieAM#Ej~thbk&>t@XR2Dk2J zOWq{8#L_;$khfxHT^{@J+fVheC;sHY zsu_-?*VmS*oI`5P(bO-R`nuaVH~s((>F^r6c2i=XGr{hC z4R;e18tCeChWjr8wi@i_ZU&h=&cBg6oB*>;huQy5qri8*= QHORQOLGDzDooO5V7xZb5U;qFB delta 1668 zcmah}ZERCj7=F*a?RLw?)~@aP<>;*&blqT$4w*pQE}NLRC?<2FG10F3!11-3t_Xg% z2qsHLq{oBNs3b#;@ys9043YW6Y>X%i35j49{nu_WYg#1%4Oy8m-`jg<0{-wM?K$t) z^M0J?+~h-JP09Pn+}Bklf<0w`69iz)mvGS3JkU82=;`rC1Ro(z780W`9C(&xnMk+Ey?G zp7k4ft`V^sMYbTW6CQVumm4}r1EvZP7tQ#;=c!`C)wJ)5TAE2QG&_0ny({wRuWpPC z|8Qf(l4Zw@n}ox)>_=k@aj;88Rjj7y3*s2RROB(DVxi=+En?ueUNZ8ex?Nqpz*mGy!9{T8PDV!NvrSL$hl241$s7zXBVKp=z+|kNi z*r9V8=*QhaH(xEIx5Ua(=}72YrU@n_QQ4fCht1M7%8e)=d5aGbIO8?J^%Q|lFIaQk zrO=tFfMLm~v(&dMzU082cqr`H`8NNYuG|rfE;C(@`sTFtlaQEOO=04HnDfiTgf#xWv&kXdVAR+ z%dH$Kn)E4N`YFyy#FDzeJBx+7w5*X* zUX6q7q@`0xo7tb1dV4d?%>%`vlE=agv<&ylX_+-v)Cp-H3sp3gsBePcn;`TRcB;Y_ zPt;RvHQ5U5sZUIFtod7OwU9kgtBxYsBm1cV_oE~JLSWdK=(u@D^Hawrq0Kb2Jzr;3 zCWZ}(jy-l;W1SIcgE|w8c9?#Hi2~boUumj=st+uHdwmO$aweoeA>oDfS}|Q4Xc&Q( zmb4_gGYeUR#LkXe_Xxf^wq*6H&3uNLufg-Z`e#sYLH)kn-O+IL)nHE~+7}#*_5^qBj|KNfdK;pH1q9ix z;puSjfK!w4<_l|kG#ZKG$>Y-vmAc_xN3)Y%sce2wc4_g*j>4+eW#n{W-J7q*#abQL z+LInF?2zvMp)QZ;a?GmPwSRz>{g_=leWLGpA+I^t-M`|lCxw;l{T4mY&CWa52{Ue% xbvC#EpZ7U^M(RA(rS1y;zSQ;Qcvjc9ZWeVt#wJ~|;PkLL*G}PO58LA2`42fO)4l)z diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu.rs index 3f502ee02..eac29d8d7 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu.rs @@ -17,8 +17,9 @@ use cortex_a::{asm, regs::*}; /// /// # Safety /// -/// - Linker script must ensure to place this function at `0x80_000`. -#[naked] +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { use crate::runtime_init; diff --git a/rust-toolchain b/rust-toolchain index 9c4710ceb..23eb56819 100644 --- a/rust-toolchain +++ b/rust-toolchain @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2020-12-08" +channel = "nightly-2020-12-09" components = ["llvm-tools-preview"] targets = ["aarch64-unknown-none-softfloat"] From 3a430163b5675f7e5b24b4ea02a81caee3e6040f Mon Sep 17 00:00:00 2001 From: myl7 Date: Fri, 11 Dec 2020 04:10:42 +0800 Subject: [PATCH 004/214] Remove redundant spaces. --- 00_before_we_start/README.CN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/00_before_we_start/README.CN.md b/00_before_we_start/README.CN.md index 8d609d00f..58f0af400 100644 --- a/00_before_we_start/README.CN.md +++ b/00_before_we_start/README.CN.md @@ -2,7 +2,7 @@ 下面的文本内容是1:1 复制的文档。在每个教程的内核主要源代码文档的头部。它描述了源代码的主要结构并且试着去传达各个方法背后的理念。请阅读文章以便你能熟悉你在教程中将会遇到的东西。这将帮助你更好的浏览代码和理解不同章节的区别和增量内容。 -另请注意,以下文字将引用源代码文件(例如`** / memory.rs`)或在本教程的第一批教程中尚不存在的功能。 它们将被逐渐添加到本教程。 +另请注意,以下文字将引用源代码文件(例如`**/memory.rs`)或在本教程的第一批教程中尚不存在的功能。 它们将被逐渐添加到本教程。 玩的开心! From b74a931d01b3e23c3b588159d196165f858efcf3 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 17 Dec 2020 09:46:56 +0100 Subject: [PATCH 005/214] Update minipush.rb Fixes #89 --- utils/minipush.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/minipush.rb b/utils/minipush.rb index 9c743bd32..a6ccaa73f 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -28,7 +28,7 @@ def initialize(serial_name, binary_image_path) # The three characters signaling the request token are expected to arrive as the last three # characters _at the end_ of a character stream (e.g. after a header print from Miniload). def wait_for_binary_request - Timeout.timeout(7) do + Timeout.timeout(30) do loop do received = @target_serial.readpartial(4096) From dad68abdc6721f2d9e0eeedb35d39649649bbb21 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 23 Dec 2020 22:41:10 +0100 Subject: [PATCH 006/214] Update dependencies --- 04_zero_overhead_abstraction/Cargo.lock | 8 ++++---- 05_safe_globals/Cargo.lock | 8 ++++---- 06_drivers_gpio_uart/Cargo.lock | 8 ++++---- 07_uart_chainloader/Cargo.lock | 8 ++++---- 08_timestamps/Cargo.lock | 8 ++++---- 09_hw_debug_JTAG/Cargo.lock | 8 ++++---- 10_privilege_level/Cargo.lock | 8 ++++---- 11_virtual_mem_part1_identity_mapping/Cargo.lock | 8 ++++---- 12_exceptions_part1_groundwork/Cargo.lock | 8 ++++---- 13_integrated_testing/Cargo.lock | 16 ++++++++-------- 14_exceptions_part2_peripheral_IRQs/Cargo.lock | 16 ++++++++-------- 15_virtual_mem_part2_mmio_remap/Cargo.lock | 16 ++++++++-------- 12 files changed, 60 insertions(+), 60 deletions(-) diff --git a/04_zero_overhead_abstraction/Cargo.lock b/04_zero_overhead_abstraction/Cargo.lock index fd6d5c737..235957ef8 100644 --- a/04_zero_overhead_abstraction/Cargo.lock +++ b/04_zero_overhead_abstraction/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -18,9 +18,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/05_safe_globals/Cargo.lock b/05_safe_globals/Cargo.lock index fd6d5c737..235957ef8 100644 --- a/05_safe_globals/Cargo.lock +++ b/05_safe_globals/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -18,9 +18,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/06_drivers_gpio_uart/Cargo.lock b/06_drivers_gpio_uart/Cargo.lock index 7b2456e16..115e07626 100644 --- a/06_drivers_gpio_uart/Cargo.lock +++ b/06_drivers_gpio_uart/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/07_uart_chainloader/Cargo.lock b/07_uart_chainloader/Cargo.lock index 7b2456e16..115e07626 100644 --- a/07_uart_chainloader/Cargo.lock +++ b/07_uart_chainloader/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/08_timestamps/Cargo.lock b/08_timestamps/Cargo.lock index 7b2456e16..115e07626 100644 --- a/08_timestamps/Cargo.lock +++ b/08_timestamps/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/09_hw_debug_JTAG/Cargo.lock b/09_hw_debug_JTAG/Cargo.lock index 7b2456e16..115e07626 100644 --- a/09_hw_debug_JTAG/Cargo.lock +++ b/09_hw_debug_JTAG/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/10_privilege_level/Cargo.lock b/10_privilege_level/Cargo.lock index 7b2456e16..115e07626 100644 --- a/10_privilege_level/Cargo.lock +++ b/10_privilege_level/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/11_virtual_mem_part1_identity_mapping/Cargo.lock b/11_virtual_mem_part1_identity_mapping/Cargo.lock index 7b2456e16..115e07626 100644 --- a/11_virtual_mem_part1_identity_mapping/Cargo.lock +++ b/11_virtual_mem_part1_identity_mapping/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/12_exceptions_part1_groundwork/Cargo.lock b/12_exceptions_part1_groundwork/Cargo.lock index 7b2456e16..115e07626 100644 --- a/12_exceptions_part1_groundwork/Cargo.lock +++ b/12_exceptions_part1_groundwork/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] diff --git a/13_integrated_testing/Cargo.lock b/13_integrated_testing/Cargo.lock index 8930bc96d..0334159de 100644 --- a/13_integrated_testing/Cargo.lock +++ b/13_integrated_testing/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -37,27 +37,27 @@ checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" [[package]] name = "quote" -version = "1.0.7" +version = "1.0.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "aa563d17ecb180e500da1cfd2b028310ac758de548efdd203e18f283af693f37" +checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" dependencies = [ "proc-macro2", ] [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] [[package]] name = "syn" -version = "1.0.50" +version = "1.0.55" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "443b4178719c5a851e1bde36ce12da21d74a0e60b4d982ec3385a933c812f0f6" +checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" dependencies = [ "proc-macro2", "quote", diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.lock b/14_exceptions_part2_peripheral_IRQs/Cargo.lock index 8930bc96d..0334159de 100644 --- a/14_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/14_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -37,27 +37,27 @@ checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" [[package]] name = "quote" -version = "1.0.7" +version = "1.0.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "aa563d17ecb180e500da1cfd2b028310ac758de548efdd203e18f283af693f37" +checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" dependencies = [ "proc-macro2", ] [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] [[package]] name = "syn" -version = "1.0.50" +version = "1.0.55" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "443b4178719c5a851e1bde36ce12da21d74a0e60b4d982ec3385a933c812f0f6" +checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" dependencies = [ "proc-macro2", "quote", diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.lock b/15_virtual_mem_part2_mmio_remap/Cargo.lock index 8930bc96d..0334159de 100644 --- a/15_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/15_virtual_mem_part2_mmio_remap/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.0.0" +version = "5.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "626fdbb0f6fa1e6be6df0e94795998894c4ad08d3221e12209a6c0b4f546a0c4" +checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" dependencies = [ "register", ] @@ -37,27 +37,27 @@ checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" [[package]] name = "quote" -version = "1.0.7" +version = "1.0.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "aa563d17ecb180e500da1cfd2b028310ac758de548efdd203e18f283af693f37" +checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" dependencies = [ "proc-macro2", ] [[package]] name = "register" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "185fff3cbc031a1d17b86a980f8014002da35f9187cf4370728e0eb1eeff6829" +checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" dependencies = [ "tock-registers", ] [[package]] name = "syn" -version = "1.0.50" +version = "1.0.55" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "443b4178719c5a851e1bde36ce12da21d74a0e60b4d982ec3385a933c812f0f6" +checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" dependencies = [ "proc-macro2", "quote", From 18c7259c6065fcc8444eb9a7f116ce8ba5bbe248 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 26 Dec 2020 23:59:16 +0100 Subject: [PATCH 007/214] Default to externally powered RPis Addresses issues in #86 --- 06_drivers_gpio_uart/README.md | 27 +++--- 06_drivers_gpio_uart/src/main.rs | 7 -- 07_uart_chainloader/README.md | 85 ++++++++++-------- 07_uart_chainloader/demo_payload_rpi3.img | Bin 7384 -> 6712 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 7240 -> 6592 bytes 07_uart_chainloader/update.sh | 8 ++ 08_timestamps/README.md | 37 +++++--- 09_hw_debug_JTAG/README.md | 7 +- 10_privilege_level/README.md | 31 +++---- .../README.md | 37 ++++---- 12_exceptions_part1_groundwork/README.md | 83 ++++++++--------- 14_exceptions_part2_peripheral_IRQs/README.md | 82 ++++++++--------- 15_virtual_mem_part2_mmio_remap/README.md | 56 ++++++------ README.md | 5 +- doc/wiring.fzz | Bin 0 -> 61259 bytes doc/wiring.png | Bin 100836 -> 81105 bytes utils/devtool.rb | 3 +- utils/minipush.rb | 27 +++--- utils/miniterm.rb | 11 ++- 19 files changed, 269 insertions(+), 237 deletions(-) create mode 100755 07_uart_chainloader/update.sh create mode 100644 doc/wiring.fzz diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 08568cf5b..e3d502b60 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -49,7 +49,7 @@ init_uart_clock=48000000 - [bootcode.bin](https://github.com/raspberrypi/firmware/raw/master/boot/bootcode.bin) - [fixup.dat](https://github.com/raspberrypi/firmware/raw/master/boot/fixup.dat) - [start.elf](https://github.com/raspberrypi/firmware/raw/master/boot/start.elf) -4. Run `make` and copy the [kernel8.img](kernel8.img) onto the SD card. +4. Run `make`. ### Pi 4 @@ -57,15 +57,15 @@ init_uart_clock=48000000 - [fixup4.dat](https://github.com/raspberrypi/firmware/raw/master/boot/fixup4.dat) - [start4.elf](https://github.com/raspberrypi/firmware/raw/master/boot/start4.elf) - [bcm2711-rpi-4-b.dtb](https://github.com/raspberrypi/firmware/raw/master/boot/bcm2711-rpi-4-b.dtb) -4. Run `BSP=rpi4 make` and copy the [kernel8.img](kernel8.img) onto the SD card. +4. Run `BSP=rpi4 make`. + _**Note**: Should it not work on your RPi4, try renaming `start4.elf` to `start.elf` (without the 4) on the SD card._ ### Common again -5. Insert the SD card into the RPi and connect the USB serial to your host PC. - - Wiring diagram at [top-level README](../README.md#usb-serial). +5. Copy the `kernel8.img` onto the SD card and insert it back into the RPi. 6. Run the `miniterm` target, which opens the UART device on the host: ```console @@ -81,14 +81,16 @@ $ make miniterm $ DEV_SERIAL=/dev/tty.usbserial-0001 make miniterm ``` -7. Hit Enter after seeing "`Connected`" to kick off the kernel boot process and observe - the output: +7. Connect the USB serial to your host PC. + - Wiring diagram at [top-level README](../README.md#-usb-serial-output). + - Make sure that you **DID NOT** connect the power pin of the USB serial. Only RX/TX and GND. +8. Connect the RPi to the (USB) power cable and observe the output: ```console Miniterm 1.0 - -[MT] ✅ Connected +[MT] ⏳ Waiting for /dev/ttyUSB0 +[MT] ✅ Serial connected [0] Booting on: Raspberry Pi 3 [1] Drivers loaded: 1. BCM GPIO @@ -1212,7 +1214,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -116,16 +126,53 @@ +@@ -116,16 +126,46 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1239,13 +1241,6 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs + use console::interface::All; + use driver::interface::DriverManager; + -+ // Wait for user to hit Enter. -+ loop { -+ if bsp::console::console().read_char() == '\n' { -+ break; -+ } -+ } -+ + println!("[0] Booting on: {}", bsp::board_name()); + + println!("[1] Drivers loaded:"); diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index 4e6840ad1..9e58e80b7 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -147,13 +147,6 @@ fn kernel_main() -> ! { use console::interface::All; use driver::interface::DriverManager; - // Wait for user to hit Enter. - loop { - if bsp::console::console().read_char() == '\n' { - break; - } - } - println!("[0] Booting on: {}", bsp::board_name()); println!("[1] Drivers loaded:"); diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 0b48ef90a..8872bc18c 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -25,10 +25,13 @@ Our chainloader is called `MiniLoad` and is inspired by [raspbootin]. You can try it with this tutorial already: 1. Depending on your target hardware:`make` or `BSP=rpi4 make`. -2. Copy `kernel8.img` to the SD card. -3. Execute `make chainboot` or `BSP=rpi4 make chainboot`. -4. Now plug in the USB Serial. -5. Observe the loader fetching a kernel over `UART`: +1. Copy `kernel8.img` to the SD card. +1. Execute `make chainboot` or `BSP=rpi4 make chainboot`. +1. Connect the USB serial to your host PC. + - Wiring diagram at [top-level README](../README.md#-usb-serial-output). + - Make sure that you **DID NOT** connect the power pin of the USB serial. Only RX/TX and GND. +1. Connect the RPi to the (USB) power cable. +1. Observe the loader fetching a kernel over `UART`: > ❗ **NOTE**: By default, `make chainboot` tries to connect to `/dev/ttyUSB0`. > Should the USB serial on your system have a different name, you have to provide it explicitly. For @@ -44,7 +47,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -53,7 +57,7 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 7 KiB ==========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 6 KiB ==========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now [0] Booting on: Raspberry Pi 3 @@ -429,18 +433,23 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -144,35 +148,52 @@ +@@ -144,28 +148,52 @@ /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; - use driver::interface::DriverManager; +- +- println!("[0] Booting on: {}", bsp::board_name()); -- // Wait for user to hit Enter. -- loop { -- if bsp::console::console().read_char() == '\n' { -- break; +- println!("[1] Drivers loaded:"); +- for (i, driver) in bsp::driver::driver_manager() +- .all_device_drivers() +- .iter() +- .enumerate() +- { +- println!(" {}. {}", i + 1, driver.compatible()); + println!(" __ __ _ _ _ _ "); + println!("| \\/ (_)_ _ (_) | ___ __ _ __| |"); + println!("| |\\/| | | ' \\| | |__/ _ \\/ _` / _` |"); @@ -458,8 +467,17 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + // Notify `Minipush` to send the binary. + for _ in 0..3 { + console().write_char(3 as char); -+ } -+ + } + +- println!( +- "[2] Chars written: {}", +- bsp::console::console().chars_written() +- ); +- println!("[3] Echoing input now"); +- +- loop { +- let c = bsp::console::console().read_char(); +- bsp::console::console().write_char(c); + // Read the binary's size. + let mut size: u32 = u32::from(console().read_char() as u8); + size |= u32::from(console().read_char() as u8) << 8; @@ -475,34 +493,15 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + // Read the kernel byte by byte. + for i in 0..size { + core::ptr::write_volatile(kernel_addr.offset(i as isize), console().read_char() as u8) - } ++ } } - -- println!("[0] Booting on: {}", bsp::board_name()); ++ + println!("[ML] Loaded! Executing the payload now\n"); + console().flush(); - -- println!("[1] Drivers loaded:"); -- for (i, driver) in bsp::driver::driver_manager() -- .all_device_drivers() -- .iter() -- .enumerate() -- { -- println!(" {}. {}", i + 1, driver.compatible()); -- } ++ + // Use black magic to get a function pointer. + let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; - -- println!( -- "[2] Chars written: {}", -- bsp::console::console().chars_written() -- ); -- println!("[3] Echoing input now"); -- -- loop { -- let c = bsp::console::console().read_char(); -- bsp::console::console().write_char(c); -- } ++ + // Jump to loaded kernel! + kernel() } @@ -583,4 +582,18 @@ diff -uNr 06_drivers_gpio_uart/src/runtime_init.rs 07_uart_chainloader/src/runti pub unsafe fn runtime_init() -> ! { zero_bss(); + +diff -uNr 06_drivers_gpio_uart/update.sh 07_uart_chainloader/update.sh +--- 06_drivers_gpio_uart/update.sh ++++ 07_uart_chainloader/update.sh +@@ -0,0 +1,8 @@ ++#!/usr/bin/env bash ++ ++cd ../06_drivers_gpio_uart ++BSP=rpi4 make ++cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img ++make ++cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img ++rm kernel8.img + ``` diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 9adf8152f7447be4c0d35ec20c96f7c67c377510..cb83285f1b2ca41d0175a18604b3e5cca9882987 100755 GIT binary patch literal 6712 zcmeHLeNYr@sHROz4H&V7q0Fi#=5E`-dbz8wXW)_E(6)x8W@ZLjWQ8f8BrARWsfFzR$<+`FNlAdHV!~{@^gnQ_Vz;O_KNrP16=wg`*7)fs~JO=&(f6 zziF@0rJ41DI-lfp4Vk*Nxiq*3^i|Z~Fyf`~5~AW7lDZcQ7<)N)gUwv#vCr#oScCcM zV)BlGW;{sYa;B5z?maqP{sx<^T&K$e?XpL6y-eE%9;Q!$e*AF?&jC->%S9V(ps^M_ zaEIQxI^Ub+NrHYa^MNM?p79zALxz&TuR^fKVz9$OLMmaQ#xarN)g?orH4jqJ>ogrq zh_-mdB&S`BO%PJeCTg_h$YHXQvaE!PY=u*L#6V|8J%L1VAB zgC+?Yr)|!Lwad%ARf4tJQMTSzoIrAP2EjhQy}JnkZbdBAyA66jN@0!%U^94#Q6Z(dopO@S9YsjAHI>6CIu@ zm(!rv`DWIiRg6MZIiii~wFnl8!zcBB;sSE%3?^9Qf4>Wx3c)I~P z9Do17pQa{8epWaLbTm4N_2qo!!duSUPYu_Y=B3>A`Q_3Nsfb=a z^%i~CI$XGVPC1z=R=C6+6A0@LYj1I`9Cn-J@RVSWtt+HU@*xV>9;9&H5eonKR}^l; z8d>*iQfdwelP6J!c&&}Yc79)uH8=rW%YHyb9l+B5%E`BO08jhv*5SyKf^ypuXY5h4 z9CiwFc%6+dT>;Kh(4z{d8Dd`iq?y7sPMZ7-a@U#ZQq?&s90N^Nq&j4S9;?|Fvw`1g z@TV}>4*m%E+90FNBM<7uKd9<2FG))M4&#SZ+GnfLeCCMxgE#uwF!b{z2GZ<`5o= z-VhHS(abd2E7&g_1ODw;yZcPG*hayYa^n3&&ca}1R1TvSoL)CsqM#ncYYM2_S@KSq z(<~o)pJ?9Gkw~cBmLHoI%j3<0Lk=5!<*D6&LO?po63dn;$0VXSI z2g|xwM_Ygf|9X_d+n{Ucw&gDNk^dR{O#}N)1N&0oi&#*@wA$g&?b_i*sApjdO(qZ< z#@zx8*1`;CS+fK#VDDmF_hNYh7>-U`g5PIn;-)3o`%Il$0v&9_2e~wP6gl2j#C5S4 zK3i1H{*X1z_fL?W#S#RTQee3Z`ag%|1n61ki@-D6ztVN+eXdTunVZpz`=o?=e&UYa zQbVsL@EgcMEUVu(?z`aIpTM_o;+@weauRF5QgK!+P?AeOQWF*db)J;}MBV1Sn`=g* z?w6lSKa5W`|MvG&)RLIuv6;CKO<%{JeoZvRYvk44*Rd}$hJIkq`t&IBM1o(gWAB); z)J-$re1Xm|%?NUe$43z>5qZP&&E&Yub?{#LnWle64hg6$CRML-R<13kIO}sD=f6Rn z*^PMC;JF|77ZG!Qt|+F)*c>?>H_eZYK}Rf?&RjvB&c*zrrE=&H_SgxrFn$sH;#Vcb z-Z`aZ-ity}^%vM1oAashfKYIu9dcB`7P}(m$2qP|LP4rO5A{JTh#y1`@YReKct^0O zE}|D8*#7re*C%bHOoIL@^_fLV}{PFuxnDZ$9b$0V>Kpwiet}M1`I4;pch#6 zU55VaEIr1Kf19DNy`?oyVSE^|n}97%LVk+Slcb<;2k`tZXFAXaJntKL4g${s;Q3d` z-``rcp%1!0hRz3oX99Ar01uCeA~l``-w@{hu0#%91brX$GHoGfzsk}w7TC$Op9%TZ z?C+DPWBWnReFV#Fhd-Y}pRpbDP4Fe5{@4)n8ezUJpYJR;zI`mzBna+yJdeW`5&XOl zaIBgHn*6}@`jH;fz9Sstb7pEBCOXrHJQtv!X_DYs2%f#Db)V$W86Lx0EpEHjcZi?$ zJq7Dy0{z8~$)-@}5;=6rTvl`uzq5h~wQt{%FK}O1^7@heHkuqkZ`EhArq&9!n7)=w z))?nb9{kY`o<4)8NwB6`G_8^KMohK~HF#%x7~5V08{5M^&g41QM$Yv*mGk_@^H>W_ z=TC61Z#S@_&x@Y~R*uI(VEqNKGR~{`F-FF!;oe>nJ96p{R(mR9V2zvXvG;-XX<*$C zo_7o$&IQIAL!EUB^Dm(Hi?Lt&Q2WQkSXmy{KG*+tp;W;>IBy5`fc>_D*iHDXAA11T z(Q!QIz!p1d7Wab-aDVQe+Ql~Ro>fOVXL$TqcWT%CJ9~pHQ1F)xb?P^;m+NISe70Aw zTfE0)>z3xr)Qvi<{fxi#I)waUKi)fLwaTbh0(hRMh0h~H7z94WHK;QMrqurO7Dck7jxhugNdO`|VJb3cb3KplJc74=i15BLnj zIh`HL%7HKMlXDJvI9m>{%M34iIMb%-5!CiJ=%yvI%C;oT6y!aT>sz9-POXjgsW(SS z`O8(LWcTytv?ai2w6@aweN!sl^$q*uj(0R1bsLk1+)7V z>l2`FA$0ZB-PPAWV+nAtbk7QY+gDT?tFX!H9ZbgF!uDnMj5Ub$lZg2f=r!uq znM#~FR%LpPgk=ytDGeln!6|qYZSyl9}r>`MD4)2kazCAQah~-|?1CDW((ciFtB~3fSp2gb9 za1J7}>EN^RSoGz^6FVSbtop;2b;9v!_Za#T~W|u>Ryy z(qqOtog)t@my1gDGbNubN1^Z9d!owgB&p@7$>XGq{|B%Hv6hdURShv3noE9;d5Iyj zisiICsC+Pra}Ca>92Z><*8f;#j`}U$wLa7g$o{4;SH<4`#?b=u!?y#>dw6pdc#7jm z)QLRAvJGQ5HSO!f&rRw+4+a2|V5VL9uvSb&f1Pr{E?7`p*X*A`HE9C)?i45m|%(U1DT{LIUA&WeY= z*MICg_S;eLz_)&m?^?tcXO01Vy%pA{58xZa0g?g^d~e|A-)T#O2LyTWnML*KQ|MVP zVqZDyr|#v#fwE^7nbQYBbC9ZZdVao$?-K{`eWIhJKD{5b9VO;;z9DzV?WMsh=!*#Y z&`S(p&l;darLsj{thC_#^wXimy1lpA&qiO%7-Kj~*$V2@)n<8cJNm$CGtSVrke3*9 z{0+W^0IQRG5bUFNdb5G&?0xjx$HCRNR{Hmk9&j&YI7yx$-4D;@c7SY zn&&=+?0nP%p3}~whIlER0?xhgxAl+jMTgM4vLEs7Nm(~ptmIAB=xZ;5HL{P->?;J- z4nI}{zmAW;;VAcGyn}s~O}5-U4NPl;9=78IVpa{kdH9Ynu?1&j$l}=7l+YmOa8<42 z!>VS`>~?&ZI4&s5&fy(2Y||RfkKId@30{YNIj9Yynyf@#E;Fm6=Sf)vo8(;bH^Ha; zozQ03F*cXdY{&nF4LWw<({1@a8-Oos1INvJtWueWf{-*Grk2;WJ1P~)u+%Qy!K*ek4Di}UZnyLowGE9#2}_n!7enxye*MuHcez|{m&fIG`CNWiz!h|bTwyoZ-5$5s z?Q{Fx0e8?Ha)&)GkK5z%cs)Lk-xKfzJt0rn>+-t29!-L1FnEO;0bsGzJNav2m}M6Kse|Mx`Upe zH|PucgMnZ$7z&0%u8=$A33)@lkUta%1w)}wI1Gqks1HMQ7;Iq-O)tFPPB))|abdsG zuli`TD~hLbW4zQDpG`N@?HH=ep=g-_OGPiSeFm&|fs(vv$tr zm-Oo~X`^?ORKKOIiy=okcfQaob#}dE-{{#St!itD_DC;9BfY(?ohzk{?oHBa-KHMh z#@4MX?HgU2q`I!IUS{qB4RkLo%h)9w_DaTkoALhV@>VLz@VSh8mJU6)er0A+RK?8G zR^y(fGsy1bar4{^-6w`#^ndzQg#YFEaol!BBb~hnTtiFG&dsgS=q~Bmh~&-C9W&yC z?+5y|rKO{zb*t3e(Ax7tNAJp&FLb^XZP{7nY=+@dOQ+O~=r>C*wsgGEs?+~(xCRXQ zI6LVV-(~#zXX%|CH~Ukq(azQmX=h7kWXnp)uG_)SO)s@X*qHkWD(t-OpL}-h`j(!a z)@W}8VE?Km0!vm(715T?ZLN<*dYW2y?CjlD<7i*qPnE(I) literal 7384 zcmcIodvH``mOr;|I_b_sl8^)pI(gT~ zZ#Z*xvq>!?di(J5D&>l3iB5_|(Sm*evR*Vg#^DT;JOql;r=x;QQ z-=kdJoTDzr_{%NXYLCIG6f^x5q~T^d*1g|Rm_GCJPPjrTqvgt#*F@EdwaH|LI(ZB{ zo+o7jv=_Rv)N7BM)cIz!I{gQOnmlGyr@WNd4f>Q%ZswQu#rP^2cfg_?K5Ef$#(kaasLo;TYG0G@+W&$#y@wSlrOq6RR{16Ws$cvQ%-31 z8<#ZwHl%q1pR9*T=(rMdbJ%-j2_>e%ck&M@-pYWZ-$-8Ge?ymJW#6ht)*{J7KB~dyK4c3m(_nJ}U$rvRxf1XU8)Dqp$9rbJ zOa4XJX9kSz>B5Js>vEk99e3Tf4?gYZy#x9_jUM*R)Q}(F$|k`axet1%x{DUSLG#{e zF*lkk$O()KUBLZ{V2%$NsOes#>~AoW(pF4`EAy#v71rN>M(iHITC)ywf3fU)6YHz7 z)=^yQ{CrI8KBlk77zfsQKm5KZE?R8UqB(jiu9Mf3O_739C(}o47hTX{ZpC*LFb4{X zG?=)UXM zkwk3pzB&^PvyWKG5Q&RIw7q!L_Z)n65c(b$bD}xm(GDK_z*7r&uF8=8WhPQOpsRZe zsIVD&-h}xYGROPzURfmj2C;S=I&OxJ_vve+c-yenfVH&##-Zah4%0%isRR0DdehJy z@>|giT8C+aek0KDP9fXY4E^@QR~as?-*&{;$B3_;&@c0v z4gEGizuC}l?R>I{b;i8)97+dKdc)S<8hoz7CVjb*s?zLHdoLH#}m()tu$i3 zL;qfd`Igi-@J7wp$GG|CJjnUR6mTk5+%Qu1XN{KM z81`>zp5Z@CCYu}!$)EH2$vBfIgfk0HO8hI}_cK5L`C521lZKI-4o$)44S!joG*(i;ddaKiocV&V06!Va=dp z!P@K4j}3HlApgvEf=o2RepgdWBkXsTwU&1)p%2?$%e%2-f>JexF<@A(jo4=$U!lz4 ztVs4{;QU3tFUR>@WmG3lkn%2Mk~7Jjt{=?098-%Z!E*dLWYBmAf0``cCIk3O%fR+D z1;Bq+ivP(fQPt*NU7<|k%wQRQ1{vsD5-IDJ+`Y6y*&eG%s<%p0gE?Oze-#k_Sx!e*35cEY|kBXkWLn=r@r z`c@nN)9v#b9Of9O1^x}5H2kA4zwoK_)*F|HBEPvjR4{#+*JD{DTAec~qDGrtrfBCO zWSFi;P99sP3<@ii`(fuv)LN7Ef)d+D%D9*jomeEO$v#n?UX!WLcnzo*GSu?44oq@_ zaU)Ni;WIXt1;2ofINln@s}#<2=U>?P^VG5A!lecWi0F0DSpZUzqeT9hwHAk z@m0!w8Xu^)kpH+2OyMx7TnEOTz~vNG#G!`gMy!IKb?-{ljPI$jECDqpDdovj{3YuV zd&?+sb#sR5e6UU(7YQ~=M}?-^0= zWvXAI?&W&;5y*c9@|iM5Kh@6FW2jBflRqiK&vGacL;dubnTA{MkQ2QR%c?~vj{oY} z{YQS4OT+uKW!0K1C&u+!nDWT`lkUK5Ho$)fvn`WLqJGrxUo{(#~ z&Vmo9rhU(-CyCz4htEvHZ_{&We4LvZ--P*ge8_iyf{*ie3WuPQxOJZQeUO#fM`beW zTj82DhC1$tSu{I@ej!#~rv@T@>eBP1JXKGM3B3Ws49#bGDuA!GKVr1G!oq@<<4^qs=hX zh_mvAK7qO)5e7;_LQ!PDknf2@=7)_m{5j+t1OKRtqSNA%=yzC$*gKGk{EYo47nXRx z#M&J0od!A4jPry0HYeadxz_gr@55Ohoi^k}E@ICmF)#Yt*=FCF3fY&RSz2hvZ7~?S z+lnJcYVaN`9z1dhem9Pq;C-Paj+!Y#pg&_MiD-44v9$1gAqD@6kR1=;tg;Fv5v!1M zDouYc>W!V~4gLmo3;MREjmD+k(1h&gT+;B%D)f(*XiXk;QwHoCY-XO}qFBI~% znR6ov@vdl#U^$ln%p|`3nr%3Sm)a&E|9R+T4`Nvs$m56YV&a{!+j{7G9rRug{nx2q zch#y3SsUg###@$iUMP-EAO|p<2!H%G*cP&!`>{tJc?x`*P{XBg-%hvHLnC`8$p6<& z|Ld%qGQIl0%QUCU#5Q7?Dk0NA7AgCawP*d;YtNRV7UKAcAqJV>0QfK!q~bh)<1fI!<4|LAT@vT?KexXWR4o`^V7l zQ_yb&@I!_CQw$D)`w;-P% z5b{o;&zM!wYh6Hp#rA0y^5XqDh&ROPLG%L`7m^;1v(xAcI1b-JzjYk_)-2#J0srTF z#qJZR&2hdJU4kxW#q220m1C~PkR4|mGB1a8Uby$F|2hT#$22_Vj(wGauf3LRx{Ud= zu)z#uX%=$hgq$A-y$M?k%C>42X!ae$b4Pj%14r1#V@7H^ zOElaEzZXECVfa7SQta7}Ty`>phI#J>jN6b`d7a0v8*(DY;p-RBuN_2QeI9xBG;-u6 z+@B*)T8jscB=K%2e(uPddS2}_-UP1~VavC`>)YU!`Q&pT z&13p3IDLcH+&KC=jn_B8>q_uCh&`|9GI4Lpyhe~W8-&HDHVP&!7W$Cir^QHV4(gr; zBm={12$ll>Lnc*-!ApwgNgM93sDA zko@9{~rcJ%WM_4$o>3(2l3ZtrZX48=V{af9|`zJ1RwguNd9Ik&3**D$YWge+FxV*#8{%4FJa!#?#;haee3>%&!-6PSwS#@V|u5bra5dOEh{t z`SV-=vY;eNDwt5nz3)yvqsI@e05fwXJC4v#uryI$1CkWITDwQrTy zY25T^+y!^6)wYi6aIm9xN!qnqn$+hr_4$kXIL+^$;CF15>RSWh9_h(&sJAx=)>8PE zrsQ6%q&6-h`ukaucyTnp45{l{r(qY>Z@O0Z`a3l z_4i8;-dAn6OMg|{*t9*+-X7c`wKN8McDMJgUAw#U$#7s-rL_e*lmeYn3(VFcJrQW% z9n|^E&q`BBO{DesSxIW1?X@c$>g?4GmVakzK3Csgi1#Yoje(wB+k)Zn9_hi53N1!vbz4^&ND9sG$zNqtqyry0JmT_m_8w_^_+ofH+r!1RJ>TswrwClEYxYy)2 z*5~wbS|63Jh(d4(;V#B)w`$C$|~E|vujN_ zt&&}#H66i@uJ9h{u)Z{5b~+A$@G474%*5Q6}MZ$ECY5&FkyhBp&H;Iv#$j^O0Up>*sg+e5i9r z@JCV?{JZNhXpWWy1Iejjl+vo9leLkPx z4;K9(?+0o>w)rvD9@-WTg!io3-W3k6VNZFiqjyb5*Nz+cP`A^vl-RTd2R!?9%Qxl& Xn7;?}S{~M5r5f#(ufMgy&BuQOPYV9h diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index c3b1ede6cc6ba9075e528da9dfaa43ef960e33ed..9abdf41e85b09f016e2b760668f110cab1de31e2 100755 GIT binary patch delta 4030 zcmai04R93Y8Gd*7e)i6UkmN!V63r%O2p9edkQ`RC`N5We1~L(yQieo~+yP0*h1O!R zd!QY)hxV35WzgCPjFgcvrc#}b!zHwxO9G<_f#kqs`+R#Bv`jnw zX70ZI{+{=JpZDkcKB_of7S3Pvu7&nXB>mN>p&pUsxRFNPmBe=(B5Ld_Bd>>o4IV3< z>L7AelGIVHy(JW+o)hTQlWazoI}T~*ggI6RPp2N!t_$U!l@2;NSVrnf&Jkk$YAdO8 z@+tEujhCa(7!*<>DV!WH(;l)s%4KP9Til#WJ7*~ooj@EC3QDzaEUT?UW5%+x+Cx@% z@o%9bM|7f4cB#?Jr1lG>zRJ`7$vmhlaO!DcHIvtyXa5zl0)mo20;ZuNvdJwN@Ok@J82eB|okaKqOHZ!|#b$kCVnHFBlzxs2BP zBZ0P*V8&N&Qct|!6&MyA5zjN{U-wL`>aFCb4=q}ek|#Yt8MN#8>-0@aZ+h*tJhD#wmCbjY=ooMOx7pLYqianvZR@v*}WN)2=dagpz z;}E2CkAI zfcO^6-RwX+FUOZ+-oeTHwpoJ>^Z2fYX`r%_qmCgcdjT%`J-2se0e4T}hVb%1!%D+D zIOqAdV89le9Dk8_gw}J8ArHn!EIH-=e%WhB+?}XPcWIZV-K~rZd66cr;Nm#wkKwxv z%0-3DNS}~h+AqwA?17(~h?Z3#9_(0aDyr7|gse!v;EIT;h$F5NXQ?zQaCcl>di&Md9}i{JmtJ$WHUkEHQ+&Dc8%N1|C{2`F?;!r zhpY80pfROR*=V@SLXEx*IUa?9eL`+2Gf*~+CGP=yN@_Wz&Vba5*dK+98FuqBS^FR@ zl{=z+oR*g=jRJXHwLWG8vUWYKKzS4AL2~IE$$lqnei}9(z?db6gCz*O>8Z+6V;hmwQd}e5MmRzwHXMU4Bqj1ilX2WOOeFU#Bf{HS@+2 zZNF=_M-*lFjzj#2k?WI2wo!v3>8AM2%~m9b4M}20kreeuwc}1Vm#y_UGnE^6(a9H_ zvhGQdPg!g`07E;AwB;$YY&lr&tk#}Psh{u0{a2GHCsBJht{4i(AjFftScTid z>>3-y?ybZ3seAi!w4XUVO4+3GnuP2jN!ED;w~W9h$YKd#)NnnMc@!V5x9DYSSRzn_ zWgL$Z)yC0&cJC%Ci7T?sFkTz2M@eIMa82S4j+q{2XLyuDF;WFjOHwuN0NmY8x#}Sb zDh({7ST>mbQGPP}e;w7bNv>C{OI3RCpR9lpdE1Htt0jAa$e3Fo2N|;)=v3 zjgv7u>A@mJpQ^pFXE`-$w=ptKc+O;?bXasoidjB)FU-Y+RsR9}Jm}_h0~hxIUJwIr zfoT~oq?u|Ij15jCSc;H(p?Gds>?l;valwXI37J?ku=V0uVE>)hGx6PdXqJkS6&ATX zt$>r1@V74&PMrYbr_*TZHG$&hrrmiei?b{oIK>y43L1)4zpmI4I^dOa3kr`zFUEd9 z`YXWbN~NW1f z5vhp-`D(0J(0vTVu;|8D2_ld*antzQ&nr&-goTNd%GYD>EeK?8-}R008WUt<1NYh# zRtd!ml-)WCtvcEcBXFOa4BWVkc)^cZ@E)_kPUSw_!8f4XwQP#Bc=epgQ|p*Q@8BuL zlK!Fm2Wc1T>M=IOHBHq4LkHE4uQs{jQW-EA>trxCdX&B z+7Lg>`DJbV0!r4yUs|+0VW#p$ZG10=+rF^GQ$DMWzmzL?w@#FlU&W<_C=C-8sRH^< zO_GX=7k(}(nNphZ6uHGXOI}P&nxm&SUTT%QLwJ^K$#RvITB%#aoP#0_j(<`&OF8yS zL{5py=vd|9O4LeD9P*=}%mv{L(Bz5D|ANNZ+BwwgEaV?sGd@JlmLZK8zg9ZQLOroR zQ|q+*EaAGh+}b63etIG%4|8h&hosKKZY@P@SHg8?3RR>~d=O9Bgd#BPugY;5!3M?v zZdP0k1rheG?V6Fl)=&fo!xtWk5Hy#Q(SbYCbV@Z*s+z99P#oB&}J!aM2>^ zk*bD9qJL=3IoU$YE^0#F$4W3~3daPc?iqu~CBi+q*=e`5Ub%c}I zp2vSGew~3$3tF4~ZSw=2CX}?Tz4-}$+h*yBfd3~g0cE+gTrv;5!X7$k)6IXP?A*i{ z#G7q*avj-gwzlLLHOFjs?td(~KBe%QAJReC=xbV%ESS5N)NPPLw7qgYAjfrhUQxratme{yPQ**Kx zGS6%e_}g~CH4V+3+aGTU1a?Ym{1S_4G*42^w8W6K`8f*<{~vUYR+tMS?>Ks!T5WA^ zwqy!!Zu4*2_*Ao>Nq4Sl3AD9rleV+TLZ|jZ?wo{|f?7{*_T6m%Bk_oV#1;wGey7zk zkjZZCc2(>4z4zR6zVo=}o~t*=|0kH<<{ykl1 zI5)X2PrbR;q}~v%u@a&q`$rZlqhd*{cRP(y4mFn6kU|x*YMLphNT|NhT}zlRjZr#% zbf=*>8qXt@Dtc8@VP<}GhoHpsa#b_YUfZ@xWpq)FIx3Q40q^>g<>(vL)zsIJs1($6 zuR*;e3d*HQQ57&>(KlC#*O9W`kikb7qxfQFCi(`_F^l54$<93W*3%|+B4}2>US|RR zeANV6)4>JGheBV&=pxc&mIA}O+kxTF2DKRT-5=N$jjm>%vIZD#91_(_Y2Z6ViVgkr zA)|W9M=1@<39AynNXiKCH9$ViAkPhvQrh~8`5L`uuMP7x9n-x=^=5BwMh~q020D*p zZ5%E0r~!|7A$b-<&STaJDY=7o^gWFg;^p#3H*uXJT9E*8gF7R%B2gZC8AMU+ba zVw?IEFn`r~N0;%7z=fCfm=3;!dE{%!l~d!ucViZ%;Oir{gFjH}8Q1SbHNn{aY*aN} z@5TmQn8zgIug^%nDSK|SYWCz<`K}$a_8Y?*e0$jbYv5y!r+mBxb5^Bx&yk0AljOA` z&iVV3^djGGK^}T*J{2AJP}fpncljd~t5Ro;q&!lQo7w}KJ+xS(=kK%WWuT{_w)v}4 z1E6V}pPS0^ZNI(9cLjS#^V2a&5g~J9Uxku`_LuSkO`njlui<%pPn{5h7t(cf&O~x5 zJ)MTC$Rw_b-6m0?1uPAUW$*yUiH?`(6)+DMa z(2sQq7b$B5bMztTe*Ziwev>AAXp0hJL99TKQ*Q8mO)w`14b=FcQ4Z9bNog&mq9rq^ zXesE9za)0`V=k!EJX|XK-@<%*X}OE}^qv&E4(ao8=7D)$k2o($iq@)xXpY^@>-6n( zV{~S@i)CXEmt4_l?&kM8Xm-ymaV5pVDyBm$Oz|2pPkZd}(sEZ+$gk?HSt#;8m?qPW?XYhGwqu#dV-ze<6S+G96TYzyd}G6SnOBA%&V&kwqU9&F)q*K$23Ae3{6htJkJQSG zlu%Az!$JLgnWFs_O)&k3>2#xaGW{DqFPUfhlyGj+MX6tde$Yrq`E1k9x=^LU=h-e` zwEXMwNEAHdm87m7tH${zsH>5WT1rb?W{NeehFwg#+Z#-vTQf{UtRv3lA!{iOnQN@a zG(ERjk6ECj<)rmk74)-UesrULjvid7aL!*1Sqa2ox)*0yCC+7>ty3|0NKi&jk#dOj zGLk2#*8n@QhUGHWeqgi!Ut&6?EWlT#<6~JC;A1_^!1o65F{U8KgTS~N7)`*o1{hZZ z-(uinou=?r0AC)Z5{tylxg1&u+W;fZ$SHil%lKYkeBY#zVsO@fz95rd2P-rAweLQS zhV$zJ-9LmF`59sn=Sa+yBV+%WVmA$h5XO4xhl-%b()Y!JxpY;k$|w)9Z#F~MDkJ=W zG4_f1KWK>kYXyJsD$!!-xBEVm-g)!-VDyjI2WKX(^H@A@h-;f8?04Jm=P1}0M+)#J z>W~w+%~1wU+LVW3EB3``9nSw9qzsEWv61Oq@8E1)maAU&8PriMMEszE2v!CB;Y$JtkrPwy!SGADnVaUtl>$Bz;f_@r)SX39D7Ss0jy51}r+NP)D7dL*AxanvuLo9SrFG&$AtxU5=*(&Qhu?>z8_0y?_W zBCECnIW>$pkWkl9A^Dzrq?YTYy|Yh|^6Cas0vGR7Zi!_v(~jUV?3*EzC$nDoy|p?q zj@s#O^Jr`k^?AIqRt-ga)!7%Zm+DB#*5QK_d0wtHTFUOVS($jpX5B@1>?Vuk>s~<0 zE3nr%Y}NF7@02{8MZWP|8k3+4%M&1v?*X&#lGitx=SBRK95<9DZ8}|lQCYOYjJtz= zH)taAO7Ji1$!EaLwW|>WKS3<4LQJewCp%ZDxRYqH0-7VWCzf$ODAB$pJL0!EwnC4^ z$?4@2-*yUEhdMSWf=>?a0jR-6W9%#Jx35JpHjbD=jTsXRRA z0(Cth^p}T)lITuhhBpbFA4fj_0(zbVe$*wggg7hq1GXX7_UDpv4C_x%&hmYYxpBOk z4RR`Igk9hV7wA5@)w_?^;Vg|M4299FSaVG*jD0#5^q*TO`)A~q7dh}+4Ti4P(&&L| zv;(CB2d*LRhH)kz7s`^TS)v5_mkni6t&TF57abQe^sfn)WC-VxO(=`ng#0sE`P*>b zZ$>@-Q`8MMe6Kdn_Jv2}0OykWE*tJ^)>w5Rb=gW^E4LZteQy%Ycqkn9x0(y0DREw` zNw8i>fo2rn0nInOhi6xfK>v%d%U0}VSs-r!zKe@<;J1~q_X^m(4)(8AzwNA1C-YWK z@E&hkzmj7+u9i2k| zyH4Y}PV6JrX)$!_&m(1Ly5@Y~cFp;6)FZrq;@E?XHv}Bn`>mYL^ZpU_9%IYovO9M0 z0Y`j1&v&lIBYR2mRIq;i}}9;{#iUe;3)&13!wj> z@VpLsmia6&Wo?_4+jIkRAFoxhKQ+18efq|f+_EXT3i!a!RWIx9^RVv=ux}Lo$#w5z z|KKi&eRC80rdnRng?k`!Dm4y5cGf0H`l`F>ppmceO`jx$uykzN3chEj{VN_ z?B~p0ze7J6{1?QJ0C@(~2Xe}+=l`3KXT^T{3TOK^_@)})eQ2MB|MQd7qs{UC#UP9o%;g0po;B%822B(S0L|dJh|R~P^JX!jD3aLaqL~gt^?hM{sZjelSXPhPjs{waW6nV z)A0XFv$19;a@pw|I?8KTq2G$U%JbZR!;l|6j99;d`_dld)t8Z1&mu=&!}|sDq^-37 zKpJg*>8=BB>3Ox+ke}rHE&G#WC!g=1pzqb`gQ)X4ud>{*!F-16e%6Pv;T(r9tV3yx zZQQ5DIp;Ty^IAQ>A-033v{-KgR;K53Ks6LZ-vZXF@Z~$edIVS*C!YgZjM=l`>=f35 zB<@uj);EE539t@e&1;d!KCelUgYvBq4nOCgILFH-^qy_Gd%teQ#q(Omd!yp z8+^ZRs8zp$uAKLpV6&ZC{^5LnC;w#fMlJS!#$KE|GP&X2G1RMA$F-#ac1C?anVy9^ zv{rjskS}PoG<16r#?B&gcTdT^#rEx?DNXV2}fZIBcCs=Y+i!Fbq&Ye1!r<`zg?Z zJka`EQi8jMG1$Kmb$Uq>_YLkZ#lE=$To3&i|BV34z)Nqvx(9d$Za1IbrB0h$wXiDs zF#o@T{nu)yb60`$Eb4)e1pHTn82ZFWfuN1XJ_29lv2$xq(#!;C)`ai(bM0j_TB5k$ z1Z?{H%a|L$`dwJp4<47$PXG({_1^n1zEa3PQ#)u=DztOmgtOini``ED=I;M2BnkB% z=qvi{%Il!xyv{kDBw01!TgJD^#P=}vnJLx}Ut#a?y*OsWe-O1fG-fU(|3Sno`+H?1 z(ixH3IzyYn9ZyS}BH?G7Bg>`b(ko@Sf#50Aeu>`J`@pI_-I0yU+C$-v2O`~C|5JS) z@vc38hu=cHPukZ@wVjdnR;x)alw|nw60U-L&UP$9idnhNfFX*Su-D zw$!fE@dc9~NqZiUp3uj0GsE;Vy`RPV7yO;;rMi|-q+5C}67K0~1~hHuUVUA$&gU=I zxz zG6PvVuGa0${@M}=cl78E>zkdSFVNQ)pcs=o*`KR@M)*fu< zpFFx|ZK%7uInon`IiCuJVe92mMI_Ymbn^q@?qGBKmY%JPZEUg)-CLGLvL@LQUe?~+ z-Wk~n8~(G-cfO8e4O-M7RAA2Nk@ZHl{nDg=>GN4SoUhuGrTeGc%-2!9T|6(-AJygA z_5O_V%y<S% zpUdwGxb1F-+v#?>-ENQD>-M?*?tsVcad?~_m&fh#c)T8;$L|Sv?Ouo1>2-PCUXR!7 z^?Ci?fY0u8_?$kM&+YU0ygr}L?+f_teuv-bclq6ZkKgO}`ThPt01yKZ9{}qBmIW}> z7TypEMYb;6*coYF#*y+=d(X1=&P`MKaBjvwb2DnwIvnsE)1Q84JcRN4Fs|icjaHh} Mni+blb?*H9H)|Tdf&c&j diff --git a/07_uart_chainloader/update.sh b/07_uart_chainloader/update.sh new file mode 100755 index 000000000..7d57d0a62 --- /dev/null +++ b/07_uart_chainloader/update.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash + +cd ../06_drivers_gpio_uart +BSP=rpi4 make +cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img +make +cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img +rm kernel8.img diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 3148aa3a9..9ec2e42fc 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -15,7 +15,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -24,19 +25,18 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 12 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 11 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.586140] Booting on: Raspberry Pi 3 -[ 0.587227] Architectural timer resolution: 52 ns -[ 0.589530] Drivers loaded: -[ 0.590876] 1. BCM GPIO -[ 0.592309] 2. BCM PL011 UART -[W 0.594005] Spin duration smaller than architecturally supported, skipping -[ 0.597392] Spinning for 1 second -[ 1.599001] Spinning for 1 second -[ 2.599872] Spinning for 1 second - +[ 0.543941] Booting on: Raspberry Pi 3 +[ 0.545059] Architectural timer resolution: 52 ns +[ 0.547358] Drivers loaded: +[ 0.548703] 1. BCM GPIO +[ 0.550135] 2. BCM PL011 UART +[W 0.551828] Spin duration smaller than architecturally supported, skipping +[ 0.555212] Spinning for 1 second +[ 1.556818] Spinning for 1 second +[ 2.557690] Spinning for 1 second ``` ## Diff to previous @@ -754,4 +754,17 @@ diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs + } +} +diff -uNr 07_uart_chainloader/update.sh 08_timestamps/update.sh +--- 07_uart_chainloader/update.sh ++++ 08_timestamps/update.sh +@@ -1,8 +0,0 @@ +-#!/usr/bin/env bash +- +-cd ../06_drivers_gpio_uart +-BSP=rpi4 make +-cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img +-make +-cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img +-rm kernel8.img + ``` diff --git a/09_hw_debug_JTAG/README.md b/09_hw_debug_JTAG/README.md index 93b66a66f..12cf0cc1e 100644 --- a/09_hw_debug_JTAG/README.md +++ b/09_hw_debug_JTAG/README.md @@ -157,7 +157,8 @@ $ make jtagboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -166,10 +167,10 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 8 KiB ==========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 7 KiB ==========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.372110] Parking CPU core. Please connect over JTAG now. +[ 0.394532] Parking CPU core. Please connect over JTAG now. ``` It is important to keep the USB serial connected and the terminal with the `jtagboot` open and diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index a9035cec9..84ccf9248 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -192,7 +192,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -201,22 +202,22 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 15 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 13 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.703812] Booting on: Raspberry Pi 3 -[ 0.704900] Current privilege level: EL1 -[ 0.706811] Exception handling state: -[ 0.708592] Debug: Masked -[ 0.710156] SError: Masked -[ 0.711719] IRQ: Masked -[ 0.713283] FIQ: Masked -[ 0.714848] Architectural timer resolution: 52 ns -[ 0.717149] Drivers loaded: -[ 0.718496] 1. BCM GPIO -[ 0.719929] 2. BCM PL011 UART -[ 0.721623] Timer test, spinning for 1 second -[ 1.723753] Echoing input now +[ 0.637617] Booting on: Raspberry Pi 3 +[ 0.638737] Current privilege level: EL1 +[ 0.640645] Exception handling state: +[ 0.642424] Debug: Masked +[ 0.643986] SError: Masked +[ 0.645548] IRQ: Masked +[ 0.647110] FIQ: Masked +[ 0.648672] Architectural timer resolution: 52 ns +[ 0.650971] Drivers loaded: +[ 0.652316] 1. BCM GPIO +[ 0.653748] 2. BCM PL011 UART +[ 0.655440] Timer test, spinning for 1 second +[ 1.657567] Echoing input now ``` ## Diff to previous diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 4d9299a27..0533361b1 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -264,7 +264,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -276,24 +277,24 @@ Minipush 1.0 [MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 3.085343] Booting on: Raspberry Pi 3 -[ 3.086427] MMU online. Special regions: -[ 3.088339] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.092422] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO -[ 3.096375] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.099937] Current privilege level: EL1 -[ 3.101848] Exception handling state: -[ 3.103629] Debug: Masked -[ 3.105192] SError: Masked -[ 3.106756] IRQ: Masked -[ 3.108320] FIQ: Masked -[ 3.109884] Architectural timer resolution: 52 ns -[ 3.112186] Drivers loaded: -[ 3.113532] 1. BCM GPIO -[ 3.114966] 2. BCM PL011 UART -[ 3.116660] Timer test, spinning for 1 second +[ 3.175017] Booting on: Raspberry Pi 3 +[ 3.176100] MMU online. Special regions: +[ 3.178009] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.182088] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO +[ 3.186036] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 3.189594] Current privilege level: EL1 +[ 3.191502] Exception handling state: +[ 3.193281] Debug: Masked +[ 3.194843] SError: Masked +[ 3.196405] IRQ: Masked +[ 3.197967] FIQ: Masked +[ 3.199529] Architectural timer resolution: 52 ns +[ 3.201828] Drivers loaded: +[ 3.203173] 1. BCM GPIO +[ 3.204605] 2. BCM PL011 UART +[ 3.206297] Timer test, spinning for 1 second [ !!! ] Writing through the remapped UART at 0x1FFF_1000 -[ 4.120828] Echoing input now +[ 4.210458] Echoing input now ``` ## Diff to previous diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 04f20cd65..21c0f0eef 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -399,7 +399,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -411,29 +412,29 @@ Minipush 1.0 [MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 3.006343] Booting on: Raspberry Pi 3 -[ 3.007428] MMU online. Special regions: -[ 3.009339] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.013422] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.016985] Current privilege level: EL1 -[ 3.018895] Exception handling state: -[ 3.020676] Debug: Masked -[ 3.022240] SError: Masked -[ 3.023804] IRQ: Masked -[ 3.025368] FIQ: Masked -[ 3.026931] Architectural timer resolution: 52 ns -[ 3.029234] Drivers loaded: -[ 3.030580] 1. BCM GPIO -[ 3.032014] 2. BCM PL011 UART -[ 3.033708] Timer test, spinning for 1 second -[ 4.035837] -[ 4.035841] Trying to write to address 8 GiB... -[ 4.038006] ************************************************ -[ 4.040785] Whoa! We recovered from a synchronous exception! -[ 4.043565] ************************************************ -[ 4.046345] -[ 4.047040] Let's try again -[ 4.048387] Trying to write to address 9 GiB... +[ 3.090618] Booting on: Raspberry Pi 3 +[ 3.091701] MMU online. Special regions: +[ 3.093610] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.097688] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 3.101246] Current privilege level: EL1 +[ 3.103155] Exception handling state: +[ 3.104934] Debug: Masked +[ 3.106496] SError: Masked +[ 3.108058] IRQ: Masked +[ 3.109619] FIQ: Masked +[ 3.111181] Architectural timer resolution: 52 ns +[ 3.113481] Drivers loaded: +[ 3.114826] 1. BCM GPIO +[ 3.116257] 2. BCM PL011 UART +[ 3.117950] Timer test, spinning for 1 second +[ 4.120076] +[ 4.120079] Trying to write to address 8 GiB... +[ 4.122242] ************************************************ +[ 4.125018] Whoa! We recovered from a synchronous exception! +[ 4.127795] ************************************************ +[ 4.130571] +[ 4.131266] Let's try again +[ 4.132611] Trying to write to address 9 GiB... Kernel panic: @@ -442,7 +443,7 @@ FAR_EL1: 0x0000000240000000 ESR_EL1: 0x96000004 Exception Class (EC) : 0x25 - Data Abort, current EL Instr Specific Syndrome (ISS): 0x4 -ELR_EL1: 0x0000000000080db4 +ELR_EL1: 0x0000000000081454 SPSR_EL1: 0x600003c5 Flags: Negative (N): Not set @@ -457,22 +458,22 @@ SPSR_EL1: 0x600003c5 Illegal Execution State (IL): Not set General purpose register: - x0 : 0x0000000000000000 x1 : 0x00000000000858f6 - x2 : 0x0000000000000026 x3 : 0x0000000000082a0c - x4 : 0x000000000007fc7c x5 : 0x0000000000000003 - x6 : 0x0000000000000000 x7 : 0x7f91bc052b2b0208 - x8 : 0x0000000240000000 x9 : 0x00000000000858f6 - x10: 0x000000000000041d x11: 0x000000003f201000 - x12: 0x0000000000000019 x13: 0x000000000007fc7d - x14: 0x000000000007fdc8 x15: 0x0000000000000040 - x16: 0x0000000000000000 x17: 0x0000000000000040 - x18: 0x9e06782800000028 x19: 0x000000003b9aca00 - x20: 0x00000000000003e8 x21: 0x0000000000082f58 - x22: 0x00000000000830cc x23: 0x0000000000090008 - x24: 0x00000000000f4240 x25: 0x0000000000085248 - x26: 0x00000000000856e0 x27: 0x00000000000857c0 - x28: 0x00000000000830cc x29: 0x0000000000085530 - lr : 0x0000000000080da8 + x0 : 0x0000000000000000 x1 : 0x0000000000085726 + x2 : 0x0000000000000026 x3 : 0x0000000000083bf0 + x4 : 0x0000000000000003 x5 : 0xfb4f101900000000 + x6 : 0x0000000000000000 x7 : 0x7e9198052b2b0200 + x8 : 0x0000000240000000 x9 : 0x000000003f201000 + x10: 0x0000000000000019 x11: 0x0000000000000000 + x12: 0x0000000000000006 x13: 0x0000000000000031 + x14: 0x000000000007fc2d x15: 0x0000000000000000 + x16: 0x0000000000000040 x17: 0xb557f006f276cfb6 + x18: 0x0000000000000003 x19: 0x0000000000090008 + x20: 0x0000000000085510 x21: 0x000000003b9aca00 + x22: 0x00000000000003e8 x23: 0x000000000008160c + x24: 0x0000000000082264 x25: 0x00000000000f4240 + x26: 0xffffffffc4653600 x27: 0x00000000000855f0 + x28: 0x0000000000083f84 x29: 0x0000000000086810 + lr : 0x0000000000081448 ``` ## Diff to previous diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index e983470be..e3fa7e512 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -665,7 +665,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -677,25 +678,25 @@ Minipush 1.0 [MP] ⏩ Pushing 66 KiB ========================================🦀 100% 33 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 3.134937] Booting on: Raspberry Pi 3 -[ 3.136023] MMU online. Special regions: -[ 3.137934] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.142017] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.145579] Current privilege level: EL1 -[ 3.147490] Exception handling state: -[ 3.149271] Debug: Masked -[ 3.150835] SError: Masked -[ 3.152398] IRQ: Unmasked -[ 3.154049] FIQ: Masked -[ 3.155613] Architectural timer resolution: 52 ns -[ 3.157915] Drivers loaded: -[ 3.159262] 1. BCM GPIO -[ 3.160695] 2. BCM PL011 UART -[ 3.162389] 3. BCM Interrupt Controller -[ 3.164518] Registered IRQ handlers: -[ 3.166255] Peripheral handler: -[ 3.168038] 57. BCM PL011 UART -[ 3.170078] Echoing input now +[ 3.203172] Booting on: Raspberry Pi 3 +[ 3.204255] MMU online. Special regions: +[ 3.206164] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.210242] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 3.213800] Current privilege level: EL1 +[ 3.215709] Exception handling state: +[ 3.217487] Debug: Masked +[ 3.219049] SError: Masked +[ 3.220611] IRQ: Unmasked +[ 3.222260] FIQ: Masked +[ 3.223822] Architectural timer resolution: 52 ns +[ 3.226121] Drivers loaded: +[ 3.227466] 1. BCM GPIO +[ 3.228898] 2. BCM PL011 UART +[ 3.230590] 3. BCM Interrupt Controller +[ 3.232716] Registered IRQ handlers: +[ 3.234451] Peripheral handler: +[ 3.236232] 57. BCM PL011 UART +[ 3.238269] Echoing input now ``` Raspberry Pi 4: @@ -706,7 +707,8 @@ $ BSP=rpi4 make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -718,25 +720,25 @@ Minipush 1.0 [MP] ⏩ Pushing 73 KiB ========================================🦀 100% 24 KiB/s Time: 00:00:03 [ML] Loaded! Executing the payload now -[ 3.413865] Booting on: Raspberry Pi 4 -[ 3.414255] MMU online. Special regions: -[ 3.416166] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.420249] 0xfe000000 - 0xff84ffff | 24 MiB | Dev RW PXN | Device MMIO -[ 3.423811] Current privilege level: EL1 -[ 3.425722] Exception handling state: -[ 3.427503] Debug: Masked -[ 3.429067] SError: Masked -[ 3.430630] IRQ: Unmasked -[ 3.432281] FIQ: Masked -[ 3.433845] Architectural timer resolution: 18 ns -[ 3.436147] Drivers loaded: -[ 3.437494] 1. BCM GPIO -[ 3.438927] 2. BCM PL011 UART -[ 3.440621] 3. GICv2 (ARM Generic Interrupt Controller v2) -[ 3.443575] Registered IRQ handlers: -[ 3.445312] Peripheral handler: -[ 3.447096] 153. BCM PL011 UART -[ 3.449136] Echoing input now +[ 3.486234] Booting on: Raspberry Pi 4 +[ 3.486623] MMU online. Special regions: +[ 3.488532] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.492610] 0xfe000000 - 0xff84ffff | 24 MiB | Dev RW PXN | Device MMIO +[ 3.496167] Current privilege level: EL1 +[ 3.498076] Exception handling state: +[ 3.499855] Debug: Masked +[ 3.501417] SError: Masked +[ 3.502979] IRQ: Unmasked +[ 3.504628] FIQ: Masked +[ 3.506189] Architectural timer resolution: 18 ns +[ 3.508489] Drivers loaded: +[ 3.509834] 1. BCM GPIO +[ 3.511266] 2. BCM PL011 UART +[ 3.512958] 3. GICv2 (ARM Generic Interrupt Controller v2) +[ 3.515908] Registered IRQ handlers: +[ 3.517643] Peripheral handler: +[ 3.519425] 153. BCM PL011 UART +[ 3.521463] Echoing input now ``` ## Diff to previous diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 6dd7d6301..312bb5f19 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -244,7 +244,8 @@ $ make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -256,18 +257,18 @@ Minipush 1.0 [MP] ⏩ Pushing 67 KiB ========================================🦀 100% 33 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 3.041355] Booting on: Raspberry Pi 3 -[ 3.042438] MMU online: -[ 3.043609] ----------------------------------------------------------------------------------------------------------------- -[ 3.049466] Virtual Physical Size Attr Entity -[ 3.055323] ----------------------------------------------------------------------------------------------------------------- -[ 3.061183] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack -[ 3.066476] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data -[ 3.071812] 0x000090000..0x0001AFFFF --> 0x000090000..0x0001AFFFF | 1 MiB | C RW XN | Kernel data and bss -[ 3.076975] 0x1F0000000..0x1F000FFFF --> 0x03F200000..0x03F20FFFF | 64 KiB | Dev RW XN | BCM GPIO -[ 3.081658] | BCM PL011 UART -[ 3.086606] 0x1F0010000..0x1F001FFFF --> 0x03F000000..0x03F00FFFF | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 3.092462] ----------------------------------------------------------------------------------------------------------------- +[ 3.064756] Booting on: Raspberry Pi 3 +[ 3.065839] MMU online: +[ 3.067010] ----------------------------------------------------------------------------------------------------------------- +[ 3.072868] Virtual Physical Size Attr Entity +[ 3.078725] ----------------------------------------------------------------------------------------------------------------- +[ 3.084585] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack +[ 3.089877] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data +[ 3.095213] 0x000090000..0x0001BFFFF --> 0x000090000..0x0001BFFFF | 1 MiB | C RW XN | Kernel data and bss +[ 3.100376] 0x1F0000000..0x1F000FFFF --> 0x03F200000..0x03F20FFFF | 64 KiB | Dev RW XN | BCM GPIO +[ 3.105060] | BCM PL011 UART +[ 3.110008] 0x1F0010000..0x1F001FFFF --> 0x03F000000..0x03F00FFFF | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 3.115863] ----------------------------------------------------------------------------------------------------------------- ``` Raspberry Pi 4: @@ -278,7 +279,8 @@ $ BSP=rpi4 make chainboot Minipush 1.0 [MP] ⏳ Waiting for /dev/ttyUSB0 -[MP] ✅ Connected +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now __ __ _ _ _ _ | \/ (_)_ _ (_) | ___ __ _ __| | | |\/| | | ' \| | |__/ _ \/ _` / _` | @@ -290,19 +292,19 @@ Minipush 1.0 [MP] ⏩ Pushing 74 KiB ========================================🦀 100% 24 KiB/s Time: 00:00:03 [ML] Loaded! Executing the payload now -[ 3.376642] Booting on: Raspberry Pi 4 -[ 3.377030] MMU online: -[ 3.378202] ----------------------------------------------------------------------------------------------------------------- -[ 3.384059] Virtual Physical Size Attr Entity -[ 3.389916] ----------------------------------------------------------------------------------------------------------------- -[ 3.395775] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack -[ 3.401069] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data -[ 3.406404] 0x000090000..0x0001AFFFF --> 0x000090000..0x0001AFFFF | 1 MiB | C RW XN | Kernel data and bss -[ 3.411566] 0x1F0000000..0x1F000FFFF --> 0x0FE200000..0x0FE20FFFF | 64 KiB | Dev RW XN | BCM GPIO -[ 3.416251] | BCM PL011 UART -[ 3.421198] 0x1F0010000..0x1F001FFFF --> 0x0FF840000..0x0FF84FFFF | 64 KiB | Dev RW XN | GICD -[ 3.425709] | GICC -[ 3.430221] ----------------------------------------------------------------------------------------------------------------- +[ 3.379342] Booting on: Raspberry Pi 4 +[ 3.379731] MMU online: +[ 3.380902] ----------------------------------------------------------------------------------------------------------------- +[ 3.386759] Virtual Physical Size Attr Entity +[ 3.392616] ----------------------------------------------------------------------------------------------------------------- +[ 3.398475] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack +[ 3.403768] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data +[ 3.409104] 0x000090000..0x0001BFFFF --> 0x000090000..0x0001BFFFF | 1 MiB | C RW XN | Kernel data and bss +[ 3.414267] 0x1F0000000..0x1F000FFFF --> 0x0FE200000..0x0FE20FFFF | 64 KiB | Dev RW XN | BCM GPIO +[ 3.418951] | BCM PL011 UART +[ 3.423898] 0x1F0010000..0x1F001FFFF --> 0x0FF840000..0x0FF84FFFF | 64 KiB | Dev RW XN | GICD +[ 3.428409] | GICC +[ 3.432921] ----------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous diff --git a/README.md b/README.md index 845aa87d1..00cbe66ef 100644 --- a/README.md +++ b/README.md @@ -120,11 +120,10 @@ provided container, please refer to the repository's [docker](docker) folder. ## 📟 USB Serial Output Since the kernel developed in the tutorials runs on the real hardware, it is highly recommended to -get a USB serial debug cable to get the full experience. The cable also powers the Raspberry once -you connect it, so you don't need extra power over the dedicated power-USB. +get a USB serial cable to get the full experience. - You can find USB-to-serial cables that should work right away at [\[1\]] [\[2\]]. -- You connect it to the GPIO pins `14/15` as shown below. +- You connect it to `GND` and GPIO pins `14/15` as shown below. - [Tutorial 6](06_drivers_gpio_uart) is the first where you can use it. Check it out for instructions on how to prepare the SD card to boot your self-made kernel from it. - Starting with [tutorial 7](07_uart_chainloader), booting kernels on your Raspberry is getting diff --git a/doc/wiring.fzz b/doc/wiring.fzz new file mode 100644 index 0000000000000000000000000000000000000000..55f80ec8d4dd865b1c77762c04d5d520cad12c5a GIT binary patch literal 61259 zcma&MV{~O(&^DTM+(|m>*zVZ2ZQHi(bj*(Jq+{E*ZQJI~zB%XI_s92+`;BpbtzGrZ zSx>E}YOgWpTuWXG4E!eu2naL?LTkFh(5hVmBn$`$w>Ag}`j^+x(8STv zgFxPbfQi=B(|#~h$7zopE#&%L&D^IptqQk>^ga!-m@_V|xcF2)Zt6l;0rIDPbS+O4 zbXN8nu*PtG;jS)J#dSp6>UAmu&btUIBCJkXoZ zVE2hVW4;bV%eUdTI^Bz;v$L)V3 za{712XA8z}qW#Q%^wzBp#^mBl9M&pGh=+VTz!N*H*@M<2cmEAgGnbrY$=SLw`b z#zLdAlqB!-id+}gbLd{unIAj=fgm?IY+zYC?Ev!wDCh z`JfQKZ_^Ndf%{XWEl8nLSb9P9ebGh-!E8I-7Aj0KvIs_Kztwt=Pz&Lk9~KH9u`p?t z)#f48FKUhhYMUWW&d98yH|<{9mW40l4!mMkHm^op#_MDWM`x=51Z!MdGM_R1`9oXb zO;y_j0`4}y_3UKV%l+f@_0|q|%@o&=ihj@WMDvLo9YUOMV^?+O={Ua1=Ln2m zTKWk%!2G~}xVst#7R(1j1)A}jmpctO)RZWsSQFc)=!0F&IUSz-!8O--5WekQf)64~ zouBe5x@P(25Vl;kpEQ5{34!C6@KbcBK&S@+|I{ExR=6B#ufi=g+b;l!H$&^5^FuT> zT7vQC>j{bXOxp3c31d;`TL!Lfo$o^^o06=5G%l06D0Z&0XfAgOCyl2&Ctpmg3OPKG z#erlAE`Ll?S8pdS{4GfD>lXA;XWg==2vioQk@m2rT94JFWwysL1{$=Ue>g0w^hJO; z=!j4%O}HgN?Ru&d1+s=O>p>A_RaSI(iZE4J#^_wTMIa4b-10;g<Lw5% zP_o{Khh{IYfY@qs+0{mQKh^17T-YQQFVeFs%E0ThU6wR!2f@~@Mv?kRPG6Z{zs0ao zB=KCbfLwMvqVVWNS72Mt8r}_%oL!H*OCiJNuz?hbSRuSYB22w7(V2#;-E)tpity4w z+z9v5)sKQ1+Sg$C`RfOYGGh8KW+AXJ_D~6-fyu{#igqAF&X9stwCXdwkz|1?Z@!8X zgLN<(Gx&pJ>JLyE^w46YPT*<6LCygfL2Jsrww9nls^K(1Rsuoj_mV(nC_!GhP2v51 z;(SC;+}f$Qc}wlT+>gbveA4vZnhj840-p}SLE2H7KrY6Fb9FYKZK9!X#^QeSYa#wa z4bGVYol6NGrwq=y=(9Z=_D^VNXKM#o&R^9+JEyrf79I* z&ZD2GxcSKuLwx0{FoRO)k5nBT=zHqeW4QTqifqa8ttM3-qmCer{h@p~uNnzT7qia( z1a_|{Ow9;IBZd2c$|02+&k^TM5xxMc{l{4Y{OfOVrfA_TA=N-EGJT5}e-(LZ(Hvi% zNhEnnR2Rr>RpNN9FlNf&PGKEWBH^}ZaP|FNfh=93ZbS==u(j|Ca2AZ8A$!Ih+BN*C zxk#x|y#RpqCw&zQb0xv_Zsr|7f& zD~DbQflD}5S?x=lZ1q1${u1sK{Yvteq^?ml*7tJ%d~(zXtY(DM2HdX@L4pgeUm*hW z8_tkae59faePyC8;rI&YJ-(=;!4;0bsE-`jrJjolb@rcDIc>zg^27^AX~g|Xl>XOx zjLNhKQ7eMzQj1TM?D~J2*ogh3iDVOTnP#l#qg~@OFr>HP%He&5p zm?o9el1~3-|MYxJ?%IWzv3K*RzI z-d_R{<^h2Oc|-U0F6psf3B?mw^41{>OSG&=`ZYPYgx%wV6jv+aMf5%g!DsIs47&)# z--k?eu#1`V!4`qJ{7T^hj`Zu{4eZe_mzTMI{(T=eYG^4J7{FgU{vpEZXP)6l@WM5F zgw9`0Dv6u@Dc6FPx*F?+unjk-w8>;_i>(>HN&UTnw;@D)`vc}3GHtfOS(P?H zsifSjwaDU6LrR^3S^b_RlG)gW*E|mAkw^kUdNp1!tUbd6whYz9MULWQ7H$s*?}ZAVYQB9b*vnM@(I6< zM*GL_swv-DWr>|bnZYb58K^Te^do2oi6n@(sN|MFi&r!`#DeJv&vG+K@?| zxZ8)8k5N1bzjc@IDU)mA<~^6`L{we;bk@ z!R%_MM8Ml4WncuSL2_Aqm)+-!pPqGV(6#S9U@rcFozHG?$C-Ca@fI&O_`0_NUj?=v z-!AYvATEeEkidgw@5glE-6g;Vd-QI8HXXmjL1U`)IT2Ev|7fV7xfC^xZn&bdDEY?& zJ469l9@tR9US&A49gH&{JVjXhI}?@BSb&$(p{Hpm1;3q**pb z8CM9&G-0zbk~N(IZ+@eiVD1L zUzDvykZ2bML~`FT-;Q%vE@;&Ff4}NT_MmaM+iMJByQ^Y}w&)(2jlg|x^fox_OzWFt zb9Bzf*hygQPa(AjBK<%$+O>-43s_U%lNCS&=a(5_Y0sSncP}I~KYC3z z3H{Wy0suuUmBsmd(pU)~&Tr$eLPaba1T1mzN@tNv9Y_jIe>=MZ8$l`5V?!6XsTJ$t z74L7vCzHh|i3xF)#jCIBu;KzA-Nm6oML5laobBBUgObIe!~`W5B2JxMH+z0o3>!%< z^b3+&VwdrW@IkBItj3&g1qKC@qN)RA?Lc6chyzKXAf+?|NX0;;)&fZ{d#|GIa)zE5 zO(~G}?Z%6(9&j;28!`}x13lNa{RWJ~WuLqWss~N4KqltmBp`C%FF|FhO?E3WV zE%cc598Fwl7-$)3zeZhlp8p|fj;DPAa*NnbB1kl)_4eSJ$8#3PGdmfxu3;t|Wp48L znGc>3X%t+Q@h*x)EMQ++T_jYtgN0G~hfVM=nUJrNsSpt(+qZCS@VRYteN^-LeAK+( zKHc7L_5;+q^<~~8H!lH`{qODX*IjRee!yox&K122UOt`e?(WYUd%yQZx+{F&TRp#f zy{?b#AEBFk+gsbbx_ms`UF(1+KtEe^JD=~%P5&|Zo8_?cJnNoBv?Xudny&tp;__6QCMR4(Xuco%m$K&Jq zJc#c1zP_E_9e%WX+fL`c-lW%qt=5izZ3}#gMd0&sS-e`qO(}C&6CAHY>eIRG@^bG8 z6;o^P-t6>y`z)uL6))!KZtq_6@$-7#e3sMGxed{x#q~w?uph+oZR}s#_W5-7ae7}| zEE!&(ZjR;K(!H!HlH=34yxQ_}d3==12fSH6JH15w^??HB^!z%C zG^>5@^;~bZQ?Bj!wl6-+DEYR0z22d}wE#-m5t`e#xpn(1kvL z$^Fa6cRRc){B5?LAi8c00MWu5geo?#Ux!bduOGmXA=3UZ0$72j1`AR+lWH`MNpH11=yr0nX3+ zmy`X&u?WvOr_T>$b6>pu2n6FWNZFli*IhpDAMSvkW!~Frucx8I2+Gs%8FaRM8@?T1 zntec6t)Q!Qx4ydrHfbzZfIqswnGQ+L!E*yXPF;=U)Z#^aAXL0xc99wxZ$IBNV%@|i z(fvHy-fwneS0c;l7=^cad|G&lA0>G7e36W~XY%L*kwr+O+LlV+nsYjl_sHe>F{uD| z?VCR)h5%;9Uv+-C?SGuVq(gucWoDyUen*3>He90rEYI8WZO)i^)WLa*<_B{=)VhF@ z0a6svaR|P20aW$?Brv3oAKm;5q31bv_^Zno=hBlpq&ZbNU2=;-EIz((uVCi=%>*}Y zz?mk|h8t4l&k|hpj`%;9>sdC+RJ+&LuKBei<2voba*&^M^_x&H10+^a`t8^+)YvvS zoi-=%sAo4hd>8EK(I{w^ZL72GI#f`WL}+{Om!F8`KwONoCI>|bbgC4fzY6K3&+9|!opNMT_Ua+vM`N5U7Y|PkYn7_#^7^- zv2;p(?w%`$pkAU>ks$Qu9Pgqu7f~^tucWCq*`n9!$hgr0Q!6YKfzR*RK8G2Z+Thm;%V+zV zk7o-QEJA9Di+|d}mIOwEwV5V{v#H{oJZh9M*u|V@HFMG)f)K;a&F#~X1pDX@w955w zbfsW=-pxHXgXlGr#z2qaYfx0jP4iNyIC(ZjiklVaZvjm7>1#_AE-a^M*Vub}5~@>8 zAL5mmudS*URQfp2wAEE1B)GF9V!WH549D^@v;6Az3jX!Ej6ZEI|)VREfOyCZEO3r07%_c#tFaA;aJO!TU#ZvC!dFhG^rp` zY8=e|oaWx(!{yxu0G@C$(84mK8lL9mx{Is3J3rl@caPn={k&Vefap8D)qs}I+RTK9?SH1)#mEix||pU!CCO-GHvmDD8Ya9o>MM_hDSoGCkh_?y_7SsbXLSs`dLV z)}Wb05!4_{?z2jwt}V+qs-!HQ$b6v6aAi&R+ z*yNZ-NlWF!_>l87R1A3X7r(;SV=)pH#K7}n6lRU%q(1_YTRByLGapHllb_T7H{p%? zxl`<2W$RUq zWE0r^xef?ys%)X;$k#4w1l<7YG#hKGOYFEJ?;jfJY_#La-7psgU$7n4J3V|ni63m; z!Zd-@ZE-DvQnbnw&5rs#PM!DsN@%!fyJ|P6#@cb*;rbA7g@Sg+Z->+W=Cz3adPEZ* z%_hDk-eP{Ljb=Qf2`0JVxZ}N(F~Xl0e28zApF$3|?$=GK>OO!X2T>gV5 z?2h9P?->m-{+2n_Hh7p%Oy<|X9x>L6q&@hsKYU_iVx1-%1z79p%><*z;x8x7QJ&5P zi9WDTNqk~mV%;VSMp%Rs85l{^@z=9*~IGw+l(g$smKX-gigabU#1?% zna6ns0rGpI1JqOYEZN~jEo6eG>Jw$B55Zpr$H;G7w$&C*ZxdU=MrC@oh8=3ER=16u zrqvg@@xjVJbct?`do+iz%-FY?*0|pV$qWO~HJVWN`{(B~^yCC(pHQKpg%cr>Rvb1! zrkiIgAeTGQz+5w##0~@$^o>w)S z1;iwwD@|1n?klhf><|{Ryo)A+`#(aSU$5#S4qBxb)QmQ}h#-LMR zFEchDr?V`jt0f)PPY(DOL3qk~@e&e}KWIL2larTg7qA?^2PsC!9aRppFtKJ&l1BR= z#v}tjQ~Wk=b7lIzvY>mOY0MkHBp zQ{U7Y7~DeYs+f#v@C3&j)|?2j==m;KJo>ZYMcgpV2&ea8OPCM9^#TI%NCD}2XNHo2F$gv;qGqnohF!;SZ1D`_qfcRfOBAF`g_am9SsixvkFN#Zd;P%(r z6J@xIKT&Navhpf%FtIka&R()HJNC>Tch|zv&e{c+q$f-@3(Z`9qhL82Rv&)9?=lu0 zX*KPvh?k*CLz$PLJ16gSF2JzL3oqaP_@wjO0Ng$MeT;Vl#_fRDdOAm4l%G3>YY9RA z=okK#`fDBrQTR&02-c(G%`B7J_WBl;^mZOQ9V6<5>al@)Q^VQ0e4g^~oS8Tj;H%6> zR9rocbqyBXV(#6_5SQIPyxmUe?A;k#w-7e4ef#qi!SC~OaA5hPmx}pTqVi|`ZqI9a zKwKpqL(dBmdK2o|3qfmE ze6VV+!4Qld60qzAh^{b*?_$pb{18Pfz8C84}&_T~sTXm*ep%XAmPy>^I472dsebS*4s z>Bvd8mqbryB)j@xTu@~M2e_mB0mldyh4m7k4jrDW&OvOl$H?d)AYt}4;)o-QZnRVT z#9_nr7~-n(J8zf{>$o7SkjhB~t}z{EaN#}9rvMLaRfG^)9n=!D>Osh@6eJNfdN_`Y z$P+TG+8~h>^}>#a_O+tw zhNLe^tkscpm?<^d`i>id5fHSta5B`1t&`=uQbT0a=$SZdI;}lDF1w?F5(A{JWYr;b zn1wakiG7qI6xJ%4bpwMg>*^w%$GRGAMM0%*@)b_J2qK-+6BARyX}pD)ZDUnfBD*wJU`rXr8`rpqq^H(>_u6bUy2NU8xJqtilUO5XsI$Z z4T)VnReK7Yucl@KjGf7J*9|}lh=_KYbeWKJ5LquR--6PrDwa}%^)y=gkh zX{#oWrmh->rf%nm)3Wz<=$S_*trohf4(TaH#U9HwPMp_V#%2~%DV+zIbmNn+brDWo z)GWvC1{G%H{;N{5AoUyI^yTcJ6N)H3R_YvCNO(8(?AQ~!j;yJNra>BmtNO|5@LQ=z zBXaP=OS%y`t?GZb7U-p9;O!30EV5HO54ah|CsFDn?gyw@t(aEE~qGHu=6f%^v zTTCb>3~6a-rVE(TaRf~`GL2O1!7)lr669PRMb3|2kd`E9*m9<+WXhVBo*q=1HZV7T zG25mc9wloGwwffTYi*6q%?UGkjfR^hXPBF!lBa7}=6euJMb=MUkeX#_*e-odb}=p0 zIF8Py;Deadb^M-wSe&adNMW9w9=J6&J?HIged97ER)X1;ysqcl=KIF0<>Fe5zxLiT zU!@>@nU;K0x3PKgqsN5o`SPjFlO81+2Y!sxekE>i% z-lGWG{$=|l-D8)9!$|RPNPjY6uz)SAoemw~%Y4PP_-i%SkteaoR@o&S=*VVB;oNHC zrRSoxb8)6M6lv;DJT01u`k=>lP-NM#%dg0{XMkfxc0z^oh?Vb|o^74Tn%E0510n^nyHOs4SQ-h_N_ zTypO_u#O4KTwgyZH#U!A_eQ12NGhAwa^#AOC{GJn6o%e^xGB?43N+}wio)W(YFE7{ z&wyV#LR1)*dng4Ji(wk2bmIp$Q|*lbITo=yhEQ()fJbzt5wT9_iHf!0l^D%Qrya<; zLB(}!ezqa#Af#XrB|>hiszycP>PkLgw8dZ4DvO#RQ+}~L9(fd~%heEp`vp98`~yfA zXfa+!<71mxU=*KXS-DQ=7!R@GW9X+&s2Qyitmz1ofGL!h)b$VF5Z1C;n$q7X2&_ij z>xtLD3u(tzqQUHbe)vNTT7=v0K8nZ>Cn0SGugUY*hd(2`jD<+&sj17WUi1Ad6?%cN ziz1~JGpX;QG7V)k)KO!0+GQFI`LzYLDjlkfqhQINKVi&Ydg}j24DQz=!Y@oMr+(1I zXmT^`;YD-q*Suy^?zCI?xzk(Yp?wM2p@h+BaxOf%3T!!V-s~51?(kW6qQ$WNQkDMO zpHOO3PT`NbJG=eU+dLb`xB;2$-^g#a7q8~~Hv#iz+9TlLmubxA0j}!b;=oLzLuD_` zS8*{`4K$eMJ9F~%mJ0M_PL?0^!hOS^zqVsw6Z2M@43yQmdW!{mbT{}!--04gpab*4 zg0Nm~dTITA5hBn4u4$e+v7Yrk0FKAC)R=YtDkWQWZ2`bru35ZidyiLUrD$LxDgHN+ z261+0H#9_E|Aa1*%ro5ydE0Y^E3_Bh5}Zva_O77-BVos03HZqIg?5&O>PSKbZQPJf(h(czF-%9Xe#r#yA*3mu&f=9^X1+ zUO|<^;{`-jVkP{^NyQW9M)!E>@0s-dzkXTof%=G>aKF_U-ZVb~t-w<`KCIvhx_8nA z2}n<>9hZ(#uxQoA$@!zhSfL@b1j1IF@ktSWtssJf|5`=AfzNq9u_XSc;(zoo7K4G! z3uYejBMpP$rR#+@xa^d;GmUHxKFk=5Qix!uzmdPl6;@dh#G(-BUvV4Ta)-~2(vDC$ zJSWuTmz%1n5J&`H*;IbJP*ONgT`_MJkph(l`ru=lh&e*bS8wofI@EG$YIvrY%$VkX ztP@}lYrkz$OsB2Vx600uN~mfoVFK4k(RmnvD{~s!{=QCKB)=&Km%5+ZmJE$i_wl*= ziKe>~K$4LLJK&6{+{e5k9mE^`{2SuC94|3rW1G?Bfbf&PBXmCtWF?%L&czv3TnuLc z>pCWo&4;4?Yl*Be1OKt0*sLB53Es_#77cRB9f@>CR7se9{yV;orwc^nVh~bKW|0cJxmx~< zj83}gxlZcRMcV>ax+s=ul0}(;Nsjt&2nN6HE%;`aqpA z+PSZ(NxB>qG!WhxG^&zxh_zM-Dg zI8x%$QRj9Zyzuu$Ya3&hbSgDrpnVlfW@&GrtG$nGC|t5d10xL^rgTwXuwWJ~t^ryx z6M|B`e{(%n19+uMdlkXuk9X7OmIGM6ePlEshZTyd+=%VvoO3eK*+zxb3~ZaX>rGmh z+eUO-%H`ad##*IL`jokJ(Ez})oB;ksMvZ`KSPGv^Ry&K#fE;PkDa;s@ z84o8#3}@Qq(rAl+TjS6m`LkF3&A~%ctT|uC!Tog8EWdLFONuiDz_}}MzdJw(W#6V$ z@~^)c*cKC~gDn*H5(2Bz+*mtJUr~%r7qVMXu%iVN@Ir$$amC6$GuVEJ4K`0ZMa;_x zd5kpwquwk@-tf6kTd<9Q(ErB9a4QP_+|TO^a{mtkz9=B~HQ0<~8+$#l{!SdvdNG@( zEF36)XpLK-exy(SIdxyJ=zrMOF@~L>%*rM(ovosHFFa@*kJMpqZfe%Sb8fCs+DuWl zaOJ+lphC%Ek~XndHGCD;^Da=k%-|_7tD{FsW~Z1+Y}}kf z&EU6fIUt*N$YRj0Tv$(kVu+`C>8z!u=}{|2v)d%Z-#k~ z0wg0rTV{Jm)ZW^W^!`+~5v;wa)yF8p&0Lv{31HXbb}U~(_J}Hq)&jED%YBUr=d*M9 zt)6jYFML|nCQv3Gh&j_WRBUJ-~T>6(j@u5Qc7m@Q2lc05}b`e0#MF0v-eG?xDf3vjI z#&m?(8ik$04OBh@j+_bNnEkO#0Wji;jQ)hW9|Sxg0p-&@Kb6HUrwhLTw@Ou+zXb=VA{z7VO5gg3WK&h9YGz zC&ccaQ)He!C<&vHn3p|+&g}$wIpssfdhWuk;<*;bUerN z66Y1?bY?Qmw^cSO<>m&0iCyQMptL^?=%hdIIyF1RT%4+l=L?b6;8o>cmAUJ6Us1Wwy$+vgB$&f4LtJdg2f6+`5an|!ACpidzZQ}tO<0~tH#t8)YUuXp z==23n_CM~wSFbLYBUDmqc>MN5LnEYzu>-{!@Hqg1aDZ(Jz@SBADKJti7TM^56i=lP zE71aR?PpSR_0Q{_h9A%TadEO0YoezUktF{7)=zpzjPlM4U;1_0j&014xKnSn*K^bi znBpc$B>f{JY_>u=iQAJcNg@}Ng(6OdH=ek@ClFwT!UHd2DvZ4;`=^?p`5Zb#aO?=F z)!@1TG1}_UGqRcg_UF*#l}&=Q?8?OZpqF^D+vz<JCNzvi7tI zMj3Zf3~|JzMho|EJ(!3=aKFjKRpth9W{E~@`OZ)CNXAK}V3raM8I&BQbX5IHW?SJh z4=>0vC#gxuH*4}tytDc~B`{(t6dCt>ouIKko3Zo)2p%3(~yS!Idae`TGYt8VV+_xtti z5MDkfA4}YFfXfKWvCS_(yFbs$unUmpUswY#bLYLKt)NVZPB?NJQt9zV6BS| zsodW{U!IC>$kEL0p9H)!3B!*nQ`s`gP}#Fj!NDo(cZ`DFAU{x(ZC+WNI3$|ZOlA?<73$`;^zVnM}%HL{2Iuk~MZ>h6i0sH#+pCst4y6*B&~pjp!LK|^m>(S|`k+*xN`{|8atM}3N z-$S%Ql&UL}r#(C3pYR!nvBNEgu}@zZJUV7?FDop|tuE*LFL!VG4oa!Q)R+@Du7v4w z%=|w8o@yMq2hNbl*4;P|3GY9h;t8>DL8pmRg{cFloe!%RoPn#DP1Vs{iEHU3eF@Q# z(|6^pWF*FHW%G6LG2@@2shKHeDu5U23*n4DM3?4kHIq8MyI?K&fw_4LjrIE7r znA2Lu6ukG~1(O<>HlCvQl~G#_^ylKD=xSyTnI<*dXHM%aHF)AL>*#yvyQ$9DsfQYa z?bA-8vjD6C>RBc$bUg$sWl8F_2Jv%ov_n}K=(aEllf~1$FgDAnjKAXEh2r_S;?xBa zwV;|LtHrUN=sM_4byk(j2KVh zQxCvJHcyQ7vGd^y8`~Gsi8Q~3!Ey$FUOkq#1;;g?ZeSLCDr;s&6z!6V?J5@AW;Bq? zWqM;sWqt8IBTRG$ASQI$dmZ5EFG8(Lv@q5}mf=gI++scvRR1FQj1t?rK_tD|lBPf3 zRNM&W!->)Hn2xtjE4{IdUSHVbVMHcnU4c6#puTyafKZ*vtG>SYut-qix(Mf3K3FyM z8#M6;&-BqB2fh+SZW_f>NCP}|H0XOc_k2Y1&}iM;HSi7B{o$B{hT}f?e&kv74?HGK ze%_b&fTj%dzu;~`67$h{9}tm5&W9WrtiPY7;82PcaHg}5TkHm&M_sv+z8xDMWa2r- zypO0M^tB=_hxidMKYUAI#nWy2BXMeOpiv$s(Wxr3B%CEURFkVH6s@WHt3&oj#l3LB z-TRjC$&eF06>c~V54eNln%>Etn50lM=8`^Y+eDu!JCdivQ-!UipO<}(mK>9&1V66~ z0!ik(dL1TBJ={IXVh@NxtgHayt+>vVi_yJy`>X-H)*2+5wokY{!C zp%NfD&TRwA0HoCe66YKe%Z`w;dxL}Sr4Kts2gEj}Az!cim7|DXn!KNz*bS4AWGb~_ z2kdxmtw}*iseIRz@1o)wlWARAjA^9EIjLe^VY(F#I4WDX3Q*CrP^*`kkL&krvrwNk ziUJ-j0C;caGP4m;-@%w-R5ygFVnkcHVhiedKIaUU$~JqJ)*TG=M+%VWF!67rAU7(R zc1X$22vHt`!&|0t;ab8WGvOQ$@=hk$YU8d&MSYbqR&H3Hv^wGdT^A@k9z1Gi?hP!% zf~MzEZ{)3O45%S-5-r$H(ZIc$_-{(xUXck@e~z|WiB8CFe-f0Qa>`z{O&iEVOY&?u zpFlccI2TK@2M~2hpOd8)$B!^TT|kkQFnMWO?6tq(q6Gcv7Ly+m+?CVUzYN~1PP?qW+Y$Z7r`Y!EXiQA~6(8PXz|9CAuF60>BQa@QNXA*~ zqUa=48lJZxP0nH&*S;daNiG)RmXv~potQY&x$u_Vnl~f8%~bh%rN<9-dlTO28f|J+ zy85Zo3&|%1J6XL6GZ{u0-}SC*IyOG>5W$|PQ9>A6YYHz?{Ejv$Zw!Wu9X4^!BliZv zX_VXNsF}d8n~*dPQvGl{Y6A@Rj(AN`o`q$l$CAqhTT~z~Yvpsp6zE|VSA3gcx5iw;j5a;BRSVd`kdlN6+ zmCn2VOn#`f78(=ld$Z)9E5Bq%N3-nUd7aEY;<XF5N`2%U{W&-*{nlvCT!lFa`h8G{utY|z;*w6%j1~8e<`bJ8p1#xz-lt}l z){^YhoI$EzR(%Y^{;b%5@@Ly~P)&7#o%+1cLSAda7HD9@-Dkd$vppWJ`J_@M4VAILu&Q zhlIgOt{>#NXER(`8|uYsRfWsdT8xm@cGY6n&e?f*oxUo6N+-50%KXxF7v=t-ojyj! zFyE!@V-C2404^@FgB4uasYNixm@=aJr6Cid8rt;ZcY!YSy+Te=OE2O^QAZot@(n}G zkwXPMyt#++vjq5p{KqtR0cf=DLEa}z50Dw$jl<&3cF{0V!d4roGg$9}Qxz544{B)R z&IStuSW?OfonXuh8vFhdfe5pBxi3=ZL4$?lb6yI&0M+uq*&;BqfcRQr@4C%Da|Ev4 zj;K^exDU+KcuNw=+!OnI!bLYjxUos~x^mK6c+1}~>mSwj2lwE@LU*9pL;Yp!!QnR* zLU90y2?i$6&F(!Yr-~mDNFdyBzGN7ia>6KP?R){o@}R@DNtgqw-u`p147I5E1FFWL zL$xYDp}NkqETbyDOEI(>O%b(TpP6ctO#4^Hei%`;<7j!z%_bGOT*oU>1kf-z@WClM zvn)d=jStm)8AN`9&`5S4uLL{aZ@iB-S{c4R+1uy_Hu6z`gL;-9>>#GuopxV2r_{8C zEoHfdexS#O#s~a4y`^s&L?OAEK?0GZ!7V99fd6B1t;m|pF| z4G7vJRcT?)`JHE9yQj7ISW3*8d=*Gs?<#4d;caBMxSidlF+nO2*f;LlzsT=}Y@B$o!v+T@s{~$SAlVHLO&G*k0!F8JK`!+f}U2vB~QZCM@ zNnmmw?>$ty5OXl5&AV@HX;u(FoRX)c&QAsF6c3Twz z^SLNIqsmaK9By-&FZ(llu+v(sA{MYbX0hnFla#G`IYWyNu6t^+D7e#TsL2AU>yR#M zdaNQ9pyrt``-I~1jYUJ}NM(DZgR-GS#R2QX<5(rOj;II|i;<+fSoJqJaABI9&>eds zRA4sM*Ai8CiQxWX!BXdKd8*1)ShkJ+bf=6K2fdKm)!-8R+<%&#Dn#UlTStL4q&m4z zVYbf^T~(%T?=XbkzAaxgauF%&HKZx$ASfkGLf7b!T;qJM+cmq9*m3T+1eBVKz~W`! zTImvK4g{`>Y>J_qNNAOT4f+Fv&d78<5}?|=DakPYLz*}be|D&y;?0m$wj&>A8A6u6 zBZ*vhc9&0WueUtaMdqn1$6e%A{M-wHu+XqfccsmC33W^PtcXnKT!iWTNXN%4$7-$7 z@LF}UqYv8E`S9NStnqE4oMNIw_2RBeLa4)5z z!v&4R@H3hh)X(}DH+jl8)X%02{<{((|9iDe>A1>Pv88V85O$lUWJ_i)-mG5>YBLsG zs&>@d)SeAfZYHWQ7LDN}bg6CqL#WYx;pO5)s#NEj$I9ZeAb=&(c8ARSRF(MK_50R! zFt$)K7L{{()3Xu8wObzmMJoC*=_z;}I~i-doVxrHpx*mlyS5dJ+JXOnF_ONJD}egw zd+pg)Y*NQI#c>cFs~RIoV<$e92j7$97TxrnQEZ})dC3mI-R622_KQ@AH3f_6rl#q6 zAL4r0%8O6;#<%1Dk8$VQ(e}JgdOckJHRpdBABP=!*-*oHgY7f2z3<^`TXH`;wqeK< zX<3rh)={SJv@I(LYX0@+{JjJS|Id*Bn~~U&^Y;oMJo+Afwk7w!4R?;~6mrYS+fK0t zr5oS6rso8RYx1xA(D*-$xtK3Hw2qvLOMr3j`{@707=0i8Vo3fPb_Fp0-$u&4uSY^b znJ1shjc-}gvl7I0iKW;7r$MUnMWfvNUiw9Y+L2TCzm3uN(l45%FXRfK{J#w+kM2jh zBe&F&s_MIUjAAvXI@|Hq)Gf0jI2(>;-)WAm^Ng1H?`gZ!PG*0q?8<8R@Je&49(41w z_u=Jym4G{S)!Y>ytub)2FX zC%L^x8tp&Mjw;%2uZ&*1*k&{eo*zcM7OC!-4nL&b71MG)GCvODbDV>+ox}`ABR-@(+)%QW zk4o5&-O=N7Z1xLz_GBZ+KBRlm|8;Xm=eQ%vafg>oC}Fx=%Itq=u+#O=EBBavk#MUV zFDRhCQed0KaFG3FapyrZC?>!F#a9RMQ-aDpEE=6M0b3gz(IzBzbN{2$)U2a z9K-z^A%F0rANIBZk$N#USr$HJ>6Up&3Ty07cV@MmDXfgQaz=?6e42Vx8Z(H}qavsh zBfjNc!5SP7i>c=D3;RcoU1ESZ5aHE|UZQc5g}Z~Rk;@05SSPj$-3xJxIdRJIgIqNj+s^u%ZxIuVwG z$&*%F#~89lrk3L+rLUP;cO^_*s~jWAe>^S6OGsZA{pYv~Bg#-vH$FWVCnVML{`-~Mw*65C?jR1-(0@~-!M5oxqhM!M&Rbj6Su zzCA#GDQ0?HMzR;~f9i68$aJif`tp}Gm(R@qK~ga+jfq|QOOm1zaEWq_H%`W|G$+x2 z9mjBqx7CT0z8v)r!R8i#zkWie{h2!KL%)8WPZuM2 zTs{!m9c?jmekL6k!o?P9lOE2jL>{BeAAhEP-H6|D)m_ww4=&8mgM!#U5^c9`a|d*L z`a})uNvtvW3?zJY2b0H@w48pO`sns@e5iDerO7|dn&q|RM0I;UB3&$Z&-kSP*)&sv zS;Igzu}T_m&&1aazjDc`WNU@%iG5@8vCsH7Om+`<0djYYwdq?0=RZAj&7I-#&2Lfi zvn%Go^v#|3KVPSZ=$FIW>_}MftBHQFmkB#tX?#c`=cXB9{L?DK)7XohF;V767ZP9i zkwemmZ6jWH;7^OA2AMC1seRI4Tb3X_u_1kQwP7}+LtQPD`&#cwm?XVdxs4iYCW0OX3`}>5)RP|43S20ZrfD;6qb9S* zJURtQo~8DrJ~8!QBGfylKUdgDwJVp{Hu$C?a``AX{`3EHNQX{va*%I-xmP_cUmDUYbwuY_ zMORXS6jq}O|7OSkypJ+|{M#TUST=<7|Iqc%;gJR1`Y;^Zww-kBiJgg^i9PYewr#Ux zYbKu9nb@{%W8R+g{JwL}_u{(xpQ^6CcHeccdsTJ!UaMRSZ4}ZDAy})0d+%CIFt;@N&sCMQevd*a4HD#a#p#*8E)xASbaJYrEvveV z{dXrSBG4Z#nIIi*ZefVkn4B?=@>*QdaNMZab=kugFWx-3787x#*Z&(~*aD??1~H#L z#b;rk^>0L;J$&grLta-U1!Ag_Pe zPF-CV$=CR}t2;1YkOA`JGaeK}Ug9=YzJy-ki_H3CduM2q& z1gFhs)h%f58hDDC#?me);0sv}g2eIM*a?+A|LkgusbuJD4N|9`kX`n3#p1K;IjHC9 zUk{RyUH&r-mme|V;M~*tzgwm&t|;7;j{+k?Wod@asi%`faa%l_x~bRj{X=NHf;Ja%!B zRdt1b?;YI#-t)0rvbgSLtbpr=BYKk>3 zjQ7zlij|So_L$lA8E{J)&>3>GrbjP}TTsQvSB!Z;jO9vi3muw8^?Q!B(;C8z{zHz) zfraU7WVDltDGGa7l2);?t-*9SZ@f^fllWvkG5$4a!vu{2ccsRceS$MBI(556=BEJ- zU%{#jdr9Vbl6`i);x}$h`|tI9s%?q)=qr867cgdJ<;{K)Q7|rE(5$+gk=t!(jDk)=WQy8g)|aqO6Ok`ZzD7Ny`we74 zv1U=|o`41N95y2F4JSz!ZIq6qFRv9hxtz-3C3UUa1?1;}R4S1o^?^G!*Mvq)cUv-L zC*NzhW;yf`Dk>)(bP120!&kjNC~!E?)%_b zz)^LTl4o<~w4PC9+caV@p`|Fx8n#x^RvbZwCCS89-$+_lE)R{tre0Iimr%6sEmO4Ffwam2n+bI8kDerej8NPU-7F24Jh48#~J%VUu%xjS?s|c$f=UuRPBK!3AH3DRJQ=s=B zn?@MKkSt(GqOk+-YnYTt!tYcWBaLxqVEcOQ0sIDHq{)^d20CyM{T$yP8$Z4?M zmb(;bX51ovm1Q8$W-cBI(jc1&6$4uwGV)N{9Sdn;-RxUnRi7xlY9UGhn>b@H&<$Qy zx|KTQ+G}@!go2hSlp4^}TQe27#Q7BRwCrH}t0bH69=N`?(P3NZ#! zXY4uq8M6?qT&vs`A_fS>tny~$MP=3PG;3{(ER{B6CQvPjI?>DjW77>vU!@>T`|C9X!D6MiXpcUMJiN*;-(&p_TK7$-%aLIK~-mZARU|b}XHz zGb|4TT-egC$$Hx01dV%CI06ul2_3lllGu!EDDT1y)V!*Y6y21zVT{1PDO4tG^TkjTXpmwl9*Xje=r6N^^ih`NX4>Ff4}N2_{LdC)EM{jiGSM;{GG+b=hEK- zDU2=?uDnVKDH561YJsG=i;hoM;4b#(HLO)_MmJd(BE9+;8wO_7UAB*MAJZsxTM{5E zZ+)cC@sQ{ZIUwjvytgTeP+LyE`D#5QJmmI{IJ*Z3fX+c{aA1eRN(1t+E@x9)6qxVl zxc1PYKJj{Zpcdp!Gzdb%J%&JR=93~O)yvfyb3s@&>{^> zfYt0L5n(;JN!4Oy1#mhP6d4#DPg+sI#j_iSpuu4D=t~L&DUvPV0UQWURbL4T7~&m} z9uTU-5T*m^l=Be5#IWc)MMTji0y<@p(7az*djn9E3Bl^Hpu>ZPe9vsBDOh;xx&HKH zURZ&_?qkmxABUE!1n+uqu443okKw~Lj7V=W;o1E(PFqH$5m6PXw2TOFY7t7H+Mq5P z4!xbu^)N4xA}tfm6=DW|6OHYWA;%9k2fr34kW_#Z2P6m>2e6ScLpYKM26;YZU?G9A zVP=Nb!qzvre*@@vu}^I;Vuwp1*{!^=tBw};LWM@&n#3Um~BCxVcb zMMDf@TA@{@W{uHqn#U-V&v6Tk!t&Api&5<7G4kpOzAD!^0@ThjiFh=sgh+uQI`?Hy z7o-W`mD3L49)E+Y3u;n^^ChQ42wv@1z7{7D689NSq*mmJk^oxK$moKtIf)8@(~^Z8 zMT&g%y0;FqA*6?6-ZuifVgvv{`JmlE+3i`T5^g3-em3|9foJ7+PV48yX1x;rZ4_Bxq$Q| z5*BLQXCtwdouo4$sZR=Y7?E~fe2q~MmS+ojI49Q{>kW-St7@CXQN}fQ2%K#q$8+ud z3i0)4ZvSCdd|wgaPriLzVyXUp$QaZlqEO0dwu_o9rA8Nv2_00~URa0x&!qKW&EtGh zYbP+ldd5FT)P=0TP~VuYQSR^@3D(J)NB)StUf4Ydo7>t|ZDN6dTh>6mfNMUg3K)`5 zN$2y4CS?G>Ix2DouJu^Q4LIfnMI4(DEY#&#S!m(kpqooj#iK%NZDOomPtucL=$o=q zT4yF=a38bC8vW03L`H={&r@KLGs})$^2~(Tlg*{5R46EcIpJlq*&ZP%;LTvsO;dBw z->c|hG`2@ZU4p{Pq`>_nd__owOfAi2JSBm1FdY(L5)$EuYmqXrYCOe^MgVxc;F?Ef zRP|tQ3CV9!VC8g6qhhmj?AGnbZ7dN`9o|;rc_|5N%1K6izIv$S^^`J^Cki-d6q2uS zXu*HsdvPfPCVNm{ROslDXt2ECSk)7{!PYuyQx4Xk0z?(Se$vA3sj-5egx%14Sz=#v zj1mZ;A!~XOMMta#V;o=#q5zKLzyjmn*-$Z!yvL*y#GmEOBrMQMUW&1P_U)1Ena<%6 z)5XqNnVA{!8u^q{4Oi#%Yfoh}HNNhPfRVfq1uB(52=t;ygy%6zJrd?(Stht^<(xCW zFe1HhfcyeHzTk518Y$4#6(nG}^jhAI z{>4Z)4M&HNMUB+K#?nyhf?ls6H9bD@#ToE41(!#u-D*g1GAAd2=|uemyY?L$X#shPNf1fFM!}jFa{s}SOxzr;bni2Be3V!1uSIl>?G;;H5XtNz(f)>me!VHnj2mBTC zu|iiCd5Gc%TPzN%?SOV)btx?aMIr%fOywK76I=#<-**p*l8ju=5Q31(F@R?jzNC>u zgNW~0p4SDwcQiKzfHE~*Lp9fSh?NDAqRI6{Ju=zu9PlRDf z*OTHLDbTSM>PF@E#^IfpLd)z4&^e^I2d}2YQ}jmBgHn~{Jw;%7A(G9|LXQ^*%n%NE zrcYae&=>-t5DHLeFcEzF7&Ui}6o>QhV&dICiiqbS#puZX#o^NoRKdNYuQ9?Ic6ioE zcMo|Lz!j-&OR$)|yT=S`U<`$2c+2#57cuo%Od8a&s9)5dwDX+BeV7J(Cr>n+$VD&F z0BrGh%brWe_~%}0)lH0{5~V-e^8ke-St2BWNDdGxMH_1U#Q7dL8-=OeG6YIre>V$v z)$uw$52~WiU{HwrN(^9Zn7P*>WGjN>ld9;Pm7$(e0;~u=lOQuM!3?uog za8A~z|NC(D{OT(U|19hBDs0v}tc%4xczwusFe9aCUvWe^GH5H!j1;kcYV5KY!ev1G z5HjUj>^C_%^p1Xs*n9_Nl7Pa8$2{osmd9XlWqk7%b-Cm($9^$Z^k)zvON{&(h`|UH zqBV3V!OGYW^yf(nn^SzUU^Ub&VY@JObF=FeLwwp{cisZpI5EAviikmz1H7_pY3xD3 zTb%Gvn zmt7N>ze5dcNs3eyExQFU@Y0Dh=(iW);oToEJ3H{#iz;RCRJy_z*x#E-^Y8nM{9V=& z7LN3-+AKCyQ~@5{mIphZC)Y2RS1ngi{5*P(w89_vHm|R1pbQLBKfkBC?zg)i{(=VU zvmXdrRRy#NUmxyY<{mnsdGER2ye6zm%GUDPZ91jK1}eDT-!Dnz=O18|HshD#-Itz} zimpDT7(1m5ef_*Zbx>aX?)8sXu1HBQH?KY+O0Skc=_K8LU2UH(-q*sfF5b>h7aJ1~ zbD*L%+k)Np@(2*LSu^Zxag(9JsokfZ0md_?J6* z*-4{*v{>+X3~pD~^v$Du7+o>md%D_)a zA&8Zz7%_q@v9C(OE{@(tN5PGeND?Vihm)vB{+~BxP$Y_jBldOBZ#1G$qbYW&(M9PH z9auS{<{hvqLRE@kie#YC+ji)jY@`wi<0weEVI+6pfrHS89PeJ)WZmYi@=Qe!bU&xuhhm)>@y@LU`&qUu; zuzn$9{Sp|2_*HZoVJ9bzKEZ=FfheJyV=814|E?ktjVuu@Fr&J|G>4uo8&9EQPo{Gi zso9}&M5$v(rh^^f_MI@)3v!TOsbTSmDAWRShDNEemy-GbU3WU3V%45(^)RwI24_2e z)TI?!;y1+1V@^?monu(_HgYcit={_H2(yQT7FF9e%C|8`s=V-Rz-9vWI=N}%;i-Vc zj~mc96nu_W*}ea6sMjrL-g`##m_fUbK=nI%kQ5@KGzh>UZYF?tIUnub;(Q-L*i4XB z0Nx8Q=h!)hS8t<|4@hf!krcMHifdkn%g?PYsYPfu?j){GN&|F>K>ObT=ebU5seS4? z4^V-=LT|MUZKoLBp?!uwh?@0GaZ9`w@Jc1ZCYKD?e zX&H&>>-#5g*4P`m0@N&?f97^h8QKwRe+tmZ+27u`8evcT>-YGhK)y`yXAsAb9-+%g zZTxrCEnrg>tK*;)tDoUd%d?!yb$T{Ce-+(UiZ0-JTmimOBjv|UR$+>}z^EWdec>Ya zE-qt7M>>=P)Hv}+`SinO!m6fY#VAwcehsplVqT`Oy=RAm zRN$Uxq>!zo1I}PLA{bk}%@T=Ut71l*2171uBHemU)dsVV6hlCn2Ml73nu@-tc&^%` zQz=SqEyN%C@SEcL9~PWt$0^A68W#v2^MU?$QcZjr?dnzV3#XdC!93Y6>Zqfc7*Wu> z_8!suY(*=?9{O1iAr|fz8C{t(=S`+E2O^|l1S#XD>O|giriAIkBrGw4D5ml5!KhOX z0u(9Q`!)JJw|K)FVu{frAy(h^Sxh&P!jdeDSvhL9PmLH+MOy5^N+TbvXVaAf`?5J^ zCkko=5LV{9zlRwY`+fza3KZ>S8;EeM>brJ-aqaju!#f)s`56;I>`s;N`S}@kTVPXR z(9%Q}y{R3(dA>?$p60 zyJu9#9&$cpYtI|ID->>*a)3Qo(+~LegA=F{>n{>i4*lQp+ZT$x;M-$I@G(yj2R5!X z!9g9K@a@PI_^N#K;RP_}K;5Q5cXpR-)Q*6)Rp)K)=7=?jxuCMx5-?f;XUZJ75(@jO z3QW1y8VTF!MD5#EwIMr=jYSN5LN(3;?v#JCLA|b-dM~-o{r{T?v#_;Al}_}QK{=l; z=Zn=}lj$Cu-Id>?zETk*VY3Ac%*pcmpvjSZ3Bf3cx?bsz)rJN~FK~S2#6jyQhcoag zkmj-uaaRuy6+?zzAJXFj?1FNACLG}$*Vu>{+EBZ|TQa5KXY_;E0_wp(mBz2vL2MErFSVECxWrXR0bOFwrd`Qk4E;PKv!X09Eaxlz9S-SfX z&`&5z;qoTP)aMpeC)4jDp*)kMV)oM4W&47M()iU<_Nv@cBUDbmzKhwv=zrwWMbcfl z_kWKE0DJf)un?g4HPt@S5_AI5CYrRx>?KV) zvj&p1fb33Hr|oG8YZ-;`O$Gb-#9K_{EtSPAGzjB1;qoaY%#~5psAOW1K*{HGHtF)fo`}A=x-Vuy&n4nCUDJ?+TtNcF z?2%k43E35;;!fyooot_IPG5qAsjtYDr0o%ueqp;*oME+Pa**@1MXX1ZZ~qs5ED|0f zbQL-*5V{9lF?(o!XoHkB++_%-D%l4NC&EMyYISLOtW5^8mMl)tM#SR11PR}z@oGnC|a_t8$#BjKn`#sXN%sZ;i@k; zVkbWnqq6~-_iO>*rU9bA?SJSeZ*c|DZ@&=+OKWqXrW`$@u-6%f4rj@xr;IH}A`*fj z8hyQD8mAmAO;b99o`L2Q*+YfprB@^sQ3gxX6@+0wx4}TLS8gXWk?^NL2#Jd>c(pRNb52Fq`W$+`bSiLlp56M+C)$rY z1tR16J=TaKC91rpREMds8KOZ{;T3~H8NW%xRS#t0AMUl_ToCRi5N`CQ*n6XDVbgzZ zU(oyyw_UrqUFG5@RA4`Ejd_QaVQo3c2td#`}KNsdZs7Ew`bGvpG4My9c>}DXYDa;6yIB4c^1T_HubS$aE>tTHi7u= zv-!t2_xOp&1e~78X-=j%w757;R-Jv zIUqw@^N--I6@I z>|3l2!KJRplTC)KpHn#9NqBY>3VGWa-`qw-M2HmxtXo%ZVK^~Pj}oqcQ;ZR`fI2{~ zzAp7&K_0X}r~-A&c7&{l&Bv4z9rr5-vyarQE`PyqqaHR8vEL;cnP8@9>1o-vF&%}B z>e&LLGpUFG*&GDf#5h}^O6!0~P1My`Y6@8&!cr(lrk*>@o+z(Dajhk09ckn4-1paCqWVxPiTDVg|KO%@8k;*I*np%^-Umhc|yVjbrHA2xAY$q)$kI|w#_l4q*r*hZWl3}I>uj!x`9uqm7v(a#o& zSsEnwH5Xg~6rhET!}|67V`ri{mbPDQd>=t*V*XrjY0To^CmZ{ zfG0z!AEj)N4!dDC6(;lnm`im%2>Fn_@ET}jju7W!S|Ui8GoUqXTPog2DO?e>rafz3 z!F_M`aqr7EMJI3ldekERMk~$>D~#)sIgZqqdv*&&<__QQS=j-Lk}vqj;kv9}szABN zLBrJnegGS8!KmNN5N#Y=R8Qk)y7GEN9(@V-ZV_m6GpIrB1wVv5il4}*99e*2Zlu=< z?!g`>htdq;^tIZOVI$) zHa1xfC%gE7-E3+)t@KA~CLYbuNQBsO;wQ7PgS;ag=tWaLgA0<-gJ>*mM#3dZ?8Vca zuT{0j=#oqr%MsA1;_Y5=1p&eKw((kMi!L66xLzGa(lVCQF6D(GCk!N49X>^U+2+{B zzja_cezfX<|ER+Dc-X^Q$-S@-Js87=P}pP~4jLvwQ|B651`{%=2)L}Y-ot2ToMRCE zZl;BCt%$$#Z>`yhh0nq`lN&gEHUU4-p!R`3h&|3!=AVvBR5VX8=-lyOpOr%z4!E0i z-s`(fMND+;WpS586{d)X{5+15tk%*HPOIO<=ru5IFVre!->f>@NxR+*Da9WwIOf~z z)a&zcJu&R_!OYD#nTyz#b+>4}`etuz)G_S)bI}jB*#{jm*nJsn91IiYtg+n$jHBSV zSDp@v74r<bv~0GM5xWs+8Y@)Go|64`C$9bM8Sc$m%z7Gu@*M zakfIVsU4Ki5EJ+(4UIVSWvyZ!OF6CkEcFmy*W5LVQ^I6UN?t6;(kK>OWkdbjbkQvt z^3p$?rS7xY^vB+5zX>yr+43u!bRqZ~V_x(qIj@wFeuOi2fS}_S9D~cyNnl}~-n2qj z?g>Vbcgz)JC=-Z~bA-tXMnx`Qu^M)&m8{&Fqx{Iyjd)?mWuD^JyXeWVkD2Becdzv( zUcMbf2`oMy^(F5^->3&qxGfPltg4KQ&Zv{AwA%-h&Kyc*Gt~H#&bF11M_<#0#hUNW z7XMN?+Zpo$vo8-vTs|aIFq=`gDI-99Hz8d(?DM_tAP``9hqki#a-bZ8H{c99ETllY z&qpKG6#Wh`eb!XM=%v75($zj`hkq96@BzGDb!;<4#RM=i~dvsZ3^d?=EDJ&*k=z`8VuCOH_%E$jvJ`SQ>>{#R< zC@sHou!E4L3(#FRLhZx4q5@7s(YVqL0WKHH-LpWL#r_XwHBen5 zM(T_JF^Id|Ga>k9p8^RodXHOWI0uQw*wa9Dx&siku6hGLlBzSU6{51&82eRUpDFf1 z`99T;6$Vf~kk&M2DDx)8@>U&MKd|!h_Oj!};-4m+enIJW=}`OxyzXlw zJySaH?#62~?Q~YQH$*()96zkG(*;pr?-=NJH^V(iVNs7XK{n!U0s=bQ2UvnS8@R_g ztz<5XS=6eZm|EmnH%tY!?EVhjN@B`4#}b-VQ;`CYqX)s5#bzMmmEkg|7WZzVrej@w z;T_{H9O77z$r6bBDc6F_loi0w7QVn|0}KUjO4~xU>t-FFZ5Jf5GT-y+Tg$l}1vcAi z3ysFyRiFwBNn84B(EB)tT2Z2&RY@*1G)BxDwaBnjsxNA>O#1*AyX9YB*w+bXISRN^ zAl#e~vw1;|j`aLLjvi6BsVyMbpH2`>a+iDHh5!m2l4g2G*%y{_65Ed9Vq*Wut{i4R zWSk|uFQME5{Mac!+fEbRiYTVqS}M`f+0puk*OR%%&dyyP9@K(1c9OeR9))OTklr6 zVzO)Y2^`Em#M$lo`u+5m2Cj~oL%w<}#@!jyq-!>`Zr3D50+w$Zi-ZV8cs7L?ef%pZ zu2GyaCRvAhQ&0heP2%bpcA|klWF}KoIhL_@w#LM}D%DxUy1=y9)7PQGO;Ab)q$B_jzl$nxbC;rfsBj$vgfR;}auXMY)7K z8xYG?a_dl6#) zCGN`qi2F=V3v0}q=aj!ya+-(6O3Hx_MjhLbx#yp{`~H`@*Pobp%vhK9gjDRVhHk0x zSz3MW2YUS3T;=0hO5 ziD$G%muy=_AmOa^D2Cmei_+6UM_f@!8eSrzC$#Q_MJO{6O>3zUr>htSs!xQykpl;K z8S|#Jl5ZvFc?!QP+H_M>bZGPD)zNH3mxI{Pm@cr6!D4XcH*8951%H^3GU3g+V#@gO z!X7))JjoJnGC*eEh)aPJ{rzr1J<*Uc*E^2!3N~*6jc3V{UcJx0c-NKwsRWhIPyTA zaBCnTibYjV`A0C~dY0x08zIVU5oNHk!dhi)&h*q2e5^Z96T(|2s6MTOWV2r62)tcW zy`};mx(B#Qw#tyT&{3$G7=Bw4?jdy}M2{CEVh0-1eNU5=r-4jIrf!qTWR$qvIPf~? zOR6%GX|V`Pw&_sWS=rSWi)Gi33}s}_4=dQ?il!L3X7_8Aub0m#jo-jE^e%J3EAW7h zWm${(rn;T(U}=rP!(Wn`Tu}U?qdjyazvt}OA_dHKCMzJS1P0OMKv6#lH=+F^Y<|n1 zc@W_NYH*u_>8?1pbRK!tY3%6?>zt!2*Xki&EALi(jcLfWW#Xxsb*hsrDmHo6VZ>$~ zx*(^rZTi`R1XtT*Nk{ugV4D`oLhVBvI&_TUH(JuLoiI$yvl|f?f4{XCG<}!$$#_z0 z+o~b8Eupb3`L4hd$A{!`N5Znqs;A(pe~CJQWE4)@Ig# zngJ!54BL%5cKTX;fMol7cOL)Zq0t(@d{Ze0Qr)J;cE%$Sq|}Lg&1Ajf_YfT!NDFJc zNSg_)Ta)-${k6R!URG~U0id*g6fvPR7^s?}h$Q6K%kGmReOg;iah$$y=+{&syy)6K zAIX;8rEa`S!Oe?d%vBiOjKBKp3{k3UiF$Yl^3oxHkC0Qs8fc-opc@&Wyi0ilDkaZI{By2 z-UT38^sU$Cmz+m#>cHc(PLGTcKuCWw#CypFT~ZZmP5eBPE^5@lO*ZUEvO(}qUPNIA z>i2XX4+E?+*i|c)6t&&+B#fzO6Ypw4bV)YGN#<`i@3tF{mfjRmCMG%h7<-;7w)wsP zE+hP&z7D!<&Ge#y?$3#%n{51%q@!0OC=GzlVq}F{JP&v9(dgf0NjdsS`<^QEam`~^ zPHwWvxMq${t%6^ZmG)KSQ+~-6c96xdvbnQqOfpy82GLoAD(sGehNa;at9&_UE*o2+ z);G}3&5Zi}p&4(8>aNvcb0ZSm;eeKU<5CK$-`Seu;1l4SsG!PSA@VMG|~&|#~Z=&8TbDpgsnvX_qJZNOcj+*xT06s7-#5Ac(1J2Fp>+Bo|owFA@JVau#d z#+a?n>YrsR5#me_z9MxS6u_MMQXQa*{uvo8UCd!zooEM+rlC-D}P)1*1ai^#Uokuau2G!W)5@6;qu7*=3Z}c#rzun{r zs*q_=-i{?bn=b;%ay(0o-W=#z8lQ?DDB_BF4?P1BUXFH6yd5(MN1{#h5XxC&HT4`= znCHMf9mk9Qe%)^ExHZr#1;1s_l>WeG|2w$ww43xYl`qSIde0b9)Cys!cl!<+YWjw- zOIO>6n?CPnKy)Y4)h93KNZr+Hc6Ya%0ehMH?&&EDYTev{%$#A)*G}Q4$M}lUtrgxUDMv-$4^qn-qWx*s7EDfO0nFp37yF13t(iQ?7rGlabhYm9 zAY!cGRD8FPib8xGq<%2qZFbQuCpl>Ntagt-XYP|hC{hG=efGF}%KknIZ2-t0(HmnW zU*0CUOd%GTBy@eggWhR7qE@AqF!YD(=%8A^>s!VF(uS(Yw~WcP3$n!U&mEn`2dAf- zc^lxZLEK_>0m@bns2B2Zr3NeR9?JU1)Lj^CxL>ruuGT4XKh-!HHUd>-2C)$eD2P8Dm4K}X?={l7hSX?a^GAI2dQ29ynrI+iq{SghD$G{(x~X8 zt#Y|dA(l*zb2n1iQjgczcB-4Zq>9!VLG1aIsV7p3|fc7PvAKDY8s21Kt}c{Za^oK0aqv zpwDJ(`#9phlVt|let6%%a0l0jdjVI$H>}U_^ z*Bgi!CHW)a&+DDXjf?n}C)BI0zpwwN1{LY$<&Ph=!kur2;bmnz?tgDUxeb?l8*>+X zbqyHrpH)??<-52lNn}}y38gBE)^U7|(jH(~MsvaWQs}yP&2d$9W@vyT1|;6tTOK>O z*!t|R-+UeP`wQ3v7!=tieJRDAnE$Y zV!wK7U-wICyJNg{y@OK2eVhBeau?-YphiAgAx+6S_fV9X)dE}_8-7FnXjL6&1?z5= z$;!NB-1LWb8|Z_L-l|`eO?^HL;3f z9wW5Ck0CY-bEv^c;!v@SMsSXG7R+vY}-GYerBS3Wj54DYF6}AH#cn zA;5rAb>6=pypmqT(hlMKd>cI_+?W6N|8{U3e{=PZ7SIvAlrvyp>i^GknEv06kmE8n z=4EB$;W6jpWaH%EGG*toFy`RpV&~vsH8Ey0WjEwB{9h+I{X4Qr-^zJcChJqETTITs zJP(iR$hY-8Yk6h!@**OSvfbPI%VRWTuq|e2FS$corQb)_k5ds`u|A`op}2aLq^0Dm zM5Vm;=d<*;$S1y-!@ZUJTVmvqFDyAI--~YUA8$9h{r}GJTy?yduo5HF{Ic?RvFG=G zT0-@2e@5MU@`{eibocy$9NMbS5_qMcu zv-sEZ6q@(jY><6)H1N`>9G^d3nH9Z00| z`!~adJ+GbF5RSxIKF{-tu4!UKG-^ciE-$!Zp$JWo;9mA0dvQld2_14G zLt6ff{v}pPb;L2w|KdZ+3{%u=vQ0qdyS=()>0EpAPbTco2dbsX%<)2hxD_y|ArWEDJ zqoBo2uM(Y9$RrUaE^gZI!kHDYM7}y;7AVX&XHZl;1XH9m!F5M_CuRc3HtK&4o>{fm z+vLxF^&|Zd`IFvx1-C<_fV}&;ao3BE&nTbYAfi*7`Xro}A9ye1C6vCs&QyZbEo`BQN+^0yYcEy1bq_kE{xAe1h*RBG`u0GsH3|1Ci)FzCW)Fod_XxPTN^Np= z_G~m3R>MYEk0nKgv=#Epok?HrQa#*dHZE(ITd5-zDM?*@&ox7(TJJ7;n2;;iJ^9uvOKJR*F((Pm8BTl7yJxF*Oj*k}F!0<>t?D*;f2jmZz!DIpTS{TX_cw>`@18$q#_jUq+va!_ zh2&zoex2rbdW(4Qg+b#(u`uO~Ln~L6;3(E-tsV4J>oby05`i1#6|@ZNjbYK$9e3l^ zDY=Ju0Hd(2(bQw9fK`uvp`s)e0M1v}n4wL%v15vSMrJ_+FUCGo^|_7N+qQ2hgI!v+yHxTisU66N}z7Sjm>VHo39BtLfkg1@e9Vz&{dJ%_mD1l1uT-PhkW6wum+0)KIjnaC1 zUrpeO6*!aheJGf9P5ENDVhv%~f^Gb~P9wGHd3gob6~f(i`9q|08ZYf1ms~84G5&_h zqCHsY34Te)Gc%2Nf?tXiO(mShxviEE%!KcEutUd@5Eo8TL2ehs&)^d>9oVI`*sOxR zG5jv2;lBM0(5troi>tf3DuNrDW6u3Ml*L}njN`#YUNt_Npd%+NR8aG0UIF_aS+atT ztDsluLvNDTp*G+s>&})rRaGjLccSFdl=*=(!Q;1cg`5}h6`TS+H-@b_y^3#sRRAmU z`t~)xTQDWYTk&$uQuCmS^Z6nnyK2S-@8h=47OO8Wk*;f~rwq}XhDQze@2WTpc)0bn zEmj_~r_(pID%OVgxC7EiR*<+Q6Sm=)b@$9`F#oD!;iJ z*5LkJNc>&5h{XZz<#1VLM9(^#7qa!X(JZxO<_VYV4ft@fdCd8maDcTq6?rN5Q&1u6 zqudteoiptU&-1CuT-teGgiFEe%-4-UyTL82GQM4@;R3fu4S4aIGW|h>LcO>VNPR9mRDQTL}XgRe=3E_$k57 zhn}+Fa@?+k16M=;hv{{9ktTv%3apc;@K~dVZtrIpok}Gau7_&(rdFpC#(F%iF(-b1 zw{v1xEX>_+Q7lnCJ?!dlvz}jw8|9w4Q;1xo2&6*LJF_&5%XU?{&IuJY^s2RPva^@n zKwoB(rnURYg~u`$Jh5qWPO~0(8GvfPQu5W`zrk-IzzP;-B5!fp(Q?0;7hJ%`94>oW zzVVG=ozAe&Jbr0aEvIFE#lUZs=iVt)*jrW)q2)d$*3i1Z=tAcDF)X!4_YxWJC0(*# zmmPn2Wz3gH3DxsT(2TGwh&j6`cl>w^Z*_LijCG0*YTA_3FJe;qMk49a5VEN@8-&nG zflPh>kEu}^LJJu+YGF3jtMZihR901G5wRMGy72uJuoALM3X`n>b{(65q}McPgdb1X z;bXGlzvzE^Q~ zoVGUN7}h4TJD2mEQM{}xBet{R$W+P*5ZzQD!5N(75*G(|9lU=z@XK3ks1E*WQZBJy zk5PrlOW2aOxLf?d3mqfLz+3aSLNob2-=EPe?Sww2E}MCv+AcJBT5ZlH(?h>A-76~C zz>CW3kBqdiSh|i6#)`f8d#)+$)|`#Wjp#2hJbsiY=PFZfkSGSQdUoT(@$mC&US< zpX-U6ZuKo%mYX`8lcM(}=RAVnZ9x5_39@;B>b7!kX>VnXAwl zzT(lIA)f3+V3{LlEDw8$tV1!~ahWN?V^Pu~zYUR+K29|B5(ND-&NpNnRo67o!1;Zn zsZS@{Fr1>bOO^BMOzQH8yj6jn`cJn#6x^Mobe^w1()fnKDz2RJnM}%oK^Z;PnMqfI zg;0W63`!8d-43vyjyo+lW?pS^zaWxGjp_7Z_*{g)z`(~ic_I=%OGuiN6jBj?L5tKk zL`sCUhldAul3}cvOQaauuWYt6t#+1{idz2Ap6J=A?{QjiKP{yLzyy5qg%30t(9rWy zjqhM;!uTSk>259)mB26-;Svu6fZte(w>0kQxr4k3Tu`!|%NS@+naJb{$pf^e z-HDqQ>8J|@JSPmpIgk&nOun?N zUb&*^sbj>>zl*V29lg+M%@J-&wYR~ISiiC~3%0AQc7DP1?WIw^hE=9RuVK!ZNaSj07isgqWH{?=wa1yO)wgp*boXd$*#OnvB zb(;mv8+9%%yJ z&+4Rk_XI*kAY-tRXUWIXyF+?>hD-DLaWG=Krf9&oF~lQ^sV8Sh6g8^Z?a{S_-h8yU zgYPD?sSmX^W5%Jqn|bbZmJ)CCzTIXgoVF0hs6{G6LI+lRB5^}qHKzAQ4ijw-i+%9Uj@;GI(wX- zmAG;c45S7YsR12A9pS{TozKGz(_$^aEw!UPsV|KD*yxkI%?n9A$^ep?qhTd6_bXCXbp2l{QNze6` zw%0q}=Aw(AXK_orT^=`&ZA^xY3&OXyr){V)arLs>>X6sY1nOtC9Fi5zQCt-65U!kBRm@RPcl`TGy68%8 z*oPz-j96OQQ5Qr-tlH9EwTj}5$wpD@an;N$Yt%WZz0vkLVFJ!Njw4jn^OA;8pqZ4f z*YSI**HJ68)Yp8QohJFcFikIQPPj zs`+)kM{j1A16c>ES?G*8(|ePJDACI7t#>#LwiNlgzz9%bw#MRqZhp6`g6HmFKHnpV zu~zr|`l#i(k}ixo;J}g}Jj#i1H>kjR>xJOKNx0DWc3MEO*r6%DGvw9P%ry1Ka{}wZ zkb4X`q`}(h)p4|QoV4UeU5`aYx2Bo{O4Nb8KSWiJMNOYYHAmu0KD&Ai{BVYplL!)T zpUZbHinPRQuwBicA4lG|RK!@Dpu}djWqNDbNG+r1Dh^?}mC~&R$-5FsS1-ur6Cw8&W63PbObPKsm;Haunv8+pi17@@d`iab2V$<<(9r9;mg`aSBE!`urZ}|IhPgux2!J{_8xnfusOs<}yIm?TBMGzNqMQl=_YD z`(=JV4$TKbZARM$BhRM8qjsZZhGEj_+(xpLjAy#B0VnB-vE@JhiTHalnCNmTi?6gc z)mRFuY`PuECLQF%2z09Wn<<+&u21 z$-{+?m>z-`A1swB;95fwDmjo9>LK|www2?y#5wc4qv6xKnAJoSr<0jOZUNG?hbTfA zCRq;ZXfebXl(0j@Ny$B+P@6`c+%&9(Mw#}tU7@2})fl%Nx+-qay09kKaq%U8uDmvs zdrDA6<{MlKu|w4AA@=S_zMAf(o9H9^Xx?P*8)sP1U*A;-AJ(FJ_NkrhQM&b0yY&%! z^izBEk$UwLd-W0f`kgQOkBJOO8xRtBJK)jHpXTrHegOZcVSOY(r6m~_0D!3UZ?p8j z8>N=U_ICe^Rei6{mK~0GlJAT9^Afz4F?F_QzXb{WSk6NXAgsZk?&c3Nh0!t*m{GI| zG}5;#_PNOGTdK?wO3gAD4P8~uOvKDYaym~b`R~u~zi)?LuAZ)KeID-;^a_ zV+Wsh?}y!&9vzz6m>l%_(BR?f<@HJR>@slw%xamBn`47OGW#9;!pBiH`}oo2#bEaI z>gwgAmuti80)8Ji_eRf{ia(O-1ZWFg+=_y4}0W`|BsK`xmI;J`gpxM&Y^*$ z+vD^8+lN6uzR$<1&^M1~??H*omH;2 zc{w`q;KkF&*~V_ft>NZ(EkC?HH9Pt~e0q92>R5X$jv2_8-ds$**(h7|h2_XQX+=#= z(8!E$&4HF&_3P821!<$JH&AZGjb~3czx-IHHwCq(W`R@xK(bBYrym*UJWFF6QwBfS zYSvOu`ZBNHog5aIzP*msvfugKXT#sE7O-#D+39Rv_=89DYSHTUUkP^@Qu}L-v%C5J z#Q9f0>AuiwNBl$PG99;%W3Rux%K+q~Wi2C!-^-vB&Iv zKK%gJZ~8D#d?FK9!;Umfu<8f8YD?0@H(_(J# z8ZnK4<3-&z_^6GHSPVfcv?9RT!}3_FB#o$q5DDxZFSdll9@b!U7~?9^4Z}HkgF|2| zVWEzNHI)Z7z0I`w3L)t;lR%E@nP4P3P4Pr!6}u#|*=^d8F-PU(VM%mjG~lu8jVrGL zt+&Z~6V1_BisLTEEC|2s@UbteXYp)Hdzv|1Q(lCD@Lk;m4#97ZK?8cL8S6X_DmnR;u4 zA=;9)Sn502#lhH0)pm4Z*vbUG6j}ooVo#JY3z*R6Qusr= zqwUEVMzQ`|5jAaVp33N#tl4slI531A2U#TI(2%K@51|Mfie(_h5NcMC-wgqbN>r+| z>e5Hg8X=e(<~qqPQIS|(MjfDBWVl`wC5s}S4UayK72l0%rbEwhty2oH(4iRfr!ml5{NZ$rvh_@mTr)+Wt+G)yP(6; z2+OQ#S>+gl1F<~%H@YWVPA!oD5|zaMiy@@lNBYRzRK6Pvg=Va#32VAuaj$cG9(`Ar z5xrlDDO-!{p=IjBHqs@-NS)oDx~B~yVwQih@q3FNGrG4N^om?xam=)Ba8m|&|L7_1 zj(0~<7`nw)0yK!R-560N$%GJW3Yi2)0$KwJPy%{R>B_IaZV+iia<7$<7&E@M249h@ zHG&Tl;C_7O%Mj-(LO@<8TP$9@DGiA&K(hsOK!iRJ+6-bZ2r+|r5kG<)5Z08mdBUQ$ z{d4d|!V7lohT$Cu8|`Vt4Z5#@#vH<5CPV8?4F)7|7?)8YVQE!}EGH5$BTGG(lWjRN z;Bwmv9IQHSMjJ!f!Q&(?_%ucp$7oH1^3vG`HR)|llWLn&i{b@!t40zl-P5cj~Ei4PI|2FTTtt{iqW zru2=qQVoiw4;~;#guPLra20617>F6?njsJ=gM}C094;)c$#?@iGl?45F9CWp!lJ~X z^WY&%REN~gMTDiSen5p9SJRb2wH_?xaE@72g0zjKK5S;ccF~tBgl!ZxLqq`lUT?`% z7@SWqXr(n+WYLM=qCvbJ4`T#UZW*w>)-A7YFmj}YrZ`GtY`RHM2^aVuJ5j#|iCZJm zB-h2BVNeo1S>KGr~soL00__^GtmV=2rN{=MqAx$h^d;2J42WOnkmMdkaY8@nqJf2h(-{jYMwA1x;h_Ok8!q*TCU*j>HiD?hsi?x| z1U*KVSJW_}a2ynogL$JDLkLJtc>u-h`ag^*gpR5RtN@Dj1Xch=mBAI61r#63XF)eK zubiTaV4`}4C=Z3UP{k0XLRSn>$X5(p7Q50IxMU1a?I);Wvmj!y1}!p%E7y)FTFih@ zP+T(YfyFtCaHSHwQiLf)7R|N7m4;^38d0fMa*Zh6QInH!(u@L5P-PLwtn7BZ01JaD zr&%bni^QSMT6xw(Iro+7oYy^k^K1`w*knnN{OHv3a03kC*CqskB zo{iahgNqU$fMn6FBp5c$UWu(H2%CSjCGsVG;%>b?gQ$l4as_%^$`fJQD0(`s`>OcO zf$?WU#9mo@(|SILOHY;e@C+`Gddqet--(vkCNF%j5xWOzC5S?p#7eHD;J9bK)XazI za#65dY)JP*A{`wmARP8FtHWezFM6yOW=s+W`?u^ob}=z~(aVxF?t&U*!e;BtT`<_z z_->NiSQD!`$-rS4N7Sr@jd*5OGiN;=;2F*?{7&O&_=H2TkXv6ucPXR(ty&s}*X5#8 zF7mn9lQ`o6Sxa@10znCZjD|G92&>i=ow<0aR8g^kapC;IH)n=O1K=^q#A4c$au(H4 zsve-Oai<~%_h;*8&)*K8@Ob)S(hF4Dh^qxK8DEh$KMV$c=VDAWwXUI?IQG0~Ki|gd zq}mU756w^!W>8lY5(J!W#^Aj2sQVHgsXp<0r~=VT(uhh}^wOs%viwjWk(hV2i9or~YYeWkmFdT! zOEO_DGLzNbNj48vSeX`VfeqAa(`rrB>~}uF2O8h{a@b}JLLhXGlu*$zlvI?i2S^Q& znA$GrvyRj(tiV?wOEf5!xKwgv2$;5A(HE7WcH6^*v>$C@ zr^xVxKWItMs#00RsBmd!%E#NzUiSQ*@IYI)e(oR|j-^Bc24;9D=}l{6*4X(nGqaj4 zXh{1ay#-q$rD7WIUFH!X^4Ry90Td)h)mI{`I|+5D^K~Z80Sg$oTp?~D!4avc8j5$5$Mc;1+}R7EPH?w=7$R=5D##HX$&3UYk5emy^yHvowtDj3{OU?gJ)6}2iElLEs4%k z#o6dgQl+yiW|YW=GoybYKr^K;MEKEdI;Y74Rv}F3Vf`Y}k7Xz^1oilNBBe-p&zyuJ%W5v73Uqm(q=_# zz^ez*%`z6Vx0?hGm48t`1MS~7A=FZ-AV9^vYtcQ>a=O>be*XMps>dId2d(G#fZDcI zPPjyJ37dqGNJxC5=9tf!uLem<%J4*RJSHkPdj^7SqF?XjjQhS)8?#1^BCzI6?}j8d z>=DXFQ0m{v*16LEIS-m4XZ%4-em%ZFLEl}_ZyKgw+d#iKcXxm6=6bzO=f4`2_q#~e z>u^8vX8oKMb7$-Ldk_on=j!Zc?B4j(sM))>T%u^ttE+D_v${c#RCAa@%vy$vKX>Bw za^A={Sv&Uc^ef-v=lS*T##{Kff3Hq{?kA1mPCkG^d_Ug_zF+IUAazyAlTU&~w?vTY zz$jRl0L&g--`Bl@UV@;0l9c@#T(UK;JGY?O&m#0V#)IR`$mdrRM5k);RdwSktt%j z1)8BGnrI0<*;WoGrA`!OLtW zjmf%VgKMgyQ=$#h0AmR=B{gbVM;QV>gc(&yOKNWf&fq2t z5g{D{XJSuQw@0h9Ix#18;gdwBOWmYbvnI5^wk4yF{~**z<~bDkNg)X-_+gX^5T_st z>fQ&ARPPs=9(f@Id?koO`O?mWisG3JCYFHIA)1k4&CP41vl}SdM*9Tx)be@vcRQ4VO7Uv>_VMTtcb_U z47DROF9I|*42n5%v3!XwpXVZPZE6wY6ptnRhkt2&<_gl zMn{JGWTi)2TH-85{qcbU>Iin=pJ0^0~ie9QRm5C{vhJCAr2YX%fh;1_nc6 zd5jVYt+NJ0#6k{17|{m-?>6hQ*p%-ik%rmn@+Ucz)U(FF1eN1t&?N@jrj(qg1VmEq zZ&;L$M)EPeD$3Ewc7wfdBskh+UvLrJj3mkc?=rx;^Zye{2Iwp4XVGU3kk8~HgqH$3 zFRs`lbBHH_9%ytJ7BZ;7iNwYvzmYPYFQIQSm44MVAPY2gy;xfBgPp()dLg(?t_GTc zDY-)_%{F1$B9dWGiXv|0sj^Gekpv1>-=)ZfE1zT+qF}cn<+VBn%uo`?wPeQ-4uECL zfr3nWT_8+8v7yaHet;DvwGq@!EqYIU5?O{h1}PQslP(o>Ur{lfBv+q8um036cAiltUiHCm*U<; zf#lQ;F>bbq%+(vE1Oah58J0!#c6CXrx01Gz!YUR&NMmc_mbh0v#0&@0{ln6C_FsMMtOe3L!9*$RBDmZGC+V`ITxJ!a{=@}X8YiHi2+16jS4#8 z_yNsENDfeeD|Oo%@v3=@sI?l_gMaO^TP8*{n3BBCCn1&FkW-v_I)~7V2~rO7?=z8V z0WRa-4}jI8YBZfznD{XyNbj!`1u$PX5I9I*!OBy|j$pFl!fAn*h7}j%TZ2M~jjn<$ z@W|T+{_xBwSU_1`&`CV83uiT_inPR8n!#I{L0Or>S^aquWVFpN*t};fGlcR{Q@ARF zR~W8hJ_e{T^sF-4WEf;|$d`lwt{61xVJf$D78RWJ0kIlj32iY1viu`Y=`Jz+I(Ko( zmmKR3_M4k-&CM2L>@C5p zO`xQ;gR&H0fx8q2S;DA7kGn)s&vX!-X^@p^vU#A%jOLMG=~+bY6h>PzFSo~QE~#K_c@ zG_X^GJA+y0Oq6w)w?@EHl=ii5JIu%i!a|8wHTkjcVL$+E%!2-7Wxd=c&dL=0$Fi|} zP;m^hHTzxhwmjQ(P4?k98| z(B2&%w-RCxQ0}lmDS*eNWPBiz`l7EOrHKZjcwp%&@2jbLOfBLF_2}<0b-GbZkmujager!LWts>NkX0`hyOuV zDsMwc(2KxNASp3)S-a6$+RWQl4+}SONs&mZXe!4#k2r6_Ig!AUSn~qnNyl(bvSEBR zBs*x{SQU9Yzg6Pfsri+{0Wn>;lR{T*WkC z@5hllr)VxJP2m++B zmSpLj=7}isBHrjYJqQ&2W0@818AAA9BCI4_^%oULhfMMW7610(O%b*Oq2SM z+|{$*2+Js!r)G2lQ9#MNfiD*gOTQQ{*_W(nx3heEw&=7h|Ed@P8bZA0$qnF#`x2$? zDmdyKK53c?L3X{4vPjS5<;(Fh9QjfyH+yb#`!&}__bUkszMeVK^a?#iBMrC8Ld`mK zypr|`yaiV|4M=SakO%OK#oVihk`VJX=f?FrXx`EkQ@=DoFbq2Eo}2hyoJsP5_ur61 z0gwhSA0y&dDr&}L`>LCrx4v%Gt)L`iz7SbV5qkNe^b>H%G|$L0dB1z*+&$(V2}YpK zIvYZHo%6Yxhh>vIZwlQ26?UQ!`s zkHir!_&z2_EKVbB&ii1gY4N13eSFr+CSDRmw2BmpszV568B3~dl~r!QfgLmm7Qf|f zh8kkPsLZN)iea-AT@9Y_mALJ)0DonIhiQDI+qVmtAUZKt3Dx^t#bp%wr)Fr#Auy)9 zLfkoUgiX*Ki|A}lA%i0jNh_ti){}90gB)!i3HH?A-vX!X0rm{?L$KJc|Hc15VJ3xS z(=Fy-K0b0i0D#2*4Kw|JGV%WtqWVt`zPA05SejqH{-7!zS1{s5m;Uw*-n9+e20-S9 zVA2ijztIHJO*E}Em?*pMr>mD7BGCsbR9nq7rY1n`lFNUqoh#}u@czDC*`WG|CY-u6 z^t+1@J@z*?KV& zCjOIy&xH@7U-odchbDi|MG%O9sGKq_5GnD?!N&m@SX`xns~0<(XP-PY1&Lpl^orj2FAhE%d$bAM9T?!L%MS}g-|3I;;QWET_JRM$ zhrIcUmNOgv&gk^VA_pGIDt`f@N!Y{u4m*1A2Z`R;WV*-wgFN7;PsYf!V}h^Syrd5< zIYIsF#{~@>-!I6ENqYTA6)B#N4qy4f(5V%ZBpE?DdjwH+GptFfmiLHBmh0aay}W3W z5~o92gV)jKrZDO4779J$FrX;kHPp##Rydvd##56%Oo+)Lxfhdj+ z2G%LlKPMoPeDUbV_U^@(J74zHA?toCeo-r3M@uMki3Q@x_!@>Vx0pq6ycdk{8QVAn zgBe|Nq6PYc8Uh-(uZ-XX$v2}kb)U`Sa_>Kg3lbdyb|)527>{xg(`bN`K=>+5RI?B0 zK_khUDvSkL$!ZAbm!Na;7aL|KwL!S)9cc>5NT#fmwqIL(@Qx9BGRPrS95JAhz-DA9 zT~jVeP|i>>j)s_4sy+&%pI8i6u{tIZp&B;Z&Vbp~#tBxIJ@JBezy+Fa2PittAd`${ z8AuvyD{yK}(Q+dpLu=Ry@ei6O;~G%4tdQ!)*D%l#67C@r?9rjC zjuuLXjk1ngbO(m*VH52kQj>{pkEL!;h&2Elx28|ra84UwC1`kcO`GVGqBWq@ug2xZ z5ub#eA7L zHGu@$0)-GR9-8B1MdCRKM>I?+O3I{23U)zzu0u~@>p;(eLS`+4dTg=26Wt^UUODIo zI{+9<_F_!y(J$>@ZNUwdlD0L~Mfg^6?1(hJW+@C~{8V+Mvt;OAX*ibO6*TyngLuZO zFYONy1W!Y|vjWKtQR9p4h?Q_@D+TE}Au-ZB|BxoqX$7e}5~-e0wdo?g5FX?eM{K^5 zF$XNWq)X|D=ZF4i6-*JkTny_!sSM@$z%V1DW?<=VEN>6Sadg zv(f=}=vfgnwZf)EkiGh2vO#|xIrHM`lW^u4H+9n8y1GBW_LPlrEU%!Aog$Bltq|r0}H)^>2!flQb>@5nC11WQjEw^ zZmgb?#X6JKOFn>(jcU|&>$B)6L7^oU10Euk8ww;eC>E5=h%9ge5;P#Dj6434%b2A#nG^P&T;mde`mav|=cA@BE( zBGcc%&PS&MoGbugn=qgNTVxb#^%K)vpV|NDS!Ui_UT-8WGTp*o5?E|7GGWvl-&&cH z!#S7GE-X4?EDjoKth$=wh*{ZE&T_)U#RaK@puZ+EdL6@7L0_--7k}?2_B;OE4rm6c z3?P9pU$TPRt8@pv&z-uPor)s2#LuC8g%TG+uHml;E`ZQq-4a$VpU;j-WOvASG#beh zbCCaQX8*VREA3Iu$9cw|V?7tmh$DrmddL#?gl5b!F3-7_lv`V=*;ZgGI-ih3DIJLw zrXw0+Y4JxdHP=-m{|`OyQ&uyg_}F|;8H^~rA6wVbx=cSUzw&p%ZdN|7$z3&zdtbbu z7Q=l*F*wEDl1#*%hgx|#+IuxekOXe-!geuIC}KXJa#K$L^Ay z^?e7PY}vbX#{`2$VlxQkZMZ1JBO_zcHbSyq)G0Ta^sC1JM?KY2tjep2Kb`hykhPR* zhL6W$*Z;w+x5g+YQ5pGV)|cv*ET|s>vBR};2flcQbX#JPH&cFcqgT5pFFv}ZkRYIq z4{L+5z!w2J1j3847L1h|ieNBWWXd{35vq(3)Wd zk688|tGA6jZ=Z-an!G{Q)$0(l8^(uW>R=L?OAPrslKh(Ke(MngfxJ*H@Gqe$!9o#& zJA#~rK+hT$0aT%B;t;XbNHs&~gV7uvewuo^`?6zGM^6_{Y5H@M^><2=?qXP+@gs%m z!nt!%cnX0g(#ZAS&kXF@%xfOAK#|cqxXS|44>U^~v{7o*awX!JsyeqStq4DJ($-VW z12T`bRBM)XW~_wtKRG!=`bSw`GZahGin!xd{{Pg{cUaeN7&IR&w1eaXVMS&NxcayHT z_^NdKTKS2&i@-}{;=F&->+dR~&in5sLv_X+Rj+@x&XHZpKR=PFpChv%Ls~vPcT~Dd1$|x#agO_6uw|;IyJW+Vt0fbA1}ocIeIY z?a`k0{TOr0r7N)Hh-1H$#$0|~(@+ynZRbI@yHf|bg1ifZe0YeZ(P@G_-_Lzdh?rd`T7Y9Dab>bAiga)ATZzWjZcrkz6R;XHgEyK+UAUYsYUvVs0sjo6W9^E9 zb(|p69d<(92}WDNtf1or+i;z#EKbSe(QF|>eQBC(3Kc9!*|sZWu#-_0N%x^BqUZqB zzTo)=ccQ}3x1MY|^<1hzibDH8qnbWQ*0`jeq?%|bfCzW%7Wz_1RGiGF+%d!tOGfD) z-QL!*91C5?)}+!2A(j6%OtmkQs5c*C%8vY-tLws^L%_N{%R0D|4!tnw4PiiwtTA2g zbT0sXiVok1`owP9AVKqYGLhWy!eQS_U^v5r>IOR;+kJ7+QP7}AHEDN9Zdw`Hi^rtU;g)8VFN+Zu}F%YLjSr&)Y z=OIu$-pjaCXiZ!SvxxCepE4_k%`0N;CZT-Y=%sl|EjCgq2bXW26M859FRTb&&S+ zOs&(rz~&yOPH#a1@#zqt3vIv-+F%NBpMTgJI17uBs5fu^j7F-H2#3f*$2N)`mm0= z=CU|r|89Ex%qnaDF65!(Oa{3khJ!M;b*TMzOl^!t1sX3dLscp)o(TdHk%JE~)>U4t z>u$TZdZgi+x<@<(N!!;#Wuz|5*c$a<{(!<-MTE~O67}FXcU9z11yr0mO%hI>Hv>_9 zgc7d_l!(lYr9SHce|PY*kf&@-88PK5)h#Ia^c)iq7sx8`+(1w!GfZ-?qk5(i9%dx8kEBWIj1y~$xug`e zPTd&C9l&DlE6DPFg6~=a0t; zbQ)N+3M2$|j1!`E5gzmOmS*JbiNSi$pHem3W70-P!ysJFwPdd ze0tvn8E2K$r3)0-THJhkUj;LVYQPpi%`u==^)B;q9LnFFUtiJPt`xn)lBQy2rnpT_ zkEJ}1gY-L3)CRCOdol9;nze$Wa{eXFP8Jk`R|~T9k_C!k#b*a|_=drU{;3zh3>o1E zlBr5fC9xiy>>q^45wBuvCTi>DemcV6`!z3KO!*9|iF%Zcna?ZwcMITA%I4Npq#VRM zM+d7-!OklhJ73DGC}?-aB%!{zPux9RXfMofP_8@V zMKbB)-nG^TnJm_~AJ4~7;X?$xDc&xa68$HqMN%%bmKS>9(d^}wwXi!c?e>>6uI$+A zEUn~6xp!!rg1bd0F3f(FVYvEjzKXs~ul{{PaD^ht2%oVh3*e_(8HN>(5sHI3=%6o z7E4%vT$58--93sz_7R8Sxbg4{1GWxtS+(537Qh(VgOIjI(*E_?8=9IMUYrg|#R=`$ z8;Y76Zk!H5q1LXo-PShrG&lG-8+MKcZhfJpxuU7rP|`$;^hw&;hnD7sre;G!e~F~k zdw8iCzTDxHc7Qa>9(ac zJCs`!jZ%ASxyhw6Bs)KQiYb`}lzvAeR`_7h6F2al%cja>+hTtmCkZ$c#b%3PxYuBK zvalzPeoQ-%lKhN(+;{vAO>72hPQW!6kT-KEHCxPlrf_G6YC^=$3fiIF8gNC0=lg!% zuII~}CF(`NH5>fbE*glMzvT-w9wj7+qbU^;Af!@7nxCIr@#M`C^q%3G_sN<#m6|MS zd@2_q%o z!G{ujqs)7J-}mu%a%zF*X>IC7Jes49TA>%`j?-4gg^;9Xt_)Hpoun#9YPtOr^5GU} zsi|`;+=M-_rB*V1ovL#j?B)>ZRuBuAgSp^##Q=3Fu0J;mL6@V21LRTmj=fE&9aGrC zN@PzBG1ytQ)B$do`L(4RLiV+-r5O5~HV5sjT;6Vb%!38oXKn}!HYYU;NJO6~458TI zG=WP8MJ9RnEEA6-@M7lNCd&^=^#SshnPbRoHulFotXIuMcF;L}5P6JEs? z!(}zYVGYA&EyGzI!(lzczY*%muh8Ke)bvr{F9*XLkI_b8?jQ_RC8 zH+^Hs7;mHe)FHRuAe&)GxB!a<5K|eIIv!_f)voy-1XW2o*9>nX!=-nFmPd}01$P|N zTcA_i>Yu494RZ5+)RIcsHg^2VFgiV+=S|=8>d;4z59FOVePHLHg45siCMrA`&Awf9 z%Kkmz&<`Tdf{ggLYnm2pJGJ=r*60&EXBjNfMUTeE`iP&uWp5rJtNS2LJj~=ifL74Mb-MVjLtPQ?F*ffhd->{;FCS&{?23l#c60s#2#g*=K&e7+ z1C=nI;xFM!UZxz7v^G=6*i%J;O%p{ja4DkXra4rF@NoBN4 zD+HD|o~o_9zD+BPnu%krToY;~7Bs`kU2`Tdu9`|^;FKSobruEGSm#N=x{CGH)2oVV z={3x#dkY+0qRX6c#fdh)MN;kh)g>%sPi8y6V-;#fHyo}CN$8=JgV;3Rf(XBv+6G}Q zgv>ItK#;^jUSZw$8rEO?$E#tx-X6m(r4^W|b{^die3B`-Ai25d-#Ex@d^k!A&*e|O zj>GX#BsD-Yw%m!6INe?6IrH)vHBRT@2rOxA**yCvCp+Fz7hVOZoDK_Yc-K;#j_;wBLuK`!#=672YTZjH9aw0u@u<{f<5-Qg^OV9D z8iS547lH0BwC6TZ4`}xadIwfI1YnnAB_oDI-J|o}9)LT!Mz!R{x__X({(yGOx>GhI zgJaV&unUdKnVne}&vamgPE^e5_`$mqm%0VgnXwuh99=I=1KF$CreW^(-}SW-)v1pE z_y_B<>EV=?yMEY) z2VHh;BK8{g{l32;UsBJ0e~-4x-fVn*%41_azvu(1-S_^Vi5^(Rsf_5~jMjkw03h~% zWwifKq~$-7;V;nAn=WmC#D)-X_l6quuPIs!3BO39s}0>E8UWOb;$0vCukTLkPBycv;(2MD@R7H)nTr0|?c(Qa-I&0q z-cBjE+_ZFcX+8DF^>0JPUK@PgSzu$`nP=PcufGs~e9>1m_b_TReQC|4UOxUWr@wBZ2hmC!Vs{ll!g>0Arc}|fS3ZlYjPj>S=s&( zt8x%RRDtcnC%x8_SH9UmKYNwhsVwK4#cl&KPCB0zZCAQ*BK!P3A{opEIbBP#J#{g6 zE|LCwf2p`>-lkrP+{DsChejmoP%V}!C65aJ$lTZ1>~pp}Z)rUajtjTEbwXM2!h$Cj{+UX^;; z<-TojG<|A>GBZS6>v|$>>8nQNSuFC|F6X_;3UsBoBzrSIuP2VJ6HRaYsh*aoH^$ma ztT3(5wb2_s%9*;GHBq`uB5b2HtHJ1=ksM0 z)B3i1edw*esIt;bJ$X4H?Ynmsz)i~?mE*q;U{d$(Kg*|Ww$<525Op?&&ClgAUjDV% zN30zlw&MIv=#l%05_=hrxqO^`}U$g;*G z{BVf9Bb{gjj0*#Ql(Xh?pHxE}gX;GYcy`bUiszl_3Vr7sxn?=#_(6u!DaS8B6^X>F zC+aF770;50xo)z`ldzT$biY3hv)j0tWkKAwp4(E{$yQrysigOM(=3bmh9Oio&BOX) zuA8m;)xkb`9s5VmM0@F%k9rnN3sd)6{f;sEE-pP91lgQWs0t`|`SYv7OAR_1$+sJJkn1{uF8LNfVNlYizqbBLWxvK zZKYp--N8r%q)H$JEK)g_o2;4%I03+%;lOsxv3BTx+?-kG`c9kQEVwv52O)eDPm&Lw zjaWK)JRj=)zr9TKO`b*#oES0W$=@cUC!a_6^uA6`{Bdzj`%jX6*feC!$2a8Bm*YL% z*zo)_>7(kI${Fw&`jZyZ!Cw$|is0jD3$hBY9@d%=|J; z&bp{DzY)yx8ueT_+V8i|3-8;FBcoIg@myo!d(v%YnsDngUKh2i(wX{$S0tiq%nA`h;v zQjgOWjAt5KxA9keB4^B>cN*jR)^{v8o`732eUrgGTTf|YkGiC`yk+Qs^zHU7gY%7x zeJlPH0%^26^Nss}sA*QRbTau@i2(wFk6alyUD~AQMk@&ithJ7go#8+c@?DJ^6ZR9% z?%OtvywWaHGTvVz-mD^t+hT56hK`;YjeMUR%QMZUS!+9+NRL>9HS!#09zlH;g2E_m z9v9gcCt}kEz*;{qpBLVP8L?D&Vj`nS%SbxfrDw^ktiw8=Lso10^5d2&R7T>G~yW4|w2c{!!j8y!Rj)?$dS{NB-*R^xNjzt+2T` z)`6~LA*U=VXP?&*Pd(Obxh)e`JhT1S1d3kfy2Y$7b|d%a969%v0}O}ZZiIrB9-)ry zzCASy=dH9m4_T+R1mbqcQrg)3PYwN*-tng4cj$#UKB^*>$nK;NrKiuG0G|$FK3AYj{z;E5gV6Pc}In0xO3m~R{?34*D z&*UPf|5X$d>#xa@M6G7WLr#DD!*2=$BfhFn^k#Jc3@eou6?cQab@4%5Yu?eS>s2Gk ztAZa+Fv|kUezQ4BgbW=4)i!*p+ivGMn^-a@hzFr(xQD1F7xp~MG{IF)9rQMa8db;C`KD~zU zdh%MVVa4=3qh%R5RlQa`ymNXL5I~#%Vcu}Q!JGbPvsAb<=elzm{;v1Hq0jf`W-CWK z=ugK0V|2l*8qi9{aO*2|#twJe3L(^QZF;WTs^NNz;D@<{aIZZy@X;m4VQqDihhgPx z=7JF*c_aYaY3iKp-p1v8jfoMiL0&7jSox)K7_aq=!4-+6Yuy7H5s%eumFt!fNXol* zp4}$R$4_^9aP!CXx)s+YE`NrCGy(a&h1yWy1MU5<#L=C+r|UsAxGR>3PrQB(Z5kbI zhoDjXZ&MvMyy6N(bAZJq!>1_Eg}k<{9ukgAHknJAInQ6{jGp(|$UP3#pn#fw5&f~M z8tISI3lwRdaIZbTR-fWCde&Vf;r92DAq|n)yKs~pR`;?UJ5)>i)%5F(V$*w~(rPP5 zh6@qJ3Wcp)bzWW4V=c1fs)mGgNfN`pCAMdILOJ+C9Wsr(Kt1eV1KBt$0yK>xe{UUK z-48)X8$PM7>Bj~$ahQkum5dB64L|k1C6`X`Y&9De3+MBqON8~ zXuHm37g&;tHg*KCc&YW>cRy>8^NwIkxIkBWkC6$&!HQkglL)F-bE_`GeGniTjd?Qt zZ0%av_txImAW|i>vIq{&R-*;LH1#nK#(<^+DaJ(s zBN38-U%udEK0ki&2q9=>)<2TnIXP*MUg;erXA;nOBwdWe7#zAOPL+pIG;~O^Psmxw z$1#ln07obVf4i>smgECuyaWtHDEaXZjmQQcoxUjH<}6W0U^I4?pDnszOzY-8De{Rz2&wuY_V6N5%>{UXVXa;C;kG;eT**0m!eM5`4~Jr@0;w>{ zUIsPi#^cFhVPkn1+P)rnk^dFr6>~)jg4?9hA9XFE0;!p7AgU_Ad;t{*fFM{LkbdaS2PaW&7=(JqTXCGuhyqBL!tz z1(`v+GH`U9^_Gw>CLnlaBltAMjf>-r0GBB_E9e1=%sRS1shYnC1az!4UPnz_DleiTjYhH?rpj?6Kv*(8rnoJvO@ z#GC@A-${=GPrq@{y@19`^(bP$h@#n>X3!jn?-CbO9rd6g9X6ya52+!NfbT?mzzUAs z=hH;Q@ic@SF6jhc>a7Ql@@z^10){ce*-A)O`9r^7o4lUr_}!3)_lhMxgI#7p+2DlEEp*~}+ z8|EmQ!5_sgKoTKFP}@k`KqASJvEgLV{ke(-6A%rj`p9G8zb5#zN68yQFJhA&mGRkRCVuIaDcH~GywzL<}K zuG}I&WV`=WlNOh0S-vX z6RAb0l|Rx4F;+hyUPzX6!-_9`3rBU}!1~u_VWgh^-R@VAp5y>5q#>;I zO!NyS35!8`5Pqle`uEYJkj_K*wYUFDoP-p)rUr|u$fWF4VFqJAgGPnKP25xhPAXw3 z8_oPYDn8r>_0H|9d`3{1a8(7#4$SNSO(Op$vHwZx7Wu{Q)Vw`4rq)1*7*UU}U)gK4 z5(3PqF(xF;r?)rcC>^gx*qy~=upsItXHuS^qWnkvJTcOP0#(dV;@r%Lgi;U>`51 zMruoP9Q3)0_6-V-Px*w!(s% zbXCL1j5L+xXiV%9O5aXOpq5$<0w}FP=%-d%^dCcjrsdJm`d?s9t@P;KLiozcXo=Nc z34O8&56b^h9W+~Ol7^l8oY(a4yo$UOcpyzhqe#_L2~B}M}XU9>Opwuq(%x{(@4o$RZfXkaSU4%lns)y z^ji>}#*k@|m8-uD=jW$NCOm7Cp1-8J#_tlMAtzTY!DP72v{E2B=O>}d!Yl5> zXdKFaR#dItCCy0XSwzcc5!K2S`$Xwr94+ne@G`##;ANtd8)>Qy(^%L&mn$>H=4#A^ z@LSfsISp zEf84GoMve9=@7B!tq!28^LielX@jo6<7qWDDx*y^U_l*&bYm*{GYL>rk8uWDW;Ah* z;aDS-wya_sdqj^&sJE&mQI5^S@&YqpUptmAxsB^%h?pz0ERMVZkw7o9>U=C$*L^-j zv=W=aXa-@t5V`QhpH0+7ctv%z5*62LW2Hm-YMkaJD&#T zZoi;@cK}BboX{*z=aGus>#zB0OPWx97CH3$xQdu27A1BQ2q;w^E&Zs#fT%+ip|DOD zj8`uyo}1_(X<7VSnP;$(ws{gF5Qnjoi-1zhF%zYs+eEqCjvb8!kYLI>k8YWsy5qas;GGXvxvW!-+dU4qRJj>MGG@$f3b3rR{~Zc)WUX zz>NuhC|-3stpiR^m^NkFCyvryFa{>~q=XGbri|F??FD;m2M}{v5njz;q4K3faA3&k z_%y!yH1gngmEx>de2z1{mGU~83XC*yPo7kmNu|B-uz_2VuvJyD>I1YsZXX{vxCFK! z!$IM~!MBsu4h7o4(%y6>eD^6R6h^lcIGjvy6nojCDYkO!(7f+8d+=bp+k0F7{R zVs4A&B8w&*W6g|Vp(R~rZ=0EH$GqH)cM)Q)EK*sv75E|mtBEG1NrzY9Z2lx<`dwEl zLJfe%e8|nH(;KlM7AY03mQ940s(Z>@Ql*UE(PH_}QLWEW2g*!Rt-A91zi&*U+(B_#GQ@(9@j5?l5hdx(vM*I0pi@B?C-7IC`Ud1LXuDTvfDmS_kdF!BksCeM22c3e=iuNqm8Q@k&+>`5-Ojlu z?4cQ98|-tmqKngz;6FvslQ%#7$VIqia3%$3P8B^da?H#^D5XzxxDY{bbm-e;Of6P?ewH1>6 zy7Q_G8m`z-9$l}Y)$Lwdadk+rszVseOv#GPq zfnKp`4j!~>>Tu{AuIBJE*Lp*{eh)S27@5G6s_Fj4agbaa*}4x?8q`^ZXhErdI=}b5Flg2NpA!DJ*-C6-j(mRXjiOt zRcZ>6xVmBUN0mIA>6Q3Gy~)`3fr+K{+=MTJ?3ClT_oG6m&P6%6HSK*3@ zT&2rP-_rqcF=~Ufn|dFT{XUw;ma|)%1MXl@Wv~3jb=&ol~ zJL|4E(MwckRuF=lcEIeKonT8Lda$1>D}!bNTW2(w56`9eN@0lGVw4Gsp68qHz9{38 zsPCL>hJ3dS<&LmLUrqQt2GI}kl(H|Fqr^m1R?aK==yf${a29%`+(^{HHhu;ErrOSh zejlxrE5)oWLVt|T5*+Mn7>E-iPP8oEW52;%fM>RuUABbmZ>J5ln4G|5PqSv5o2l#y z1Yl;n0&ajbG0YS9+;c=!Re%n?fVm0G>NG}n*J6s-PB<7AHcWSpgKUsg0?y@N>uG|LBPe;RCK`4f)MmM|Iqe*C0r&Ji1 znDy?w^6}}B{NLR+k`(;a`d>gnwNpYtDgTdI|NnBS^Y^mB^UkuAzZN|EPi}C7FRMW1 zw8~VpZ>!rn0rN(FMmi?sD>@XzbvHIbuiN2Zq2O_S$q!{BykKre@emd zxVUsoK_~m5A0q#m*@oJ>(=>K8P&cBSHERtch5NNCleuH6tZ;*gSzCr~L>Q>A&ydX* zWQsK!TkTpp!{C%Y>y^2v)$U*x+N09`T^SF(q1u@HSavJ(yP*1}->U_ul)>Cj%J`Mq zV>dSXo|WIMJ&zLJ@uEr~G=Dxo?=sf3`4UDBT9Kxdp3Y+tAMk4cpY!P}RfpjaJ~8aR z7$^T=X+MjW6g)aFK8w?uCY3n6@}NC3W2E zpSNiUbLrzA=#cyUsJ7$CeO%d{>N|TKAE^@`O-l7e zsjt3=1fIJ=e{{IC0Vdh6+=mM|V6~M8xV4*LpF>(IYOu|NR)aRitg@TP)9T~PuGV^2 z>%uB01Dl{>yVgZWL2UdUWSdYzO6)Fys3ltNB1Kfpb3A|M=RB}#KLt_(uDozuYIw!q zxxn$5iPWrd(i3(EbgkDdNtH{c&;g?zLpDSwv~gam@0G&Pqy)yRB^rNS93%1!D%e&_ ztc{!zyY)%=#bQ~RQ8R|}f_bQAui?lw!w|;kp+`~B0QqGhbCg*DOlBxB+O*_jAc#WI z3MW`ROBe%A=t9Q5QB7oV?xaSqylC?F%swU$iH+_O6=r#Lic+DI-ld?|tbgy2w44)i z*Fs=72mJQMxPrN{(fCBlMFNXp!bsr!h9k$cu_^ldvYOqc3Klv(o`lFyAqmrtN~c3r z22bfy5|UC5f$E_O`#ZV>S9UGJ=${>`i^=Ial!Sp1Y#oJVZQCgCXzS&G=aTZu)H^{! zmO)-td8-L@yw$6`RZ43_UW!vEEKTeJjRovK?ku-f+fscPy4dT1Q9fVRo7`3ADx037 zaNY62<@c+WVgqShnP;l@KL|h{&6UxHJrzmEKJ1xVdW1M*$<0=m5BKuZ$%K}EZ?MMk z`57`_cMFJ0qFRyE@SroxGSbpT5BkdJiz{6ARYMTVIi_=uDQ@y$@dVV>Uo-wKb;z}y<_lMYBs=c~VkA>-~^|wgPMdYgYN@&K2Gpzu-w!LOg5L~h^fjU|EhPfABl1mox zzD7SHUyK|9Twq+q)w)!x!)Sw$x>7|4V)Cfpv9#EI^p#zGUIw!!hS@`$eF66KD-XX<)(U zsV#`ZoYnDQRM3ZFv%`*`YC*|%a4p8lyoyuT*@EzQmP>>D_`3Ux(;@lrVH;M8d1nPi zC5I|WR~R8NlzKYMv!KMTPu~CXaA>jsc%}afr}5%?pHxm%jVHQ7 zQ@X{v6dbt~*0$EBaM_Xz5eAOVz(;k;rW#ar#MtH)3i6(lpY#o0W*0>}uHUV%98-hX`r^l&js6T=^KMb35>HzifP^8OwtGt#HZ zJUXVvw>w;`HaAM}Ed$Z5oOM<$qFu86gK}w4LkKioMZ=w^Mi@=zThkl?N7Gc7HcezT zEz~I$Ux*`76VH_KE4B}rgeV#u?sOcPh$&;paOa=*yH+Mu^egBg!;>VRSUts=Cc)Ft$!^%;zjx?{M_IFC8GMlG`2FUm5;@YE~=F$R%7UMQ^0w}evMt1VeS1hSo z2Pf24<0-8cdRm=$9f)CC`h#b#{4My^c}5!tSB{dhaKc* zmVA`UpsYIj!8rSt!)cOJy12d2NNHCt&&5Zsl|VXX^P8r+Z{q^eh|iu%@4f#b7<#~t zO1DF{RJY{xW7`4#+T{MGd2cQrM&8xwyR4ShjMS^pZT1hf28j0R^&g0W8-FowM2>}C zDT5eVgnwePRW5^iWgQXsUk^1j+is(NrVx`kdMuYumC6tqb|mYjIkZlQA1WRQ?wJV; zT&E&SUT_F*=;!c!f)}GwUQr-ss#KNEZ|qgkCOcG8N5$+_N+mxmZ^_;S4N3xkCMEl2 ziNCB!>xmp2?)qSTujz4!^CCuXnvc9zOTe1mX(WqcS`E#JW6mvFtB=EyXh8iy?9DEm zhekxvq?Hzk+x4&yhgNJtIH!w29JUa5ab$1@S^)jKbl?m5my%}exRVm5Z~e<8UxXpj z0JJ`%UdqDscp$cWzXcQ)V5J#_t~x(CF0!oxkKmKbS);+wHuWTiF2cUhi*xPXt?-^l_sD?rNq?M5Ay`L9#4`zbG0hYWtQj&S*30dnf6F=4*!Bv7?=) zg?+z#<7=sn{L+kOdGjXMSOVN^Nv1H_ii?*+f^CJ5VY!Q;MYD*DJzO(XO;OYA@li`{ zp;_yup&@^1-$+x1Q|#RmyPa4=WFy&DW15Z1jI~+19mjy}yV_>OMgK`-4!Z&W85aE! zMl7+H6(h}BB-$M=-Q{{W4gB8rX8Z$|JnfP18&4Cr?hKPV4#LvJ~Jxy6PFWeC>3^KjvcdE^uobZleGFiovKVF-LkRk-2++Lo@{ za|rxMGDsuWn)6bfAtVCZ_AXVt{cc~-lG@oPy%XY8T(#!WN`a2Dd8p_7hD@uG|IYg>^^N-m-K%@5^V9)Kf|RS<-cxwf)t7slROgTtW*f&CkIG@(qa=@T>)E}M zw1bi~_a}(Y7BmbL6yyL9C`2fX=2Ru{f_)r3WCRw}g@S@OZ~gUw{>LvAlmpPog+&?Y z>|kW-SPBx@KcTL5chuhyVooFOdKACJ1B$0?ar#IXKJ! zoIHGN#sGF600+PfV9LpEW@H95GT``6%nsR$F2pRMplTOjq3HjO*&!Mf6y$}Gz^Aresj%e{hRCmd0WSycz?Q4|HiZL z{EhctD)A4e>OXn@>C5<=r+(kxJbzD7{J&SRKj%t*qpGg_iTYpa^INR{P{sfJ)BT%& b_}0Jt1y_`N0sowY0{K!yS~d2;bLf8oRwP)9 literal 0 HcmV?d00001 diff --git a/doc/wiring.png b/doc/wiring.png index bf2d084088cfb471151496235342516c2e49c123..d79dfb9d2f483f4c87ad2f2a1ea529ceefa02867 100644 GIT binary patch literal 81105 zcmeFZWmr{R*EYOK0i~onL_oSh8UztULP3!3?rxB7P*RYVP(Vt$yHUD9y4i$u?00Uw zuIGNgAK#DX$M@qqj+=w6d+)W@TyxDi=9uR=$2ka7Qjo#KAjN<{Aegc*q+UZHNXHNe zq9z(D_@N#_nQwfaZ?^EJS#bpOJgT$17lNj8xdM) zeKRe!xseF18t*HPS9Z^g&CFkTI2gb2P*5@Surz#XL@OqWA?zjy8n8BY(x-N_wz6>) zbQ7VyYgZ7whkwmZOMQ2Wlcfl)`YR>sXSNQ;)I4lFY#gl8Zssmrw4xZ)!VX3zg0H2X z|I-BcBtmQE@}uBX|>k z+mJGLG;}bxb27KJp@z4qZ(!@}BtlCIj#K~BID8WRr*#|0e+B^z2Rr--J0}|l`~Q8q zlex+N%jxhV|2Z9gv7nNok^dBH`r?C`N<|7S44+_3-8#rhR8}usJPAS&bzy*jJKj5oGnb7a%sLT{}3C9;}R&2 zoG?!D)qnY|tMz!@51;R5nw4+w&uAoRxUvRW%>U9Dvmnr#dMNIKhJc7pM`!ejF7A`O zN&O<#mOR&h5zQx-xQ1p?_r5t#lc6-BZqxO~jX6hT*^hKMxVX-(BfKxmR*xTRGo`-% zd?4)?GrC1J3KDg&Fn;W{=fG?cLjS=O&yCro;UNWXXPvA(DFQ$+#pQ{XNL}C zcZkwc)#NHqQj9JFj$e4VxXS9X{(&ZMxF0-0jieG2wJKpg8hePVEbw@YUeBnvCsfGB zxq`IT)8PW)Avjq|FPVWhyELq_gLcDCp@Xt{BCB((@y&$tSL;0 z7#Qk9RYy;ehp(`7Pfkio3W*f@30zCbJrhantLK$3v5;bI?V8c_f@;DrMd!`vep3`D zCxxh*?Wp1FVLGdy3q#mRp`oF~>tUg0cHxClLb=qoiWT>EM-$C%uG(?8XHhT6Eq|i~ zN;WPuH|HqXn(-V2Jf@}1Mu=$N4GS%pyzs9R8KlHtRa{t5hTeQdz928;W$ejlXv|sg zqy^`_j}$qUk{UCWpfuYG!tNy6^ltn!zm;W6w)#F%=UmC_{letS#pzOTF6lK72Ig(~ z0eg+vfb_2?&JaHj8Ie2@F-+qVV+ce7=is{Y=Ss?xN%1Fds(eD8?x~kKqwH&XQ_;yD zCCwidvJw;UGZ}fmoMh>QHs5-yN6&xZT&hSFYhSFsr6{rS;x!^s#yK1dQe4^|{r>z1 zA1UOwosivdDv|!1-n`5si(1l)=7H{vW*GNv)rBpMZpCI6Egi{6yPwVbtveSbNDv?A ziIRh>`#v$*-)QA;kBxk%*VNd$bMe?PF9bs@u~&YWDhAB%phD&ZZ{A*;w`TbFC1}VU z{x-#4Ui5@f961k=a`P=T;$j#1Kr;3(e#{^9@>{a$)U2YgXf&Zot4cor<0M6a;+`Xc z@TQj3jDcT9pZGMhjcU{L;cAYP1HQ}-4u!#LGo&RY8O?jx-Zzthme%9;%U8+~Ad~VcW+Iu+Lq zG^_Z+aNSU4j&Y9TBp9px+J490sLRA){G>4UOp?+{fegM5hI_q4j@ro#mL_V3cb$+E zIJYrh$nq>5)CK!3o#xL7L3cisP;frz%QJBk9%`XxhnNK5q6VO*#_*#)s5XTP;|~8y zAV>{L&07cr9SdPr)0wuZT~0Bm5oUYwF)+jXfE&Dnur^-qK((+#gbsxEzuKlB!LG{D zla}h2p?3XO)qy8d&WLsma@@BKLT zZ@m)<*h)s^4f?I~O0%8Dk$G18H7xq#^M?(v658BHpng@|*A>`4!31i>EzC(x{+5f4 ze6+KycYZ);_;kTlSJBhz2bN$l)-Zhz?57o%LC3*nED@yc+g>BIDgCs@jKWESH4?QE z_EV3;K;55a6$Jv8V;$wEGc+lQjun`9L%TyF{ryC*CS-1IlVxn8s#Aad+7yn9pK*t4 zx9nGcYH})v-TutUl->4t2p@Gm+Doioi4@YCNIslp_npn9W=BI@k4eW4nML@-*zTS) zWLa&c?2HRrE9x3k>KXMQdncs$J_I2&=@#M+U-b_h;eNlx`)Jt?nV8!?NS$-qd(_;{9c7 z=#JR*msQ!&@y_L3O!f()Vun%#*aj2ULM$Bl)tnK z4rp+T)*j+{Ijk_iCrU^p+O@hd;!)zG*f6e!<`r+@Khw_yj&r#KY)s8O2W$x&!l?Rv zcK;ws%^t4{#r*B2F7lX&I-adLu6!G=z8zHZu?Z(2x#cY_1`g0nl*wl{^bMad83h8rO z_QI#*hPO{qSzPWX7HbFB<<+^wxNjV$323C$yJ%%sNRwJv8((UW-0J z&Ef{nxLl{`C6?^R3z+1qu4htG&t}g$7lEf1yvWc0)6kgUX%7~to>QAyL({o4n*Xnj z?)}560Z7Tm!Wq^#N*a*-?&-BNQzbIHiO`DD_1~nM@q%8So0^c8pr=Hq=WdET>EeBO zPhPmwq7XHE+b5+T1Fv?tG=y!x^GB1D)7+qktNm3wr_Gdrm*a0Tsjt~}UIR^(?u+h5 zaI2KmE2%4~p5Mc@n+aSZ{)7vKktOgxVB`1W%Q;eTHm|GV9=`IJecZ$0p+Y+n(tQpq z=UQGe!@6!;SMsFI&azhQr1bLsZ8_(>t{oZ?E|j}*>v$!0*j7hmxAukEev-uX?zFccy9%H<+fVh)N>8bI%__jX(2lI6%aN3m`y<|t<^Y(F42_-VWq8X8@N~V}9{x;w@ux{@ znsp{mK2tJK3PH;hbRBOuFYMg4+&uP1)vH`c;I;w9@($_kjr(BsVmIk$e#dKnxYvF6 zsiM@aZ8^4EYf@!HY{A$vDGRJ#_<|}#o0o{?7sigsI1}JYu-g7}>i0f;2Rxi|$uQBD z{$Q1)X0|j6Y1uam&HBduCTibrjYrl<7k^Sx62)~5QEn?5=XG3lV@*rIZiCPg{D|y8 zdWop!s;FMsxbTSG`z^Gf`$}lx47*){Gw*lTSANO-F@3NK5q@TT5JhsVSkJD>n`*w9 z+jYD_(4~2!=X7m0$9KaAX;G8bMTy`0UN2Tl=H7mMFta>adVH?1%eaWEvaWu^*-&?= z3ZHqmSPjiL-c#DFi>?0Dzkvr@2HVd802h}TVnU956AKGmxC@#nw$4}T zm2$P1HYiz;Q&s)P;I<-P`Mz!j8k-2(!m8k8iR()jHW4s zEzdGU0*BFa4tFYiDV*kX_hAnqV1vQUk7)m&>>mFJ8ePw~LXafGU1I{WDKEK#HsI^Y|B=q!G^ON(om4cVsRei)Ro#}kFUDdSp1%!xLe zhF(OFLb)$-dTWQ`1b8~00{L=I5|gNDn7V%;03kfJDTsZY(w#IkOf*I!03q6WeAXAT z37CPTj!tYD*~kXdNm=F~{P;6KRU}GWDh9l)hNrcXtup|(C()OFll%pkW)8fqYxXGd zs=ca8J@~h{Sddr?xgcfh2k!DWSI&tC|7$e7fz?2&bFJ|BaUmBYmByx zwywUF#e%D+O09K`f~oU^V3WOPB?Mf*jQx+7-HfuYdoCe?S}ETf9kmQg%+F4)1bfa< z;a0iLSw9oano~tx%|YZpb~qy3*ywWoHTX1E%t(HGDyb=kiUmYd$bDq#A=e|<5-?E` z5auwlSZR<|#mq#@2E|c}hfbX-7nXGH56oYShlQg0O@~hlh^>2VDv}yMj_XxoF6nHq zr#iJIW1tFQj#O*W<7inq6sM1qV6zPqB8@}kw{Aj@`$(@3EWP6eXU)8w(DKSkz&20n zV2$AwMX7+Mrw!GFXyoeZYGyfVBZ4B87im4I%SHA4_c{v0*InZHNJLnp=FV;<%OYON z(}KgZs^1DZSoKNcnT*BXCCCN6I(_AqbB+nCI>z77*O?REbb5c5-M{7#k8XTEzp!{w zz1HVg_NxJhKJH0o5j7f4HuBu9;ZEb_R=Lw$K{IZdeVGK5eTF=WtZL5B#P%h7y`0jL z&)&@od5rY&1v8%IvbJu!(lY0%+xc(xAJS4Netg!7XT#|4Pc(crP715qiJ#PW7%^hR z5BDp#s3f)AnKUrol7m%6#1rYe%vqWJNbxJj2tss;qY%?P;FiB0l_D<^4J8RDQO9Y7 zpa#%n7BPmKz2;wnxm;)XYE}u8O%i{XkPjV#I7X7F+191NcXQNy4{Z?_6KF?ZHaafJ z=FSOgH!v-v00@GE1Xe8$v8Iv#0|#Zw@}hZ*ka3HkRz1JzIS1W(t#~-B80RnP5}I9* z2DW{-Rwm)rS=4i{W@#Y_E}ogkyuJunGUrsqM>;k7<+>Hk-wV`7&qqWlq0L^llk+ z5I<1E3VHqYf#_EZJ!1t+j_8Nkv0r-vHHxv^*%l}-s$l7kxU^k;-Bq4!=@iE-q;tj3 zppHgQIm|F6HYC}S9!;l5rjqktR2%nj;pBv&*P+pd(>||%zT!1|aclj=0OR2pI?d0Z z0)i}6v%~c@TZ5r~M{04u2j!W3bQlJ9WFIAytkH*%*af-a8<_gLKZLt|GeL*K-34i1e@$ry zvti&Kh~>USAMP)aqRnn)q@h8GV~}4DBlbs@X!-au2^sU)6b_p#x)1Q$IhW5vLrF|Z zoR~jU3&p%o4>gNbE*wwhx206cjpRYKD_6=*%=S zGU7&y28_AFWRTRUeebWUSY|yW))o}Zo=m<6iFy_=CsGS{6SV3U7EC5T(+BY+e8R%Q zh=p8sGd&cwQldjcJqE0J4F{Gu{l@HTw|IH?_GgLl@FonmWwn)~8RgkNM^ZWZtcJXl z!q*?O;Qiik)R%eE7nb)bHF#v?H4hID2OM@Gh|6maJVAj|DSEnDI?zR`XeFxXy6oMC z?vJ6254GKvf;c|O3*+LGgE@vjXrWpOB|bDHsW24I$h0{|kCWnmhchxUm*RyHV0seg z=Cq7VO^>CN>L23aahePyOC0Llxo~enK*XdUDtXb&PC726QC>C`MTDA_CZ|0m1EAac zZ3biL3oiPgx$5;6(ys<1?Bf>`{8mon=12g1$Y&6wQ4v1ga z1*JzSLVl1^{0J&`4*J&3>~?sw3cbj}8_qoITC@GVZq9!!g>&+-Wc6lI5r?LTd)qNt z1Q+*&NtfN-*h2URDK9dvgs({jLT|hL-szVN3vECQFy%5CY1!xlw}3nqjSUiP~94f<3`f9 z1DzNq*8*{~<==n!bSPmQlQ52=CH#BU6;M$f9Ka1beFV??^eyv8W|o#!mUuP3Ko+&h z-@Au6p1Rty-Ihyf@nJ{dzw4wvh1O{>gd79o{d-r&Cdy;@mCkSV2EV&_ws|kr)r+hl zIFGBpxAiDt1CKy^2P*~NFIZr1;);D@*K)1SZ(!enA2EQlpEP)z?^{TA6!$5h<~95M zb%3P2V<{O8C?dpyziiWm>Xfy|Vp3C^oxLsHH`jhhr_!Nv$4&lixq+PNURC6R|6F4& z&f~I|67}p()!+LzNECr1>yYUTt%Z32jf;gqpoz zVq%h1#+8v%hLLd^Aa_Mm^uvcfAW7Ac z$WH!yO|8|<=Y;pWSoDy-BP>|65SYV)xyXJjwc6t%FBS@~F7YDM#rJ6q1_mm%x6A=nC`RE}xXKqQ#h99v zw>1RGb(Tz0woo75riLYB5YMxaH*>7sx>hv?+}CM_IC5>2asnGwuD^4B`f;xl{3FXF zLq&WNG}oY6XKl00&cMrYZcWR=acT8HN`NKD3OhFkgCiE;Irv!qwa$eP>S>`xBSoRU zR8098qb`ciT5J4u?*9S&ZH5SeC!l|@$q2!XIN=*LH1vB+SHVZ+w8RRN>c##d_K}d2 z@n{&5QM_Iem0CZ!s7iDpj-kw$B7uPWQmq z&K+@*$X<{^zO0CK)}*u;Cl0;8%@A@P*WSoJ<~s3mQFpyj5E}#)*XpN_$l(6tuJbTd zck8gKd2x^xIAi>!Sn?Ge?J&s0yZ*gsw*|$o_qTxNA-5G6LWNHVd`BMhl@Ogb35F2i zFd!%3A-6CpcJaBst48vk%{t-TDvPBfGMIg`^Ws&ohwUdZcxnV!n)Ew7m}u;qd{i>| z6*qVEqP-rfi2r`djQjP&S9_;3t_dY-I#jvAa$hkDy4BPqwRn~dvJoY{Wa8F9Dg*@) z`GxnnSaw#&nmvehT31d2Pn@UkTFl-Hbe(y~8jG47N!qjKAhm^m>urM_OI#AtzlK=@ zv14j&vL0uki0+|}p#<%39B;?cDu==C|) zE}IY;KJG~hU?J{Rch8%ig!za5+58iGCAx59--1Eu>AJgiPYS7R<4RO!da=JwOO%ZX zYr`ZXLe4VRcQ!eYQ-FDe1@8I6koGI_x*%a#7a`M+NV;iEgcFZf)0LOB?yH&r|}q%J&R9)zgtlak@ylFl!7A zegabDkOJznnx5lN62eOEnubF5tik*829)3aKfWDS6NBHi2rU903AF9MZ zR>EVq8S#gqn9~*+nS?(26*`k=Lpdd7^(gFI#k6vf7;hy5nZ4U0=na>6mJ+R_l$O%; z(aK5ejF#|bN$!^D)FA>HI<7K9_Hs6d`_Wx^vgc2o`Od7Xs?Pe>n!YvA$yiW@N%hxG zPdt}Z!NPCfBEqqrMAK(E!^jD$ObI z-qf1X?;g5LyS3&tco57S^6aY$YK96*tZv+*M|2RQ5ffL|+rnq*5huJmO9>k5pK)mn zioC4qHw;W~6nJXLu4f(o6DF+0-rz8i6Wm&plt1V&GoTJP_*Glrq?$eTc6Tw6Hji2& z4d+OUM=ok=zI3*kRVPxWoa^ao?Lm*o)a&|A*c3G}ha@NMYHd%?z+E0OtJ4+R%eaW3 z_nSI?(pWHyn&#}-3e9PdhloSbTRMvB77|Ozt+Kk^1j5Zy%>7%mlGV<8 zLZU5+hMLr}THgU&DXvkrPxkEs!ExOI6BwJqg3hY6VtrB7^l)Z*HEKaY!A)l#6HLC< z>R|UAvkv3S{@B=99F++xKEt!Bc{@SP_Up$pFJnJG^x~Y+{ve+|k)T4(-{%qm}Doj>CtUKN>PjHAS^WtuFq$AD465XxI zB4g0IOC1$0lYgoh8GJCu9WZfJnj-jvcvSr(_(H&)oGqg6!QtT{h=2tJNl{Qx%ue~L zkzFL-OV>%ikdwn;vg9E>ob_&&er5=6r|T^05WImLbWu>i*x2BhxsU_{^>L1##v?2w z5eV#y+Y;7cmSmmn>xMTNhcHK$TlI{LZrtHVc0NNF@Z`nk1baf3SA-xT#>WBQZp|g$ z@BXU=fCIw=$G)^FE|>ctdc*jnHlaV=q$SiO^lbC-3&sfcXZ8;EB~%&IXvwhxtzsQM z1|w2qQ<>!9klwpQF)Tf-K&b84+lXVPNP9bQ%v|22D!Y*6oYf4`$cMv z{dQD=MNu1I7ww?U&`#HoCXn7W34o)QJMM|hC6_`|#X$x8{^%d8}0dHvqW1FR%<& z#omGAur@Pp$t{((uTIYgE?#lxFU>reG}ubXz4hGX#lU>4Rf(9SP&1~xf9##Ha_OrR z187jP$GFnuLc0P>f(Vusyc``ptEC=D)x0?*w@ISec#j+_Ne?o>+1_wnsDg-D#Cp5HM(cf6C-ULwW04;Pn^v zyu>l!Tu~<0DzipZ(VZ&<_3GPl#G{n(N{eMJBrEetUk$!Om_1M2*w08GNB9pqj@0@Q5DbN9E@F|I zDM7zuPfdhASE*ylvUrPPrRI@PaEO`_!Q@I{HTeD{EE}zw_&hxgTt2k^dl3mM#IWV& zjOImy7Fd}lX zTnT~ymc*C@muMA1eEoy|%)a>)j|<-1(L%;69UNq{RnjSK7g$x%6w7MfYIb?q!;8lk z2v>E>42Hh`M3ar7NfXV;kSXORq_-giu%?T}nv-_Tr@fQ_>{HVo;e%Xf%@@@#G?lcx z1~Iz(iH0-j=8a817T8U$5mjT)iZEM6K8u(uAGVaYWR>}kT}r)w&Gd#N+X@*O4fjSg-sbU<%f^XD zCKUlc20N<<`VMwUcN*1V*p2Gd;YP+aiK5N{9X!feW$AB4vDJcnqHcF25}VxG(H2~K zJur|Qrfi?{YibTn8BN$GN)idOn*i7!Qlq=T)`O*rjqqJ}xgaZlV_x38H<_V9n}@u} z*OzOrXZ?p!fxTyevtp39bS6B!8wr;89{4kwrcxo<`q1oplq_o*WjPiC_ zzrnk6Ha4t0R$^|yJg2DXfvxr-VLVr^Z@seN17U3QQRPXMkn&(IK9zqKcJ^EP!ms^H zg54IFGiq>L9r9=_B3ysuDQ{XAwcXQuP9P$Ib@+h9{%`wq#z496l0|HU2@`#p zlTR2~;W z4;JcLoJj?l3g$-yM-(#k6y>uMs@$|#I=x{vIBUofe#{~SN7^!1tiNK&_^^DN&VNeG z=I1FG=g|D?0DYVE_cSUWaw#Je4q`2R5vL+EeEvf@;sgW-?JJ78-x<>|*pfo?Ok`T9 zoh5KMSlCdR63A0M)zw_*FAmdcj#q>{uLVJFP@YV*xPPsA2B!MU`vQ#49r1vlX|3oE zcEcsu{poCZEN_*a-5r}4B2IdWIU8GXMqArZFSYGHh#r?&rUMght~F@ksLg7m<&J1U z>OAi_JYP;^uhY^w^z(;%$iHsVQ%c!Sa@d5`>FY}DLfjIxJZ}34E1RYM_P_(4_yqyE z{fd}vAT^qVzm_o^qOfKDU~B(x8I&)rkDERn7( zuA3K={pdFG&j5#d$jn|%U3ny4ym(_?++p0${>=N0h$1Rz3#a{#M9LpyLXlEl@-0&G zSUPm`{K}>(t}U1A`s+l=mSs8=C~&4r|81)chs)658gFz>y}0xl_9yxGMFZ!V<-C=A zQ+rsjl{XX?z7E<7y=avhQ9qViMXnMUNPnNm&3`XsFv@-q&PPO4mpwf?w{{-~uvj_2 zTK3Bw)xk6UC27tGZf((PI*v313?B$MeW5pcYH8>g6EbzR!&LZw9)H|M+nG?N`N4*+ zfDMgk&3*u=w^&{YD=2*%ILCm7NEoddy5m^VJvR0goD?6OQqFE3hVi(<|Bc2*SA|;bD1( z|2`GJWEH?Zj{x?m*|$-2*PR%rc%1k!-ij5y-!e*;I6!yc9A)UdpJZ==8@24+4wvkjDvBcvi(Oeswz&yj?ZxLHfP@ZPG8-jr}5A)1@Kq z{SLzo(e*{j!U$=9(X9E6R|@Ho0lfEw(yndRVwiDrq9n>WKVc?-6d{iOU7$3zq#uH- zOMx9EGi=^-`bx$12yXrN;XH7M|ygA-cf@ zpvr%C2g_rBp|7zgvbSSO->?h_OzNFkh$|pKnbWu3Kc)1qSJ0?kv$3LVCULJnqCW1a z!~bz5>AEAxwTmVfeBzS~J9*mNRdBAHe0)R>AXZVmW18t`;8 zqwdv5wu+QSI`06U%0ZX0r@MHOF~-YnNS}Un`xL&$n&;iY_-%>ZzU9U;*ib9p1L-52kHD?O+e}cfqO2ep zJyi(Uj4CpJNB2U2(v>{qHP*?yGy+V57reQ_bFOc@-a#~iN2@)XL==Rt<}2Jz>rH9U z5FaohImuC2?DB#fV+CAJdDo_M)U&mg#9__YdVZp8{HO1mxT+pXw?|z57@Fj33+Qk3v8D*+d00j9Cp?GKPO`|7wJJ zgyu}DYcjHS=mAiaVkPAK+|jADBoLjxL4SxD))PA9m)qSyCeXaPF+1X}k z4<|uB%IVOCCHI=gP(qf)a&p(AIwZdQsHoV`P@J6T*0~?G#Cc?>NwX*=XJZUf6nW&q z#M0eyeS$k_cC(snq|ARX7_gTI@FtugWqn(uK3u$k;SW*9 z2jTM_?|rsP-YcPHt<^zZ!WfoZWeb=*kkG3?8?FFiM}|MYQMpYd3cR-@avpZn^4`7~7KDn#Btj1bbJ_G>co#PkZZvtY zE!3ZifK^VBSp2RlSXPRsV~+QNP5#yA@VvadQh6VXifKX2V=!#D93`*6W~wj+^r;uW z^-V4*EDY#hTMp@j$E!E2jlvXP3BG=Sb#8umWa~^_b%X0-p|gy*8NC>-oIY`r)QFPJjkn<3Jd0U6zm}Ev7h6ncY=g)1-LXq zR^7l!qULBKqNbiM(78afHH~Yh0mUVk`r2@Gc=AxW`=JG1tdDTxr~?pE2+lQg($iaY z8x2;u)adGdv31R8%V~)hc@rxp+N}0-_K~TWKmYM!N77aEgDIT_rRaLm+WUzNEN<(w zrij5KSHR=Voi9Xz;iJ)=t%d>a5_nL*sQ6H`(4LnA0nfi5E~YBnL3(Zxd$P@t1N0XzSPBE?hRdpLIwRqc$w?Co`hadJtC;-MRhO#nFA^O=X% z`R|*my@8XQJu9>ZWt_f3|P zc+Rks`t#h^61f(&INt%wl&JPq1qV5+o$C%sA-#wm$5@mr4+{RZ^fbRUoV>8I8!qMv zfrz`R%<7YPrZg@h-9VczDS*nb^k_Q)0s%tJTzPb59Az+qFOzgJ^1FNCQS42h5tl@k zl9PJZn#0AUxZ;Z!;;&yH1lABnbc$VzadLimIeKTw;^U{37u&e@vBC-+!J8a7$StXy z>|$QeHhGy)Rvl1CNv+I;5S4I{J35o~xy(v+N}zTEKBnOKi87O$ zFqgXZsh>DInz-C+#CW- z=>vK*Poa*r4ICsqcJ+;b0QQPoc9x#i0Tei44{{_bCXWTdgRY6NU2-h!>{p~bGunU8 zIk_i+W7j+NlEZdY7{pgVIMWKYYPUx)Ocf30Sr{VG+Al^P;6tXTK&-y_RfTEONf3fo z+a(M!tJLUOW5pfgr^|r5Mn*|A97NB4p<3!+In#S6;#O}yQKKR4gv4kCvD;iA0!5o1 z&;_cfGsgmiAMso_Fl0j{GNa)H4=4~*V}IVvUo^MFiJj8!;HAsCox_a?*qpw0_&xkv zXOQ&?ZDZx+#042^ip1WYp5I}2iMG~nr6JGu9`zzQQnrL{;`ZpIL?MmeyPh3);7zYP zcyhh6?Mr$+_F%J&K97ga#|Ee-S`qj&&?_E9Dqzey*t2 z&3J_w0m%M85ktXd)>$%iNdUy=WhD-}zZ}n0_m6EC-MtuGC>ML-%FVx+f5Fctgz_Pz z_&IuQ!-kW!B%`$O)vd!$;?kwnSa_G69KU!B7JKdgfW51=u0Z0i^Jzkp4M>#@#*z|%Gkv5h?4N6ySPrk`_=_$wpp@rdj!3641$V{l zXpNp~##MAb+pT=e+CNCa&rjV)7^LkP6T+gVwuoQKqU~G6i49>d@L#|H3Yt32xO{gG%q=N*gQGR_9V_76^mEMNX85P!s^gkL_{XdMa# zdOiqnkTnLK1aQ#A$?QltGPqSxrD~|SM(NeCV<;dpKO?X5HE$gDy*(GU(5Q^6H|=Z-?C zSh7c{+R*s@qy|3_3P>mQ=L>)E$FP*{@dh;y2kjxx_UqktI}u_Cuo-_R&RG@xYuv02 z*+i+`w$B%agc*gyvHDNfW)A1~yxA34pSx?hmypLXhT@>$lF9M?F1*nNlYyp;Gs|jg zjpeP`_I;(k^mj3eHY}2kos8*Uutr_I`TVxV>vPkUQ;*1pixXOjwhtYT7^B$tDg|}a z&k#W^N2%Mg!S(v*#owt&i@zK9pi6P{8Nj)09qqtbO%JFApD;+p(#YLqvP)4sYROPm zFn)hv>4?uB((uMkC(fSK+tt3y1 zNQ*~g@JyU@=wFITp%yQ-+p23!z2V888-}c9zaDSv zy_vntz5P)3s0@symgu2_ynIS%`6U-z&3~@feNSN6%|8mbMl;AqI`p@N<5Xjc`^K&U zZ#~gZf3sw5ezI{po{jPFato> z_7EYl4vND4madOc`8-$G)}yxVK#AQ|KfBmz>M0h`zNhzyj+QiAf&X*#^768uj;_R} zTfEln&E5$iwyI4vnX|LAghXRi6(M+ISAS(Yt@anD4$C}$H1UwkzMIuUdUbzLx^)Us zAij|(n4QEG-N$jrmgA{=xme5M$)R>W`R~~oXD_1f>0Z2|HK$1Gsm1Cb0$7+oWt$+^ zLaI)(?8n2!d`#PsDZcPdoN60kVae`?5cz^dI{{TqA;4iUkmxjTY(hInb4 z#``4xhDi7es952lakKDFQq9glyI2Po2rZnimRhQ|(ipx906&`o#`E2LVT)+X->g5>Td^;6Mb8Jv5Cx#Xw0QOFpq z9$eLWSdUt|fmh?{yB_JM>n4Mp<8dWj{+#!z(GcOJWI@^Z@8CHyyA2Xog9YYexXhbR z3nYLU74o@_)JuwsrMl`2!+C&UDoIsXCzg zB(C64v=%^7yYXB*X4>^@kP;XT$%rmw(2eS6(1}*pX1RTHPBnKrY<+%~?t8`1;kmrS zaM0>Z^?TA3Mw(Po^BEJA`=OGj4o>Xv9`L1Fw@|vOsHpUm?t2!gFtzJ2z~L#-NY!8w z6e(iOgG7G}qPOk@Z|ofB3RXkSf%IeZ_+al4iYj zI@)k?`bu>7F!xfSSzUMO{9n?mp2u#&=iOqr?}5D|xLWyBiF!~}f^_I&F@n$?ss|TM* z7>L20o#IrvZ5}`hW$iIoubK2V$M4I>tgK;|*V?onLpmkdh{`T}oiR}fCNKtGWVChx zWRJU}LSKT1j3Af$P%sTkqnMEE(*y7x=x>=4mHjL!vvokxf@}^21*9wWk0;0R z-g{n&yGVIS2=3n_MPifTBQ!+jO!>`v+|6`W=-hVLxBia@d$BJ%QA{H-oQn8)b3hLF z?LcNqt)ikCNY+$?fR&Up&vh2vM8sL1Ea8a-l$0En6D*ff4?p7Mji>JCiSC-Hw3WiS#h>5R-_Mx1%3`$xzbPoe?PJ?TRdw~=Fedt38RjQkb>a?E!dAxrK(Jot&I7K6wJ@%O?qPk>pQ-C3v(n zG`|-p6owlC=Jp+G_5*Y~lkrXrV$j_Eq*)|{rN$#@0^O#wMnQZypBO~=!0pm?GfO^R zf(_N@W}k8Ky~-PlmMSLA{-UO^;0}fM>*7({APM6Z?nTjdV1+!PyaJ@9U%lRhwC`e4 zKAYNh6ij1eWtDbu;Zqd7yDh;Tu+Bq*L}>*4SQ>5>Qz*hxXNkJ)&KIQ7o~;Ag0)fX! z^3O#chL8mGO=wt^&RPRnI^Re5^bw(=)&pf6po3szv0|DJ1auT)UoP<*2*ml7b6S&$ zi9Ef-8wWTT}!4ze>L~}8{+Ua5EW5cpc`%q&d0`Gt_@j3mEmn?-6jn4z-J((pF zEgb|rg*z71)UL}&#~8;8zG8YS>gN>KE(RYrH(po^3!F)S8bWz5-D?qu9w;M}Sc8k* z#;YvI3q*fyS+W7zYw}es7F^EndEvq*NSiHT5onH_{^>?O{u0TF=bCMEt(-6 zZpXo-x-qrB!I#z{lBa>dJOBEDzAEy)uxX_(+e2MVFfOY;dZ7u%0k0rgwGohIUS{dQV&6o?B!+hzP~-KEBUm;eWLN zc9;93Sx!&t{~@?zT_6d=3AiUDM%~6MhsTuNBwMD`y|IB)GKTOB-^(s-|AA!r>#cgR zvaE-=(gIPE0Uw4R38h0ynErHb4h2uhg!lEy73nmRiudn9fY${s*P)gicwpwYMSjy( z!JzP<*T^5d88!>lDWS5fegL1H;TE(%*+c+!$5u0AV^q@`EOxiX2QWskMidw-aM9_{ zQ|#DfA0js~U1IPn)=xM81k^wrk#JaZ5$t71I~M?+mv3DCtLS7)r>#n-H4;A;f3xr< z^j#!}(8r;J-{(2QFx$9Z* z5KLo)>p`brmuJA8gIof<yh}ZMta2Bl%V3&XSGpR3bk2 z^L9TWwhA3U6$?2hGX=iZC~j*UOd!IVp%;@P`tF1NIVf&xBWkgr2)r&&5ayDg6%?{A z8rX&x1Ona)ePAdx58j}rryW~$!(BiOlZy15Cs;c~x?Y?h?I=Xeph`{*^4xaK9e7|`kQJW9qlgYQHvB65H#3Niz$ifE~_yt-7fOJ zZI}M|g|c}FE;Z@wU@kqoZGY%a|o(CzH0Dm!}BhX{9^c45ebL2f+ z3kykW>ta}m^{;A;;v0|MSFc`~f$!Yi-Ft2Ak*rTO^+snr8k^?Ew1iy`9R*k~ctD2d08V+dY~NFQyVw|ZMkm`wz+BQIox|P+ zBtsH86}Wu)n6|b)WbEHPaY63gDjvWdMl&8#ItGSE;E$r2Wq>p+(7ibP2K@2P*<-+y z-2=Z&%+E@abCeiy@bChFD4BgVDDw9r`_kq^pueWk2l5m^lzznOj}+sfXyeQ{%_{lUF{&@cR1syw< z`t^^mJ6~i!VJp^(fSE#@+H<}=TJK^?e&t3rs9%g7WOZat~p-}ik= z+g{DJx%0dFTe}2jf+PazGQV*DOh3PuP&zvBmqXBF1wW;KlE1iZc|c-9-*|PK&VaqT zzk((vz5+4ZiU(yI0cf=|rN%>nRi{y${Bby$>L)Hgab(ji@fh)Rl5 zB3)w9-6=|kbPOTgA|(ydF(BO?N-Eu*LwC>F!}Gr9T<80)YrLL+e4Kmsz4uz{7i;Z_ zsw1~X(4>JP-s}Zy>RsJ>zD`X{?DLx572T+;rqzBMIi!Q{#ePSNH4X5B?Bh@-z94bO z-0{&J6Y;sdY#o>6j9$+4r^x8j!h~p4I1(SOuNuE?etN#v!_Oh&R3u2gtklgY9gkYr z8j}ZUuOAWfQQd9H6u}mDD<6{Y?Xlzi0fzRgVrubE{ zn#DxA{@F%-gi-kQf3Bc`LnkV2q74;7!y!8(O+(+l>&XDuAMP1TOqmWQRti{4nL2vt*;kk zOw#(-q~A)7^gsQHW3usLL4l_fqNhnuQ-Hzr@F6i3RWPzS&J%zk%OU>6_n#tLcRwVO z+y(vOIE~bxi9Zu8fm^oIQ_m-=b2!V?!-!wYEI=g@<7-yg(H5v`$uQ-Q9Fh$=rh^kH z|7>tl6A$rjuXobQJm&3_`O*TE3OY)^8oDfQQIsB5v;E!6_FCu@1f0oQOz&z?XsD!) zj?R!sWVhee)>hQfakz|ey~xK%WOmp3pqcd*lJTZhkge}+d968bm#SKo;83-yE{XhA zqjbavy3kSQVuA@i*3b9BuV*Y&S8F-dhaOu{(sf}*@|O)tc_iLE(wjQ}22JYx+!{x! zZ43XUt$41&NWvTZgpcgV=n>Oa*Lz-;Yn4M%_K5w$D!8y4p9&U9XeZb}jDa0j9wHj- zap|wOM?L-B`%vlBB-xZ>Hz8)J@a$R*5__C%j zWP9#sMt~|dfcN1wwS7A!_gqZRm920CBu|yT~d&PQy5X=J4@!J8&6Zo z|0|;oTbwBU?+?ld8d)1Djw|J`Gm+>G;^~G~FJB>GG7c*g6ef35|FBUf#dPkLi*hCBWKQU5LFaM& zz=VVf(z0VG$sDwQlA_fqZi%yfJDm0X5F3W9#e@qs4+(T?$TI&0nHR= z%v&NVMoHC53=}GD<_Fwk-q^9UqWPmezcj8H>$4r7-dB1OOQMowNo4Ewmp@wL>&p43 zAStHp%i5A%bKP4(S*Z_D8VYojg-g&$Gtw%>L|mc8(P6|L$E8mI7I}CxebhW-9G)?} z`5^jP++D(oS8UD|QZJUlfn@8T`sv}OsO`?H!w)42729@Bt2A{tAu2denMv?@wvHTM zrszq@^cXsiV9r23{SNmVIGq@eJYagrbm{-QxTmvUe-5scYnqzP#85G&}5C})9U(I-ukgTqekF7pUwBd13blxy>Ahc%^N&XoGJg<*fYr$IqK z-3z=8K^?^BE{N{%xSRgGd4qSYvF9v3b8nkv{ zJmHQPd!%Y8^$%Ofb`wo@jKz4We%QqhaG<4EoVA1wUHzYV`g$MPvp2dvO+_ra3e~Q? z;nSfByy@xb@y@t93EX#H|L-T#kqqr9w8POV%*noo934%pK4;Jw&s6o3(VH@uB+Bu(Z-$jThv@=Dmd2rpY)5xdOs5M_X*G**t*$kQKRMu~R zt&uQ*B}SJop_D0-SJb~IA)Wmj;psr}vru0h9O-!V&?kQ6W!$aZ+^W-tF#hq<7ZlBf zCnHAfW%v`emT2_XhvH$fv_4dR9}U*h6J0+3d4bFZYP1?Yt5=Du>(PHu=w*5^8^p`=}Owd11vz zWAa=x{^o|~@pd3}+kB#78Zs_Vsm}U3*>%R|Mg3M(By&qm8`hlOVmx(p?7Sbp93~a= z!c6i^XJWgAlRtTW{m4-Nw}@<~68N&8Mo%$22wKqMMZDK@g(Kw&y+Uf))-z|c=#Qs> zFzXxJtQ)N1$o`hYrw|?CbZm35e86Bg%-c?L@Edc(n>Q_nCZ4^I9C?5%v!jl5#zMAMh=cAkd z>aR_{RPLA7x>R8x(S^?bqYV3;saU-eQXg&b_34IZL`oCDEIqoCq4;n9%cjC<&;XB}yUmOFeecR2X- z@BnOn>A=OHOv8XL=|&&0^`NbM;i`oGJc;CV)ZDR$@wyoQ>Mjpt_%v}U=;wKrT7#zt zsfT>Gc7y)f^nSZc!Fm1jl%8q5LRGy;22 zmnB=N*HX)$E)V2%U-lhPV0*W1*_mV0kwj3h5<)aafe6a%w4-rC}18=be?I z8tkW7Q(eCHH|Y#p@{!%c*-(Td|KlIjr_0OBf19CDY~Q=OVW*evkng1lzgh{04Shup zXWG!MFiJZ~;9~DK*FozpVd>s+6Q(-@>rSlUdMC8zOwt4V(F0yJ-LSugmjANEXZ84t zdjy^#?!0N$YB?g^ruq*mt&jte`@2&7@)#TOvdw^W>z&eyr|fps?ivfJ|=x_%gTb6Fif}|3ZZ8R6M(QF*o9;#cZGimg0*EnC0ZfJqiY z-?*(9_J-bXlA*8bQ6pFDl4fZ1sg70;<0Vk;zC^rdk^J(72N3)1ad!2yhVJL^s!7W# z0?nGcDsPHsXpgc6ZJ8j_uR?8VX8@t`hdVh~aMDpeqi?%%-&QI^$ z5t+Io-mb(yLj}!1H-jL%ke7@baZi;f1!7RK2fw^#gxC6-A3l9FXvS!fW)ASZal2AIqs$ zS6B1$^2DsHI6ztj1$y}5q5b#R*h)8gQ6%?OX_ZdlT4p>%5GKq+PU3v#n)FnbQ|!(A zH-Tdpk?>m|P05cakkp2y-%+M&>qXW)eQDfjt>DcDye();<3x~3`C=ocPPri7y`Cl2 zdYsWmOCX;ZfTY*z89y_=a!6R@mfrgEbd-1WylO#vLU(bV;^XI!Ws9#8HP{2x3jM(c ztm~WKwSyD@@Q7zX$4hj3`mH97@7)y@{k788+q4y=Sp<>3J&(5z*R?YW#S~$g1Tg2* zUz0>b`^IKMO791p~ zk$p4c>y$%Hy5{+Uu$wJ{{~U31+O~sF&-H&sV=9{Fq_4LD^!?pP(kR8~)0{%j%r9%Z z5<6A9C%oT*TgvHEBiPhgHFcDoIZJAExf3d3hA@Km?_SEbKew@~QYW_NDS*jwxsc&? zczpoHBhd4f)NC#Cr+8JvwjZ_O`3;#+)g`xw-3IP=?cPZNbjT1GSm9M1)DH{?R#qmv zvQh#}b%Xb=ycu5yidl?0eX0yK}~@_nsnoJrGM`d6@H7`id;3Tk3D6KLz<4Wr`OU%hAz9d+f=1k zeE!9=aQEZF*JUQBBn|G%_2*|1&}9|Jee=`Rs>5_Eink*yOAA?8VBC&^VWHT2DLfEe zgCc?grr}*?SfYLpu)QP4pAaj)uG{-|wZGQKZ|JCHcb4mH?9loVB9^uxB5o?IM<_=v z&*$R6ubUJx&eDWg*iKy)Ryh&Es#11<+*Uz9J}R_6kV?$_p)>c*wd71YW&XDDe3^BS zL#(3T;FHkMl@t<1*JEu5=o;FLAGsZDQP`;>3e&$8*sdqMl8g&ZV~8w|E61l_KP9ft z`llCM6ko0Y&zIg^t_@0J2z;V?e=hZe@38lXrjZf&y{%wS*e%3c)|Gj#ezQ@g7jY0O zy0n96q;qNp)#x|DdjxMlH5z<4v5!oNQXgVeQonvOULLG0#@%16sx&QyA^*#Je_L9F zSY{=>Wh_H`x^fnolTR+-H0;VddDh1Q2R~da1)J`zj@ipfk!+~Cfnr5vvhbPns!sCl z^yh}tWs7H{2`*(|o1_mCdPW$Nbt8kx(L$)Ce(B#kp;j;=Q|9jqk&Qh5>~t}!O6#}J zpB>X-3=`$-8S72!1*4Senv=HaXB7zQvR9!_Rln`tzJHW3;!FZikz-beif`yJXIEr5 z^s7Xsw-mGZdlNhmU=*>YT`V7hv_0{$yRam`E1(U(RGwX6YFL+b51@uYgQ|sSRc4y(A8?N8#F>Mh@mjD3JSKPz_5=OpAtOZ0mj&f{ zFy7-rIX<_ry-2S}vm6n$VhLR3@Ox?W=l9cYFO@xFkH}bl%8U6Iqe^!%>4iy2&FbhT zRM)n_;Ljz6{qRS_1!xKG8nEiN^POZnP?_ZO2W-BlUP(G#&OF?<1kZwTc`2^Pj(S-6 z1Gliv?xxo2rf-dtcmo)k#%{dmx^g1?K{@;Y`}P@Bv&+=w=#roK>`b6Ws01^z9A?7s zOK;%ttUEEX8W7r?8Qla;b{X%UvIA@ z)t`x;*{^>;AqT%@mgL^S$e+^7UpU-ki|!Z1~`|QTW0r!*NUun_Eeb znWQy78Z;Dd9E%~U*$@F6L7&1dyPuqUh-|yML_+7gmT5B7x40iTK~B@=<~Pc=&K9P1 z`1&q9%k}hNogxU`z)#d2|K%#re?2A!~cR^8LpU+jNvT>Ej3T^8uF+(60B?qoKY0A?Gv|+ zq(W=TMjcLnB}SjJDI9EHXtM_MgGXf)5QBIvIwv{L*R=vWMhz#;)ujNe1U$mSwd9YV zW$2(G0U;Csx&79_9sg;+06o1#*GIRmH=^7Nt0=ZtGY!BhE0`dY(TaTbhotmAY8jmo znRG73y)=|*BJ_iDp5&2oF85(K=hSl(hnVbgxHLsc7>+I>?Xl$TE~ zIMpv6U6fi)SMe492M?rg^juXlaAbukHjFbJgZ{DIT8hVlAYemqU!?r*5lIS1|A4Ajf69wQob~cM0g;4&*W0q?j^>^aFnfsEh2J6n z7}|#LJc9zXr!2T{%8{L4n?xqvVbfC^mWHaLvaNJt#2%Oxv zx8k|6-6YaV4>a(a7r^FvrbQp!c3} zbPp@11(zau>l)*j+igKo)#uEYA4W z40f+>-FuX&CryWfrHELR4@TDM9AIE*=}+OQweG^5lD44lU{Wp9LA;<`sI1RXs>LSYn0;hq+LQg1*!e^XdIU(@~5$~RNYCX~-@ z`@dP(0y+`05V7i%h* z;tnqOq1KU$zz8uLF}Gy>p=sUg(&1T{MgSIV{rRP_;7bl#pA_Bj@7w1?)07r{3ahH# zRYli!g(d$>mXY+JOVq%G4dMpNq6TkVPuKP?gJzKciA?M#d&dH`KZF5(1%AGbBcq4H zyGAeV*H`)6it5(;v;j@3xoxa7<&Cw3HM&`pM;@ZoW#@ z8@fBh$2oGbYJ?)?fP4%CPr+ zWWwB|X$jWY%=B~!85tR)`#6B(b)$P~pq~=R{(2{uKDsP1l~*tSC*Elf?S9XpmWW1w z99ke5=P>G~CEOu$w%*;}d0%y*Mm}u$u_PBnMOm4qG{n*_!vgYuTGxMm_i@JmuT-GkrG zeM(f(Wxe$}n5Ws!7`RIqb?<2aV@QdIDa%a4u2Vwv&SUCfE~PMZ1gf1ho3~r z3i8YS5BCXAQ70=fcw4zlBzJ-4<$0=7kD$wcG#C%VT>(`6jgOw=&mM+-c<(Qb^tuJd zlvSSt!iA)V^9F0skC^^DNcI)}s=3QpUo?P3`OT1pe3HEJK~-l%4=^OZ8~((TLhnX{ z*Fo}TXy&Ajt5^+Jk>&&1T{g3(+km^x!OvR8^BI=3>-J~;0rYRs&npic8-FvzY%rN> z%IMxdzFO=zxGtH=$9*^Zbc^7p(&yYXzxOP-3DwrmC{f9iNzpU3t2pk!%Ma}(Yx{JY zA{~oS!sGB*iy-+=h|Z<4wUW+`CepHvTxs#p;SV73nsJ!f?H`Nl8GssKBuxCp{&mHA z(nvl0z-_-rJ#wMGVXrOjrIH=`3-y7oy~{Y-Yan#itI&K=8LWDnjkBc*gWCZ}yb(Q4 zard=Tzt3ZWN+M)_3jcZ|-gw*QVg*ELjFU^;W!^^}Xx@Y(w7U*9{nJm|qkPz2z>cB% z<{X_pKcmk-@5*bWHPgpdz0EUW#S5YN3dhoqylSd}wGvtacDFLo{5|0FC7F{6D;k;H z4Q;OtwmBwovu>`ZIO)@{)~Fr){_o&u9BJ`RoQQ~uipu`hO-1GYeVk6N-OD@kDqmSW z!a8h#DjfUuvUBsR!;6Fd6w*CE!oW*fU&Yx|hOfxrH(fqQFqQiALtD*=o%nQopPnyo zt%dEFeH;ZPu_AxP!Gw&ZP!+eqG*~sH?s2f89%(du@S$HY_5WA8)~V3A2&J-|4dD7h ziq&ZBZ|i`Mo<#CLO7-;=uK@+8pa=1HIn!z(D1%3j+@MTsRe^2kgEWY+&uQB}f{l z+d4KcTGuu@C}zMm)YjRtQFm!QAJbosN`j;-@L4F>rwmO|Mfa=On5Z9qH{X6HfT)aN zyfPlpL-z!_#Xzg7fmi?Q1z7H`U#{0laIVz>bZ5=Y=`_%lCaoxvjgj>jeLpUj6TwVy z*`zU}!|%F&*Y)hR44#7JG|xiSPj=(d&W7&~DVaCFDeUBR9sF5#fv#Tx5)RNTqG=yScwU>8iS_7Ki;a z9?#c(QXPkkE=cYVi_;Sl>3Kt`cXKdW3WFLgk=oX!)a+zU`>M5z+Fn99^}cXd%{Jp> z4bobkHoOPoU1M3k@j(xa{+k|1h|1g^I`U>kB-yAWT!J@psXE8rx9HlG8r znmXY{T72O1@Bf3Yv_76kFGh_zMW$YSJw#XeT%eEvpI|&a%I)i>>ZQVSGbLiRT!)i~ zcAOt|h#I#ZV<5GYJSP>bMn3SwuViL@xNX8(0ZX$Fb_c4+{nOA*tj4#I^=dQz>LsrJ zU>)*Ek0Z9i!9$|82|>Omyp1yK5er~^_@FO$Lx~zBnY%nw(zO!i^DD=l!mT#RQ^=> zsF)0NVSYCRZIqMTgX_XzncZmI0%zi_WdlJAU|EPUQ~k~bF3~e)DE8^uG@d;q@9ca)JurC!Yz zM$w>t*)~H_~X;EuAmS{%C=O_er=bx4N zTDt6AYI1^R(C^@jwk&Bj_AkL;~^D zi(4urbop^Y9p8-s0VfYhMhI~q+!HFlr8;)aAS~H%$1oZ#6Hs(Sg&&T!g3ipO`leR4 zB)vtTXG@Ks0$6qD&>@bEjlC_k=Aky^p<3 zrrVe<;EaadFyuS`JeDogLu`ykE(N|674!C{;A`-XJcDk}n<-G0nyit4^RfT8DbfY7 zt0#~y>W59D7s7oJ`&q?Hvj>)W9|7fzzo804m^nJ~Ht5*|rp3sPoUaqziTLV{i*;V5 z1FD67sLpYn|F*>Dqa7>#4ezys-Pytay6M2C-gTlsf%_9?NI~fSQUHabDPeP6Q@?3@ zNA=^S&O-f;roc0#ass3`AMpdfA+8*Txyf4sa%){mCpj=~!jTKSh^o{%oTL}m3YVM2 z!;=b(6FaA+KC?z;U-BJBH@^ghgxn>wfq2t0F?Q65EnbNCrF4qPdr=pTGV`vz%wNPBJcBl@ImJ`_*w{!-gR}g*L%eHyULH!dUK&UZ-xz$e0@NbI8D#ZVe zKZ7lk`PZ0=PUhP77)$R9*eu4iv%67|4uu5;zQ9E`y|epkos&E?1IWOj+=GrG>SeM% z{~L0`Iq$HC<5P6K9rC@iuJ3LA_0Z_9NMD~%9 z)F$uH=RvLnpiMJ9mmsLFw<3@Kty&mOAjTQ-F1$G3P5;iVNc+|j zV8=9%!+VEENA)WgzgG{_=Z-Ju0n7U38Xx^O1~`XGn9tVfkLqYpQ)XA_jA#pmx`2llTq)#v;(5^Zli$GMt3G7V;M?p&!oX(R=$FAi*QNFEjx@zIUVwLZ=VV4e zH(WAI94&V1TF41tzNAi+V5t$7laHG0w&J+e)1i9Yf<2~62$AY;KWg_6@t?DR&!H4> zwmN`;F;TCxfZjGn2y_X?>)78@ zVU2|*4R1<$d5M7j2+sQLU;2F~cM=<28y0xk(y)XD`cJV+A;V?`H}m|?ju)5w1dtZ^ z2_yGkNQxSn${0)h(y?Y16kW`$*n|)wuX}JG&~KgkO*gn;>aX82LkH=+kPn_D=B@1k zgYBN#+4CN7wM5#eX=h1!y@5kO+6#ul5PZ*v=>>m8u^L_}7mk9CLa`(m$(_02>k;nM z2`hWIT_kq5{8zn{#<3l%6QZlSIwPE3yPtg5H1)%)GG7`W%xkt^ynYnf)=Q(`RVdmN z0=R-1=;qD2?Oq_nyVBL7>~dgKwjY^lT5$o*9F_T@Rloa&jNtY8nSVor(03-6+uFhP zcHp2$S1Sab6dL>%l3;X2MMYg}PFElOyu9IZnaq%{yps<``Vj76r&(?QmahL9FY1P&O3y$~fTD%vxcP`Z zZ`DjS24^+y{HiBbu~H~h49@zhTc17JR zKdmSL0pfUq?+F*z>-i>6SMIyS<~|{8%il#yjpqOEua&Qd+#apUC$_1^0-AssZ){C; zWMTX$2j~EMG6(`h%`StB>t*k=@dUfGkotqPS%gnFe6bV!yIz2>sUO(&?Wgh@@W^2z zHP3Umq?xef13dhb`+yXNK$<5`Oz%pbe%HklEVpPFToj{{sQ}gV@)toq8^F`ExlUJp zLXaY%29kFlx$evXkr}mv$`>rO<<7NtaCg{#%Vy#&%Fh6mqL8&V9T_(QD z9n7TV{3QMg=nSAT2ASJe9KbTK=qkaYo@|Nzwo#Advg)umtT7!RXuYC9Sb1MD~(g zk*W=HD@2mh&`g(fJO9@`@%i%x%^68>2g9MGARTZK05`ShapCCJ$9dbkhBIqZm4PQu zl0$Z#mqZL&f>HKE8g>MoZ^Kscz*2GUczy;`ddnwehjSS}q>-2Evc?@|0xzS(9vRtW z!s!K1jqsxS7pU9Ba8PWAQ+=A%{z9vO*R=ovogg}Fe0uSue%2c3%;}j-$7YZ+jctY| zs3Bo9_`zDm)-k!*Jb?>W*(Kya`RYFfMd87^4PwS1D)Nhz(;kBogy2{hk~P(%??RD1 zpeE_5&>xV_nZ5nvdCkL%9`-Gd&~6TyOrf&r)aG%xKd5m2ev(pEB`YW>xQ6mmhuhZb z=pHTmJrFBa3&g(y?~CNk^lA2g%4SE<4Dkm7q~8>+Y5wE0ifLEgdQOICD;q@HgENH6 z%z@QFZgOOfmPK#WBLv*(@$qr`4Jh{?Y>X3LrPS`HfqEaf@+U22Kt%2-1yd}p9gZda zy%LLDE%5IEUSW2soPSr&XQ$F8TX*e%%dwG1=jeDMf|S~UHmh8qA1)6!I4n%c#Dw`< zOpM>@%anmq@p(%==dFj^TU-n0O<`SK`@<8^biE-|s}oUEBhb;=iLv8QKMTCTjDB9ASfAM)NMl*H6|mhJ zd5YY~gre8afIkq)P&ua7+3JdFtQVlIzXBQ>FdJ&oZO{}JTDxnH07h17HV~LVK^p>N z$CcMC(FN}M`nbPW?(k&Bx;e1y@K8&pvfceNE>ZEa=(MA$xMLI|6}w->(V}I=)N1oP z8SA}tmF`8sM$tzd+ib11Ma%ttz%g)RP8!Edd_R^W`A3*uKDX%Eydt1=?y4Z^y*bc7 zHNA71p02N{c`xfPcH!t0=laO^(~n4eMPRs17bu&H#-z9(h+8HbcCLX&yPBhh%qVAD zA_qYg)5PXqerT|xJpbpD(oMAi`0?(S2EyK-{cl`JniDn=PTF zq%-wHC2kzP9u~uZtCxG+K^pp63U{Xw_sEBH%e(2EJDFZB06aybim{V0zJ&$IxqI!q7x&@Nk31C0!8!?O0 zjtj+77Y=f~fKPk^dk7iLq_DAg{*S2A3{+9 z#$U(=4NG#yT%Og=(rXk&I3Xra-U>fAer{K>^rVE!79WQdw{*25&Z*eJU`EX25-$H& zv0A=N=}}O zA@6a7e(QV)1q#gcN5_aaRDLJT`h>_yf<0`5?e(wbIsx=hP*C*-)}$rg1Iid7On&_O zd*gUDm`vhdUoNjLj6Z~iBSq-ZmxX15W{ydz6kwm+MGStkC+yvf>gQ%)vKxKJ816U+ zm65*lvKTZI&z(l2OeDJkbwF*5Yyb#tNX+qJms;n$J+UPx&DbieUDtTm7x-+}ojnL3 zNWXrK>tuiNE}8t_?1kQW&wHTp2s{ze%7RBK5@m!cZ97uB-S1dD->8a= z8fC*J{)Mf>zb}aVgA|21=fdfvUzIl6zG@OaIkeXd* z)HmU}|JEicNm5eR6|Qo^IfU3=^-QBc^i84uN%6B`fZFa@#}W;kUhXQ`Ry|+ugG#WI zEN|Vaw_g`gu@S;Lb{J*}URy1XJ>#{{kKTd@ z1tztLFCL_TmNO{e7H5fLoQ;aOw@DgB9vk4!S%($#;_-5vX|*kx$N#M3)*Bsr^(;mj zF|i@*JRJ+*9hBCkkItQRG^2bDfed;gXhnq2KleNb%-VO&UR!uzwUN^Nn;O$|j+)OK zA8HB^stFY?Bu@8tu%946n(B+Yp}SqHo}ArCQCOq& zv-pG}O3bBPo5grj0ONsV=|xk&%Xx3$!J53yp)bJccPvV3dY(FQ#YYZ27ReYtCs0XO z74@>Yl!;nE%i#?`rG|J5IMNo-jpD~u3UFvRx{x`gO@$xleZdQ&@m|jn4$McS;#800 z-x0#MPOJ0Rut`UA3XzE;l&brd;^L|(w3P(+ud^XQZ7o9FxUtVR-k&rpH~d1$2mq_{ zR#e^_DbSON)E%PE8HqwWAl_+mw_kzmIlJ)k7BC!PuEuzC$rmiUO&iwi$z??TT@oHC zN4#8a|GTjF2u@yp&A5=vsNP7Tt#~Ej06)AZvJG!01iqq4#~Zg5m|d3#CV0W;)xRIv zBP~g=r+j3~U4+&h4ZZ7zNV*cy;}xdMLq2>Ze};7dNX^}wZM`DF2G)g2;OBY>=s^gG zc}rX|xBpfgIC#kU@{@;mI)u~a&>X_4BHz29Gt69~$lMs;2M29g`hWMLaH~%r8f7#; zDs^Fwc51Wyz9GOArB68ey{tkwF@SSfu9~zwZs5>{;z7Fpd(oBHb$`#D-vu!mO*SuC z%;dUSYJ6O-e|6g-vd^yCZ@*Tvq^JLSu)quc$Gy2@>y|3z(cDr#8xDfBmw4sFc#TSt ziodgL`@JpFmBO|Iojgw7x?i0yf~fv!pI zvnzzSQX@h}WsvOTv(-HK#Iy~U3>}rfth>n>&f)%|!Mxuy=>)Cg%cOu=bRY&pkhCzC z90LS0Qe*Lq)m6pMkY0d)`{s{xNymXGL$^Ncnhn78qosgsesl~l9vXgG{fyZQ8}gu8 zh~cOyF%b5v*>{}cT2`c3BGfEbIJ7pt@5Ba9H%e0-UW4Cz%5I@`b?yN;5!w~e=tv7J zolvlKZTI`c_4Yxi7k=3n*>HoqLZ((;89dcG8lPiuBAWAyG4I{vER~f*a6gSQF2%3rq z^pF|eHO_0`&qjeTY+pdmyqs+OBCB4ECfFkWTq%BH?*N@8*VFSgyZDyEWn@UB(WV(Y z5BbuOU)plqV)a7Sv`AYXG)w|%?gJ%PMqVCvtSC&Gl!7Ag{QNxqIcQ?D6&(B^3?I%V z9KmN-c^6Jvb`QJs{KxpvJpQ_v^w+ps(!AUj$!`qPqPuU7C5OzbIqMpN)5ypUj$SU8IpX@_98L zN#xI7XlHCUV+MPMs^}32rVwwyu>E|UjkSYVX?cvoHXQf@7-warl@iR~1IARnrRj^L zU{lkP#S>;6S`l|j2a%-VZ9++yjhFDkl^6F7G6>23&F0xWIdMPOYhLjRdqdTZ+>LjB zvpNJ#C^iLM6ZZ4!qUq#v2ufn^b7Fm}`6Xiyr~MZYu?80U$Kj9tPGSAJp~0Rkmy%Ec zkjbR}JhZF0{v@U__{?MhOoIpl)^>=&9RJb6Va*TERBfCpbgSNHCI-`J)3&oMaB9y+ z89OgKNpS7+%FoBHe~}OXwd4n=f~}}mx&fqhEpE_Gg*M0BED#Gl31zi7wScevCXAp7 zd-Zxg7`k`vtdNMbWt>^N5mIn8VAI1Muh;=U4j7-Tv(6Ac{3!-`uwsbow# zr-=&fB-l~=2=2+Lv6kj4gnCMAxip~Cgw%yAPG+UsKCL?$e74X5ap}|pX#K}YoxS!; z;06b|`d04}xA?cNAF_10_&jI~7EdBB5%j!49w>QP6L^6T?Uiukha69ckh_I-I^Zvl z*x~P{0iM5v#)YmbbtVK^>0FIif7g1K5m#sj*7=#)OtSYOxKc8oEwyc%@Z%8!gjkVc z;lOq3I>D%nl1Z`MuMRk^MEBoiE6FmqDk^ivOA>v>>;+@&6ytN$>bTuUsKPesuD@eR z{=^NCrI5+_p=iy~+5L*dHt#E&r_wmTpc#(}Zvjher3t1;w{FV3*Z6tOpO)CwYAU?& zrjH9{J_5UJ-!hriObp6T~;TzUeXCC-k~_@C)%h9GEG}4)_0f zdn%RPcaCTxZKHWP{zznX5q65FZnh7T=1p0|3z}4K` zF9c)8Q}umeM45ddady{K`c6EalCiCgX=tRj##YCVl-5CcwI1|qr?>C{*UlVP70sDe z2I|1PzoM0^>mJt?r0}x@w-MfN)H^LV@o8UWHf7$P{+1z;rru5aLSjMKZLr#;aqe*~ zpIDbu@vTCJLDJdKo3IjAEOFzHA~MnxAkhdxL5sj|iCfhEp$eNd#ex+5(IaybxdI6& z0eF&c9gd(;)lUdPtJAHgyu2&*XT{Hufs?+$wC4I5c zH$8Gvq@AKBbUoVc0v*2+m%L@rzM-_Hv7;gH*FaBEqWBWV5OM1b>V*(M5Frg-b~`N= z2TQ74OdvB962O*3vB4@;9Bs>Km~$wE#DWGROCUO#vqnQ^_@36Oc2C&%kUZ~;+@|rN zc`Q|VUNdc9#Ms`T!)ku=5;ko`K(*ElOp(WBm*fm*fsHxzWNXTq=KCA57z(nDDB>bH zPio?eg&vVVoqVjw^9)}ZD^`-}(6DpY<@RA#$;HQ|Se}-K zwIeh3h*ro&%=&Wly-4)$_0ZKDm!n+%n|Z{SrN^``$B?ev$Us>l06qBN*VdHwXe=i0 z#VWI?Nh#zzn!b55_Fj01uuqrcxrdw?3^VX$sGGpNZ$JhGQ=;(!*#v5bwjb_e3!q(L zVK!;5GtQR-*}cVp0X$iDbtx`;E$%L#MkH&reiGjgn{uF42&s zh8c992%WeAG+!8D8HK#P@u7#5P-5R#eOlTmHrVykI-4_nobaHat!42XSc0Gk|0^w$ z6?ovej^jeO4}0h%1E_MqEdOZglCTTV%#n_OOQ|y-r2@PQJeRb=z-iGsx+{j+$P~ie zuPW^RTJ$cdz!FG^eF#wi)(eaz@`c*ZgNs$f)r)?@3_KSz0Ud;`_56*vkn`Ec&A2ox z@K;K1=+*4?81wU2jnJBpUalhGJRQW_xzE<@v`Z2W!&8^XAAX| zv@a;*AL|nU>SD{R4v7;9SGNP5TFeoXn!2wn)$#j zoVK~Pj!VWwy4|v{b36&R%-WsXDC@h+c~z^6*Ke-`tM$O1t_?Dm<5NbL1JSE!N>ZO` z*j8wV9`O?)aTaFtVJhJ24@SfE-XJ!2}KSf`YVyv_nGf4*f)Dq!I|4s^wVI^B>0rbEM z9pT0=zsesHLnl`HQ+NZj=oM=QCP~nM@urWG2aP7es^!)6{5{PQ;5jF|Qb#}OLK7>_ zC69!839Z26b~FQ!MY{hBawD7HdZ1m}`Z&3yuYrVN{+7D}hr>(YQnEat%|@UKrSO-! zA!xIMc!ziMVa7f!k6WBP8C*Jn>vano@3rcW-&RUT_CI9GtXe#AJy;+lT$;-`TZ#~O z79(&i4482jqK9G==H}#JI@Ql2&JPq=sC~mdv%$9PT=LH<7#MT&_@`vsuhVwI8KM!UF4x}-!8ZvHydKG7Fe{6 zQ9N~C{^hj}t53*O63`*}Ie+__`Sc%VY~)zJ-DJDN!QVf;8~PP&20b^XiK^$P3V7hH zSdI7}H4W&M=35f^*HAnn2Y1R4`B}LO-5$ohe^-v?sT*1U@*xVb6OF{|KE*rcAl{8d zc9;y1!hbTJ^_Rg%IA<;wi_iDR2oJ!8?EdrNw{*Z{qYBvR=LXi7syDE&>Uyy z1~}o-WOjxPeM9w;Fw)&R#CB2xoYKJQA50+%&OX*<^;vWH8DV4OyBJPs4-+AC3yUOS zXDLhX3vohvm}&orx9rGUd3U?tibl@eqt>}9 zy8~%|f#7i-W@2ybQ%<9o96y8Yga?ys&Yo#$<(}H?^ zDTQXf74RzO$NXtaCZlhP`n7RGhkb$Hk~@F!XDJ&#s4J%z4scmE_7tsX#N)0@YDslqqnNS0zvQ0z)@9 zkKq$Qm?}~zF6XzN`OlX#)_J>k&xT@|34^|ebtKssvglNeFcL%NoE7*(?xr)BzQ!+{B@7;(LocS< zUuxM^wBF?WXhApt+UtC0W1i1V2ZZ|Jwc&n*t{?|ccrp|}Eqn=Vo=nwGjRfo9N8Sep zZe~_>Kb!9YZ&C9Gt740ibyO45ipx^0Q%j<)PH%mIWx zyU|BaK|t>G$y#^6U7O<}O3c^7>kaE-(3|jm__%exWb|9+vq0jrV3aFP>3cY}1 z>wyaAM9SM+zbyJUs(I$43RfkvVn=FSHkQ~wH9k%D3yBjwAnn!SNz-K&;fEAq6(e=g#kbJ^kq{;UIivHtA3qiDVx&o)zyga3`^E;(yz}(xIUE-Iw_5 zMR`)=dh$ywZS85yo$}u%mSZB-ERC&FiovPJ+a#E0$3N8*XzC+!40(Q>TODayH*p5> zPA}KGx;%Kt-=)c@6-`h)&=TH`s#(h$F@w6f{RUexUq9ZA)+jxyTgN{z#JZmsg{;);3r+q4S}Ro!4H6&IBwB%(={1W;AAQm&YdW z&nb0w*aLa+?Fbi5rhdyb#&OE&mR7PNW?&X!?xMMeH*##UF}Pybj9;Uk%{OJ|<{q_T zG1#7rsvlh+4Q8pS(;rSuXsq=ob$g?rB|I~fNSa()1M$jIBZ#bE-RCE{h_+~n>^Im^O@VX)a%UUX@TcUom$!mu^C zZKJNqesyvfgP2S3>DPa6a5<~KzZHLFF{`E)Z7QEK#wwhxMaMUGa<=fTXDJobu$)tSo5NJW5)pC8yZKp2gC0rbWt&oQJ%QC z=dbj6X>axfI?2H~>YH&mIq|he?3)AceT!9B*T1K#_)BCcx-fnc$^~&V8~!6vs3;*$ z=ogNv*%102N5O};x-$odw%iE#_OisA;3if5Gckk@dy@q>nBE(9ZO(jl?I9n&c>3wj zU#-C6B#%y$nQil3U8u{3tpmqJSmg^BkBenqOjk&;c(1d2^Fw<*X1H7l+Cnf}T8q}` zK8-JHc1XUuJTF3|2dWUhYQ5owMxP?o^Ypi&*^Hbk8gdnj-aByK{@(Xt z5h*F@E|Hdwr4f9RqR{WCA#HPp`mpoyVzURV%V61?t;F#*^TWc2<4GV{O%m*3~g7+P#WtD z;EQARtyp%QsmekA{WT4dimJJjTNIQCBI1uw4bOBier8hcig`{Nj`Voq> z`=$j_;c~np(kF@{L$xi=01dXIC1IN}n2n(W?|}H1kg?5D*x@O-6ycxoyU z6f)jszBf2InqvW#+T6;c5h(noslT0eUuZGDq#T%KqGL>zqCrwnRirWLKpT<{y?rsu zOjn8rbPVmX+An7s>|IBWXxNMFj{G=D|KNSxF8z_zMa*BkrzVqR8kAM`C~09%?+*(n+CB!_pu9~hqL7OQBX zEq>hiC{D6uUH%G|LTc98WgKW}AcWYVI+;K#&p4MlucK5rUvrsLoHZ^w`i)&WUOl0= zP=eB;y9fz_1)-$4U=-R0*PZ?T2z9-|Zpp31F+(6F-(ew+MlN~a2{W6dG=q3>4>OK! zgg8*mwvP3NP(gs5n7Rt}YPy`o>-eHe=Mmn^WM*r5rOL{6k@a3qqrR@2|Mvpb>w!<% zz#OvL?p+6adzGIWgpG5U$w^6{fFx@DjraCIOgm5rkI8nrzb|8B^HVbF%h#ci5mWdB zEcNn17`n-p19R5Swzwu&F1;~KiNnnjGXCA?W4mcr5BWyq{|Mb!Uxr0~m^`67dRTd# z)WnD{7yXsXUZa@bMuAYAa$`>56!<9UDOn;nFlu50h3%46??n^aqW2*t+V^V6-R;Y2vq6+;_88DrqJVX?~juwpU| zELtHCHsV0+Id zG@Nzh>g=TZu+Vrfg~jowiwJv+SA=oggPg56v$F9c4|}W^PU3K1h&FF=An z_W0dvs;H<`ruw`X-|GLNl$GZ-k!1^g`SK+&70T|i>3IwD*=DbR{~eFm!xZF(G04$8pQAU#^1`p799@J^$v) zIf*fZZ#P2xpyMEly9~xTCYZe)HMdkQFQ~a1^^g+H=66@)x2J1!7nAsPubylG73_}F zEoVmpcb^BKWBaLL!@1w-d!45tjELH zK)Z6`W)d7QKr2DX39{2>PEw%a%RcBW(GSSkm$$3Ds-CQt4%n^F6zlDu4@3LQz>$Cd zYO8LWKiTDV;jm1aG%<~%hnVQ@?nZyN6;?u0YTt`4gct2Ka3n2CV5}s?-}+K? z9WcE1OLbvmV+uz8SDHu1$G~FF*7p;EYE76__NZ^07bBmIrOM>lP1$v|ZM<>dSjak- z*C?4xgKmGumJg!8q)c;EbNRWkTZrG>2VYPjot?Q#j&k7DVRi#6lvb|b(q zwWJkUb|x<_F9RB_QZScOrge2!cQ99>cI-f}FhdpMDS zERgfZ-2T;_ZO2J!JIu0jw5X`)Ppg|1^{?&~&UA&Mk`i1Bit%)At26;O%SH#VIoeWF zdnLMRMxZC!bc)1EOVEaC&LaMj@xY=b3#|`rRyT*L{?DP#H5|!BxEKN zfj`qfg<=oPahaHJ@8gcwA#~Hk=&5+d3!(f+R$k?@5kpVgMGg;}gu|i@&)YaOFJHXA z6@=CK{&W1r?@@cJklSI?)Q$#d7b+T*qW6bt;q7*uSj$04CaH!|`5bUb(?xn$Fz8<4 zLaHaT2mHZ%&F7Fc3kdkgeHUv?p5*IcLVsydVY@ z(&U!CmIeK5(^#tY#ga>@uXgyuCcx@(4g*Ho4P=Kdd_F;q) zdzU~NC?&AUiol9Ee<=oQj_muZ&&IUwa*gYM-9P9S0n|GA~s*BP4AHXhFt4*?cr z?(FWS_df2@vl{nCN*J}*izZnG989Q-NncDyI>X!k#)X{Y?nvId3E%QB#V#F#pJHCq zqWq%BTP$gr8dd5vb@f(h^7wh44i)lQ?*{pH($IkNC$5qXMj79;G!R>fw()1;7{}J2 zE`g-hhK8zW$vScYgvNqcUBUPGVH%<8?x;*J#4oI9bp%UWksi;DZ}L03@qgh!5LSiS zFaW6r%d0yhwe)Yi+Q|6qU*s8g1t>1hhoa$m-Td`rVqq}@P%ZGT3qno)ia=elda2Ux z)q*i-;c~(9>fan-KXHNXt^H%+v)_p3mX=$mr<2hn+%*nW5`b2V zBw23@+6Zxu2v>hRW=(s(&dGjH70#dJ_f6F}=56!4=CxfBLGQFc_)f0!VrGptn-AbE zws2dD#*YGFEK%d(Wr3UAB$(Fq47}bQuftv+_GQ?K;!Zab9bW+(hR=s+#{ACkdTlj>MzLI=w7Ftdv@l0 z(z-`!IiAsP==oPNSJeOgwE!(`h*Kx*E3iiSr0WSz@6fxvoYCQX1E|tR9UWr!_Oq8- zT3Q#5_yvdp_~35=Q;*J!wp5QIY54S5;S?pmJM)GQ8Rt_LAWmX!*{GAKZf@RRHuHO7 zj;Ef4ler6!d81}iD+_7Q>XXoxFr8K6A+`w!#(v}g?W@r0vMD+$m47>naaDH}XeQ_6fe+*h(T?KYIzon=D z`m9`>W&=ZQs^736I{Tc+pD;g|Xf(k5$DN0%tJR@;cp9@hlPzFmbnN#HX?wPNBY-!o zbn}kUlb~7`UDVaQ&c1l?n1l{F#pxsoC3=6oPhFj?^G!wA&7MB?b7epWzyG{8@68K# zELCA_slUng^MRJOEb57}KP~50e@{xC&0F9HJSy@XcHd_n=bNpzOkv?1z!uUhd#807 z09&l-m9mEVBJlJr9)T4+a{*n>dWGy5jT!omtzp1dasLfSRCI0Y1_+ls?@xqURYLb= z0u3&0ktwOZE&maPk9k>ntCum+tub^CAL{scG1U)F(AP807|3J)k)4V7d){_v81QKU z=Y4zq1O0D(vh}T>%Y+esQ z*+yyO@$S&a+C3kL8+O%2o63RL5YJ%);6#6BSaK3pv|qL0nfn#;N3u zJeQ5pVPmd?`-{PTIgjFZDRcfIjR4c}VBGc8J2#OG*xR%=54KV+rP}=}$@guZBzT?K zzr|dhblyWBmiI_-IYPzZA!9Q@uoV58tp7)iD(3RuW5hkvR5H(xqg6Dw)}FoB7jjUq zfMcj@{xiBxO>_UUFs-`wN=_0ApSzYReklo^uhcMj@OOmx5Twpzsv>AH(his6JO{hmtg)sd2! z%zdFQ_AraE_o6Ln&7iKB`bzFVk!vESIA1btA#JFuU`Q;9o#pIVy5sYn?}uqW3@M4f zAWr%_KTS5iB-}SvY{pMVpyQNH7!@{&gZO`B=LUz3UO{&T_X-@60kT)ml-`YB` z!g@aaR2pto5W*S|s=cq^w(x1)qhFSdmmdDTjX1T>dwk~+N*`LFy)O$aQopRzH*3C_ z)l^pf!TJt>LoG)wdZTxOFjt8ivX+jUW57tL6tQmYt{XR@KCwT-aepHH^mE^qZs&MF z%4vm=@D13TdpEIkxV&ff$Z^f-~_mT7B+%a$+P_=zeurU0X!5dZObG*_CkpKa%+z9hUzclE%d7 z+i0Rb1FI*tpfa}cXioiO;g-xdjG|0sn!kRp(-&D{ic$O)eJ9WoWob~98f>03s8Uul zd36$Xb?*b#cOmn(_j8L5Oe%j^^U3IwBivIjr5wr&_F&D`fn?d@g9~k4e3LUjc&#z@ z-U`#A-k|Ok=z1OecW%VT$zRH*vie2@X4jzT>LI(q;7&R_lWhzt-H5{<68 zPf!6M=F?N)&WJ7|DEh7yKM?CCbbFbrtjeSreae>uMAEXrxfaD7#G4<(kTaf2NtG8(5#2J#sWeDoo2c;WwClQ>hB@i*7z`=ajB`TSz@t;n6{h1}su^7?0<8bMoLG9`zNv-)r} z;Nqre;x6{%rOT69XHw6%gQ(J5!ImN6Rd~RGLPNc21hZ5oO-o{(6m=Ipk) z9y-P>sv=%BPWL^5VRD!q)`>~<79l+zpTv*1Tcm~f8)SYs{d8c;xHY;z_@k_F0187v z7X8ft^8YAhg1p@pV+F8IfakYyB<%voPe)@fA<5+eWZ=1$WtB>@8`2O zM>0_6InLW6c2aic+p4IWsaKBxZU#7g;B>(CyCV}oj^}d?*lC`DemqB@H%G=sYFAxG zV{FAkhLF6V*Dq8F?lQ~mnFd}Z_8F4Qj!ADx-cx9&x+H!Iac2A4%rJ!&#O18&laE-9=KHmBe3A8^ zc5ax{e+3Ixvs0B6 z(KnGJed8`aU{zGGXka%dAdq3<>U6Bg4iRB!h{?i9Ly!d?DdF^^?JB6SGKWQiC9}-u zU6HDRrR2_|^5%)Hait%F5T@rEPl~bDPgMCu3rE-7HzY$In@bg!MOr`4uUxDGzjqsSGSRe|Ekw1Q zC%kn^Xe=f$wkW~4X!p@31O{(3xpJ&G{y@oGJiH~~{Wt#54&y@eudpLOonT`>_Z4EW z>+ZWq@Me$FKE52%TFyk80&`TLTye&SKy04*GTZ$2pDSw2$zH8Onky-C7K-t?D7GvX ziOq8c)f8H2KS7P@aylnA1FQoa6Z?R%2i-nvJ+$Qdt}Jn@(=n5Q zBHj7w=`R#W7LpxM)Nb8vlKSghn2;VkC3#*1Qps6BiMGA1ufcMSU-g=awp8umOSCkY zg{s4vh(5Dzx$|1g(-J;-@ikOGyuhMwd}D$NxRSGErW`*B@o!)=5^QX(qFI%W)B@{P zXy>(TFMZ6S99W1|ekQfFf_tX&UG*(2q%>^l_Aw~#ynq;`so&8||A?>k7JyLSu;kLS zv&Um8gczBbe_4|?HqZ0!=gQe6alg|ulu}|s4-l9T+3qFnus6ylELIDW)AX!fU;_(A ztv4bQX#M*tvb(jvqvr2_EhykpAc6Aq+5?l5;W-3%(G72lzQt&n@G7!4>q?=>Q^x&XCvli0qUdXNt*`% z9I95!Rg30x?FGWTJ_|~SkjKB!wl<;8&dyKL(r#OQXbSX+E9I^ssrtF8!bG_^GyJ4I(Rr;p1WqpO9b1YnszBC<6JJTob2Z zrGrm{`EGOHIq4Cz%Nx{ifiu9=5WReTx3{)FIy-X!n^%GfPcW3yP z!RTF!(`pn;tzi`~hA{2U*k8BOD=IJbY`FB;?LaC^Mze}i$Zc;Dc4{yfW@yU^d z%@f-h8H9@c1S4}vTW+;4kgV|Q3s3_d5ZFiskW@9bwFuNKEa)sOEclTrb_2lH+&>wQ zrpb9)%Us@R3&m*Au+q(=1=qPbhFcToUt0wZ6TSUkOC3+U_1DSmXI-I0RNLLMI0|j` z3~qtKYF0}UXtApTDzYEV^ID~SQ_V8@@kkY{KaBT|ktot%2VU_iks^}7lg-9YV?Dmx zeQIC<9L*{MT3Zy{D*en5!Tm9a(5C1&xJNOSl@=_7ERB&9 zqHKgKxW}oo7H_R&umrU!V#CSBNeKaK+Opc?rwPQs);zn$4fwAmL*SaME0hhXkNA~V zA3U)uX64?#%=X@DlDW>7*^63N;<2SD)Fh9Uw(7p%lr2c^2N0(=Pm~zyp%=Q$> z9GQ4nrea4Mo~TE?7`HJ!b*2wH7AS6R{=v^M-}tA7RvVf#6LGQfm(k^UM5fC_i1?(6wrS#p_rqU`irYhjM*8cv zxMFSeeXE2ER^|XLr?~C=xzaRu2Wmi7jrPC`RM5P*K+y+%A!D34paPY$K>aT^%x2pKOljl3K3<~cAiPMaJZr4 z@()v%1WWjHWOWEpd4}4pkTqfJ2#13?AA#EA_~DUH*U3x#Sy8NZdjLwlAqS6!VF*0p7$|OJvh$%C5#KIy_Z^r+%8pjIw~p~h zixEZQ8E(VSfgvTTDk>jjWo0u@KPw6T#{2kf-h=0ou{GOuMR~DzDF2(p6?lhXnGKSWjd{R)r02UjW{!%L4UTt;T zzw!|l5!q!#2fWLGp56GD9|%99k7?g~TlnTT&f^}Eu`rx+6ZBKYVu zpURQELwG}!Fl<5Mox#RsRLEgLEfX%Txa^4Z$+8(M0KjNz@9|Sp!vz-Oh+D)+(E?13 z;ystOLkuvV1DvKTE1O?YF=nc#7u1;LJypVpPk&Sqqw_NwKc#Gnthh<0VMdJEhZivx z`-6pb?szJq`b$nm%y&KvBg6Q>(|;O#4gL5n1^rCK551M^T_)=3+asP`R+(seBv}Dc ztUjT0&uEq~iwi!2;~nfs)$!jyf^Y_QH9l8n*;J4|AsPn?~PYLd6n;Nq~Qp1-@(7bOVQ zJhh&n8OYbXE@xU%%cJI@!AC?^6IR>VdB(&a0jYZF<1xIqX@aKnQ=!kdBylUljI--Y)$E>Y-V91gO5=KyKw?c!ODmy%aq5d!I0TPf1$?jlW*!T%mpEdc1{9Kjc(sXPGGS|vHW4mP3Vs=!rp$bZi1O5 zz%xFP75ymPm641GcK;ydjs2}-$5SjR>Hp|2sZ)JCfbhkz-7rMLU(tyC{Ox=hd+i-j{Bz8cK?ONXOiPmq2nYZ^8qkSaI>YNLGCVw7R9pE!n z3c7agH|{7KVRLhVH4WTjfdpSLL7^>!8n)_1GQoq0P5>F8QZ zrVRVAzN6bQZBNObs))gJg50FlqMH>u4@y?rWwO>WcFPr8@7`#*CcML!F5*TYzhxY(M)Nzn0$0Ckj!*lzFD%JanG^Jdv%#T8q$8(ZrsRq2jw2wy9ZjbIE;CR zR+brEWbQzHEBYyiNlR$SzyfLcoL0IIW%W8h*D2Q^+7cPONfj&T-k#2>M{FBFk^rgn&{Z-dd!5`IC2NvLp4BQg~`(FNB2*e+QcYqq$w0hrun$wg7{Yw<6& z*7*P3MV(0}r_c4#u)b4Ae=ridjO@>&cZ`gT-)v?|aPaYGON{IRvwGU~B#MB5AncaS z@J$|ME^fQp1Ohpp26B_hMo{Z9^}&cLNJ7)`sHB89^I!w?XUjEW0UhB zp3mCHXA^8|SQg)Kjj6QNrlId0)yn6Xdp@=^{hVGd`Pm2s^fL5~1M^rW=~A0k9Hq$Z z#VGr8LVjs!$;;athkTVUR~pzg17uJ){{4#_@0#2uvgQ2}3X25Wpl-@6VzJ?Xx?}X3NG;n?}#*z*jrgXg+C2d@zzQKh~5;R`$@>W*M z7B#h7a#vvZr0Ptpmzz~Ev1zvCy}Z!={sqlCad*(&@A9UaHOmTCyEG*T!9iD{Rw+Z z!0Qp|;UvZcGQQiUd%=1?A@=jSCDl0-Wj{1sRi7Ogudkb#JsDsbsu_v*uZ@TCM`hY5M? zK^>JS*Re%y+MZf#T?v|l+AfE2&A7>=VN>RxeyZR?a0c8KL`)N1jZExc=}s<~+#mRMRrlfEM#~|Jds3ZoqcQxx&RpU>JG>NI ztSZXpQpJ-bY<{*$`<_Q6Q&Wgt-1w!hH0lw7;?a1n%M+IBf|2d6F){pQU^XnOE%4XM zmJ2UXAmQ%AgZ+5-hR>GFh~kPFuPc+0sNvanyc+|Bp)pSDHXkm0c5qOXk*R4lD}_Ja z0h6@;84UFHW}ltK>aU~#JWBr33e00h7k+=id+Q5&Sb~MletZ^Ak-?dz$KH3Hh8~gI zwY%82YySS)_7!uhR~LkQ{-QZVr>DL0xurUT3cQ2E=igm zu84p@oI(gLURu7Ep8%9_(Mn?s_V(xIbD&_74t;??3i$1y3W1a2I;u37 zNqAk~ZEl(XBN1v$w&?)~xx9P6Tq6{b)BH4WKZ6YYqquvu=_ApbF+e4lJp432a0aU= zC%!(jB|oZ27xH-1JiZLfSKnHKwJpIOV;*YwzRsX|W3H9UzLcCaUN+%j+-oFrk0s8CAjb>LJf2_+yh{I|Tq((i9?X z#VJR8>#;GM83gIah2I;>J2!bqSyi-!2xW_hmuHthsQ$-U!%FJZ($iMiiN0`kMDj{6 zgSjU>?6h3C1T%nHP22$wS4XRj=yrB?gz#I$D8eU=$_J4r5E9_%e0&6r17S#~rX0_J1AH?VmIr9L5L_Y_wGTzn@8;Tw>#=NzF3osW43&df6p70E3WUFk2`KKEV0}+S9zr;x8CVB7Qr}}*XA;ECzNIP;c+i+X^ z;a&St&e6;VnP3rpdos!@x_$Z2UHGP00~*-1;^(`8j)JKR?F;z(SJ`+6J5MY=Aqbh< zN&x8N{T1Oqw+v&ncpY*0+V&~>jFlt&DbgZy+v}eTy?f^G0>mg=R_@LfFQuaFcqhtd z+isoxOo0eKL}{7!C123zhDl+LlYn5RXBQWyn+5%TPrXkZq@zL>Zl0dijSuZNVvK>= za&(lGlm>3&a}bW)xBt8zXa|`exK#KupT$6m=h|0B&pYHDoG%PaRRW&(`Cq5k@4g@7 z$fH(}D3Wa-;~z9&Ux=aL)E<`8aQ=wIne4n z1jKXkVc#<0A|dU5!C)GFW*lm8Y>@6Vr;*cgf#!I(Ne_cOTgPwnXwVo6E2D-^GN)hZ z&3j{zR;yCz=0n`=DQJCOPb*)A!wv#fAQp?AZQyzN@E9TLF(NER!`bZ@JDDvcoyyk9 zm@H=$u*+-Yze^WOB{FtFz~Xh)2oluzQ9Zh}<2OOCuOx)4eou>>-dYSHdlg}WDJJDp z7(Chh>g?gF7SJ$;wfcwAPT$V$uKdo2c7CFz?VKm4>W7~{8i~u(lU1H;3`IdINFmiVFm7k2r%5l6qqD)m9gHC4J)DLaiLfE<`5qtBoW*~piGzz`7N?H3pF zM@#joM8;{uARy8Hn-*K_lc_1LWWOZ8fIvb<2K{NQ7RKc%oO0w!go~IUy?bN-Om9@G z)ZKKrbK>2e*6GrHipUKj2b?)@K7gmGJ4iD);Fyqq+{XD&*D9iK-3{hkN!YvajnseI zPJgOMo5*m>GRgjB*0FZzO?EoolXi?}C6ITnUKRWOL3ON+66KzD(rzJ(k&)iiqVuJ34ML<2G%*gQ-1&c zf}Gzg{v5RWEv^|^)cweB}M+4UZQn(z93Z@6)kUL~Zw8b0bYnljo}$XhKQ63i(+P)ty2al*_#s)zkziZ~F(_e0+0K?3t96*ZJG7N!6YmNO4oLn!FNz5>4#zl> zjiT4cygo_s^fq}!=S_*bijg!?*~r|C3ckBHZcQ^A0$J+&B;^{VZx4q}Vv^mu z^&&%0_a^!goR1b=pbf@>`j)kwikYB#8>pqg29a*9R(~uv+fhp%n!jB_-ui|3)9|9N zUG{>S_>(E}HH3dV5II>kOu0n4%}<$BQ|kI=F;1(}Vb&F>(c;Cfpef}qo#(oqzh2Z^ zdJ8-PkUH*+!2c5St6keG@uPUU5{-878{A8F?>|leOcO1Asgw^;S0m@P#^Odc=1$EW zNoDex4Sqi2f;M4|Vg)wSpKnUg1nx>~wPw!sJu?&iL%tTRcF>#3Hvq%MV+4ExkJA4-8V> z_=yAyIhr23SOg{n@K^9b-mqaYrK;xW`8B_6~UJ*s3gU$US@TdIWX@%Z3k$H$Yz)s{L@Oq^SVkv;hR)v~*&U zqR-?bs*6ic`>^@1_eL#iKkjYbYDSz8;K3iDy58=?h05;FOyPIWFQ&%4!4gLDnc`jKyAvbq2myAzML&Kb8py!vV zo!#hnTIul_V6*~I5J}JTF0|wR8i&FVhGC1lsi)SjdgK(p-r^AAiHhIdMX=ro%EDs97*oHII7V?+s8T@@g^^ z*DQMUMm{ctELv4t-Qkp+PN&BLPMk*>b~_943vhBWFpQ5C&8h0;y9Ks&s#HJ41 zUtsL7=b9`Sj069FH+%Mj{{!@PGFcX2dBM_2&0a(9V+;#xb^U$k6osm}H)gD#J=hu^ zRzvxmR#N3ZS1bB&*DTB>83YH+1jE(;vJTqET`jz{O=y?D9q;Fwwd3MlELx7|W5#Ur zYO|c1)egM^N+0w_ffVvA)jzT+s-v>)a#R=eT0g2%@@g%%LPiNu)Uf}VaJVa{#mHYj z*xbTgu%BXNR1{DtP8=iU_6-HdALhGRQBCw`BYy53hyhgNyN$APqYgYWM?}DrEjBCb zB4ZOM7{wtb{!(7fqN}S5{N0O+%F1&nbo;2)%>lw6b|>H}sSTR9kCQG1HMK(b=Vi57 zhjPzlyR9{xO$YaFcLAsm99M$KeSmzQt_H+j14@4N2tj$&dAcCel3k;Fp)gTf|2{8x z-Df9%cs7lJm+p@-@lGK{Sv74z-fvTQd_{$i-N7NZ)1ffTj1DlX-{i?41h2!$llh)i ze6D<9Ru|HZ0uPCbLib$vV5shZ)9Y2vZ)Htn-m^W6{Lb*c-~x~YRxmQkfq52A0W_^B z@Gar^!QNOvBH%pTZqL5+xn(sydA=1r0h32h>~tRs2W!lbQXN~oZ z&ZYbvoPn8HneMt|#>m_V)(UnVd9aF%2wD&j<`Wo=ZKL zJA6$K0NYGKCDD^zUVneuRe_p)%e9A7@wNMPlPJBr#Uj7D3%T>F8mnbPk@RyH?>Dka z>~Fc7lbno6q;6T|8Q;(2Ku-JGOHpR-tJYt%*x56#x4U8*v&TV7#U|bWzq;@)OnW@0 zfjsKmC}PGb>D}FF^T8@fckH&Pa9+GpIU+?5@;R!Lp#B&xp||pnfHL8(8&)gp-zrhAVnPxJYX0jcXXp9(RMA(6GJ z{L}czMG9I+?`j%YTY1e`r;!NgbSyzN3dqA|_xqKK#9d{65*Hq!%+$B0H7-4gWwDIX zcIyL^g>q$H)7QhI&>k+V-bVoF>O?vBi*OX=>I;yvz&@Q@vIqw*Qoh#C-a8M7qw)>d z#Ki_mAD$wqu{rXN(j?O(o!9B+2n`~cuGpUZY^>9&mp2U?LJ+(ES8QW$PN37kuU?+? z3f8KwSJK>q@CUd)pfBhn*riD_&9l+NLi>RYs5@bKB0oFs-LIK?P9Xu(2%xX?1FKwa zlL!8$$Hw%qD8j_dOnG3Pr}t9`_MySaILK)Gr0rxU+jH;YOal9*^N9iZ;e2(qNBZTZ zdjQ-5P0a6!RX*jt#U1y|tuuTOSZHJ^Di$(G=^wh}wze7cw2J-Vl2H%8V^pD7Bzoe_>+n}it$9d!GA^!wrlBSn)S+1G zVxaKx5EJ@>6V2Q2+72JER`G|W@K`PF0D}-e3K<2K0tJ1M&wcc}ct?jidp;Bn%ytOu zN3ESrdfsS9Tdr7l7AF>pj<$ia@HFN*d^=X(ePd5o3RRnwTp_R)*cIWqWC{qyX1d%- zh$)HDKYX8$RR8JFCBYfe{kv?=PRB!B+6#z|+wa#_NmF&}E*E?j+ML$mq|J@DrMcyb z#&OSgtxs}^$!tZlLMFbK(q_9%{@5-h7p=)${i3Ug%ixKq#t2cLA z*V)#KixY*&9{Jj7EM~a|lT^8}X6IB^GJv%@@k1~aJ_gFwh@(>$6ux*efGK6m-8yWL z=0DwPz}L1%Vm>|HQN6$|*d_z`+#o~uzC7bbiS~7a$Z?6;Mw!Y7$=kGb{Q3xO>8_Dx z-!3{%Q0=mxG?{cx=bg=U-|2UB)n_Q~z%XXls258q0^}kIfyCq|DXA@LiN|m9L)JW; zH-oh8=T``FjhMdI^UHF>!6XviV7McgQqc_X7!VPWZ*4})J=YRUjLEO*4atV;Sog+8 z-Lb*>%ArE)MzHb;4{0FQ*3t21@B;{jon@l*@a%G?KMbD+GXv*6m0-;V!oaTHcQsL9 zf=e@A$#_H2_l7wo7qwYs6dRhD7X-Zvyz^`zP*naw1AqP;=>g#sNu!>{`O=%*w1_&n zQZH6cp1*rSVtXf@p7o~({_XX<6o9_d3W$+-`g$VYVs60fatG(;lZocLsEIP4UqO#y z3>6BOkIJAvcY16=5xj8?R|8eD!7x{LC z+E}+&n{%T571)I-1m32<)`(g(Du=w#YA+Mq(a#ShFdJLo;i1r&9|5?R&qV9Dd!s;G z;7E8Tw<_xpBJd6`A65k@G94*X02(vKXTYC3c4=YK65a=!gI`wyeSIrLeyC!TNSp7d*pifsO1sz<}_-xN1Y8?prE z2*W0#qT^%AW!nDey)ocCK~HqHWp%I9(h0Xb6$}Z!8We7+415i+PrD$hqo{pBn6J_a zx$d?Xlt8?*#Cv!uIq36)CcNU&y29YaRzgdEQ#+HtwJD?FbeqnYZ<+7zSu)^aWiQxj zQ7<*=yi;pdD_=#Yh4`6eE^1e1Hw%34Ti#jj_0b9F6nM~nzha=NE{V z&eBRn|6W;X(`9!CMn*#(E91XsGgx&ZE_=Q$?4BIRl^xq#4rdXapinp&|;ssmka!WZ{K8WA<_&IlT*Y_iKKW3 zhQ7(9=`e}9-b}>XrM1D|_rOQNSGVO*0SHI7euasj?XiJwOs7ajiAF}W*yFxhodE61 z(|kSYV}_0bE8~jdT5@qj5t=xTk9rw=XCNnMaQbS64menivXy4t;%jES)T1ou2=Hx8j?PbdLkhc*4-Gz*=m zdvASLp%gU5?St*zQ=35PH@ta9JGk9CwG7!UQ-e&dLtaS`Wj|{`6yoBr=jZ3g zjg)0%P^+q{2xkmB9;o${I$Lfp9OvxEhS!z(P2@zV;#o;|)jQ-;>Ewn5eC)647`Iq=> z&%ejGUOYpGDZU;*e~Nj<3?8%&qxFA6mJSXv_4RJ`3vRubAPqaqT&>TT`rA84QSt?O zpT<$hf?N6CMxBeykdRXt*)1WS$G#WZkg&{eB|4d z1X;{*lhtiPuag%>$CPrsEpv3iV%(F$*f_A9cZgy;QlB5;9MREspXTE&KcTGT6I$@- z@bVit{GKTItijorNW~P*6uLneZ;*4eaKEY6R^#1gDPuZl*5lU#mEhz5Z=`f~S(YQt zO4P^2upH!=;Fgy^-Vxw3<2`T>MX`#n(fwLX1LmgkwDhzC!}x(o=hA8AQC1fFDU_C;j@Ih2byoPcoD>1S<|^WwpE=Os@WX<)z^>qte#nWW6A6nq1Q>;i zgE`HyF8AgYkWT*$?2rIv(iv6T@sPfwqszXen6>6H8g0-MHT>{t$4&{epuSV`yUHSwdGpT=Mnt@4^BHaS7Y@bI7d^Ue`?GRMcJ@1K9*?~hb$AtB*xNoxfV& z{iV^4o?^kue4ryIX^B9KL;baJYF@as%PHH+G&lj@OJc-A#6%*(xS$TXUjKN+K8A%K zr<*kJEYS#%wF4Hc|B33UnI+-?)h!w7%Kx_pzO#{%t{nzt{(&e=vQZL5sRo|guR9#8 zrn?rf1&26jC*z2xul-&jH$g%QJv30~z3BAIY|SeXp8pxR>=zbZlU94;(M&zV?0>54!K7v(UY| zK7_DsA`Up*thmlCor;0%_8J!}>Jhr7JbiXHL_N+sVkYm$&x9!tGLJy`-q!42rkm?m zyb1rQVxM{{9Pw6DP)Zmd?Q%*0TRr|`@Xgqm8ZdtRe*F3BLe(uA7k zaa|Y`Fv$e(BmiohT7eJpTMo196VAWK3K@@R{nu(|$~r8;4}&P&^=Hzm{MpeGhv|}! zR8sIKWt}AY%ZWMTu$%RuRvG3s6A%0awj4$lGMhw_{52bM0l|98Ye1DK9MxxJhdm`S z7d`PUN}WDVv;qe1 zI=hB>S~4<7&>@5H3w0p#f$+l8n%wGr?!d!Z}i(w-UCZR-v+qR9Hum|hvH7lb5c(l#26is47QR-ng}cS6DT?$kcxO@-f#-I9&tK9+qFc|1)}hQ5Jc;<$vj& zUXd~HTS&q%(cNOmHMi<;_jIXH&P=a*`9|z~(~JKT|E>8Wpm{(~L4}O|21_{zkSBvc z@)A(oo0hmmN-Zt)zrMBo?XLGM8Z>?B(M*eOWoIX6XU7iA+GlGwNIA4M=j0$vPfr&X z7k8)dK=8W=e68cgYz?-|8x~?wc1APB2~jgJG4g4D86I;H{Sd5g`|2mA7O5jO+Eeqy zhhn+X+H3%GQvTfFl`0&Qrove(5*PiyLeX@Q(&n z_g0FuSVyDaZje9W<^5Y4N6SkSOYXoJ)7ajHIHlE?u*NucfpYiQ^=kcCrpOUPFhyPA z-ASt3(^gk+Z|=dZO# zF+y2dQKja5*(tt)K>m=E{)oEOC6>J z%J$-9l#7WnAa1KD|0$)^an*Bhu*O#zE$xb1NGccRJEVZ~r26W7mCWW2zh$m$?_h_6WHV@WK`-) zDSfl(*84{M?D1%>vStr~*gcudw7xiAi#!@|r`Q~nh&%qsc8!Numq^3TU8qfp{=%c; zH+flOyn(*9Nuvio7jw!G*x}IC7QJ@nxW^YSQG^~|2wiyYc8EB_oAB7qocp1A>wUn8 z4>V{nXjGjRC$jJ)n+!h+REm~GI5G$elTGHow@srrB9iYAQvqsRfM61!7_@|SCSGU7 z4P`6r{|A-oMG}hYlJ%zwJ2s(awJGzssWF@qsQo+JSTPngQ49&1P)dlhh_&yOI!X`R zy-nJwqTP77rHi8wr<#kCm@$DSah!`F_CFdh5|Pk9vHRb z81?6t+0RjG^^2vk1HFH|_j&r^zg9^vMHJP)wFTsltvD>$GE6yde)?oi-Q+O|CJZAuak2lr&OC89H-tBhOPy*w|-x$tzu_ zerLdq$+nj3qxpI&U){Egv#of!x0i?=VV?q<nJL6U z@bPiouKlL)W9hXi%At{O#q4R0Vkmt(Z~J6NS&AkCsIqrX{9~blLHN-y_04+Sa)u+$ z?g7f}dRDw;9H8=X+Qy++-?>!{|9!({I^?3MW?4o#b;kzfO-)TTsngHVC<5f2d?#!J zOpSMtSP{7%M1HGU+@FBMstP*qL=sT=7@T(g9vdgOI6D#^di@@-^~^Tjcgd*mV+Y?@ z2v%%HS}LyIE19-&UfmshV)rjFA4><5gkh?dDfM1VM(|)tYgq7O=CP@h7;h7-8pw)+ z_~)lR?2QmFlTc1@6p*xioqUr`m(R;EqR9_dI2?T?jlYR(8X(=C8oMGudOQ7|-2&ut?F zRO4>%a1PbrAz(rtp5CaYJhr%CE5;8B-eubf$admHpt!?zqo#u z+5zqG_LOs93~q_Fd6kulQ1`kdaTeqrObv%3vIQ*jDf=5Co(KeDFSJj+|s2A*mE73VAZsCBgLm9dX7N zkPr~ODx%E?DZW#@=Nc$UpX0Y9G|Y{Regg=rV%k=~9;N>i>4|{Gjw(89@t9xthvgV- z_9ev|tZMK9;+zEW{w6}+zXv(*TX%~jiQ#qtso>qu_@SV^W0#&B<79(5VdPG$icPl` zYIn)&*IB!UwrS|JowiMHNdpyLRnKl>>VsI|p7I-EHr+P(284U0It`TPI8KFBcKpEe z0~4SS_^011zEjP>jV_e2&}4GBStEH<9ZbAk)@hq&-1AucAg=cKVoNa}+2l=dpKVhQ z{EuVD!LDl)Ya)e+NFL1hz~mdwjfLBag{PiEDqVwPNTWgnHb=Hue&;ktSj58UF z^X#kShxwTCgV39l0or#I)~EV@|HkO%Z(3{|K)nj*X*r=#jQcO`m>j%aLfOi@nl&p}~_gO7}$kU=o&;6d{K z^5y+#>C^LlWZv%#W}@lKL<97wN0De|M`AgVxpO6+@W_aWV|W(F(SMjRD>X9UEK0lZSM zx~2x8zrhA_8h!x*)4iFpMt~fX$9jVCnE(?lVjz~hOd)DnXyFL86H6!zBKL^Cud+1! z3?OKXO>Go4y4}}#*g_2&cfO}NEd}o#uicXpzcNU&0El-;0FK-gUW*8Ni_nk|I#$-= zet)6Q8*c#0_Evzq$Eisiyp#s;7$tES%#=hTBk}&7EZV#~>gcbxhaDgH;CUN<<0Zou z-cEOM8WGuE1<0YgCsn1QVGb<^zy9KQ@}dVu4M;QR22ziOzxb;y@OBr+sTn1cf5RQo z4uR~u5NhCl*0!U$5o0JZcF*!g+kZKG2zE3TCdlJvHW}{3HhCtvOpS+t$e(};42ZBQ z#84IKrW`?y&H%14$V{mo09s-dPvR^U|6eaaaWOtVegHr=#=ybx{$zcKTa}tZ(2k6R z%Qy-smahRkmch`??t_Mvn1U?7=6(xm(J$zGvlacPn@};MBY?{_h$RAG^~0;Q8qe*b z_WI(G9{(Y_$l5g={WLwbdz0kYpo+luQh~06HGrAn2Bpm?w#vKz);`XW*oY7ZKpxuv z`2GhOpfoXVqNYkrdcB;grU;}pVXMb2e25AU@0_>HIygJv{;L_bHJTM02KFO+9(q{) zigr8)iKXM_uGF!Rc)CCH*oaqbt~(UjP7e}0Y(ye8`ZAs^Q4e`B-1zx32BND2NO~<3 zF?KdGlo|3)1F4Tuzy#vQ@_1oOWyh%^DT5?D&~i@J;V!OKT?}<)o}H>~@qx$%uU+gO z)Q4)}d^Py8ZgMXp$O*_;f`68M0!i!3QPk6%U@!k1{|N#`_H-lla;v^|7N)|NCxA6L zDguI?hwXO`6_!%XdH)l*M-9q>`MW&cY(oHIFEz#Y6kqx;7UL5eSqgHO@K=T)H^=6T zq4URo8(vveo^U85Y5I7IYb6vbc8Qi$5rE<>`Mp}p(Z)K}(^TzT*u#l(?;5;Ybae=mB6I+Y`|?17|WseE-5 z5rHXwF{L^1_wT;5NQ>LSbNJu-u@aVZcPuNsH5x`mv~e;qR|Prwmltc+trx`hH!D+b zop>WJf_1^Dfpz?gT^oF`(NzV3=E;$Dzg1QhDSoldX^Em3hP6;;>izls`z*vZGg!iW zDc}p=TDkGBFV2qk!jjvf#Xc>M*WdRk39U(AAQ)b6g|OJN?CEJc1N4ex7@|>~=%Jqe z_+(_^MVbZO=DdIuDV>lIDPRdsB0ZxXqPt8wCw?pE%7hae(yjBU@%8a!kal$@d>Yc=Z8YbQz)6_yLkQ|$tZMDle4TV9#G>FoM? znpAsA0zF*}nK<=CD;uoy`$LlRB>vRlq4l$~dd+~KQTkQ^m^WbuddNp6a0E4z&k)J%HpB!AOI2=A_r!dN26 zc+8N8pipl^prDAWU2gcTHw68G!g$BRZ();2@;?RUU7GD*XqZI{8!o&6 zai+uv`-WW1`4-lnVNrbhRACXnJ$l4F&|svSBK~5}|DmFP>1G@w+CwLf4w@d8iCVvo zmILLZh)3@vvXFDlK*{ykA7_zKqhq2&bWplP-;`*A_^s7zAYgyPm2tJ_`?D@7sFh1G zh(S=Po6)umL`MV{?|@hFKzO)ha^ceD*CYDOzCn(0ATVZA2LRrE0Pktw*4C%`{BA%}6B~VR&dXa=XVkoWhqyf{ApN)Lf=U5ET$0C1qeGT*Z zB493S)>wA%1el`L&yfRzZ+5Hw0zoNZ(K2H>AN7Ysyxb-KuD&jl@A>}YUPCRLCWo#W z@-!pNh@K02GZgFlg33<1@?@@l-_7_MK#_MH2LVIo*L{IL<1Gc+PFmDW?!Z-+GK%p1 zaMx_~>4Pq@mYNJldNZW?ZaQ;&dwaba%GuNCypB&mfDqXm=)U4h3Z$q$Vb`5NF*Ghc zeZR^MK)rO^yj)|UnCR%8AD?b@7J=W=ZT|9#mzTG7h9_;@=o9= ziYSVLA_mykW_gH}_kC5M0QYv$fWT@6e!cYTwIaTp$-?Z}qGPG2#b6ytC<>_bo)dgF z@}`Rqd><1>%kju9iR=F0qB#qf-<~go3qnKPS;9W!x7s9+-hWg9iUZ+x zoC{HLcdxTwd>AXa0REtlp_+Wzw-Oh*Qc9OR|B^1X@3U_Jd*JREYQh95)?BfXM4|w} z)4EtL7|==vguUQUt`0kc8PN|$j(`6AS*_})=@C0>4Z2$K2LnB-!1T#$O-({fOiVx< z{^aR#*d#-@_BVtl?j!d*DHmG^g{I zK7i$ETtqBR^U%q>Pp0)~9~Je+y0|EBMelMkNbmgtn3P|)uzCNU59{SNcWt>%V=K zC(frgOxUcWk+1Gp16LD-uc>i-C>iL*^CN3I(Ze*mEC1>iPG<)FBq5oWE)j8)sVG32 z$Y|vFAG8%uy>cr4=xo|vSxM3DJc}!GO$~U|C%wxX-qyww`9fsk@+=RD=e)sfsS1nF zIOS``Z?P65l@zKO!PJMM-Cv=eE;W>!b<0cjwtpRj$L_dvcxw-mut4v%8&t29Lc`%Z zJie^5zv~bW{w9)9kbHyhz}q+*y5^BPy?qb>-Slq^w}J&Nhb3EXTtsBG4!`;@ANko> zua-uT*YRNJBb5Fyn+T@0W0 z2alEq(!AeB7g|ps_I!_TEC7A@_VO38T~`BTsL4p}&BC*eyp*2~_ z>+_>xsSWq1?Tj{tw_^zHkivewFjQzA;dbnu^Jd=ZbI@z zxme=VN%#r?3{DTV4E+yCj0`^89aV07qA4RC=;TEw`108!i^1<2O z9=y0ZXeRyf_<}??(d$PkXj_T2QWd{?#NcJ{=8KC=w<;+=g9QmiCzDZvEc-~ngEl)V z>|&QbOfI;6WV2=I*-QJ_uP7$9tNybi>=9QTUnEt6psvuiavi{W-I|u4+N;a@EgU7O z0@MGX#|`9m!D|TF zGr3)CwuJE?i93>v=Mesx1cc8 zPFO@)!3K{>icSi4_OFHU0$!?-%a)%2Q0L3qlHWD$m$BJCJHKP&pWS?$-#*<`J}B{T zgrLBnlag|+s*aY7G^!jF6M9KFZ%99L7}>eIl4s&tc)gX2P{#QS*YlO3DIcsb06$Wuk?d`hk!IlB1xHZeu`%+Ln*$ zSW%?Wx`-MHvvl>lfxd0Mw_s9@a=X{2ids>Q3dgEp5X;9HS3sL$#rRZ0fijISz=0LG z#B?ehzfy=^5%7;BUhl`w?49Z4LZz$la|u2^A^-Qgp6ywmn1RkWv75s&pA0XTEv`Qw zB}=f5lGz2n)pq9hrWX}4*OPWGqv@^lqQRyh8p<648Ux7m#izLT9B1dMJU2eQiTkgi z0P`Dm)F0AWKZLZ&P>U=M*`PKAX=ui6i%JI;*yx6!8VprX_f>Cz-WQy!f&y@XA32@fKbcPyJ)^LdY7N_|4o#oyue8eepD#B}6t{2%X@JG-t|xU2V~1Wt!HdusRFiYy1M8YDq#S4VgkgnY9)!$D-SQWNHc#% zL{I>un)F~Y5&m?vuc$xbw(z4Cq=ue$mxux+Hin6Ihv}&KsAmxJnlK8z2I`7KNU9wj)R! zB$vt%pOF>4IAtiv3M_D-pYXM8>`|wCFH?)t@v4k&S*Ao6vuftzp2Go9VH5T^HmP{L z91%O(Kff?)(f0!N_4d*U2-K!!mcg`(V;cu1)-Tz<9gIcR`+fq`|2y&5h zcl(B~o#C@B-~<4VS$8;$AtHCQ<-nFdRWiBje^)Rdz*2wVO;lP#L5TqJRLJY!dmHS< zRBur%@T`+dFdhIP6moKsvlN*Ivaa4TEdq9`p2xk4n#tgp1k^$df3 zb-ePA+>>m^6-c6>VSE->KP{jI3JtAo*%?5`LP6Y877W;SxiJscLs?Eee<+VvTAy+Y z%h5sjX6RjklO(I(np9VtW6RfNo{s{(Kl&*GDnVFx5dK70b6vbTx|HEGNj2K(e~G0F z;S1*MW&PN*k5b?SAGvXpOeOR@U}AJD}! zFmOyEu9`PaD-_Q<}ZS}3zG0+2)H_smR>{>(V=03V95J( z|NS5P--z7uXnFO^5$jU1A969jIm7>7a)nbGiHf9TVnPd`m2fBFRVQKE=h|dYZr%YO z@W95E2=KENt-{L>+zVs_@g!GhLENZ2_;%v;94$s>NI(J-KWbg4Z&jzCto zvD#3J^|h-NqVEt~DQRcUB&_PH0)dp0DjUqW3=<+9Nhspzkr=AZE;DGLp$fPm3cpWu z51qs+_iHTXKAU6VQS>}7ddYf0EF`W#b3Uu!iG0-X8e2h9U$H-((PmDo-)l2jrHItT*;b>d#oq44>t=b}vf$b-zjvo7mqtvI~Qvqm8 z>79S_J!$Wj8M}|WDXbc*@GLc08Lu=iytyc0c`yj%3)bs5RDu+W}iYxB-DgwFj>uamMv z>?szI@thctGECv`s)wpu{;46AfF0Z_A}e(6)_SqGeBMt4%CwCakp}RjC6K-s&Z5Yf zin!Nd9*yZ^hW$}?p9WH5VYi6hwn>*YmQ#PRn3`>o{CgLi;?$DX`kO14$C#hid?&{Z zcif(HCcDh(7s~Fc)kJ?n5(Pf>ORepj3SQa_L5Hg{6O}hzHP>G(g4ZVw$Sn6iO^qE zKlb@Rsd)bXxs7u7v8l7#w@mjf%t)K;X)1%6NrWAh#Uvy;r{)0oyt6StxI;AK!M?ug z&OZjM@9{|@IIp+~6KmOMY`f({TCANy^l#LLcO57IU~CWFj!lit!tzsSt82Ia$*64X zUiCGpyjlWa-P6lxpxFMD75S@@QT%uq$sUR~P1Ox#w={b*`Zgzr_3p6xw$)mcYme=I z7D3VmgUh&8x<>2M_t&CFw$E=MBNoGiBfzg`WBlt{9^nf;jcX5VmAALk9=`Cs#(%?+ zw~G!II2In>doWM{0S|+d_LiQ9JFJC#M(V89)$?1@}01AEga<4S-eS}z#*&f>=2HV2vy^&h$HeK1I|J3v>X zP2jW$L3qhx1qft|S9Z~iwWR5u;YWIIBSB+$X43Gqwbxp4vL9;cymxNE^LgOSz44`Y z2X$m|KI`K3V6?~8sr$)o;B3`#TL@6~1WHxxcXmH#Z)amVNSx`Mb?>{(B*CiuATGpo z51wq-%$ssA#B^b>ubCM=sMDe0KXW}e?bR<1_=A~t4WdJ~gJTsB+eQc9wgQAZpiSTw zXwh$NXoSJ&Dc(%vO_;jRpIo$85VM^D-?-B1Fa2L%2xt4q9mVtV(n#LH>!p_ra%s&@ z=6cCUrAF5EuE`+S5qptsoUH6#{si?a_0&g3Cp3NHe5hfzcvqe)zRIA14e3&m+P;%MXY%r`8QSPbQ=jqOZPmY0@4bWqFRTwKjH3aFzO-ye-J`fF~$mmeUs}xx?SHWz+#Ug;iBf3?rkGU}%S4%%IZu zlZn3M+w=4SmT+BOQ;hccD;!Rbbwr@n4>_u7ITO8a8J16T-Xm>T>%HMxyYU=_4E8JE ze!YLkqq$x|=t0Gdvw(n)8FX`S?ut4|t09+wOLJQsgsCfO1{M7D@p{U#5~I z7X2-jewKIDs=HsP$$5kyNjEtemq{UrUON!``Y$$twwD#gJ_v(J6+LNK*;mOhC1+zOh$RK{ZO9))=GDK zv9HHTGHI35`Rhf|gBje}Cz~UkXG3ex=Mcma9`U6LZvus`f_Ic*z`Y;Z(H`Yq-yXEb zY#0mtT`BO|f3*_y!UZGgz0r(mY(Pxyh@w9)TD+KsKXP)1yGU+O?#Xx}SSD}6Bjm>Q zX3uH(+7>?-4^A2Uj|l(8i$I|6^W3=!)C&~lNza7RiS2lWTBHqoMr&=4Ym_L1WXmmIcy9QO<7j``Q#_f#h@N}*ZtYrskW^-^?+**|XQX#!$gcF5zpvD%Un8A)+mgE%w|Og45!|w54jF!IE3|SCo%H%@ z<*Y}i{mIBO()#)ofX>OBDHrRrbJmcvf5UN^zkjg_@Zg*wI8>V0UTO=nWblIzukk6d zgx2C+z&qS2Qhtx?0}&(LIN+54iB0cS;IogHLJA$q`-V!iUvqd#DcyMA909gpwqOPx z79`~{jI^>Y9h0=ZfmDmIr2jM@nR)Tg*v46(`>4#bvdQk*`}-C{frbIuswOhL%o~%I zyXDNgPb*S}yXO;kq?2@1mhiqn%_0Jbk&XVKw>CF~tNtF&-YCgZ4hlt#9FPcoIz$@fQ)R zF>7eAO^X|L%jU%G_$U)pSd%xpe;*k}95mSX12MxKbH)lZ67b2G93n^0pHn|2&s@up)0lL76Ad_ z>;=WJI%GUzA*TE0iM+{TYe?%51q7;TKC)hZJ>X zlpF?6`r56&N`r=?wflh_*?o`<9$Qh=zpA5Sq4(@z8TZSsNCtN|zm7c}CG$c}%4(vr zy{za0{`1$HmiNj(L(^U^A!3Ll6<_x2}byi?*?m%S&sLMx8D{HSy zUwp0SjcQI({HoETsjgDt*Fd1h9Cp$a_;3F*HEtXzAMf{$b{Fz_h1}qT31yFHY+two zKG5_5enBnS^LRPaz$4(v+CKLnAG-^w|9N5yub9){pSRt74rZUyuKZJeicUgUUxsOl zoBm%f0Fd;ddI$A4U-`J=(Ie8g&XhIGtk?UmY*jgUnTrC}yybxWnnD>Tzd*(D=4p$9 z{ppj)ZfWM-)67f*J`H{h5PV5jjQ0R1Y`?VTq;E?IAduCSCi8oBj}p23@z)&sNe?7+ z9rzhcA)~x%%0#7)3MB39%R_U&)m0x!2K36oGb;|eCSEDWll*kse;or0Nfv64%EEov z`0xlsC*6rp%`e>lwjzBq{l>9bFxr*u-C>@>EGm!j`C}*#6Me4cPy|?eb$R;A&35Te zgxoKZf?LHn00^@F^YTsZABC4dzMA##(CTujSL``1A(YRj|Do{X^?49n&(MaHk^8@{ zo+p#%bTuo~9QbW|4I0H7;9t+LQ@&O{AcL0Um;NV59g2n*HVv1sPv7{Vw#ph{Bq{jf z{@>VmrP#jl-Vi7G?{W?PH~pwNP~J~6OhvSVnXx|x9LNRADYTkO7Lu4KUJ&HseuetJ zKB6GEa^lYhdLURESd56wT&tWKThW$kU=l3%ZQ&(N}RZc4dqGAmBx6*H0?zY_;t5)hd4&0*)y9QR{ zmmK!eE2-x8>n~vodJ(8Mx(1f07Oy!<+F<$#>g(b;Wk|qX*mRUp9z?SwMcni;{+H&{ z=Z*y1ymnUIk|-=WqbX|7%|+zmZwtV5mxoou_#DIPM#QjzS*^96Tf8deh3WeJ)D2tw z&6q&wCiwydCFP0zMZ4s71)h9F&-j3*bc$X1sfN77M#r?x#TQl`a)K3Kt`gP*_eAKnsWbE4^zyNJ zsyXu2z- z!F3(0FfP`1FA8Q_R8tF^0nhqJQI}KOL(5f~C|))UmXxKXrdHq4`a|J1O>fCOQZ#I?YSjf{^qj6 zYN6~ojcA5?o2hhef#;en4rflP^dCxk0x&+tcTOUOIs`9uEOjC}r5APyNH{83tos)n zD|n^TR!(~VeS!~@+VB39O8zL^_r3q(wEmFOdl@POK?7HYQRdXMV(neg)IXJMeg$f! zrx1c>lPE0eh{@NQ&iZdK>1+FFlGUw+Ajvn}Za)p$!BZls|6TI-HB&elX;RGmcpeWA zJ)wS%BUrO*7xXTgEVPKA2L!sKh&(acG}P~s zQ2Ke`cEOiT^8U fJ&RU5D>Zca=n|5kl>f#wK?V5I3>n< z_k{Y9UEeknWmT~cAv>qobHJglh%D2om(SBEx`T&4iF~ori=-Ze1!0D_(ygDk-UzmS zxTa1u6cj`UKs8V{@sw_gFw&V>L1*^FCCp7$;A^9xryvRAyB3%5$VplW9++MQ>#EW6h*;mB@9?K`M6m z&mKO|>yrj{#5Pi3gbpAWkE)4F?uwQU7<~I^^LgHBN@^eU7Sz%gV}EFGWpkSOvZ%eC z#x0FHE`|eEw2a~X{fhQI@UpUU0L8R?`>{77U!;ZE06QB=n6{DsAe-3V9Fp|<-xpC} z2rkw&LxuV6u5!RSiO$RiGsPR+IlJX`6`~Kso~1Bzt8O=_8!%zwT?WX9V>K+{3~+Qn zcqW>!G$d>+Fj0dh`ZAa{r+50Uq5eG$m`uuCu4amOak=L0)t2!cL}$>m8){%{C4r9 zsEwL$lXz&JsC*e7oA0PMgp%>1uyCr`q2jRG|$wXw)ilIg!=th+*`z4IZ z9*<+mx<>7ZII8VB2&)n_s1#)7Er+523krZKeQLG!Peu#&`Q+n)dI;4=&nP@e6BTN) z*KNNFGL}&EpWH}0p14mYD#2=*CnShh?Iou_z7Ds2U>!i>(41c6Jx5HZ=1Y2Rf&Eok zWh#4Oa=Gb8yzw=u>*P`Je(1Ys0M~yZxnb~3xT2zAG=B7x+iWY7J!ih0c@57q4xeBh z)&HQEj5BcJ(k#ntb39fl{{}Pa({nD_rO0M;e0s3$T%q4$ zXl`ePxZILO@_|)SDm(zPs;v>&pMtR=mFr69;W20S@6P$r<0Cq-SFhkY8B>%b|5a#w z)>MnO=>@nysqIH}eS|L^$Y2#*dspKjq@4SHr`$ptwsH--9++3l#T3!b>m?+ya=3>S&B0o23(R^!ySkO*LMKi3=9hG-6cM!EZJh6v>&cLV-9r8DYkW6q?oBCT}mP zrUbR&-?K7%@R_qOx|VL0*9$HTQnvX3|1f1KV zp&yMKX}R*1Z=@L~jQS#CmILtEQ{i@kl7M|itB%5WgB=dkUdDYA!y-y~~%y2@vS% zc;0ja%qp=!e~jlQE+5I?HgFYODdI=Liw}*ahq5d$s3T_hnE8nTHsrK)*+HXK)~c{4bK-qAtb}ZY879;zx0)BN|n{&L&Rumo^}z>Q+kWqrj=Id zUn?UPGaUG{59Y7L!BPpM_s~GlQrOeY;GL-e| zhGpAX$t905_`nQWQ{;4*EJHE;hoLxm;VcjzIbBTaoJo%b#HB6_i$ySa zlomo>eSIlEEir22zYmdi7;JAF^Baca*>8R^7+rAH?DogQF(l zNlX^Ie=nKxZWO3x4M7fZ(55?wuYx~d0b|FqJ- z*{-7TvSYyJsq^K;q>vx7c7x?;UmRK5RnigRhO~pN-q1uw|*=|BwEzZTzWrD2l9f0I72eq z0DnP&ZHVbBHCDp^VC>$o5vr1NzLkG zG0!C|!tLC4Ng4V%kqx>9Q$&A0+l8tzy2V7Vn2o8Usuvg@w2wlEdWJ$gDgTR=OKcR8 z>@Zz<@G9qNuO_8Ea0{qShp^3KDZG)MBaHFR9DB<@Hw-g2@lxq{?iwzgeD&9kzG&Fn0!<1-3-N>LU>N;*faa3|-XMB+>*ATVGiFQl{=@ zmz8KZJ+*a!k#_>0Gn9WD;n-2<`t^CnP3r#yaw%5qJ(ddYj(0{Y=+#N{bv;{l@y`A{ zFUrZ;c}8y-k(_-5&r=D&*i1HB~wvaR0lHk!?P(cb#_ zT#Vkm67Z$cgAn){BNfvk3%a70HA)y1UIspTzl4mKFc!OTedJBfb{;iI1j2YfM*2>paG~Q-08-Rn?9&7fYu0qdGY>J#iKFv zF1Mf^yB+S4NGi1(8m=f__sfP>K~7VuksjU)Z2t4ROgfUdIDv^DeKn8YhKgi4=0!zC^Uhhgt`N`z#~ScYc0TcH zRX^rXjX)^BxIYQ25PzA0F&gcM>*~7PX2iSr3^qX6~+`v z!OxjpzveJj$%2W=qDJJcE-j-z^?9$|e@vTwk0eq}ZS~3}W?OC4Wp8=g(}C`yWmNFk z`wKet7_!Z3#kpdyI~zVZ!C=}un!5V>wIDpnfY&+C(^a_c)O~a?I?KnEf_sT1G>))d zGiT4NX`9}g`m;Hk`Q0>P!VChMn@R!oCJ(-gpGp!1)22ZJtTdEoD=EiKlCN$!UVC|G zr*DMpJRa;?3jgj{OL&<)XnHS{w{a`$9?=7|5Ly|Yem`BB*nMw5hU@lz*VUYte9iu| zPvzOuYW3xwm}uDeR1h9sYfug)DYFDHW#8eZeTu|`1OVytGwyUj*z|Hgl9Ia}8qQvR zX9Sv_gV6B0Kh+F3uOJMN4U8f)U+6G;7c#%*(4lKAq4sCJhf_ORmmD<5NF6?QjGdDM zmjqX)og~6j4ZKv|jr>AA#RiU`)uJ}1;j5pI76wH+<;lNrL+bZIkGj`LQ}x=>9N|J& zX0VU`(QCQx1diWj>C*0{+!K9;UjBb=ePvYD-4iYd0tN?BMDkDqBBcjJ8VNyZ3nZjL zq+7b>pp*)TfaFmSY3WW0Y3Y*gZs|M6SN`j+b?+=b@L``{?b)-RnLYE&Aud5jVFhm# z!D_dvb~eMkhfrp-x4++if!AEV`a-aS)@JJto*I2?gsW!=;(#p1O+R|Q$HchWW99hf zb)`ef#y0|5Y^v5WlxBz>LW+KP@)c%$AI0PfXB7^d_ z!sK2^Hb2H=peGuO!uLhjF;;S4@TQL+>Jj-*a`g+A7Yob9Zelk;wovkSet}=l zciD9n9^wvm+YpT0cmHMD`=Qam{|WlkDbEbP^A35st45V1^i()q30jsuCKwf)Cd$M) z$S_g-#P2l|=Yf=Et{5O~|JCjN{X2>ef7dsk0Z(z3d!obthgON-!=yWjg6FeN8JkLY zXmsAj74zh8z_I_>yWm_{MkA7sTd>*gsZ7Rk?c?Tx^+#Y42^vf-%9Kr7u@ni(XQ)Xd z`E*n=c0mk(@)69Az)&m>j>t4}Q6QTTf%XX!7oBKRkkPg8GqR^8T$Jql?W&1dd9`?D zGf3=KlYYrbR>+(fE(>mU(&CJeprlaK#9EgI7lFjXHK@YC(592a~ z9<|a8afVk;A3*>0(DEn4cGQp?5@1?FFX6mjUx0LJGVL|O0*99d)|-05IkS{=p4Aq0 zZ5i%}l55RTd&?#QKIcU2OV=hhOiWDLySi%q3eQ2k(B{@wl`)@QXOBYGSrxvsySW69 z2#*}y92DyJL-s_Fz2~UrN-p|+Scspf^b`HxHjk5}ULtB=WF$$Ke9 zxSjQ>Dyx^v?47`rhgeDeF$!qdi6|uz%ZVGjzU=1a*7v**{V4xYYcAcFW1^IwkDsPz zi1b6)I9X6hx4^J~$NbR&OImlDeRP~f!Ju`@)Wsu%-B2F$NF&PM^6r-o4-YL2Eq>h^ zd+sA4W$0oUlO;Lxfy7*litbUeL7!*?o*@Zc(C5)`NADd4NjHK zH)UKeSc?@ab!m~)%MZl@XgPz)BXT$VHVU~a5YeN1@rqgJjLE2;{mP)qlo(UiX%L)L z7eV`Tp%;D*nNRbPy(HJyX*i7ixR`M)h7j-*{%xAt@uxD3yas-P!D_|UXsDZjl|lFci@5qmCQ zM3Sprxxck9A}Sx=G}*Vdj_@UXq zasS|2ntHY(gWb2KinAPsl~TR^Y4E4yb9Wy_^h)Ox8PuKTGJ`R7t)2MoG)`MuHouhe zLWltC^{O3hHB^i*NirSdYEX~`6`=SS2DB$^rwRlX|2&e+t}oMr$X$aF@@&PGVWa6qCBn`SIsumHA!B3r)x}Xemj0}Ky2JvX=I=G zY{e6zDY9Tu1FQiIl_k4dTC}5Zu)MhDtym zoWUKYTYYN4_xqmv#e{G1FW7y4o7B=-($v0i+xl9c(WV)qq_tZ+;ZXJ$XGrhPy^9;K zN4pEpTgyhCO2vE_-7Q3uF46HBPUY~MVNI9AoMOmcZVY_XB$c<-bZxbwo?ACb=%pz6 z4SSceiAMuIRh8heKZQ_=JTvXTB>+D#2-*?iOP98sa8c_@2zXX`@DDee65YGdk21?)MZ?$Oy>u~BhoR*X%V;QJL=t#_BS zC#+lq(#B`cjefgqaj^?Vm}6XOkXu%D|*=d~SLDxoCe{F3kGcp zb=C34wTDal;$a@w$WTT5I~N|@saNKXYH^-ze)e_2vW8h754P%w;w0IG>K(dW(Idm| z3VP+s{7dGQ*oR39%vTMdL;hh9E%`%VASWA2K=Cu^=s*XJjf3G9rR?MxeGkn-X9a5(tiHtiY zj@Or!3h6>PUC#sX^y@3i=1E-qt3Rw~5V$R6Eyg$V;|IHfa2IfB5UptJjf7Yunn=H% z^<`1^`S_ONsxR}Mbmg{&;v3lbGghK>C(Ta%{g06mt0sS?7W?h~3U;af6)bA}PyRon zuh%q=#LyH0Uo}1q3L|uWc`0{VQlm*HMM50Gg(^^$1G#6ehnw{D#9{psa6ScOB6a5DU)Wbe}7_SSW<{3xYIXF0(ftN6g)r^T)6>MER}S#(Orv7t9vlKjF2i&qj}*m~{&?(#cIN+EKw@+^|jlHpfp=kukq z2dUkT+x^cUJbgNw&*ce^FD{M?t(mdr0xQGjdwYBIMBWM1IXd!1!%=asmL!y~0*|2?A9Zz2O}68jo-uEfnzqNuI=%Q= zMPg@TBl^9-nm%GSOcss>=~-E4GD>Ak@~Dc6iW*v4$`A)q7h%L-L;sk)FIC zBX_ZzMtP(Z%ibP=_kUCJ=nmOk&0@PVaF9wtMdb}>TU9lp>84E)4Ma|As>0+6JUl$G zRH+?_caa4)3kwU94<3*T3JKkOxvZI{p7+xUfs@s5R_M6K2x%L-dFr_5S1?Fw>HucB z4|md4N@6l0N5r}8x8bHId_2%#a!z&M+FGfQt+bRBaW^-SW+R?O2cdz1fuZ2n0Rgte zG=?Z&UtjF){6j(_ge{enNT8s%V|bWT;G1sO=wp_ePXe}CQ0vmllKpa#Du^H@C8c%0ePG}XlT2`YW+n;D>(d+~;*;-* z!|$|yl}oeGN@k^{t*su9M3_FZ;CUNzCU&-Rwb~w35RgvXZ_H(!>(dkgOj z*>ZUu1Jj8=WOI@>ZHxzSGxG8EaFziFl1vCPMp@{7%%EQOC>ViS$Y1qc@sUWV1O>cC z*$y<3%^K%A{@qkuR`x6!Lb_PNwK`MCx>45B z(!!MqeL8O5xZzn`EV$U0U87G=f&&JVt88d#X$b>%#tP_8?q^OY+_o69l7ohj(P4Lx z6k4hArKOrErR2^aiUzzIjkAyu%j05Jg+L%`lhD75 zoqBa?CER1Fc}LU8y7zpf4~5PI&ImGztgd!Vd+au8OS=+{&Q(<}ZV9;|!epIjRUC=BQ(*VoYZ&b~SYT{0$|m0rAHt3yo)aC5hPk>8kUlX>{?*VmN| zyX7x(5sKI1D+E}N9blEbui0C9EG6Y@;H;-N?!Pf7lX`rQgS?40yxW+!IH5rgR%8afZARI`0a1$uI0tK&Q^$t&gV-MeFDwOZ@L^W`EC z;sDX6AN1;$mk%^hF0r1FhMb9+Q;vUAD~u^RZ;g+Suhvmp#%E`fg5wH*+HPCngVrI! z>;6stp3wnK%*@R9MbE!&n(E7aaVb|leX!c@VVVlNtehP4m7hR=2~}|KT;o8`rdy() zWjxZ>CW%p;32&>#t~9&3hxqvTU?pJY62bH?6}Gmvl3tWg(dxxu2sH%4TV_Rb*-}L{ANuxfB@CbB8vrPKGKm zUcGA2+~0 zbcH73OtBFsC?E+;HwB;NZ{g`X?ktd$*9V07>gwuOH=q*+C*_dh7r+jsK$Qi(r{B`N zx&Z?Va^LyQp7W*`SSgE-uhf4A1!}9+=eX%_+<5yBH@(Yz{pL-z9CsP7p33YEt(|&051Pox$W9eeNK2QYb0Y&D4HSG%VI=TQ%zWVcCE>>xt$-2{5OAtphgGc) zJc&eS&z_*FNp_(1%zDwu0#nhvR}35B;z$J@*zEVar%#^-UXIuvRQdW(c3`4NX;R01 z_&JI0o&sfIW9QJzR}wc(B7qeb7+Kh!A&!776Dtboz(1X)mNOl!(iZnZ>OpOl58(rC zZSKfpz*tU(wBBn}3zFe$1RIwwBavK@t+D^%yhOHq*WL9JMu;Or%RkK$2L;rM7a_9p zyJ3^jW4F)FT{2WgE1V~raszuXjes|RqQ+8SOqraSX)6@Gztzn153weyC4ByjAHntP zeY3`QgEL5{$#;}$+$ph6mWZ6^C!q7ucvs3JC}4#c-Or!vg;H<4p$4k|c_RSO$*A-)h z*&+*cs!joASdTva$lzeWe{`|@=x}HdoDd(61H9m(Pm}_FFj9Wp+1>`k*Tt!nt|G}5 z0lQS;=ZNrdU9g78$-dgG&HqfOog@dp$!>OOs#)}kFa3fP&8~I z1NSIDT(XpFeG(_@O+PwX<%2|bkq{Fn6c$oM-?nsU(#SQ&0*K*Xsi~>fO9Qt+-6g*~ ztQxa|=4Uk~dBN^0f7qq}q7&XC@EKr9e#%r$SAW&qEDzy!=>16u3}Vn8m6(_qSa)Lb zZ3qYms@vMy%x4{4TuPcu(^UdM=Sq+E79cIlt`Y7+ee+N0p(c~l_-j_Hm9m{gFqGQv zyUh6Fo*l+gz-j^^{iLl}NF$4fFl6N_qhp)!#`%kvE?H0gyi6nNN`cef)8m(s!37Uh zSbCe4l@K;quDI74bQkUH>;g-KX|VOfAXmjr+;{E;6byeWDH+_&k;YVw{|I1+XnGeG zHhD1&3JP9uvf^B2VF{k&hYs)plfwom1z8svX=z3Pf&jVL*x0zHit|KR)jgD)FnRsy zhW8W|rEP3blQsg-N?OyeLykBPxM2ITbn(FGvn*{i{sZUts+UCCV}E`A`uO@x8|(ml z(Ji}sJ$KhEL@ZEElr%KL(>r27%Y4ir-5N%WMX>u zOZc=(K|pY@L+cvcxv$8U6^`e!00~7fbOS-lC9WA@&>8aXbXhKX-nl#C`a{$0j+dz;el9Y2XzQn30i_C#I&F$}fRGHI0hX;)(zUJM&r`0Vbvz zW=5Gs07A6pkwQO3Jv}|)Sjhxs9avzFdvnQvQ-?@1Zv!Qd zFc8VX23uq&QmYYh4)_4ElI-P$8x|f84j}^z3jyr4`p9bc`7X9kVjjQs-2oNxnf2XZ zn_7fmF6{dBMAfu}y@}Tn)6VT?jsJF8q1|G5O-}@@%-(=mr7_|vDm@>``vyy1a?dX;h)mB+ zPbbF5>uByMz+%wL<}b*AeiX3u01TrwxzGTRU@JLs$&9b_(;%Q7FeWLfj+2{@j~uxB z&=nQO&qfqs5?I2MM87RFwF)oPEKOe-i)on(<3n_dkilz zWN1Hl^au~r;a^#KheNaQ0BhPaLxJG691EMFGDv>g&sE9JonOCU;uN8$ z$C2g3;%bn0mTYDs;g_80_yZw!?LpwkC0WK{qEyNR$AvsSuxLo>2xY*eP zP9hc^kNfgWuR!RX#V!~&H`w&C$4!4_oe$H&KXylyi3v|hboVOG|AH**e9$Vb!2mVX z=x2k9JP#1!X}(z`PtgCaNrt}xQ1E?fXUJ9fyzDAzcL0RR6+dJNZS2F+-(ltV&T`2L wm-00-C!AB#{LP3;@V$^+5i9m literal 100836 zcmYg%1yCJZuq_F}HNoB8-62SVySux)LvXj??iSn~lHl&{9^B=if8f6N>K8?E>J+nQ zPw(#4t5^4g$;pT!z+%IKfq@~2i-{+8I8^7MG>8Z*m*wF1%g&mW=C5*R`tqB-Z9s@BF?<<8T}qj}}|Gu3bMdKYkI zwMWEaiJ=Olf#iOuQ^Tb3QboYk-s|&cF)XP@V)}bhgW~tR?vX!J9Zwy{y+e*#*Y0K$ z<}^qGk+-U6icvr9aECVO2`x?=2f74L1S^%=!)5!+LZPCCfn*1tJSUGs%E6qxdr`_(v=+_)3^GtJipcx2Y+ zzzVP=X1jXH74dinmYz&jdx~$W?9sQZl$W|f&_ru4&QOp__jw}ag!(jJ&2mnw6T`~* zzMQ_alCa3RUT?SROwQtc`5K69K6kNJB&W4(HQ%cGF(1DYVlNrRaNC+Cn)$V7pdrm)RMQ>7~t zZur2n6*0@%dY~rW)`ql?%Y`%Z2&LlbYcvP544FD=cQb2DuRhg+#Ir0l8HhxniDWj; zYsx)7aIj{H8kT%f#KNRR)G=i^*+YDsOF<(x2}=3$%Rqcls0IPGZ znKie+!1Ht72obCNDBq9N9h*#x9IMGOf@L9ge#$k%7FfNdL}}q@c|Ri(!Rh`k`WHet z$)cZE8S)5C*yhGxSiQaUhImSElkJupMf>afzX3+Qm;Y^QqCNI_I8Jn60vVzyEwZ}F zVO3cnkR1FKVtwg69j1)O=phf-svov5)w=q*;9fs@JWfA_s_3>w%rHgf;QDxLcnR7k zx;tRGaP<*H`=j>Pm@@KAwo1M|(fA984DH1pBy^vIV7=YIng1M^mGvGW-B;a?<;ox) zD&!j;jXB3KNIBwu?kOEQr7;kATuOx^=KHPVJA0L0ttms!`TXiulKv4hyw?RMKDIv! zZ-*m4lAy5CxPu~vsEl`YUXz`#ot78nrGGnnFNI}J6Nvmxw0qxFdyorF8ijr5u)fZ*+e&B9CN)@_|5vL*!*%{_P4XFkkk=HAGsDFW+U8WPfDlYhy#&(djfV5BwM_u zQn$Gm+mS18IV-Ht39?_Z>??TQyivq&tW0A7qzrcvULSPkhVRSs{ADj$CJy=S+OOTE zXmxI{32H3$3yKW;y_5I$!ajc3jbPqz# zgBCd^oadm&xg(OvPn@QopUD;xyV-=T{3T#Hb5w{!S=++u zIlX$V2Akg=tvAX_SynV`x0rm|=je!OD3_U`+Yq~?m%1VVc;0u^@UB!Brf8S%-niqV zlWB)}k}dzHsQuZd8HbMkHK2M-b$%s7V>^Q~z+!m_>8=%Ps`iGCIe{hT;(9#@JH z<~_p(eol~IkkdyHDF>Tp&FGF{=VL8eMSsW5PuRqXBT?*;zaDwFxj5Mnymm5tc`qZ5 z3BgQ9yHcjp)_O_CUp~&ZVEFoN2lH)gr760V-y|9~$^-7-p1I1o>s6b-cqUA>^pSmD z`^DNT*n1)Q{sat+MVNxg5hVWf0&1&XzST2}5rQ-|<_iI0px0WH0BgE%&loxhh@jtj zKg|khtkywvjj( zVEeFL7FzUVnQm&mT^eC3ZiUs1hHLN<3pJ&Q{p>j>BGatL54SzbauB?O4)w#v&c%{% z7r`JB4*x<&Rk8s$GnV7qrya|36&F0yWj?XmV`%X&S6+}S&)9bu#5|)S`vBm1FqPgz z3SKFL+4LVJ`a%D?Kw>FRuuz%rSJRw!^)z-kGOLG}Y zjpcXR6PYv@X~AF5WJ(VP5Uf*Z6)pUgT^lTh4*2^Ftr0&SmNR#xC*MgH4A5#c(;Ut zg(ziBH;IofG0uJ@i!JW7qq@iK=(WkicP(GFU~$80i;=N+6oH?Pw9^(W`=OK|TU1_e zBQ?J3AHOcTA|2KIWzwuD*kZ>7mZ&EXB9>8bnj`OyGA9C8(ZVF=pk*dmHh213j>HSr zSPuwRth%fJ*#k%cnTzJinhz0)^h(JX`$86}e1GRD5rb`~fw5>FZ)> zgjNNcGg}>_wCBjU`CL7VT}Rv*t+GYPMXF(4z<=3VaFb^Ww67s(kYVE8FL5+MavV2l zW&i;X0k9YOuVC`X4gQhWr7^{PMe}3b*dbuGW`|gurvrB+Fe&_2#?gaZVsGZq$LMqs z#6ia1yFUhDhb0Jqpm6{t9t#36i_&YVMsn(WTOYEHLG5<1mOuBQlW_4PdjVUbpMoI@ z-h1k4VUl*D<}nppW67*Bq*+qZ-JzV-8^6D!%%Zd`q~Jh^%oD??q`;gqbgwN5_P zB{GuJB~^(UlA0L%qh$;sXnVpRV2{jvB=~>JYM)CMg#I%j-n0YJYeP_?0-hxLSkY!2>%i zFLB{9{vVYQxzAs<8uHZ6s-6H$A?g;iNLE=2P#yR4?n^N%BxbF<|>B!?6Wr1{SPf6@5mQj@K5V`k#%p#k>eQ5Z0k)l)YI>6lHBq zQRZj=u|7W~`e797s>@s|j4kwL!Gv7+JdMew7cQU9Js9+{u2?dbq%AYobGEFb^Q}IC z8vALM5m+Vpykq$jy{sDt-pN~wKl`*m!a;?Oc`giXjb+mGJvqF?#%C4cyAzM# zGKx&;8CY~&SXS)e`c#vo$u@m7qzVuUOr|ingmBOG=NjP=*g1IK2JF>C_4F-IwdFS9 z(`ps1VIzH@Y1RhigRFIdFZPGxEmVPY7p|4Th7MC zoa+6EWz%%BWaPn6c=wYV?#X!thtFsgIi=UTqcYuEEo<2L96SQ&010<@fry`VyO3qJ z8^G`y+vQ>%mre-yMrzb^8)#Y8_it-wX7duxU5FZN(D&)I)4j9#mEP=S9fJ4{;b6&N zJs6^%g`~q{ap#NFS%Lk-hdOuLlhPR|8nLz(ChuA!O5i~j(btCqJXJ6BhT){dwmt(8 zLeD+z?R9#?(_DTEPhZB9nYzjCu!J=Y!h{X7R;w>+bGKR_esa0f72B;vStBEQqey*3 zfdwn@uZvHnSyq~(wL=i?zC#cqBVg*|Sjxyu3B-E$3+bnO$voL_Ug_Zg1A`CbsE;#v z8kRomwc%Y(<=Z2dEnaSiZ6%k-FL-SUxq+_E}b*QH@gLUwfG^x_-Ua zl%f>}HZPX*bx*Rxj#^rtu&T1$=}TX+VA=9^G59jiaMx3hH&olQwo-4MQP@>ff!OPT zjH|Q~h-vKX$@kPXP@nZm;-EE_D1lU@iu^o+a#7#cozJp7?tAtE7jx>(B{2DnDNz6O zjisT;r8X~Z$0a4Z8Qn>;ey3K~;?68mBiVHcEI}hRdZ16b% zK0iNV$+x_i{BKO4;wP4u5cEZ3i9nXQW| z3zX3yu#aY}>hnOw9L7UcUyP)LsbB#|)U#ZBY9@JCo#)I(b6fcxD?Aj9&`gNB;>4t~uA#=uF z-BqlC{7qy6G85wzIAE51{T~OoZJpLpeJp%N{0}w7T0uipi+(JxDRn85HjFo;t0{|EH7OpbR^N^mnPjrL zDej@sgSVS!Z$GY~_PAu3h+|i6GPZO)RvxPbout_WSo@uy8gNRbOfd(gj8tmL`OQ zADW9K4J}EsHfuU;+EyrlExhf0OGIv7L$tdyxcgM7(yxw(=fV@DJ&d;T0nbMOCvP%N z7Gb~^R_N^STCVIoT40*6T9|OeIaZs=1i^7}e!Z{_P-8&9!K1yFqFxKS1Wi3R@>i|G zlq9iWSyO+(OVFGXRx^Ax z(C|MWV|1y?JVU_Rpo*D#QQ#ipHsXx^-}UqC6AgTH3D^!;Z%R?={0KFLHveR&YfPs| zDI^Y7)1Vr-Qi}ZcK5{1s8-!s_RG&)3^7s6B?+O9sn*_(xOoTyVxUq=;g7_MX>|n?F z1vue%stMDh7(Qi{#1+Qbk*I$h8g(XJb-1Qi%d+Ym?oQ#sqZD-#})~D1z9Abc`_S~f2|Hr z2dtDx^+NeD_g)E!iPg5c86a=(a%zVDefQ?WbMTskvVYnW5sj>WZVBTpX%^|#5nWq2z6!R# z0Gz`jUNW8wf5Cs?0Xu*VIsH@oXA(qZ5i2@5a;<(&h_E?If{85QSJpVc^LZCP$Uj z;P|88_k~a~IR0+}+;>K;Na)B5VxJ3iQ+=!II*kI;7}VX(dHS0N5}#f@DGrGcJo^I( z79(~8&?Gu>+g-~(!qWY9dQnpyY4t}Mc?4_7)p**A^9+2Bf7Y@;K!Zvg5FKF28va`f zXCBHv(7`PQW9yDOOXe(ST3pq$rP?CX~sN2nJsJLM)iT>RNA!p9gc*0O;D zN1`74ufm4vBp73@v9S!FzDnA{#n`%K)W_rBGSN=wjp+7$i)Yv}gcJDwt=c~_UJ&WY zk(7;%t^M`+OyC@OpEAm7-g(fZ?)v7Y!#D=!Dg22F5LVR8#}!B@1MB*)(&F>xh3ZHvS>DAB|;Qh6C|uhOzRT!-_758 zy-9;ce}c5c5m6VsU3|iH0Lr3*pRwY`EbIM5#ol0>N4Ey~J5M~q&8vIc>(Diuc-tuY z$p^z~d2ghZg{x|lm)@-TN17Z076;4md0qo~@-V_tLy>u{VLxK$bs>Yiib@~byq4nt z%BvD}uVD<|cTrI&jx{%uGtc$M0h2n4YilKmP4~5Gq11OnLKV$fg?|s@Mr`Qd`@|Q8 z1|K`>5iq{N3fZX?J?L%EE<)@kF=?u^`H5pfGLQZnC+go?Yvx68$vcl| zL0eMsx#Uq>G0Y~TaU>!Ud&iA346j5s&w0Bw+3{}nOC*Cdi!`@?-AM8{6~?A_W8EAo ziM^NZ!z~VdiBjQG#8yEU^`b-KQAbp!b?Udri^Ambxg8yudUpRZLX4QjEiGyA@bG%J zFXTCfV$x|3-1GT^-}fF=ufOpjz58>K^@{L#F#24d0Va6Z?+M1Lu59Z`XsYR*-CAZZ zTX{DeEN~k0O=r(?yLG}Rhx_0B1lTJj!j;sbZkm`h`kO}=2#ZhvB?yHrA4pybMIp#B3WwOC?+lpz%U&y@|0g8HKAWii7sY%Heva@SlD*FZ9_l9l3;$Pd+NiNu9>` z3kbxC+)kNrWz4(sr)M)(8j$uvpP#ft38(gY+hhb13N-H(FJPqe;NCWw~bM*logX znCXxYNWZ;5q)*aIIvSEJF?1zo2q5$%_tEh+?vS%?cd+p&J{ZyNwY?M0xN|+xZAAES zz00-15QGo?Tkfg32pYdM7Z#gQv|6NvLJh1sgj{RWvtM`jg*!v6IDJ-vVkh6p5y|fF z#$anVH>qWwla#Y=eJ*rOoTl0i{kA~BaVJA!g*CFVjSiPC1TCd$(Q%xSO&M<2i51$0t(C$>MiGO#pQS;%K~Dp;uL z2I0;#M#_8Rm3fOAq0s_>sgn359JIWrxTG6yirZERRl&OJy-T}N8{5D#_w&Zs7=jiE zIlIljMR4kJbw3E6cSSRJwzZOEG{dbI{`vV+qle04)6xx1yW)nNp4ucR7}YHbW3Ne2 zSuKC3`o@V%jtYxVW^a>5uGwg%e|Ng7rlK!xU%^&bSV*hYj6Bcx;t~CxV8yWP4qFw& zNQwJdz2N6Af+fY<10gigt4-#x9aLma;{X9(V++v|86>i3nZm- zqQ%yHq4yh64{J`3kP00DE9WDX&fA76c^#dx-DF)ELS)R#Gytx`(P;i19&RmJR_zYa zRFNio9$n5e!Z%tk_hs>Urm&hT2=YXv7YzdcrdDed_J#qRn^c^}t5kvUo82<}97tE! zlJUAlO&j6ojQ!UM*b3T~g;e4R6ZQFF&eL<7Y0OC)(Dsp_jr`Xec_5ffbRYTsU1CGl z-BL79TOcl{p-Uo_YY$njt_}1Y z0>{b^eN3q=UaWq5nJ3L(s9F2^%PxPq``?c;^k`Z*+in*=pjTU)wMan(FT{*Kg#Vwo z8f|hucYH2&yuP&fpuQ*T8tsPr$myA>39;Xad*ItPqZN##;(r|dWS4FX=?)<3QKt8k z`f8JaoM$F9@Z!zy)3tX8^5rsvDKMfRwAFc$o;-m7QWSQS!)ePtB2P8yRJ=W}1FqU`D)u@shn*ewuFH|Ux>9rHiC90wt+)YUpi)A0xca_MPoBifYF3g&2Xe~Y%n-%|4 zD;g{8)ASb@6%{314=Ht7U1<5p==xiWI2)0Y27dZ|)kx1GC5xHvuD7QY$xTE-cG0d^ z7?C3XXHHwZ0(F@4K`og1JVZjw)|}*DYsSiScUfVfnkZ$vWAGmcYwLTh7zy4bQ)T&W zBl*Lxqj=CgX3j$;i|xtcrKsBc1SdXSE67A57siOjb{5;EnBc}K#T|dnEGs6>m}P=* z9_Qj6jXy*+!M$q?gtAmc{o7I!g`t}03ExAv3^XABJ*Cm}4~sJHnAWY2&t&npTBp}) zMZb_ZY135LC*<+R=dxkvw6Y56@=E`<^`FZ@-7WllTTixt11*;o{?ag9>vtS>*LNj-8n{yJVZ*sau_C;XjC%Q?bl`f-fm>`>qY zHWBF0FJa(w>zcNZ6f~8Fzg)zcoX?sZFRv;Tnnz3OXcImCGH$-Jf;&&!8tiG?o-Die zDt_v=i3s1V*W0fGw&517Rka2Arva_ai#Yw-&sUTn*5>Yv23 zpK=ANgx`^eSth}9dBWctE!u8Ocsw5AT#~U@&J-qOT2v4j(j5T`rk1(!Yb=eTIY#e|v8DFRJCVydpZS>*YTsaP@jexs9= zl_Rf!uET<2qv?(l_}!hPhS*7}B-n}kddG&iTh|8fy?m37w+)y;Unvd=FnD=RyR z*!!MH^Wlk3d<2&N>JPa(%hwhr?#Us82?zKU0o08PJGcAH)SEW=jE=pGx!%N;GVmjL zM?Hin)h+yW8YVVzbb#!iF|okRTx4sXF8J;*LZ|{b_*&0|NS#j`euI{f8^VA4)mRVd zEbSbc<#&*Z-a!kRvO43>1ZWwZtG;X09M#SXF6#@)_#RdVuytz+*CwnMv*g#NHq~+qk0S`WY1&JZ@9>&|{3`_ku)9ZR`bM6b#Q7TXx4@q-CMJiCAgV%dbDQx+o zrOa0Ucki4=*FCs@3CmI(r1)07d@H8b`u5FDuSGj^z2wF}72|wnu9z{A3d0PQXs7k4 zl85!G<+S+|@-(YGuBlRg(i;mR*%VY2$FMZ1)Zd&7T4)g#9#F%XtL@18b5y5OTvi{! zv~C8F=^dn_q6Eyljt-7oXl`t zbI))bb@z8szpx2UrARj7P?e7sNR-7~YZ_LJW*K8J|}w-+@vC9gm2 zaj#O-0qC#?lQpC=l9x6M{i;xZ%kSmcISBzgPuA%4u=iz@lj>_r9}PQ zNVb6Q0?w6r1r;6L?q&!^*w?62d3)@PjK(ggx& z;z|<_SAi_wH`q(5P#F z6SrUXZy+)S{c!dDg|fh2Ehw3ZiaiVMl| zXe(~pQQaQTVlH?46tw#@eJZ7mUc{yMkU#SwMy-!&c7oM-g0Ygh@-ZOlg$=BxUc$Mf9@xoO+%%sso^p<* zj9|p!1I*tICw$L=_2w>~9ie`_hXm z4Prr2@xWg{qO22!A$3c zsE;gvUi8&F^cttD+m=J=55V(qRjT>8#^W0jpB)OAKBp=aV4laBR#+vpRLD>{n zy&+u#*R=;Lp4%N#;nF|hoZEMXnA{e~-iHMaR%c-d1KtMy9s4sKhT}eqP(&L8yKW`GY1cK{)%=>X5x8(59 z^hk2q><+ZhC7b*4@ZsTL+}$chwytb6&2x3#r(VAny*Z8L~gv+>yavyhbIwZ7atlvdgltk>QfN+SS4@iWO; z43n|m#9AE2yOHl0t?c$wz%hIXS6hBSLm`=PbSae3P+ekTiJv29ie{LE|2w_j@{;>1YZUO@Y$Wo6qG0w9*Yo! z+qHJ411wH_F(jZYlOjd1MzdDBRZ;--1{DvnfrT$&)lAsN;;wfa8St`HIy7{>XRRcs zg+TGT59Pvbdw7JyYtl3>>PYD{@Us*4n{IQAc0*fiAux(ak*4gaVsCxbKx?k{m#wZ+ z-+tEsn;5}use88D^PKr-pCc>w?$gycU?02J=R1N%D5Ir%^O^a10Khp9-9Pt#V=G}R zGLN?S$o*qVBS{vRr6EE8nXQ0~IW^Qjwn)3s`s1?mvip`DmzPM3iT%+Dem})rH?52P zIk7x35r-m>(;lJzsyIPrbLLR7#iCC+L(7|)iy>qi&mCHcJ~_YYtU`5VN&_XL7y8r3 zd#_IY@elT^C|!5zRNpiKS!l%pE>}+M?P#RLz_9c@$RjxDib!>s3n+F{X8VKtSt$4d zq#!>FwXdoL_xguJc&WdnaCN@iR5|WpnGW^x zMRvaM+ca-N{@t|Oypy5_W*8zfESukG&^Jx=&k{k=i|&-DSh~R&=2uq7?oBHEiu52{7Cr_!GR@}Po0VVJFc2!tct?4j+OLGXESt;(2Vt$$( z!yAIw*9%=j^AUnq_?n!hNL!np)bI#YENFM6SHw3BwIdM)PW+DhE`;@iZ$*VYK)9nP z9R-`Uk=0?%@T1e@V}U`GW-O( zK(SeyOwQv<*cv!H_%Hy8w_yGx2@ubmUuEH3Jo_7%!rdX4sHvCmG9!X5u;aIKHvEw( zR4Bgb1lCcsVMYi6y?lHit9;ZNt>_^sNJ&d}QQ{stxFhN5dy~^Faq`&QjsorfzZ-@; z0LjR^9b4#{SuNbk9_4@*b`a(@4)r7WJeu(!jkN}$M*l9@O@gVaDh5y}AfT*;N`Bxk zlQUO83;+=-EEKwl)$?FjpWHZH;6x-6hn5S~7kbEnCMWucwR4y4cGYyliSA(Q%x`ZN_LcZd5_^YDpg;=L#}qo$ILi`WyBl8&ty@&8DpuYaf6EQmCI zDJCVZ*K7T-cl~)0{m%fVXlB_aRd0fbf1lA$KriWNl&-R#-o0WXiTgd32I}xWh7M6% zSsu0|BMZ3>1ay*`YUy-k0O{b==fDPc{EscD#pR-5q%tGZ)(ISEnZsfI z@NuAp%WmT^zG`nKSaGmbo%cjPogL5?-mYj>T!qM-zM;?_uUpP8ENt41&DLf=V@{f3 z_`^&G)VV95txVFoL2w4U4PjI7(Bek^-rKPgs3Auv8_s7)Y)R~v{Y1}&>4p$8#bmo1 z|FV{&!nTFEmZ+a!Vctkq!{ic?UqM4xB7wk_p&V{9gNz z+fG@!^r4JHuJVQYkFrRa@taLg?Y5JTYBlB%C*CLD*e-SmW=-}{~}gyFFxRVdItH*erxt0vLh?fkACuE zK0mqhO6N&hk6WkN*)ZimT=6;_UAcRCOwndbPZL8>-lUc?Mx?s4l(1|*YHho}Zpp^y z?SDRdO=o!tK5M_@@!HO_mDAQ3H6-wktkS+`Gf^EId!^51w(0n%Qj{n={MrfL<>*|% z?MsiJF=Yp<+jV|4lRTq{S88I>t906WN>88Ryh+sYm4N$M_>G{1tlst9Xz@Fb6}z1a zBkgHEeY?((@ObNoeU+eDiA~p~(fPwlEe5H$1NqT!4ml#Iw1P6ff=-4#p%YW(1)p(>)7wFG;O+@Q$>>QXDfGGZitq*l~~7QlN$MQ zUKg7TQ|PulcBS=WO+$Eg4a0L|69|kJg|B_5YpYLypZE`POd#1zaJX_@C5CkUJ>$RUF;N-;LjYbH z;h<@%z_lv0sQ3de>j`Q>PKyhP@%k{*KmNe^`n=?O^_cAIOXwAAd+&Smw`$8g#@G8& z7s<2o5$E|$0aSJA`WqRJCB=~KGth5!vNF`gcVo+%z;5?qy`MiRyyp!hH;RCCM|TJN%_Rf-C@ZJu~>n2loH| z@7w|b_pPiPAUzsfLkS#;i)e-=&;T^ z&|5!i@ns~JCvW-uh3sq}YW(Sko9|O7QurbDp_)=tsOGC3O~-*wng!x2l7!``ciGuC z3zApxC|-I^WbP{Bpw`dV=e5^Sq^cJpjFHxjYG?oTOUIkMGSG|h`hH{|%l%k(SGVvH z0XFmcPLZOGf%lPzPwQoI`)k4-!3%c!`BU^h%#xr`=hgvT@|y-@C}qiq%!^o|T;8la zVdAHe?N*ipkWZ^eoHgyi%KS!7zHvhk-|(P2m(Oiy_GTS0YJq~jlj!r%tdNvcb2Gjd3389Ua8=87Wnl|!vGw8mMMHNPklw`;Gdfb?NyN{H3XS20J zij_iTe!d)15_wDplbAJTFh$0@;9+q{NWq4KK1iNC)NF`Wt$#7wj;DzM^ zXmrH^7Mh27b9;L^RhfKW9PxTUH_SKhtbQABti^q%z;Aa5)wk<~J5H1-$74_e~sa@m=y@t?+)&8JxHHcj?UIuv8W81J|bjo*y9o}jb}X*p2kuwV7n>L=iM zx(NM{OS9N3din?rB$EC5)M7kNQ-oRN*^#PDSytrTJT8pfhP{XQDgh_VB&a%SlQ?^3 zVerWJo|8;>$dnpP#6~5d8}gm8SxKn~7VI=DBm6gW@svupG!es(W$S;%aPB?4MClSV z!9S!46Xm7Srs688B@4;v`5QWJ?l%X?YfAA@J_?ub)3_KK^1 z#U5ZXDv?tz2#?s)`h)!OfPujF;20y^V5!T(Fa1uUA-p|y^3R-NiAZi#f9>djHJuti69B*|!tuRfNty7mZN6^+yO}6?HBLvPsY9lOi<5I7 z9O8ls<{x9tBb|SRV7>cJ+R#K=MI1EH&e7^8sW)fPY&^~9Lj5}d%9fQSrxZ6`L=BtR zy_z&!b^!F2-u@q>o*Y;CT~DDvGW^41(h08GW_7gEcQU>nz0EcA?a1vP36lMOgzL|t zqG~C)>;prP_;8E!UB$*9=VG1y)cA2D@$N50$+G)x!(EcDztXe$URhe>Nfw}b0++Mf zG98Pux}OP}vmJdM=y|wZcRUGEo$j2kkYZlI*Snq`M=G);`$ns)d%SiY*B#u)obF=}P8#H&=4kGR`Pw&lzAD)zNt}+^ zECor|LFaWgc(#kPAMC8R7K)JV`|qm#p+!}l+nP$&#^(#FR@=SCOZ5SgmV19ERGZ$Uq^?Xr9fO%-Vqq@|3 z)ys2L=4_svsHy7kn)Vn{UW8Wfe{K4}_57M4qx-9>WY3?n!uaGn&_y;N0#QLgot!C6 zs|!y@3#;I#&b+b#=$Y;k=ZnA^;80AaKd!BxB$uk&zT~tnQ7Q z?&BXeW&tnMG z30~mcN5zZ>p>VgM1ssvCxE_6x)&&I|Av?Rerb%20N8pEcyJ7l%vkf4&?{8QZamvlR zfuCJ;9biI2B_WQ%XLNC$dl+tb-U@b2zg#tNJruz4T^^~SRBT?v@Q;b#P@p-13g4z$ zzUNf5N6C@*T(3^skAl}d*B91!T6wMVj|7HIK}k5a&GR2tmvuuD5N@$C?W>N zp`1305d!#^`ANMf-)lv0v`11epqewFNmrv)W)AhV|03fvAlDO$Tq_hH8Dm*- zq3NTcKnDtLgJCm4$U4B|{QjG8lRT8i9_%l5h5#f8vg5lU7P@u|eXUL>M#~EAxWG*l zQif;}?dqaemFQQ9Pz_xfcq)2Ms=Q#7vzJ}%KK*V<3l{HM&h9c{7f}k#!5h&;BLMBB1dRaeQB_$G(EUs2$-+Lm8tl(i(;IeQ(x|6#~CwzdvO%4dsW;h zps!awqcha@Ywa6#JTWHx))lCPwrVG-p^34{g1)S=YAb$IRzXKKE87(;@QLX3tTkIy z1OJ0wO-qU;g?5CoI%D-nv%t>2ZsQBw=<_Jp3TkVZI$PbPF-=+y-cI5O&6CyD_*cRB z#h#nQc@)?n$Qb>5_|p|niYk}RaFsO4kI8l_qw1JNOqd24q1~psPqZm=HVIdY|4gY$ zDY@(_{g4GLK%dKaR0I4SD;!&zU$*{`Ir5!+rir-7O%`BPTl3+5FdwoQn$YnnT{j~f zs#@~x4ey`<;~eFU8a|>YxO~JG3^=oSrXRg9sh?Uw?Fds(6!u-8cc{A9Mi77mPdg^J zDQWlnN@+*P8%Rs_JwnoE^+OqqmxqpC;RR|61p|F7L8(^Zc5f zo!a8)x!g&uVw)65(+9CFBs7Qtk9prmnqnfhaRkhp{vh2?|IM}O=}+_{{s+qMK|ftw zLqlA}3~HwT1tHDEKvWmvDwY4HIAK#VLHqVB#)-r>GXHL_x%G1@y7M8nclRdU< z7ap|XtupliEj!|Cu?2CmNsSYbih*)`Z&D{X}C^}v3Vo1Dtan>a)y$j z(!IvK2mNhAWg7+-1w}CxbNj@F)|kyXk3GT9g)PYH!*V@L;73f%_Seh1D^-pcw3C`! ztl<}A0`9%X*AKpSGy4J~AuSPY(GMI=j*m?1`!Tl9+xh&yyhjgy>yN)Dv-iRKX}8_x zZU{jA`TS?@`0GA*O^el*;H~B5{(%SoI}4EB$q*z&PKC?=bC_}<+w0@h-6WPDi&EXgxm z+q@Z!13lm+V*1iv0AJa0si9Ds@NVvTN>rM=Dx=K}Auq$dUr@58T$!0G1Pr!C%-*~4 zwB#*j{omBGl&s*0WRNP@lC>OQR##Uq^)twb+{!kpQ@LQghXxIwU&IyOW0Q+U4iN{Daco?`2hETG=9LY zdOK$Xf;fBJo~EF)l>+Vd6vg9oh(t zDf956#eYG0iR5Z9Ll|T|G%wWIgNx0w-xfm%?D~+1qg~Fu3D)m$@~J?bF-L71vlN{y zQUU4f_Kw~%aP}(9((&`{du&g~#RAzm-o%)~ojMJ$=)eJw0bD*!RNT`&`B9 zql{|Fqs&zb`sOuazNC?cm{DLqop8TI3zzK452cIn#yrPC_sI(Z=a6i|Ji?F+JVPA? zem4_5b&+7_-Uj&UF0=f+L4lHiBk?3JP3zCIZ=tFl_iA&Ek#{-|I;SghZIMPC8 z6yY;3YfKWH6>O=6FDvhreD|;iddNG3cM2%s@T6pd*aow5(?3h{zHd*em&#``>Y_ko zMOuDEHWB!o#^m)ZJXM@>O}NllG8hj(qJ~sI7Pr2^w_5IXSvWi*?IatHC!?swCaUU) zM{A8It8T5uF)_dMtDe=MTmW-&qfeM!h`&r--e)E8V_*127Kbck(5aMfLa)4RFnS9N zafcWlk)^_64?!`|U5>~6Nt_X`1~W~pw(SZu;7_1`L%S=O<^_M;4>&FiYFtql_~xa$B&%`EGH@Db5igr7vJ6ifC76c5 z0qe=eya$A{;{VQkeZu@U(UaF!-geG?e;$%jy>(Y*qaBlT0mi~dQQQ~B&VSkfPm==m zcV(1f$s8~-D!(TtwB-U5sL`Vt8}LYz~! zBCg%^1cl@c7!;4XaP(y9*az-ID@JxvVA=|lhl__U54m8U2PLeSzFBAqt*?(~-Q7pn zE-CgY9Y^-DH}+3v&OwDhj6C$XCN^15N(U`;y!ijyHRe(IwIPg3Po5kzUDmK7?kKsU ze&;2zfHF%CeDsSfwLd>LjhXZrd5ToNTXUxpMk$yQ!oea1eUP%v8XKa$PUav$t)z_x z*81;CRQr(j8yR~Phi&r4jp%ctH}!2F$4a&GC0TMsTqGOGz5Xy&7rS_DMJG99VlDJp zyHZM#w!6%SQlC+%h&11rf!GF>3Z`n8BbiiOAmo2Px*y=fG7&YRzcYQ3wpCurjitn+ z0Q{%%PENx1Q}K#_xI0WBz+t1X^O@&HG6anDyh0iZ|O%nb$|K+yM}j``QG*4%52Ey9lR#Iq&J=9qkaeJ?&_D;oZ)16Zxyb)WTF{)-O7 zVYRcwE2%cdb71v!u&0!Uv}ND?TkXTMgFN3`;tg=#>NOtbE+%BKO4{Qn#C~^%*C$i- zaWn5?bd>cgpL_e?X}R`oiZi}SqQb4wDUfp)B9X&{@J)@w=D$1c=tk9O zSYsOR@NTpl9Aa+6o&3c{zagh9wX5|N6=`Y{o2l-NK-KL9fkD^8She$R0dwow{Qd3k z8A=9 zh>UO-@Zr<1{6k~%czV^|@KAKepjq#!O(t^vdmwzNcCY>;Ow+aPZu1kgS#t@T7_4Pk z4BX(Y-cK0705E&7(Q|BXx-t#dC1{5JLG`My*|+Yf&Sb&-2)2t4%4J{?1DA>)rNhhB z3#9TGeyQThEaQ5&8_;p4@KL<_55j|q8%3(1g#eqkBXYiM%8tx+fXMZ~n^6)SRF%N` zx^XaiB84!SxXmQ`_>M(yu?NjFT9-%4&otM{n_!p9Ur-z4t56)Xlr&V|*H zx{=iW^vKw+Ro=QX)LoM}BDHXoDqHk&mBP6zSevaa<(Q~WXVEbqIFCS0mCgl#z<{Ms zN-obCFdz#lLbR&nE;~^_C&&+^hS~Kbs0fc};sT%G9Y%bsG+qct@XbSW=HE+UZdfRb zMzY}ugrUm>asrUlybG1ZbEOq&r!7*8Qu-%L&es!RvFv@*kHC%;-qJV2nEyZYeI8cb z`Lcg5A#&PIS{kgJTtZ5Ky}3?o*IQIm6Qs<^z{-ldMkQ=j&w|nJ<&ssa(%@MqlhX1KO`&aYB8=e8Dl&#Qw8lb>n7I`!BQ)Yol-c~h5{^MXH zoyD44Jdj;FHrXTja|0hWog*2(<5XjpT--V#g-RjTpto`fon3R|H(BPbKgx>$kv+XH za7A?%WS#yt$%Hx@X91z&E~Qlp4;A56h%KBT{ErY+ww+Bn9k`5NI3q48Mju*7^Ift} zP11FysPP;6--H%$6O=6H6^i1&-V|VABs@e5DZWDH*J`r7+$^{m#i3ogkF`U6O=0=Fq->E(sGRBqK~z zqJYzJw(}_3eVWo%x-bIa^=iX4-Ex8LKb2sEKD|W&c&JD+bm9jPKg!U2B@m|>vdmA+Cvtn$v2sWVuYX|Lou1LzqF&Ja z-#uM;D6C)I6@48%ileLwtPRELFumVx#LR_T?%7-%8bm60cL-Nvkl`b*pfV%iQdcD(kZgmEfr_B_P!pr$q9fJXl5N~fF@UVOxeWgB zW75;v94`&#J>O2W+H{#3zz6ma%5{ff?-#(pLb1nrS<}-?uHOA?e9Z8~MHF9eks=F_b;(?DUFPCAiN1rOE}ojgyzst0thBZfF^M)vxqqilxl;0!`QU48F} zjER$Oy5FsA?e~b9{{U47w!U|Ha{{>@axg*S`%U?~Hh(~@%cwD3E-B4u#ckU+x%Afc zbm5r!r03Ufr+cSV+EgyPbcUjH+r0am4J@SZYLQ2;7jR-jv*xcLtJ{C^y@AzeA893l zW?W|85@R||1bUyl`1Zy&hipjkiD*f&WA8v8E^UVFVGUwxLtWn;v12$p>B2|qqdy1b zFJmJiEreB<2+y~7bm+0gp^K<&=sCYn>tlOkfJ`@1gRf|oyJG;QCL~NlW;0jzQJ%TB z(stf>x+h)nmiUVllweR89qWQD`sv{-j`=_RMfTOcl?2`z+1pM5MGOD*^*Jp+$mr=E zd;4|@F6cvFl}fl;_-SlGgSh9P@SbxJ=u*>Lu$qFBP6Yx98$q{OTvMB=OfE1rz*l5+ z>ob~-R#nZQ;0j*7U-v%z?Bcwj>JFLQ3uEAaOFognrXTLi9ngzj<$KcjU3=!%uj9`= z<{~;eR*msDN-;uYBIoS~viEdItCN{c^E^*i3rVm$nZ4W%UlA9uvSI4Rtbf&{clO?xFA`;i6~r}eR@qedlHTW+OT zMl#W{X!@MJbB%u}VNs`*u$p_`Ur2eNAa~oi@LUWwN&gBs72{oS)s^PMwOd}d?O5UC zbHi56sEDcDjT~ntJt_l=$~|z&89bViS@c3&zOr-uP^sR~FMit$^*1#Fu z#05EVjEss>bJJK_TaUXS=*qu~4^63J`m+9-3%i?1^CKp?_iJ$p>s+}Kt*nvSo!{)1 z14X>#{gb5d3g$(!GzN~;gvS1~-C_(}H)Ayck=lTK^_ITfcumk_WA|+PrHIPu5riH# zdgFHpLI40Qms7dJR15o?P0U!sn2`$D&6Br#+3XJ+VkAM zkaVKqd&1ipN9k`P%MZRjnxMKCiSg$&)xFwjl3&x?uxEwW^&eG|n%U=^xtG_Cvs~G_ zkBNGUa4Jci@)(m}DW@0Re`8mjyV*wSf7X+icm0?{&7z(Ca-8LjUbN> zpgbEmEFTio45k^a`{ckUeZ39X94nJu4Gzj!6w*F=%vOP=iA3iE_vI{6t zQE5u`zNpDxj^k&0lv1QJ8?CyKtG0Q(QmOI9G6wvr-HYRp2^F;2^byM`YrU9z-ZfGF zG^5`;BFuC@zYq3?{I78uVg@2f17?kb!Da>;HK^8llhw-YWL6>7uRZ^RghoNs)HDNk zHPWv+xvIUmD(D-f^I!$h-=e(F<}*Kd#KY^O;uN*gY9EJMxydIiGgH>O@^{kMA0W#4 zJ_=LJJSnRA;#16@WB=>~ZYyGLo@^>#$fcL*<&-t{U$z%-OQ>rwLW}O+%LLossy$k% zl9=jN-Lw{pOebxUT)9$++ReGdBL0iC3}^*m(irL(5&H|RQQEaaTB_JOQWf1 zcXo^XtwG#d6-nWP>bxoXhDC+-FSP9I?Ye1@HS`RyVvy#Oc}fVH93|S-?a{4R?W}Xe z6;VasFt{%``8$3T(n?OSP@RWBHdeU`89+A!`>F?z`L||Zt=-V%9(snpwUzVuv#khO z+Xs(WSSDvJY>ci=td8;eKPoKMe&v%}T697FnU$$UI&3gX@n+iuMuyB1*R9~tHD4_J z1bPzV&(=i98*^t)LTG$ST-c}#lgRXG_%rv~?;822;N9Sus8i29q2qwHmPHvCBO;_?F;ARfM%3(MHpO~n`P34F@^o`Ai$4g zEk|3aqtDZzGu17wo<~-L zbHPQ%%5nx8ubVf0uvwW<3f)>u>&PNY-T1ZbO}W2ba)#&~8bvfxwej}v?_xCjt^F1M zadW?0687H^;ke7Ga?xS;yn9xH=GXTL*yj19++Nv~e8WD4#!tk?DLqnM%rp3O*RA)l zUJ|tB^(N)gdC=2%_k$k;_3h(H)w^-G?NXdO81Hv5#ys#|{o$q^SYF5R|C%s7W&Y~S z(L45Mhp|`a#Bq)5iSvCdNa_O-!z}8+>N<&NJ>yDSb<>#*UkobHG}@tp20&WP&TBY5 zVWqCd8O#H!dx(!jlfsk2^qbr88omKJd22&GRDS6(dCP+sBX++9n|;Er){da#?>07L zv(C~Xv?WRimDK6+vEP@3>)ITfNEl;)@s(!r>OJn3jt~#3WBU#ZHL#V`DT=i~`{uJ< zlTaOyeq?kna#XJ+RK(u6S7d^#YR%CUQ~voF{e)3cx-s(PzG846lH9Ed*e4-0oz1u} zq94dIf4QnCz-LN`F;(HZ`NcD&@(yx(m+Ug``}kJ5k? zVm*el3E#a9MU_w>qJ|HFJ@pu&Kqo{xjC6<8pNLuutr47cD$g(Ua{l!>AlX^PJ`=jZRMbgS_>*;Uij9!BQ_eIuLN~0+CcdZ#1Q|QPJGV0|PXE&J`tL~~ z!!hYnpN!*I6NV*nF&2-B&dx9o_RqA~T}?LEaVPc-m8T;&kRs{Mk8_p7VA}sezIZkZ zhR%BdS?xyIZpX!Jc_WLMYW$-*t%54KrG^$ak97Lw!j21Rj@@Wd_D%`Qkz{M3)MUAoE-?eJ3i0d2fPVcgD{CP39_BOXpCoWwN2 z93@_bal<@G&H=1c|Lsq@!a2&Fh(>g5q7(T~jJTY&Pq?a8KX0T|X?-+#(oHq^y5DA? zvwc^Q2HenO1TX=VSx1R+W+CI3YWyT#k|V?QchJG0hE~>4tsto>A29&eXZM^;|J}b| zdMF*UsH~Nb%G5AE(zlmn3=vN#-*IRs@v{9Ko$X4Cdg#mWza2o=XZ(VXI*T403yK-N z_?)<0q*6H-R=E%_^Hqj+>#v$GVcae;8Ug^pBUNRKBVZz=28QR4PB}SJ{JZ zJtj=gXv_vkw|9AoDi;NRrs*HwNgA(u{a-E?Cx31fh*hZ=-)GE1()&egh9;6+RPDEhix>F!kU{f@G5}7R{`qQBQ4_`vdZT^2@BYYn_851=k(gn) z>LW2O1mHa2q|_JVII``Ylg|_g8#mk_MV9TjF}fX#wFa(r=^Rt|Gv>wB&Wtmmdx|e^ z6%~qV9=ESwT*bz{otOZ)NQ=GEOoOFANET5eu$thclF>)MFxkkZBH+lHFm(iP;UI~d zPTEi&0UaYjDZD9_7F-Q!ERJW}X6^%SP$y2nUc5hfgk=FI6rJSY)cmynhF*G1V0i-w zQUCD0ErjYh>(CF2uCMk-SlXQ4hH6ovXuw~JTe_W+bYJ`W6cN|kVutoeQ$#MPMpD!g zmpgkl*qJZYVG2#;wV(2$uLf%wfhR^H5Za7trJO5T7r*i9-a)ma>YlBg~Y{TRL8ulObJp|}4ojHawbz%)z`XePs za-R<6_(DJMD4hkYsg&b5ldFxHO0!6f*TpVh&rN5RfV5i~l4uU@Syme2q}?%8xiOeR z1L3BtJ`D&!r&->sT-H=stUS5b<#` zA6pymm6XkWx_+{`4`=w2fW@mhK~sh)-YG;n>y1=DldNW%|Qe)D!Q9-`z?J zc;)XjnhMeR&N_Xmz+ABt>Kxnzl~3-5)}^3ks3gYAIObo~+529#t7|;CSI+TgSQ{i| z1795IK##-wYJBrtTghR7Xr4}wEtH8lxjk0fpyHy=1pH}{tz}Gkec;K-6aUBhsIarr z->!de?!1-PH`cw5X^a>umIKum~7Y_&hXH?MDbJDmgOMu>sZlf2qb>8SFarMQ6lR`RujMh%G7&UZF-u zye#Y$9jIPF#u@{}X|sZk6b$pV{V?g41l(od6;Bc6HfwFUC-yq~_a8}(%YOTg1OWXM z?sUguaPUog@)1oCGg&6to$&i>zUnu2;&a_Ufr}yaD)Yl|Bgc|eDo|ce_>!IVEP#)t z|F=OAU*p#<-Gi~vkO{ZQX4ro0ds*nXCUanH7_jj7?-69}yL{=cQ!W1UMQfokmi|U0 zLQ1tNMQf=kvFM%L;mD1Gw?gfg9l1E42*JD-ypb9u*-dMziJL!Tb^Z5W{|CozA>0m| zFLdjEEW=T0oh8x63NZqH9Iae0{gQEtVqo*~`O%Z|?w7>hJ+F+k^nr=z^ZoX+d-Vs1 z|I>TlLf@J3*QtiJ1-^)-N|Eb2Y#mE_LKEpvWjy`Dy-!rwG`(78v2pn^ zZgqDEdERfc;TuKETXX)gSy@=0C`K~s=@jY9N%f>NC? zss5|yaaYO>0ItpQiZVg$t2LpHg!hlfTZup<@=1^)SyTJoEHMRzd2R@43lrq6i++8mDhLA6oEDboaf{ZJ&L{txWjxbY}mo?+4J)Aff)A z@Zf7lA>>Edj$J^yL*Fy7HEZ;A*!&bRFUAZ@z;~MFl`GAfRK9RQS**lTQ(lL*;H+1`Sj$6$%hP-^K)`v@X!(U6S!%Lx^UZ&-T!fRW(Y*S9P4*I1VTObGCEVL zQ|Ods93ZJtOD$I06`xI(=X`c^^f%{6>}(MdzY+98$bC)9ro3L#6b2dEbg^h(3 zsujQD{@uO%G0XHVZyvTTT!?$r%zSg3B(W_#648kpNCeZvsm z6)O9*KV7k2pLr71oaa-|j(6&}X9xK5@890Gd(LHPt<`2C>v;t{>4@S8b|0m6QjRnn zj9e*rpY&)`?WM}uZH{z4I2F3>H2HVNu(h9{3H|K~15R3xvbiR}nV#tnVbi;jl($uJ zWM**P8laoF{6n+N4nEty7s=h{elmL%Y787;bKeX5j54Qv z62N@d2r9%CX9gJ?MoCg=0u#!YK^(jMk$sJ%QR0 zNSG6vd+=-`)c5h&TgU9{55r3R6`7(rXCM9>7Q^Pgj8j?1Oe`F&JW{)wG|?rjeTp>o z7_cvOvop6cHw^*&i!p0&ij2n6p@((ulqodGfJV%MT@6bZN$uyb+FshL>re(3X6z*@ zOH>BuB2C7NF@66eiw4+#S9R)ItwV8GU`PFPTi$B1sKOE^ za8gYksp+&v5Iw*n$221N+_kwHaPQ|#rG3}z0PP;J1cF@!0kqltIfnjPDDawlI!6L1 z{y5Ns4cNm?6-1xBa>Hy_JaVNO@L!Vymq~^Jzq`p+eufqw6$c&ZuxU+?=SxUlb4F5KFitSztq=&lZKNiyX?5q!!m^>mjZi8}{Udzz&J2^v z?~(gfnn95!UlPUfb4KZ_w0W=oU73j&g&T_F13l%hNiU-SrY+?x&&$LZyhv^M{w!ly zhRV33J8iP0&Sqc|r4R;j?a=FMVrj^aIE_%r&=+wf`b)Z%&*y&`+PNvA;S=ug%@6r3 zK{xKaMD$M84gJ)@%U*)S=W4I|qBYMyR-3IO#dFAWOiPNSef9?T2JBGn@{ve9d!~Dn zT`rAP(prHQ*AkWL&?QreVb&tnRpg%`fwSjW6$zD{<8W~O@%JA9kVJo)sKB_$*XTZ$ ztm3mdrcZ4W;?dLD9=JKp4rC49N-w+(ubbwULw)<#+Cx0uKxd|wjrxI^K`;n=s~x~M zhD_=l(es#y(Rng;bE=)+{L}a?=@!Msd2y3)_~>&&;^(il*dDZy;w|%$4BCfML?l1Z z;>gwj*O&_Y+(SN0X(a@J~+Q%x}2g=bdPha=S!@sxSjK2&@`a^ zbIj#aX^DTO^)GfFV-|oX_Dm8}`ryDXe5q8O$-uJ(K{CvkouNHu7B<488H;GeFCL>h z+cU&}n#=1$5HF{;xs9l5w2O1#v}vLWu=t zhGdl;PE)ulX*%q023mhZdZft_(GUY|0vznqr@K*?#A3cE)>fP2Crep|Yw^6yb?;E7F`I7|^ z%zUFCfQqwnYJO-b=xVhB@W=iS13#TnLQOqK$CBXPc$z?hJC2YdX|zSJ1PDOJfiPf| z9XMF8*ElxM2tNy$QEoz~is_(6xEt4yV39BDaCkxRA4Vhj-}!hHOVcG7@+{l}^V_~= zADkKXS_LxIjF~61AMCvvdACdwA^F7v#%|I6KcRVp$(#&DnseYtUX6`WZ&pvNVveO6 zdw6}X9l6=|%wxs>)w;FO4be5~w!7*p`XT0CF$46O#uTVw06)84#gHApTm4EP7!Q>S z&vRhvDUd1cz;Ni4fca_$PYw*=`=W4y=p$smD}h!(b)K`-4Nsd zG$(qgNcI%bHAtCt_cvJWw=i+X zeqVvamdn;i^738xC4TTdxNqDzk|Qs;I_eS)I69Dr<>%8F)e_64vhW-I=jumKBF1je z2PSaAa?fkMnu&zhMP95oz$&S*cdrCDxHduDpO4O3P9j$pd0Jw~PJaeI`8|Ch&@b$e%OO^LHA~vvphFpH6p@By~cP^p*i6gtDokXZ1{fQI% zjD%4!PEU8$&zTN(|kLxpI(`F-$(m~NQjiqi?8gr zuC49%?@SjieE@ab&OO!UW+fJ<8o9vMNLs@?L;H>OJFWg?Um)3BM`*ogDncdoRjE(9 z2R}#V%;_-BY0|+Vi_s++Iyl7V=jZn)4w;ZPk^{9G;l@qLTr@!8g<(oTEmDb)q5WrS zvlqDws#xntmEoUGuYHhz9KyQ|6}^Y!t~k{hldgI|>GU5uGbW|mc(~$) z_8>pSOT4+Faf<`BZFa+-`3Jc0jVd|#Z;a0=5{~L2@mV@9*xf!s;f@F?WZ}M9ENz7H{`Z%*-Bp7_H%v+ZQm7JAK#wg6#B9(O-~yy5;~omm?Mv#Vczp=SNnZ;wkKUoGt4@}4J68y zOX3L%k1D;k*XGm!jd{oV`3KRWd3^*Zs#LD>OQ9x<3R4PD#^XmxM)tj4vT3R{;UKA7 zVo8xLn(DQ(axPE8AOIcpZ{3SPJhDH&cI*XnJKX~HGahzXsAik_N%TuSufeD%EU=7% zo~zOx{~7OZd(uzH8I$DX4S>B^CiCN>kMmW_Oull0Tt|7@jXBeAkjn*b^ZQ&#=k+Qf z!E%a?#wv(H8Ml9!;xi#quKuOQdfNL?5#P5m502@4I_cbtZC>rF+SK)dPO)89mVn=m z)dLM5Ur_&SUhFnI2M32OV~XsoRbz3VWA>e)=flH$oG;y*=R+Y)5HoXCf2Jb9=;!;4 zB?ttPDN=cHsz87Xe8Z!h^ddl{cW_)6S^iuQpFV#q3qOK}_>(EcvNGDw+aMq<@e@bj zuU8#8CHNX~(FT@l465+27EC2YCbmybX7uphNdx^1+%@Z>_jDt5?Ed-OK&L=k+^`Zt zG|XAcUy6sm82_%HRjxiqhxU-vwvFzKc|ZoByFpDD4VjbfWXhQX-Y(hF;n)rGHA=1w z3=Hh&;Nt^UYALc#S%fserTD};vAMh4qyShqoSd8}h-!mC#dF#6v!UewIYoA1bEqnL z82gK&wtQ17Fwm0U!3*jCwvKMC$}!}??xCYlN0>YU5UV!f& z5lZ{4T31m_r)BbShqa&vFiT*{>2(Y_{Yb`=;-M9F*As-KZvswPj^3p zn!PatUyK_2SCef}m%+@01(&Qq4#1-4PIVDbpjF8ucDFhi=slxfCeeSn9HZi6>@DnR z63mie8m0^+hNGD5jX@@QwY@=38D)kw8<%|85%NbpQ!bBH_(Xe7EG%gx5yu{?;_ za`!A_1hwUvjjiN@TaFj=S}uIee>jUe@7vDyc#-{8&&~|tx*+=s2V>V4ebV4?DXAc! zhy?jNk8QAva#6W1QqZ1Q4Yyusy-!RzyMOqj(YaMxlG|O!lB@!>vE8QR>q4m12{Q5W zbf0K(zpIq?cfLtL$yc&iHp<*;f|TXCn;9mwwQuuJ8zP<>kFkHVR434m>i$DZPDuW5Q2i7-BXLR3|VW5ic4OJ>X? zIQr3OW@Cds0uWDee2O!XUfj;Y#HhtIG>9rID-{&JPakp>TTbI=K2cV+l3$)S-i1ku0sr@sMe0 zw(d9;!Ro9jP-tx4C3s|gp!4BQKhsR^rZH}tziu0tRcD#s=b4tY)TUnrtip5;7_VxF0j4p?QE5 z*|!-9TusJJY1>HJW?XBGO!j9@!KX5!Kj?d6zEp-RBiGNVG?S`eWXx5+#}l8sJ4MwW z1h;cweFzrWQy^9Q8S+ax{PWb+=L%ANaRbOEGs>ON0bFX6nb$YuM)z9e?()%sE0n{- z=`3j#GDz`K?+60W!fTB&L!?nlSX#4Qb{k{ROvSdN0NZKf{KwZgtUnJ5`bW3GXS`M- zHY>R^d&>Vf69hl}j505NB?rX6Lil{E#_LIM{Sc_i?@}4+Ks&n8d3ENw#;Z3l0?+2W zAb7S_m36jtjcmU^wHC)A%AXe|a*Ms;z3Khe%-bBDI_}gjc>{WV#)Yv^o?O7T;NF{w z2Q-cZdYuPXEDrP+U>He#S6EcE+kgCSa9@(x?a?H$A z&&tsw`7P@hmG}`2zR5g8$5I(O>|gKEWe14U)@Cz(rx-4+xLE{$6(CmjZg2MIRMpnP z#TwQz<&}uy8%Ya^xYK;>TCeT+TFK5#RB0mpj0t#QKUj>!{h_xkTz8?NjNLxWpchn= zg>2<@3`TZpcLgTEb1Z$s-ip3^0>(*7fyFRe+x1Khk;Julto8-)H^8ESTUI?t`mYj) zpqg5Y#cD<`M8|R%AZs<4$Bz+=6<5L&g9=l`;mvC0&5QbDJ>Uk|2NB?6mZLJd2LNng zL|Pg`L3QQCrihuM?AI|7R)P->y;bskUA8vGu!NWW`&<+DJE+WQpwQi6ZmWE8syP^7 zeLUWPJJ&ZPdVo>*sN!ged4+lZYn_4JrhEHsIr~T4=BI&5K2LKeLP@lmi9I%g(tLh> zD0QKDUY4DfGa8x%TFG2M^2OU+(Z52J>#x#E8|et6KHdK7WlGZE$a{W6uHC9ANM9>m zSuU-F!65~I063?EHy;*);eG*aPGDgJ$rrKpIuDVYy>6Y@lvX&3CB0EdW-1Wrz$(O` zdj15MKh6p1&Yrjpn`~51BmG+9g{3Vi{ivtJ%UzkB+Rn2iqxx+-jqQHWqf*~msoF2S zGnA=@FTAvt6YvV%h~UWA>o9h-;;-?SC(m`&Kit$ELFBJ!_P>22p~Vd+cdJn6>ras0 z$x8og-~TAdC?FsJ(9;DfDW?84W$8l7mSddn-svQdO_y-XMM)Y}MU~(qju?5fgc=jK zr^{1v`^cx%+Fth>Srh@I2_W2Ej+mmRN>0@+Qp6ObZ1tJoddp_WG7I%&JxWm=!L9P$ zId81o3j4L}hP8mB012^rUWYq1ufvzP0D?7ww=UGQf>%&NfNrsGSD>g3q+;P?! zkuj46awVC8|7u9gqSv0@)YSB5JmtM>%gRiVPL*Q11+mnO0F~|Me@~t{vQDxmQyOjl z61OR7rBEJY_ux;UKK2BLCkGW@hLGR@_Ee@}ys&J2`J0eTuo8;K92o}iU#X)@GuD0` zSn66RI<&c2h|S18Dd1qnR)8Ek=gKe1R)D52Ce@2-*mSrX% z+1%Y_Qw=9D;HiY;q7{F|s_Vc=c5Q0_%n09Y68<&RR&y;sCSU(FSNWN(CxpuHJWS|( z8eXXG9%sF7?;XrB*~&ootho4>aHC2F!9?t>h6ComhAlJYEf10Y^A&|EkrrH$XZO-J zU$|$osqn^r0mXKPeCWwaGcPwzJqrLkV#zq4uo+ zuC)c9<{v%L>R-R)2 z5-J4r5IFL5-5&uIlQaf5;$xNdjTg62Z^(_ytp8EWxb5MW5L(olOGRScuPTE9 zzI-~H#I@U8kGXMLn?7o}j1Yt_uYmhig%u&tJpZQ(+qBgS543!}!>@kpkD~4b84D&1 zRjlJ8y;1e!H+FZFQQc9+!g_-Q9(#x84&)_{g(g!4k^pL)Gmp8a5H8v z87hx|WZ<9Qr~fDBNCZi11Hxt)iM*xta_j5Ld~DWu1(W^nCkr22TD*qlbUKf{u47KO z2iTs|ect-}^vo=uKY6cDtTnHWbv~YUZhBxg>$x#+Z2a{F-;S^iw?KhPgN!}JDi{81 zz58E=m8RN(`=>sm_sc6QT5WE&#=a>kKvRc=ghc$(kM{KR#Qfg!)Z5Qr|wSVb(cYj|p7nEMSgz^@z%Ve%hJ;s1z+gtWUCPKd3%dLtL4qKV=W&Ttz>M(=z z8kT=eeq*PPfQ@68gX>(5lx6~o>1zecgI)F?hC?kna;+z?L z=L^!f_>;G*C1mt{HXys?d7=nuC;Qr)=!w#hAa@?YTG;?cDlTw;o~>ablr(-DQ*E0Y zaB&nx4S@qrlO-cQDn2SA5P3M?rnZ73c4sGBzv4z*`@&v9VH05MQbR#~*W6F?o%k7#p4u zU^oVR2H+sQ=im_M$;x@ZO0LMBLCVE4VwJowJ(}Y@8K={5R^nl^?7S`|l+5D`1o0jqD;n}ZDU|@(-Fzb@FO%6Mr z`rFq$>Tf8;m!DfyKO!5%d&m*gB8jMBB(Av7K@ThC{xGuj!z3|O9Cn5Mn|~8Ch=qeDX*1@hz|W3#R7Ycr z;Oi>4`567?2|a-FO+5TNQnl*a0ls5H&ta0HxymCg_S6H9N$ny&v=U z%D!(_p}laXDOqX?OH61|ab7Qm`Nprus55%i(ZNSrlx`YB(+OEGAz}5E4#_jYpG+G{ zT}H%6Ku;e{##_~m2H;rXeMY@TFE241o}9d;j7u6b8`^aukgL^X@m4@8Gn*4f2&YEr zUPoV))gWfkW;CNuiXPjw7}^cTkFr5MCL~3*m@s7|M)F%R)8Yb6b@=eb%F<<|4`J7^ z1f`DgbW%yA#gf;skdlW*FNIAwF7Dl=Nrp?E<$X|Ba*156`%Os4BhTV3=tk>a^R6~V z>s>v}fvoE$PiS)WY;s$UIVuUyUpodZlKQ`Tip60cS!ql?5I^vJxA|;SymCOvp<}r` z=4QfI_WcVYa}WD4K{ZmC$?hez?wlC=O{rQW@XW-YKYvg{#9xace89#WHmJnM2!T^o zWi(P2)rRYuzW$K+2_K)+i_=6Q$#~l_RkH9+sEOq_;Ve@hnu+36_K2r0KZCqayMRK0 zy}lZ&5?`|D;F_{Uj~21~SG~~qxBqbgVxGOXbE|a)3Fw9M{osRO0D*(g2;Ge)D`0&e#1@&{j zQI@!&f_;q=R8t+yqQs6~T&!ZU zYPTBNO@(PlT*e~yK*?4-@>`)gPKilwvd&^?e`%KtJX>Cbzg@58=B0bVH>A(^L z_pq3s1Ef8fsE()PZ$8t1gAdywZ+Pi`frFsg;y~kn+e38@^#NeGvd=sJy*V-$v&uE* z)Vk7TYQGU_T!w=E0sHG?J<}uB6KQ3aQl!J81_ugiwo#|x&*$x=8A~2?M~#KA6 z*w>gP{wx>K=V0ATl+r%Q88)MKd!Jn?M5tQsTftNVV=7$OEr^E$Iq*)vxrY$o&CZcQ zn;@eWd-6E65iFRHRs4yIw5YZ=VQ+8G>9drv@f(r6YT0=5!|sox0l)_M6<85xKG9!H z?>v9QvO!0>sim!^yn;0|=8r?a0F@Z? z*U?{ImD>?8I0zp+qP{V5lN?`ppa}3_lBQYWzCxb-Nh`#W@p{Q8L8dl5pIfqrdUyG9 zGRviH3YA2p1HUgQ2A&Ham7($w;(;kkla`&b?Oy#Ht4MzT$`1TpIz;@pLq^MQNK!QEI68}}Rckk%|Yvtj!M@Hf>cI0Q1 zty?Mi83w@*(-}fX?!^{k`%Bp9(eL5UxxUG%|9~PNy2F&C= z(BmFgjJY@LUo|G9YNK;jVB&WZGl^lEesUEIAySux^T4QNX9Szw{h(}rChOY<}1_lNf7uV$CqNZM)RH1se$#3<4IfX_FR;!-t zv6F|`XgO=e!$c0wIvghgBuTe(vGYbxue^FdSeyzwZC+?$w%6P1=V6o1;snb+SO&p( zlJATmJ#pwajG&K*8`S51)zX)X$nvKG`H90tZxVFNQh%rAdi!>+KtExWUh5o+xC))l z@!8Fi%S5lkc->%AEt?H+j^#|TU2W7l20V&(V=%tB(3W zJd&6Xn ze}c8F+?EIoUYnN}kj$NJoT0Zw2t{38Ic;si5wkcz-!tPFwS9S&luz??9X3Y<|MaPI za*dt6@_iW=mkXcEN8 zR;6Q0NpIK)0*Vh>R#j^dv1M8*iPVmtmDzfxJ8SPqj+-EUZ$8j3{RZ7&dT3xRXn(1j z{O}gcL?G~C*W2`F)Sa#|4pT=X_9xlZr-W76)o8`%U7+UsGvSsoprMXctSLX<>iO9- zaWPCvmH(KBs~>4F?*k_R78)yCtJky4q2R@X0zlRYiJO%G9h3c~I0y`MdYH0g>3qmK zG8Qk)PR{-N(_mMmx0oQ;jf_MFq!R71=$s9wl9^<-H|^|#fjai++kGG%J892a^hyk{VL8L8=7%|*N?N?$S5%-pT?`Jzka+~q;# z!w`e$Z7$cA`c}+etjOZ#D8-~+0+%)+n!qeXtAg_y*N5PJ(->dHao3n0@-Q785db2Q zv?JR)LNLrNE%8XUg*7CIPJ6Lc4%oBfH|gm2*ZgZc!1s?`p=iaC?&eHcNTLawf5 z?<5ZE*!^ppg|uVJggzcUT58+1tG>|Y!;`4{w1Npp=lSLNZ@aI?hPV4?i)ejaKbtm= z+`nFuK^l+EO)7_$zSvXQ9m~n)FX#?kFx~Ql``n3?;UXHWQkP%to+G#8b~4vYAvdor zbwGSNox922dv+>LBApv%wD*b>8eJ)d?o@#8#XMfSxOoNp;_VCLCC!QH{#g*UJ)LE+`I}TBU?$gv4-^)FL!xm$d zUsFTp8yIwW0_A|J2 zW!V@dv4JSyEP&-!X|F8t4$o2iP6s~NP^f!2TXPkjA7b(e_3^(^d@)bWM`Gd+awl0N z4&q??C#k%rNGGEyhyT*n?`&@eWjK%4=b`K1sy)wlse;NoVX~?*s6;ADXTH#8-|Yyw zZfu?%6Af}YgtE*rM?cr_E#wSCz0Q#)vA_OHbLmadR}`>Na%fF<6WBE?add9pH#=!u z`y#SzS;b4uG(>9)-8^-Z45W#eqp>k-M_0ewUrEiG))S~Y@xv+ND^^Lvi( z3!IS|7R{EYj=CLMci&v@Ll@WXIrleG-k0jP*1aSX0;>j!h=6@0Gw}gejF$}#zo?tm zcoffuAwq(k(hJd8@24cnSH;(cHFb^tPaF3g+{=Y*>^$YbX9`thd@KyAg zt^D<#gG)Xf>&V69l{G@vz(}odLu7k@6Mufb)S3lMRm4!$D-A4+-|89;+%A`Q^kE;5 z5Px>>QwN*TO1cVP_;~7sx7Zm>Ha(l?~<9EYIQB9LLeAFIHrr zES~5CTjo(E)A77%lhjD5_R(ta(EYda=yJ%(`upM?b+MIo*dW%+b&N%Fz^Oz$=0Yq# ztXO|s*c9`cu5h!};T1JhmHLOt@N)DV)Hk~b(Tr}L{%ZUv^7!xsdIV(wqoH#(^m~N} z`}D&<;N6Katpnkzq`&5HcuL60V$5NR;QtZ*3fGn(`uDfl1jj5Tn;tJm#OCim>~kMp zOT@a*OH2d_ow`As+#BK7uA6*YFWeXZUD$WOakcCE_ntZK18d?cOa*@nj17!>%@kEQoqdc1G?RK*t z3)_|41vdJ2gqT8Sr6Nbc>%K!9lZx9aVV^{evMOsOdaRy}6JV`}a^))0fD5C_y25co z7VYtp4mSGkf;l}wf0Yj@Ee96;v z+*CgD7-@9|(b}W}3`nHcinvxKQs>GaJT3X^z1-`Ru=b|u({-5mh)^OG;ZLv0nh?4p z`4Uci-TB?=hI(RUf$%dzGa9?!x^q0?CGK|7;d+uGaiH%!X+flUk3+3i<3XAUIg9nW zO^Q0%qiU8J?Cn7UOc=GPhu*@JyVuR#;(CMn#?fZ4*z{C=G_m)lm|A2f6ZU<~?1K+) zvXJ$`cIhc2&UUf#l2N*YBfv5(@{;;EA4NAD^Dt|*BEb;lCnHI;WZve^&bxG3z?k(( zWy!bvrpIE^k=E5E4itKxhsmh=rx?ekP2%am^$A6#vs$K2(i5cK%wxb5RYFIMERlyx zGRIIZ8)l4vItRTafgvMTq>mZn*J2c<#w0;MXJ(GPY1ENUr&vEi{X>uU`sKe5HC0Ze zGPId1{?piY4=XpoU?IZM3VIsrIY5o7x~f$|qAE<%Ix#Z6{lPKWb(+6*yE#pKRFUZ0 zb;~rk@na(AvL7as)K=;~m;@n5L|CaNM78;pR<6W7G{@JCP18V=nn%>eV>`Z=XTmZeEj?V-b1={wEv$2MZ;zoFxfnaF)4|lS z+nzQd>xMXdR=d777Rm)!m;WN<#Tc85YlQQ)vhagjI(}^XY`N>QqNJZbbx@cVkJ^cA zm9^tt&g99eY3Azeh7W#OUtj8EqlDD2Kq+5en)#4@Njig8)x{g-`@Jk*D7Np-%cgHp zGMqLmU5+-ZfT6HE9~z=e?enL(090j8hriUiwh-7A+rg#zo0#TE)d`C}4rj4icf;da zt`p8)Cr7pum?XMD2Qu_kmW<3XOP zQo?!@36me@;3&8#yZD2sCBjEP>sj~6RMi55;9Oq3-#5Awy|quEbCxBN(!JwrQ#pau zF2Q$OLMe$H%_6Q1^>r*+K`bLI4Zn_FGLx$QJ=zU2y*fD-IJv?0(YzqIT7w~{Q|B>|m@ z96hV4DFdj-DJ#PN1@=k}f!C}zre-Kt0wo?2tGy7$%n0?%5F?Pq!3HM0m>I_5hH-6c z?7p)(y8*>6n1o$b?DDD`-6IreG2y1a*rjFio~dEK)-MV$5`W9}GT4YwVhW!!UT!3H zi*RCC%v*sUa?nn|dCsMkAI7rG(r=SIkXN7^nuW;G&2a*F9m_$**Zw5`$Ta(7dJDMj zySt$rbIQ;m^H^DET}3JW1fkbS7$;OE9$K117>PvOhr_r2O($#u57?9Hr5fWD3qMWs zqD6K0?EqNs#UEuotCpaw19#8&sROaW1CnvaQ|EQ(cJ`6EIq4_b z4JENJaCVQ8DM*T8^Cim&nM}KNBRP78!sux?1y_(b&=`*8Dm5-#KWH)GRK}f4ss=u? zabE(9@BOXAf@q0WM6^(LI1KYT%2`!Xz65w>khQ**B1bPg^hf*}mQrOYUiw%3>m#V0 zB1W_;n7gdHQlF?we_O;WpD<%h{uZ-q{RnIRvOh-8ey$y5l(ED;t2`RIVh@?o@7GN{ zNnw)LcYLQ3{n@{=Pjqz8BylKshvbXC`#jLWmau%zkB4fETNySh;ZV{94!mp{4^?!g zE??k#a_)1O5p}XG@|mpWVqzTjT7xTj0m2{*8E4|R2xm$#)YnJCUzsMdF%9*VH2xd! zpA=di#2&ngUODkah!hVGPrP6ZZ05=dPeSxR1%`-k-{_QKH!tdz1Fah*KIUd--$9UH zUwZ(?SYA@uT`n=0xW^$SfHu2Q}s z)Y?iDcec>|>OUV!a1#esr~_fFkl`d=lub?=(kaKoWS{!I>OQV(aFbzr}sHQo*MAJ2P#)yDxp2GmpVbc$5y>bY5g6?~qfv15d4Wtq-6(beQ%T~_r`imP} zj94~-*Sr`tQMc&-@Ach>sGdiTt00JH+VGXPZaW?s}<`WsfhnyKA@ZIPLEYCza?iKeQTC=?3jh z(a+XO@4o_wgAq&1VU7ZFIfSAj?P69!hu((`NRoYZwm5;3dnILDof3Aj;K+RCYNFp# z)2bsZm%NfRz~WlS`}@c-I=Wd?(Jl zwwvV8hPo6a%iq-+P7!?Dk*mv~@uN{T_|7{q;HW)B7(?*w`4I~VB3v-fhr-05ZYT|t zPFbQnwFuZfGBPr392}$I^Z=9=I5Pqz%}W+7d{sUGA<>p-m@{Hgq*Yn(e0_TAW8U)U zx*z9@oSp+aZpMj7-M)#5k{efM44?baA$Pm2@6#W(B`Jv!l$qoF*3@!Owq|aAwc6oo zF<-m@@Ajy6%9Xj*#XXP;iJ*RI?EK75XxWQ_4;0Xt>f_>V4NfK%23SRnvk-|u^o6@eM7zemI(q>%NGv;#+ ztqhRLQ&JV@(v{pJs-swxhcAz)4!@o=6afuZKeH>}Pc=_6kRQ?blNE7f+3gu!Qe7R! zwZuql($4~-l(B<@opQ@b#(e4IHoz#cAm%j01rOi9qUyF^;4t{_K6-BCx#L1sh)~j4 z@)d%OBGg;)(NR%s9#31NSBc8wmX!hyh8|rF}gAYJMfqL z3r%DCsT8Qeom5d}X+4m8?CX2cJhpCLgbo|%sC#7=h-5yqouc}Mji+r{tdqCH?@1&vzIMQ z0xjnq=kvP1mikP+lHn8hTV6rsmTY(KRj^pG2b;dx@*Oeo`}B8TEmsJ1j6}`}f;&L+ zbH90d`yTe9Y?E5=(Jz9qf%1Vlx4{S;9MBDF%rbMmPOL1)OqEKF^AaBi8;8>UNvMU4U4pQ{b!l$J$8Rf&e$YTrop?89$5z1V+7YWtO# z|3(;$E_()MkMFCyp%7=B=xYcu#J}k98>7tZ%Aq4(ZH@uU1rL>f0M628o#}gBB0xzFvtJ`H$SH=U_zX(`JrI3qR1BeeTwPf<u6*`qnqf#=+cI6q!+xQ%GU_O1Ojsdy^|GMgtG z)kna6GQnKYgBhnV>GQEYKQS!xid|bNCXGj#Js6BX}U}*L_$UJ-n`82By;X01ASDG1Aid~P>h&~hUk>A zMI3$Xa*ER5Nh-M^3|%t|1{%$vRI09!0;_za%P&Iw#4VvSsh|GZ-`~%_vbh8?4@Px* z`^n8;(8v;wl`3l>Hm2FmXlK*1yXwQ~v+ z5uFjsy{Q)W;`=m32^&$LqhRUy2y#)qimH~`6gK4&@^JL-55|I`rA`?phu&PJ%K?Gg zpaaMqA(oB$-x0$|lOyM_tAmY))aKHAx+DkUSN?XJHQPN!KPim}2xN-{BGf%5W)+SIxjc0! zii40XQ(un4Zosg-6iw&B!`8*^%9`#ky}cKM>CTMJcdNEEJfim-0}so%cB20L-woxPNCc5{ zKfMAvJC+!?fs7=SCCH!2RMrhVEa1cK@8=g31OxpIAp1aUqZXLoR4AQxh+d{<@kM^J z`!?11V{FOqO_5vmewkdx|0nee!a(2S9s@UemY|mbhC;zt#dm`Zsv{cHZ!`}K-!M!;3K)#Lj z{)e5%GW@qUGTv=orJceIn?A|ztFcvE+Te%;IMdAmLU&^~ZAMjfYOYA7!Df($lB2)+Pf7!|QlO>+s74I{ zQ^5u)zRnz~mLoH?YrASG>RTg24~a{(qriN<{6zU*dhx&TdloHjr1rmD1@@~V z_}V6CXVL6P{ht#*CPryxAyBiW1c{?ocXy78Clxl@3ng?WStZz%xvD9QrS!3o#%59B zWpnRR8RM*7aSPLQRgIThn-bf8Bnt=QzDFjz>5!1%=iVq*h5=;_cvd5ceId_XKVS82 z#L}2-O1dszTn0b<_H%+@!!CbaI@2zzXq}Uf_r&@wEcOXk2b5aWIne+KJqrXX*o*Xi zZT4xQ#w{nzQ#nw;V5^%U>O2$5LXC)sc$`iC@8ACet1s=#KXXs~&RTB>ZfT!LTfMKS zE&uR}9*7yozBe+2PJG8y3miRvZog!orofk&c*2i)c}ujN%@4;B59g`9P~iTpZZqG} zd|$%}+qeGM7Tg4h^S-z}R_K}CVwOMt%;7&W!Qv+B7px7vyhghd(kHU*1>RyE+!iJ7 zU{Hno>4;dL4k~+pZVaCOZ}Y=|<_Ww!%zSed2Ak?sGd2w@>_-(Gz`v;}@C^%({lovM z+35<^{>Ku+3TLw$x3-WNqS0Vom^WGLWd>uy?0c^9XK3A#zpNKzO@LJ`+#bh%xD~DH zsQyc=iTdSThTPB%o6@b)&!2i{9iDDJvnOIAxH{eTQ@@Sh*i_fInPZO& zC)>QdbL94Mt(^LbOY-BWOIoP6hgRNXz}|CQJ)2oAlj?p(d|ol$++!NxbJ`8~+EBcl&U6{KOBB%jxEq(9Om?Q%tb_E2$H##$diJZn zY;b@eNlWcUh9;wR(Vu)UrFMPY1mIDB<8d2)UPZE+rvs&$Bxr)ZqzGC(C`*r262jIP z!aRR8XXE#x#sJr8#6XlwdUOqs1->JXor#0v*iXCu$3Y+J-^p@&PlB`$Ysl94wh)v|HsaA9MO3>adBL{5 z^*`4hkqV*N`pitxE&D@A>O>(++bCn9_4A&X#4uK`U=OD`UXDFt47t2t@og%h%FYTv z$ymJMEh46m{g=3B2%7T`rnyjFG};pb8 zD1Xs2XFIEFGQnMFm3asPC90$I^D^4n^O~bDPXy0$iGPq0v&2Grx1qAjxKy1MWPYZX z<;YDPN08;zFS{@nm%<9XqXD($b?4hW7&#TG81MmXp?67yg@mx-E#H>45;OcS?2G?tKne3R zV@aM4x6d(O$<;W#al1=+kjGd?BvBwm&?gOPfS|v;A5_K(MD;y0~cYZ75 z3ac5`QNPpsH>d^wqFqh;qygRqMvQO$6t(X61518GO&b-bxt>!AQ^@mxW%ews%bR(5I6S* za4XYZ#;vN%LH~%s?6ABmOh)tO+-%jZXxi^jT6q3kkC)!fGZ^obAQz!0C*vWJSJBwn%L2am20U7ENk;6G71;JeOMaw4LL|QQm2K?Zfy*@4$ zZ!E97{@9yS6A$~Fc?#&Bn&O=(h$rw!6iYv)g;XV@|5S+mb5F_V-c8~ecKcOUR<_ME zQLG-I_%j>kx}~lg3JTo)Kw<5%B_J)Fot@X9NLUoXKg+(38t?(hK8TpSOl7HPVV~}r z%e|*ZEF8PV4iSs+&(}}Xe|*Vubw69(U8`*0I)9h!NpYu(GEYkS*mKt0*{!p9uH4~GHj}9qD>9z{`6Ix1U1{;j%oKoIYF6}H=(u=~a z^;oiWb+Q3NAkWk(ub=gYk!q9^@UT-;;*O4v?tGvcD6Wk;+@Z9u<=T&j>M2w_-w7fj zfTrM1E-8x13JymS@&Z>lIq*q~lLH*HV|zb_By6=l6^paWR(kWw-uIshJ&^5Y#K9GeTk`g79PI=v!e02Ao($^}~$2bI;U5H4w zkQz>qA3!W>z<!l;@{-td!O4&v_^_2=%DT8u0EBLFMVn4F)Dqzyss3G(w4xnREB{-+Vn6en8_xfCOzG$$4~4eH>rDrO zg&$=Fu5pAT-KMPJeJ05%lF=O6r?oC9lNGTN9uOkgo&ELrAUKC0QjJHJj+h8^HRu~B z>g6ZNdE6xV`IPGesjBDxQ&lqbSahBej~U3;7HPbs0ySo@PQ2e_)(a!u(h)Wpl~OGU z;NvAgp1k#bQnh#C-N%e{TsaLLxV3aTBgMb9PUF6o@ovrLwzU7m+LN#MZO`Z7D>rF^ zB0;<~D@!7U%0|KDf$^{Q&K&kwbaTiR#1Z|~#FzG5E!H$}pzR`+3nBXcje;NI4?!(q zuhrWBS_lW44^>$M&1#AWIT*yEbTVt+9FtK?7z4v^>?1y)#e-x{rhEL>at z$2Mlf<11j;{cqj;Iut^4p!W3bErFDrS#NzA!wfE#4p_DS$yks!8@+k2NMk^RqIu^N zj@Hpl!^g)ZNRLUvoRy)~5Ou6phKY2hP`A_muWTkr%8~;v;J6!n%PA=y2SSl;vdcsL z*>jztbIkH&lOlb@H1hA*~fczRmG)RYQ*@3k{Oqsy$m z79YfR9F?>uU~qM#c>XN0sXD+aurlo zxjLF+zQxA7({|Zlv%T}KMh3>Ihhbq3(`0M{2c)a{xnuoO z^VkiRY~DhU;r`O^dKh4e0uPqN=jkH4RstkoRaMo_QLux`)r@X0j{vZjM1CxjXf4-+ zd0WN?Q$?@1VW>CS&FDW7$43+q_+m#mi*J8IzlG64EB^yqoo8mGk_IjUlqWkd7|yEY zMZ4{i>y3ozSl;g>#+Eouad092<#7DNM-8^zD-23z^U>VjWoA{Ls*Vk_5?Ho zSk+Kwn8ir|P2??!Q#P}q`5YdMoYwjy*hc@)MZ$+FGKya<1_{w{<`wz|Dq?9Um3k>Nhbt&|GaM)dnBxSTPYZ13 z^xim~x5KQkYF%pO*)fkX_o#EzcelCh-rZZvraRDrK2q(KUo`D1g+~|@qTbi)vl8>J z8$OK#kLYgxu+WcV}Fo_lXWGQ3d4C}P1+sS_d} zXyPQiVf_-3H zFp4slrq70c=X)Y0ShJM-kQK=C#5BPM$8B7xKNWQ|JgFCCToC8sFwK>PXBYo)hA<rIO1|Rh8lt36p}u27TXqcU|+d<)SCUn+Zlep*vWg>o&RC zQ|&~D!EndRxt>fzDX%xx_j@WnjhmCy%F|bkLQa^Hy@j<|$rJK)*hJ2xoQ)5ZR@RO^ zWlYN<<8t4IHe5Y9+2-yqX1Q&4?Qhp(oAp7;X2W#@2aG#=A$z;Kvpq3{QZ+kk^={sG zJ4>;iGGLQF=3oBLK6l%@H5G=8)z{l^5O!JZ6kkhM&&ssztR38?_g$R z$4blP<7EE*)Y!~}LvEeYprIROsT(64Y6pe4CEpJFZMH+kSbKvGnpFk50uRu}1j>SB z2ljG?++2HJw{GvW0C)WiMz+4yyjqjzg6q{L3o*rJAw9g;@g7x$9kfj`QY-=z#mUjw z3W}OG)=z90!%`>g=zwv>zPCf^(bH99I>M)Q-`y9L#B&4g)BikA!l8zrJee?m{mAkI z#V-1NZBQ=hzi}e%*YXJPYFH8bWO@eH?Y^%jt4^^6ztxf zdp&#eXpR*j*z9HB2s1}cWJY;5{f2DT>^46JA7l>~#WE?ZYX><}pt7V!J*%O1nUctlV(m zl3y@gVE$S^&)x|{Jj1JhYijEV5#?;*A(8qc>|sD3=+VtY^np8Gr{fnRvT3?BS>U^$ zhvoagcsm~0G0M{PNa28YhFRJdb$t)SW-B?X?g@?yuUn$WCp}V*ssLLlSqQGIPYTo^ zRN3J}v!O`~ko50Ve??t&JKjkV(Z~cNy&h?Mj##tX(N}YW;acIf;UIf?^h#~8)%tS$ zPz$DM5B=mU^5$Vqu|qC2HY1*K-m`6%7zvsKy=o7>|15X&ksg;9n2-R7@$BFvgP^0K zJg$t6ywh8E!wLMf%3bmowSwwjs3nnLJs}w>i|^0%OS^~h0u&WJ#+S4WeSsm=cq~C5 z-?LPYz0sU!pw*=5Y_%Fe?tIx4%&iyLexny)X*YQP-q zEAaB~l=|(i2@5LRZVU+aJ_otVd2xzW6C=ez=%4W=6ID(CPsY!|ciPgb!7entq?*m{ zYn;@W&DR}eZkM`5>jADl@(#eshq|yL?v3^+tckX_xxbOPQlj?0H1v;CS%q+84XCrx z9{B}5yuW)`a(0(6!T@~VFms_kI^YAcYS$AC%bQ&JbbW7k(%ARtiLUzJtrimyH{1j? zG@YHrT-tqSnfm)a(--bDySC-!9I-&~fd{dSu66cRok_!_d+oBXEpS+kek1U#4i^($6k286a~TguA+AVjY|{%ywNodc_1Ch~VBIZPrbcC-DG z__3Q}*?>=piBGQ`7Z=DO)oSttB<4Op`j4n0??hxi$XA9!nzrp7RFIozhu+Z<>9@|j zgxQ-p^1Oo_CyN0AcJ6%6n&2kKuy*IbHJWK%YFV3~TJ7ciN%_O-ooi!eoed^`{avIq8RE18N5E* znXa*a*J+^ebfhtRgl3T8y23dpJxIZiVa|)<7{?{*s)`EdtyGA;lq)b=~O%C1yauuu1I0p_-v4NNg9F5 zVxYW&JmMzKlk6$8C+we!EDzrzbW|JqKsWFu9RsJ?@%9Iu2zR6~Ya)3@#0kikd#|wY z2I?}L(>Qk^(m8cDHo(WD+P}<&{DzYWRyk#${+dtqWbJ*Ev};}J(ELShjIqKy&5S@z ztkGoJc>42NuiMLP;uEHVTd6`iPX^Mwtg>PF7T=7A%*(kL1Tyg1dp=jUGb19l3(_br za2(zbucZf(XFZ}!2Fl6<3)6`%^{MscsO3M>Vab{%27140txu>wN%f>EY?MKNDwVCQ zKQz{+Av8JND3S5S!HP58yWl8z=%b0fJtB*$46aVH{P>>q7=blSc2F^Y^tiSVurrFv- z+c_nLfQA2t5%avFv|2F~kep*-`YDt^Y6q+QM15yHHj3cSERd(x!-0zyhrMXxh<*D= zEgcTxvQycn_vaGET7;q)7ip*yIu>>9CKWmdZdFQ$^~_R1se?;OG%t*tLFeyC zE0(@5V8EijqOSL85q+)8D^ut6&Y)5xW0Vli&9#mrs_YTq$Gy6EbDz$Xb{=@FnyGbX zG!PWP&Ys3<9#*MrMW^ptk70TTt`TaRn#O-& ziGRc3A3E}%pE~2${YxMmS0Bx_yMe98YD6Ht=aa|Ad)*NkrXNVJsHBujRUjcFqkm(4 zEaUz<;$`u7_H7aSm;Dly+`b#KZJ^68`YQli`eiAPc#sc5JAdpPgzLD|rd*5M{WTD4 zjl4w8m_MYalH*FQD9DUph4XHGiPCtFI%Mq@sq1x$T`4xh?K)BWxP_q3@g6aA4_$ZC zxBDW4V8&3|=^~XrH&%fRkF~48^{>#^5E~SMyF&nM4|%0JQHCCwSy4{N z8yj>H^>8cJ#Ck_3U1zUYN61}s$;?L?rE^McO5{XoI_C*Dq{3f6<_%25Ql;8$X)KP< ze$*6_?V(eSWnS&R9deinAESOS=1$e*Y$w7Qa;7*P+Hx$V%kuTHU*-+=4n82d);8!- z({1KRtL9-GnOaIN23J4JRUaoVF%dd1@qBiOb>^j1 z&WIWR@{(~N13DA~lk!%ifoMvf9_@&E_QgDc7%D>;dhx4$R?3dI{^dtpYlX0y-5V?K zf17S5Afh@W=8ykbdF@BBCQJA0z^e<`DE)~AE(~pWKq&$D0a2;PE-iYG2v5F{Uk-uQ z?qrgEl~VlIUtbc5yADXn{nZ>W4+~HvS*j+ex~IPV@J&(9oBvhX-;G^+Y3V zcmsc<54(+hSLdw{j`z41O*9tR;&`R2spA=8Eq8IgQntv$jmA@p=^Gg6eV0do9gW|r zguX7W3AF2Yz7-Vj6jSd#S64q_#+EWKQscjwOI5PuB2wlIM&T}Q7&<9D zm-(aM(VGLz{PICFh0bmJ5m(G{*^C4R|2DleX6u*{qj9H0!;Z~%e*gNS=OBmj$FF7I z`#B8OogwH4?c>&lQhRzRReLu->+aN<0~#cif|k}z@<)Ni)#SUdOl3aYo~Nk4vW#I~ z@wVE>MjGzORUgkz`u6Rl>QLEq)ATVxi>FQPr*HdMZQb3j+J(zl&nJk?L#pH#l?9g8 zepb;Yj59z_aSICG8*HJ;Z_9J#f=Ks6^ z_hr@)<{@=1g7p&11q0b00tP`gyomL&k25hHQ?MqZEut*60^tIi1I4eCM`5!LIj&Hl zu{A!}bdzmk2eRPj(so%B+~TXF)wvnI`{_zE7&zZ@OjRKZhd&#mcwNJ(0LR z%eeg)ocrt_^Gyp9n{N2s#rVzarlJhPz9>Q$tB8-_6+bcWmRQv|aO%!Rfk_f^CFi{~ zxM(~vfYbD{>5=pRw^I1&I=g)DKxl|~9AyylKgzuzMvqzH&)tSgN8*tD{;&AcH)SUh z0k{+d@3(MDwd^SP$HeAB5B?SNtN95%D_bUH`@w3{2|q8ei-dA8{H5dI?4VLG3)Md^ zcJ$&qcmx8$U6warQGYGPP`Uex!iZ-L6`MSem^+?LqAa5b@MvVd87N@N~ai7~xV#n=Dr6U|7*)#_-U!R8|VItiQY<-7i zqq-Pv7@afFXEBnc?xp=d)6yP%~Q&l$<&4ORGFOVFb`qcej5zfnn*G#DnT6^ zt_o4I4_1AhuXv~nQ6mK@UFXm6BuCbczQayMW|Ig6i*t#Pr`=C7Q*ii0`h=)O>nH0y ziNvuWUL)=OM0}|jHUfSazOH0aMTz+^?TvQbp00;RYq8S-LEpS~{@S)8e++&{MI`YT zez_OgYGD4Vyy85X1biuP#~&T1iB3;VX%ap;1$?yLpn}lm{yh9)GX(~jRM05l!%NAnOQivR ztxgHaIC)qLm`@243bnP7^2$G$21-AJ^UBZqxsf9J41uI+>rCghOtm>WyNm{5u)pi% zks^6a;mPxFmg?bh@{+q-3KpTD+?mkcg#>*sH5 zHV%)19mk=nBSe;yoY^2g&7M1aUXFCSriGh{#C#XkK}@gGH4kr{YkGRaDfx|9lEu>s z<)oK4NiC_xu&(mLt#B@(eRKeggDvP>>lzYaQd6Cu^|d%+SZN_(a-;LCp1g~~?uGfu zXEgT1Ki7v&9)Amar}Hj1hhqF+AaCz>FpoTE5y}a+-8B}AFr&SsInL=`RaYIzZ;)}n#pt0 ztR@Ok@1j2svUBlE-|e;2_?rEWL?u32TeMhk2!1I1EG)#)oz9$+=j;x-Tu_TicrGL- z&gKc#`I1YOTgE+4z*a~X)t1L9?cNd`Y;WVfAvww_5 zFxr(!xSt2Szc(a2WAI*SwUqSn7*Uga6GRo%K-4oo#A!eDK4NJaW$bT_zqz4D&H1Xu z>!Q@d$6l#iiRdG$wHmfk4Xb7}S*UTRSFiu_xQcHD)LK`wd!A{rU67S}p4RC{ z(3d($sD(WDE!!!EHoy-W9!+$9AHAVbJ(}s7blcgkX1(z~C_GqJi;TP*26zbOTcd!K z<6AD;8(j*qs@l^=S5IC~JHJ`Bj;|bssMS<7t?&CUJo}ySdAq_x(TKvjF#{NHlNAvk z)DXWZ?na-eYoY2n>7$a4E;4X-r=Hc3zPv&srWXCs(Ve95!QaPmqtsGT9!K`LUFqjn zwn_|}J%fMOJjeB)=U9#Q$sv9Qk{s1b5fs##3Z0~eJXfQ|Sa(L2=iAwx`4Xv9Dh)_k zaQuEkufC)bwmh0@y4~BUcC}mPhEJF(Mt6I8&P*)(LG}$w zXULH|OVm_%+T;z5M|=G~mUxqgQ^(zvmrrXH*J$yaC3k7{%6&B`+)06s^cSIo#cUDv zB;LMS6k4(*Ni+R>1$Ly?7-xkE+lZZ%cQ-Hz?E{mFI1&HG;Zc5#AI z=%h)c=k^^9HA2AwyjZ#O9c8g8Q5(Gz&D?N{`g^T~`6MyRYE@nQfrZu}XN<{Zgy5qH zD142P_cmdzp`y0NKu`$&=o^mqO|@CWy|^z@W)W5Any_fngzWKR2?DH}v%O(`-0m|y z;-tyU9xpU!t5>*?3vRbCGa7Hz*P}9rvd6H>>+IRf$~?U{-i0GMQVpzq>7g=d#{&_kb@PnHlvPGGbzgu%g9`D{Am8OxGC#+ziencyOiihA-!kwr~J7bi7qE=D)1 zcxDR=^SW&2rHj`#K9zoFsZbBp8DeJghnq`04IYfs4*G?<9mL2~O-WO>p2;Ri;6sxs zZZ1J-^QGSX^#$$1?GHZ`)oN6!DJ|zg>89&IiMi{;ZkP$HZfr^%)gC?kB>KD_;YucD7UOoUGs4Sp+36Pn83pP3 z5be60Tyb6zm4v4&dLS7lv$;hSiW824d`c?GrZXzF5WBUsfZ4~HJL~Rr!r)Nc*iDI- zEL!lhZ~u>|cMk5W>)wYO+qN6qcJqndIB9GrjcqixjmBtfHg0S?X>7lz_xJhDJJbI% zoyj>_XYIAGbzwYv3933L-{Ris<3J`b;@irZ6^Y$;4ofIMn_p4JB>v|EMHAK4l3P&T zOvqbKg}2?GcrxxdAWCafZdAIi$&xX3synMr(zw|zK+Udd@8@uh%mr{l2OtoCds0ks z?(IOp&E?M7$udI5_d)Fzet1xOuT2o}(=RzQczwe;7I9~7(Zo*{j;x67%9xa({2a`p=ze;&UuI~@7h=0NyUzVy%(G<&udy)Z-Y)7 zddt5@v5-h1pU{~1RT0v;jWHEF>!1?2V0%pu<#x-0WO4N{x7_^R-T!a}>BBT|TgN2; zoQlB*M?FNvPvQAVd88)Cr8*6Z>8&20pR%7?X4cJz^MZ}khH1>yCydme=xk<*`ATGa z_%k=jG9sO>I-)#Wo(fy4Gh<+um4|s)oX6#miM(irlSzhSb6(K=^sY1(UQhAQd0vpK zUssdoI2Qw8&b?lQkeLK|R0SLqjtSqLwhcmGkstir{Z>@i!XHI?rWD_J0oL^6`|M_E zaTS%Gp5Fgvn_3#!XUOfmfjH<`!FP?FD@X)v2H>?RMPkvX*8S~o=6%R=MpM~(K)#i~ z#0|{yCv5s8&C-HJjUETuCc}<$I%WLmuNx*Fi>Ac@J32brs^+)bf4!M9?hB3xehut3 z!w0Jw2$Mh(_Xknxd2Ml&FrkrCrrcX(lPwn+eX+IZXvj(63_t*VqukBEkFdjR(^Oo5gF_=Bt8!gfWjeeB49DW@$-h=&Oj3T{8l2Tw|h^ zPhlK56_#6;4@_7LZ&JH(OJ=4)H{L$oWXFRXard6%M%y#vx$lSFx#tHHekQ@w)s65$ zhxnwioCzZ1G`7Cx3wTxCSMQS8nJkr(WI5w6SJ9RIBX|2M7jq9kK%Ldvy$*3N%q_Wo zId4<1_jjAu?cBblgJJ(o*w=W|q;#E~z^w}t6VqwUaoAKolsYB(suzaja%nf#`F`2_ z!dqe!te@rCC$w*3?`(ZgP8$|r)M++apmo3B)i~;LJ2PNVE10~$KE3Wy3vc%!Fz?m2 zHheFx@XW*j(YiSiBKinW9Y2n*C_oR~9jm2>|rUz>yOT<$*6 zG5X})k^jS4;ipKXi&MiphwDptF{KQw=g@(wPWz9wh(u`5C^%7BV}4(XNO4qgI=a(7 z(qv??>AxP3g!Lz=1z|Qg!O_{m>%mMay2=V8TFjv2!)g!w;U|x-_Soe zK62eluJPhlsX7Wcnv-~q4;;q&bjoNdw?2ekg}WZ+9+W!4e3+|!&xCe6RKm zE~A(~0$Z?3DK7zM^PT_Sq4)Hcw796XHT~{%(RtlTV2w zjMsT<7?rG0&#gM*a>?-Sc8ZRj;Y0=}AngX|I_~GKheszXzdC#C8Hhdh(Ck{aLtO%@ z81joil;~?Nv+}Rksof@4gDqsSg+)cd^fes`p`p;ai~qc+&2v0L-37YB<_I1;pg%4$ zouAxQ2}E;8JPdwrWpa1H5eH4)^N`rv^t)o|O=IfE?sX*6vmi;6?>ih^nZBOQ(`A8O zvDCIwiO4pGx$}f`rx%=^;Vgeu0*#EPr@g^5-Z#3|U%h)Nd62D0;RMopn7WeTYWrK< zcRb=Q^2EHJzk;RL6shbjak00x)5?8TeoJzaC++#Urb|c`*I)Vdr+?h;w>GAml~p(% zg_3uvIgVv?L=_@zK@H%uFvJs6GXCq62H@D zrwPX!mlBTYoNkVOuR%ZV`a80(c)gcE{j}Vd>)1cSU3*}#XhM{;%g0cmGsI}W-uHH~ zR>Mew4_qXmew!oYP7Uk1tq;v>7|7Gj6DX2Fjvk6rCgx!RmTIz0t-`2(5Z=a%W0!%hmX)K1|`u5$J^{am8f=$2@P?|b>px$&W6p`58 z%4$c^?-@f~T^&TrK0LiicP&kAB0aYKBld)r5guKBN%wKqueR$a(YVWffc^U8=a@o} z7Ly9>*J^v?`y}up)Yto7EbkVx%Qm7IMh_!t)>(xpBM*7_;=F^WFpEtG)DrIKBZ6eWo+4LiehY;0Z=iJtVybH;&(=b)b}_aa((HF@`U{9e*npvpM@wK^m?O z6=Ev)f0L(V%k$sLdrLTFtEz~Kw7`s2kzhcWU)3)i4mxgg*GnxjtmVWV4@cE#gG3L= z?+yoJX7Y+3%K1_)a4qxxR^)fspMk=jJh55ho9p&-a~s9}BVd9tlZ?lm;1i1XVs4lWPNq%&9u-g=?{0v-%cBhprVVhLPJbC)h{%%Au)pN;$ zx7CuEgNrNz*{Ye1z+{F)ns#^hhwb?7InzN?bHiu!UMwRJ*baQJ;n^7`HTs@5cCuE1 zWSn(z8s-l+GUCI|z`%gc9tXW-Z-;k?L+86`4P)3KtwQswPU`OAnYLo@+Pfe*)*dqP zRRAJwK^7b#-q?HCZ}(f;2>yCGJEP8K&@0^E5JUghg|E-K64`4xx|CFw%Hn7d!u zHs8+Ow3_XR*4Ni5gqiw#mRiuJUz|TYLX0fll$DhOHZO)b}O_Y#pUHEz>1Ku zYO0y~ICQ(b({e}Db;@`x(BBS>(qhELmn}|K-qxNDAXnOajDJ7|#wz)qz#8up2sUiT=uUhZPe2J)1#!`%coL z76G@jE@ws}@Ow?F*`{3@9A!t>wyVCujc3jEF%cGJE`xM?FxD@6ny&mbugT7^#FDIm zlrEk^$7rQXoxv&OG-%rv@{?E1qD7OuW?+@^KKHfSaZ5s zf>2RoV~i({Czb*D65G{E-PQz7>ji*GIAU3vJp^=u@0M>%Ou9C!laM)4n$eF_z9~;0 z?PJNdtCrk9$yQEJPK?&uT-CmQW%IZ;>3ZE~Hr9gCjMOf7FRxDnzGG}TB=(BN%mfg; zIGtw#wEBc-3S-peeW*9Ow0IQCL}@4mBzs#MZZ_!887rBv+#gq#@iTOZ?Yl==RZiu;yOF79YeH|=!jI8)RLPJBl98cbj zo}Df2s#8+n8=Y5cVj3M$4Htr$K|{m?hPx<*4BgRr+#mqjyqh@nXgHDAZgbJr`@eVO z3}Q>)8>O$=+wi*Be?G0DjtBXlmQcTT0xGcc?hoHCVA?1>4L}S~i7Y4}$7a&~jlkT| zGn!0)g=8G!1Sr7;-scbin&|Dj7-^Hn+j+JwK!McvI^*o}y2HwQuN%9^Cm^6@Vlq3K zE4Nvy4FEoLnAIkfjwnjZQvxwHPM9mN$g2!^TC5aC>6L_FDt05u!lyaAG5tEFIGK1B zc?T@8?y#&{W(sUwep|HTTXz~X@YvG?Wd$Ht|QrN8M7>gwMnJ z@LH?$2qO*dHP7hSSh}DWS80|L{9%q4lc}jG;5H0#V~6#vJus(8T4zpNwmEgOWtm!t zBf}^77`ZWRM1_cRjd2Lp2s(v@84DGp37?< z+=w?aS6C@(5z_~v2%NtOb?!3CZ6wwyK1bfC8^Pkdv3IF~VrhgrcN~~j`YLL=WJpne zhsOe>RRQ9|f+(iG`ZL2HnboKDhW*g`?%Pm5D=K*3Zf8AqBB(>}x&RZs28V4iVz0lf zLoq~|BU$@xOaPr?Jc~DxB31yM6H()*8r>jJCtCF;aKNQv>z`%gMdw3K*9~WeAXn=K zv0Bbo*qA~kx@-xtTcPIAq;w$%XPU;@$5c1^T9OUmw$-egnZE8w}7k9>N#`YXknEVYn)gxVl;%0E zT;?QxR88!+ZW!QJnTfAQe$6v0rsk8XE-8+SR zp9=}_^tJ|xhAi(rDL)=m1RsPK;fV!DtvDzEyX$=q55m_~ljv7x)C`G1(FtpDGO-Dm_}-=L1H?cB(?P29N1MQfQpumC~C2SYGO zAE!9TuaU2WD8$b0Lch2D18^nALtde`{l{A@|H9`=ZQZ4HYIWW_1BY{me%N%Ybe|A` zxL9983sKl@&(C)#0<)6Fja;2oRjE%TMw=7bJkb38 z>t@}#Ex%|%kl>`wV^u^v8EgChz3PJ9N!;bw=Tg2j1N$=fXW`!`ISp>YjdnRqO#u5C zN@B7E5LX*uGZPLsFSh#crpZpdZE#2ST{}+~(g^gUH*% z(d_7*n7!KT@7R;pl#N{>9Zd=n>I5NDxuA_o0Zi~=>FL@NV{qFaNj58?-Id|a*pE-S zsiTcp5)F%=3nri@c=`;#&NIjjkE4@$QD7VM*y6|Zo|IQu(prFN7?gAdB7F^z9Fu0; zV;t!>I9&XHkNTk5&RM6^S8R`s@{5tyDpR$6GZdurRY;d${f+C>Vu*U?E``0Zo5rUs#l4rVQmCnQ5!Lpk=^K1AprESrl?fBhBxHP`meg zE3C7#5mD^GTT<$%ACdwE%k;tt4%X$Hrqrh7{+||Qq`!~&L7~^dw)E=>03$m z(e<&0?v4_xfs&&TwX2b>E4@nR0-KuTLTDAPQixh#x|iU8aVF@e+P}Q9pQ6BTxJ*k( zeZR!=z3DHE7_?pC!{BHfdRnw6lw;ffGZRm`$KAS>1Yh77RgQ7v{?S#V!piFQN3_dZ zEjlnR#E?JSB%uIfDle~y#+GH7_^Jkzc=p?gq=v|N!8aZH7mW&p~ z-(4819UhiTlqzWr#Ldf8+x2`#cfTCIDy+m>Nu|r!=Z47ZOmJo>+Hczc**Hnb$!HN0 z_qr7^y&7hfRGY$&G96@Cu~j*uIt*p&CPD}_EvMzb8tqQaru9hb?uw(_&$Fx6ZXBpu zA}Gbj3U$>FdLKJV|}sUj|vOsTO-=MC!y+QO}IoRj+6pYDSX$jabS6b3%V`*PX}ZE#IACN=T1aU2I$SDXr6yi=Mish<>J6 z@^`6#(AsqD;OI8+6|(CKZrJ+$B`V6?@{Cbn3Ytqw=;{e&-dhw~^vc*UGzqEcir}SyZn*S9A0e z)U@N1&8r2T$G$4-9x@h*ATgKsEi0K~U>P?`jmuLW9k*Vb5`h?p7p}qRa0llNv|lA* zK(?`QSc4v~01MwY=0R+Dubspo3JzLTQo*>s zhGhQlhy$e=?Y@*coq7`6uyyANHV_Um(7k~7qX-{2{ubq4P=$AKW%KT<@OfV^68MULKAYP|NIPxF{a;i&xV)0|s+7coJ6xE7#x;g`e zFS%lCXKag;a)z$B!q8^VTS~$nP;9LY-8Kt1_;wM!9p}t%Nkbx@zt~?^WnI?ghIXnB zOD5;A11XtReFdc{#4JBMba~WX>@qC4;-zqNe2l(7!&?!QL#u4GYt%vqs(%On1dee| z!Aqy$V2xZoOjXat#(-uNW`j-zx3jx@{Pwaw6|+gEo<^>QMb5?grS@}LT`)~P%bLwt z6=uS4jeAx|1GZ87{bf5GS?m3X^;7?<-=C?(f78r;%qL4byOQeanCok&$jHd=RS-#| zu*vV`-fo!8{1~*WgCqBg&lkc=Q$gEpGANNRo;&;Oyl0t;5A+f=y{q=u#xj#F*UK*c zZT{6cn2XGw1m4iFz0NiE+pQWEJ>21>=%_!u9_JCTbiJ%NyLK>TV8@dr^_}@;VAuRb zeWYneljbI~eutepyF2>?bg^V~IyBgS^pfw>_QfB+tU0LI(6wXo`yo`%!_RY>z40KX zi@*k`zc{Z&J{Z1Q!^)zr;ER3!9?vQrfZ=dNd(lg5x*Z>wwW9RdIFCI0iB8L6T9ufp znGI*r1?{|=U%Y#G>c%?fu$6NP>GX7^dLr9=Gh*@f#JUEkiHuHakzmxclBWm;oS;q( z+QDykU!J3qdxJ+NeK+2PG+^G(oH0clVCj`V7NL%l#WyA;=YhOJAA{w6%KQ&gF|V#q$W@vY!pfH$`ucOTVug8eLJ;VFFYGXX8~X%{ zngZiC?vw?hpb44Q1mEB(l!>yiwHT~33P17CY|z*VZaYq!6`kgnUWy+(p{ev949i%=o@Nn9${IQQ6#UVbsODRm zjK&+M-F1e<^Tl9Uzr@Yy(l-G6e$lN@I{)%F%_K4jf94N& z(7);x(gEK~EP(obh%knxw(zBFEn^i&jvi#*p75;yCpvm1#r?Cgjtcr(Wm3~Z zdMN1@=TW3KF6Os1=$WI2xz?Dm=qL6btbyyGr48Z&E5PupAUN|lx2SF%zqAM5A97#0 zKN9S?J2nZu#?tIrpmm+RLs0(C4l-hjCcPJ66kLUj7?2KUigV!YPfXZf_nZ6u&KTl+ zhW^)NCf=z;ijweB>H1l|*t9OGTHsVpk%=o_j=PO4~ci_E+JJxCBucgH%&(0T0$0+)*T*n)f-Tdvp>e=4hi2c3UL|e zyE|9l`zg4%EzguOZou@6py>Sy$=&5iL3D-e^wydsaA!jNGGwfKxdl~Orj>91AL{ql z3yeIt$B_^~D~`|4zrX@$ra2G?;@-KTs26Mj0o-!%qF+ArCm=q`CN+8F#OqCbbW$W} zM(#{XZc8jxxEc@_rQDlscJwOvOT5ZS zO75#2zxP<=usgpBlJz)>;1egccfwX=#t0v~w&E-Mph1-?2qzyd2n!?IfH~ zH4d(5@>rA6TmAa<-&+*IB$WZEdH^)k=6V`F`~t5_g$`#_hpSquFnD-_q!SEi`CodP zet`q^P&1II;E9gh(bf@hW2Y_m$2t$xu7t;r*>`(4HCgg)=oL?^z-aj0l*cC>ohd(s z)+M=mU{)~Fq!v$+fn>G5d|fGs!^Gaee23l#-!Bn`-E7D9#>~nMaYdL;G>^+4o_YV#dUB=ZC;^KAQFumb^$VMJaGP_U6D4UFQH17t&Z4-1o2d7f!%zZkVv-p zUwiSP28v;BVZmsX>&jg~*4u_UT@JudvB3j+YK7fzkESWm8B}DNY?c$W@*yp(qDoR> zICDvoe#d+`gv5Y7G3c9dUQYYIX7hD);?43IW8*Mt%dy0r_X&hja(^rE#Gy)Zg4_SX zM%$$~Y}$7Vr**cRB3NLI4&;SZ2BW$(y4*%WNsEC__$rjCaOLdwS+^)gTovlPZxMMM2>$ z-~_Zy^2MaNL!vNw)Lv+vRFQcN1XEZ%y;rR4}Zsh(gpHfb)kN&@SkTIEA{&$L3DgPNgNv* zA41n0U;iq!)?_KifUt;inZ=8AlJlXJN^W*OH$-~9x`P9zCrdP^Pvnr^+Xe>avc?<~ zxD3?%O8kUJr`2+?BT3?=CK4echa zp&w<kT=N=;Vaqv4bX4K*RER7Tb!{ zgK5=h)^I+_RoFk6wLf@AIHzWglX5G_JX{bOzG^0!YKpOaE}8q+-+*URkpUugfR5p{ z2a&7-V6zOeLk`r?fV;8Qg)G(EGraGDGAi7Xu7@{LB6U+;eqKn`-~MRtr=CKUDpcTr>A&oQ%Tdi?yo>n$u6_IAjW1~cA2;_oj_ znqt@x@i*O{n29JkSsw4nDlIYm1(M;FK2QT~Mp}NSHfKduU5xDMZnNm%xyKNu-#qA* zmq>dW`w1KOiFK#V;~h=>MKB<9ne$5vVEBiJ!U=vSQ8w}>O?q?oabi$mZMz!k#>(1= zF<=}Te$RZ&@PXI?K?dfCpl}vif4A%fWR@J?xt+EJaZ2>?fFdCWe>m#G zslcrj#bQs((0m=hD*iR@NKVNRL8hCt#s?(}?PlVnDnN?Kq#@hx9pS`fSPj zx}E;xQP8s25I1nzLugGupo~!Rszn*MBpwYnq&P(DOpDgqK=G>c`uJg3{UN~XwIs}*K)oPd`w;R+_X@De?_&g2U zGpDSIM}RlhNwQX9|9_eZ0xFJ50K5$^y(f8wJ)BM-Rk*xL8%L)%jQ2KSQR+mZ08ak>veJVuw4W@>%dIFoQ zdUT`X%kwjjowj`VhEJ0L`>Onhz!GO8l{s@+VF%CX4&3}4wwqE@^^XWwu%gp3xso<# zhBk4>gzpJ0n|lM^Br-)6$|R~Sc!dW@IUjXy5|)`&?%d|@@A5vKVYCy${^B*h#eV6C z_$wG%1w|}><=Z798m-l?f{nd`>wl@b8F&iKTsd-mRJ5y?7YJqV%qH8kIrT@`L5ABTQN%3tG^3 zfFP`w2a7Ui9O7If+vNtRX?xuYZ_I>l-*ltNkN{CneAfskaPBu;K!yXmzNDm24wJlx zwN~?V3)Om=XegnEEPn6a3jAjrBsFz4A~zkDJRD9NKPr=LAj z*BvbD`_$dxDWs9bOXSbqtu$n9Z+nJ8Djk;V{la3`apIu8g2B-={FF+tX(a2z?e{(y zp?jIAQEne|&1^nZGr_^e$v7hJpWOKV{ zha%UcEmCPZ?d=--C|vt?U^em5p=S@_No1KAg1orM*I7`INYH1vHc0vlEDb%}_We-Ect+aT`XW)&i}qS$S!J2Y z{m?QbMz4>jEknQ2Ro39~QmkRt%yYG;LdzSZzQM=yCyLEpW0Zs65mF~Y|F%pVFADx5 z2A8Ik=-Z(Lc3b78IYzrfKL8|x*BRKpFb(WpMe|e}4iye(wI1N2 zWIHCdZ)C!Ueji*RaF-k5rB1#v*c6Jsu8SYv%-)#)OGNF_-Q@=*{Cq6TbF-!;;53Va z=XSd;@HT|~ewHBqxP5`+yuymna$VBAw1B(65JvBDea8qf00Tc7X%yVaL~`i5Q^sR_ z>O%vT_?J)S?E7=7;N_g?zB>(|L$O7TC`PH;7%vqMmHG>!a?~PWIQ;>RizlzMaQ)~O zW9XJOOGrQRI%YrcOO?XczQ_f=5QWb(R_|dYg(tRKHOdksCw}>xfU~2!nlZ7CbC??L z$;2!m@0*M0LW4G6IR^%0*PA`Xhy)cCoCl*=I7$?{3*D3xDJqwP_xAP z-!D!Jv%Y`fJ!@wzFMRN!$PJO+uU{}wXi#R^X`lI&BJ6Zyr$=wkB4;tj6S`|%;vWYx z>B_~4x^R8N?LNW~A`+zinQ7FC;SwN$cK=XO!RjhNoJ&2MC!8b*M`D@%4!e3&KMr{FLvY`3$}Y(YMp0jL%KB#<)WYQ}IdY{VcT%*4*-z-*pBM zx;->}nvkvDsG=<|-x}Nkk6s5=ZhbCvS+H9g@lY;Gf|>VH6Gc1Rnfw?yHZQ&U!?uNY z<25nvqr_GhyjHNrAX>ZISr}3u?UQB3&)|agI zrV7|_(tTOKC^BbXwT7pG8a;kn?TWQl?jiZOW|g;f^!0qoCln}l zNPUqdTk5FFL|NoW#JRt|kA#j3bXc0YGZ0QQSnG`TdmAFGemV0J_x&!lCO(=%0|xEO zQ1fF-h2d_}r0ExN&54d7w`Wv-n2Ke-31YRO(ELpke{LqagN)6UosU`bcJTt?4+%0y ztrD|(WqFajOQS21S48c#*YPCIyaL>%H+9zdK+qRV$bM0HSp@d(_85K<7evXAV61YJ zNKH3}#KG!c`N^6Zl^6IK2}zfCt(~+yL(P~v#koJ=3c=Sr=ko^=EHx+$riGlm(3R%& z2eR&*p(9bC#3CB8hAv1MBOE07{Jpt$enJkf^#X zj@$btyh0jxsP$Ct#KpyXc&@$|WVc^gLYTqI07kc9`mw1;<@#ETX*;FrWL{zi0K9lI zNQ=6K=ZGZO7G|b(^~O@kFSw6hl{dD>xj>yk(XI$G<=>N@Cc*RrS)=mxR(Sg^7YmC` z_tTU%ExGgopzci#{^D0Y4=8;GN+xSHC>?c6Pcdf58R=~eK|EGgUo))*2~`YOB&(;2Nx#>OC?eU$GScQwdxel*jnahyP2JS z_%K}kRa?60YyA&f_l=I=;`Q|9W!fok;i=|9^w#rjM>JB?lpSR*cu*U$9sIf|A3p=! zx7J^9{Pfi7YFAs$7b6|{n7yOgGUiNK4mGJR0;9(UuOq>d!=(9LY}gk(o^n_5ObWY@ zMChDwy0el?;K!gq{y=w6!Ip)Adrm4U+2@yde?~~Ga!lY~$!UFIS$LV@H)H||$iS!e zgT%&!+oapSuI*s-A`vygfL;qt4D~vc$75*3MY1hU4DhloZf({QCj0j>17u2&SxV{C zCGFclVkGrq7;*%xmeY9Mn&+wC)#I+B!~4s|Fi!XF#hTL%xBH_aidhz(Ou*?iFs%Lv z4Lw_XyUD$NM_?u=j=HJFqHnpsSwuyI=Dp~+c|72@KgjVqFR7@A0%$|e@9nXnRXWHB zim=2%3)Q$EPrHlm_rI{a{LZA;Zhn8g{H+|C5zkoh^7P>uaX`&vK9Ry?rogzv->oUm zW_q()+H|w8NI3lQYKL){%a=G~#7w`~WSX!Nh8CmQv4pwfLp6KM7+LRZ$NH{G(vSf` zS(F-lNsft_v~-&UI?)EdmBkR&SG$k9^N>x8X6_r>^SQ#IUN=1WrW-*4fyA$d$3E+q z7YsUQUq_Sm_#Mo_8`J-Hu;J>-XW4W;MlbP^#SqpY%&IU-DF z;=(Z=Kp4R4WGLW<#%mt>tb@JiW$KaEFb6wiI zs2;#|d?d^j-(-bo@y&F%1z)u{rbLQvq*PAsS=}#ov9;WGCJ?K&_~sv}1|NIM#*9AV zA^yw|WwiQ*W^l7qt`|Qg$eizU@wSSmjMu#owhV$lb zsPJZ9Zrp#E7`O?g5Uanz?MxJk=1|LWDhvde*Zp6^{ZX8!Z4|%YsS6VIPM9sP%?~iZ zMh-|GcItgPyt<@_HT~(K_uwH$wJ!jWn4B7tC=+QHs&uAyV}(WCH26WMmwZPmmw(6F zdAFawTFtmQ7xYDt1>9KyDZ2Z5;u7xm z&vrA2dMIM8m>5HoKR@+zT)wbx^9KERg?_O?jEoY!_>wO@%NtKlnhY{54HY1tiL){2 z7F7JrQzD!Sc|kF9WrV?n9IRNJ4yx-DahE{Q@_ z7PuQ7`uS5*U?RPqLe(g99Ya!HDW#-rv8MrDIVj$0B)m6)TNX72 zSHoK_b}$VsmYo}&%npA*K4?fJPAgg1Df|C%0qSt|7x$OCgqm=$>MEO4U5krxDFnz~ z_)41v91)rYoW|n(U8Ks)r*I=R{*B`R>GpSUvWXuu7QsUIh;8#*_{}7L>S=X*sG{G& zNe7)o%&x=j&Y<^0`{uzt$p;DmG8R`=A(4@y|2RkiLZ?(@syojSbe%>9UvEp@uFmZJ zQn|l&3nh^_E$MQiM+z9{#NJO@D3p;xt-9T#;PEaO8LfWT>TV@~{A)K7Duw4&_dR&$ z-u&couKaV9d{|oytxZO2SoN`}?(vSI2W<>Aiv+D?xz&5AIX<&(^(PH~^l_$@!R{Qq zt;h#vQd=A*F~r!|>LtVn+JnjwZrIe8xs>Ah@`o9Uia7rB9 zfAgj)jMj7I=Y`|O2qxj$r$MWl%a84wYU)QRS;7t~<^G6B>Lgk_AO$Q2RPVVm8_>>T z&3-EFN9#(?(T(YRFx`AEcw`UdLSss#$CIOP4(MWS>@yKxr(5$ap?sn4$jQ$e`I+JW ziR^;kZPZD7f|hriLhq;&G@~4EmiK#5c#8tF&Z+4&d0jt)4nshffNI$uO&_&<230Rucx zpTh0S>8$h9=8}}Np^CIXQPQ9_Vd9J3yJ2z*p+?>1K!}H-N~r0c9yvyp3^h`^j&Qb2 zgY8697K|b-Cpo38p{)I+0}6&os_SP*?-ud~1*$iX#A6?rD-~6_$-`sVG&483XXnzH z!Rh0()^0|EuZrp|WP=GpG(RHOEV+~V>AQ;OzI#2vz@aEe8x0`M6kCpmw=Q0&XM;#F zZy{fk#?!?b3vpN(&>Oy6(XJa3@Okpmp&M$DiZpKRmOS?Cpg2svI=8&XpRyK@++DQh zAjEdXwGfpKX!L31KGL~Wcc}jVr|1f~q`)Q%GdmP5iabW217%;NTU8I6yaJCU6d+2l z_r1~a?;XW40P6p3h`aTeU~@02AmWWkOVe0EBKAHO zxdRdRlq_qt5Z2Q+Pq_r$MQ4@0%|vq4gr!CBbeP7-Q}XX0))7dl31wm_tV~e^(!j}XeC(M zFVD3u%&Zr`>Mly+f{)emoK`n1HCkhoG79{Vjcr%c=$rXB5Mz^i{i0vbQg;AZEH{Md zbUppZ&yItB*mPqCMadIznHV&+TyQ@eq%8MxiOkGTPc|{$1*MRHMmoEnyA4xfut_{{ znIe*UXHP3UR*2XN;;k-rb{P+bQMmWAK9`U^u6cRn)kGmdz zb0~Shvf5Mfi*sKa6Ui}Mk!r`w>-U=}ib;+7=4eI0ghAh!=a0@Um(0#Wi)KV3va5}Z zg(Uu?P7el7Oc|3M4IU_C-5P{#M^~|P5Ph+O9(QRm%t!LFK^C0n31GaQ5N|X4C2aVV z#GI7%lYXrnLe25~?!TP{Kt{xi@=!GzuUMC70PFVsg&i%v1r_hq_>6~%++U$_rY?tT zTvLHZ5RXPq2p!UU{&C9Wd;J(AH_yyA-@U+jzwbf#GqNfxzy=rEIVL?Qb3!scl<$^K7L#tIrf;TilY7N zR!1i)$O-El_ek1Y1n$YXdM^;3-drD=0@ z%-J3X$0|EcP;0sLQd9dxle~^gYXK)YYi>il>{x1A5072fc?%t_loEc%x@r^ zF^9W!Z|Z`j8Xni;FNbL#_aW=<^*#&$^L@hm%T}c8r{vfb$KL-nDy)u-8mF~YmAw{@c3GR^eXmXF@{qd!O>!G<+QOSUK z=-6#p*@k`aes9o7YqX@3d@#2};))yQ^46pFxw7+XFSb?hQI8O_QVgiaO z*~tnr$hjm>r=e2jBV()eIo)NQV$9wIQX|d%nvp#l#A@bZlZU2tUI6MQT_YfdZ$(2@ zpbP%(9a^!q<2HZ2jBseg4?nD)#nK(U(-V(5#O(p6CgcT2?Q^Y@sPFfDWgbhfFK6j6 z8mpmX0!2pP>Ds2#`?5}#^*5P}p;uWCyZz@bWwCrq-xuW58$*Q0*(92u^HR#+aTT*8FG+&fT%1N4qyHd({5Yy zBJ~a41!dCKy40P+`E<4%y?g=YQp~scNflykRe1v&1`4@-nC- zOct%sHZ0H1p}$6~%_vY6g3A@>VwFUWF6sYkhhX*G z9VG?zOVGM)eFymw)XI@cj;w%GlW*3lgbZnfHhxI4Su=t zkkcQ1JiFXb8Zw;g9wx>F!rfQbn+t~=8LxL7)}D1ycXzcW>z9tgP=_;5<( zL_rn2O?NEzbsDy5w>j0cBDw}N6Q#P5=tV3h@|JF=O{i{qOwqj2{@URwS~Oab+l^t} zR&M$k?P8hny94%JFF=4|WA>1&k&1(?ziX=Ay`(Lp*vF?sO<5EOv(^!$qOzpbQq|@w zc?|d3Ez~CfAzl_n`3}fmbkZ(5I0(>zW39D6>-%i{4_hGpMTN%1xcS9zZLe$o21Oc+ zp=N5>t^3AyWp&j95Y*BzF@1qe*AFqiJbFiw;_E+K8}d_?tCf{ zXw@b%9H82=&=cO|G@~Ov5Y;TMjPhT0rRoCDXH2=Vak#kalWT!(zO*$RnwdCWR)l$Nd2EaY%WSF0MjfyGdQQoejp{!Arf$(0>5(w($KYU^jB++;FJWRgRJ6wQ2Rexu zoOVg$AMND~>hHSy*V*RX{ml9=gTeQ`>P!SHTt_*i|Lbi>#})pzJLRvUNsQ9`zW6~Q*b8|$f{v)kkghsZ zVXfWEUhrO3*h572S~M; z?gY2s&W{9lw>h~}GgZk84^Y*8TK4X>zGa?a9cE+AmDEcjZl2MV!%IPpRML7-MOjz? zhos88qCGF_SDZe)G%bwB={gn_KSMzG^-)#J((!A7Bj$q3Lb6)wUUXe2r7!c|8TB7( z3==7PoJ2x=*LVfwh|~lglK%L?qzrNgO&Ug_%Ge7-7epF9)z1~wMkcr_apID5wO{n` z%-MlZT#XZYf`L(0q%k!u$fq5DbWGv_7PVo+*@K&Xl+7Gi3{lk}=i`}89wu74`hkdH*?M$nh zaUCi~V8CLv8;)D=nu*O%uha-e2w6?WWX6Gz!0{g$7({W{^Pmw03Kju_^PY>Pb@zF$ zUimHxAF=B9k)o$}4l1C(;ac5##e=8YA4KIZJDoM|L01Z5K`O>5wn?KhTpikN!+gE$ zzPm*v&){lEztNc3rt+&Qs~;Tc(eDi3N1$6?!Q0XF+Z;Q7uI|bsrq{g5le0%&n_iG!02+R#t{UB{JB+H4wU8r|KXxPWyD zshEcm=+AY-b0Jur$xG`bFJDQS+4 zOy`ZnUFc79y`Mq(eoP>rCJz*~2(9K>DiIxmfVflr&4`;o_uF`ZCtRpLRHx#e1(=Ug7Bh1~L zG&&m(5kNu|!!{!jk4_QM7?p#G9g#ea&7>{t@taMqvp0>AhjvRto!9yJj8?RHay`_U zLiM?nfcGjOzpFFnx$W)+>p!nMV92TeAh$`f|C3h{NP!~1l1dm8bGo@P@%FGWKJE98 zAR{B=@tk214^$lsL#tc{)9}!vURw`>=Z{xs-WM-I+mHL>87!tSOz&?)6A{daa%@h# zuGU*Jigrv(25tf+6cqf|e@&(4$2Z(QqJfMuJpgSq;Y|qSpNchW6|A~ZeD%_F zLtS=Tl~ycbJ5wlkJrvOhu~q~KyVXiM&42HrKm(;EaL#{{#f8Ossid37UP?UNW=-#= zZTfoihH*QZZBq=ez*UsxmDbfoQH$6qGP%883Z>Py}hF_BLez_eV|pes)ra$~J2ewHpWk7KKo zNKoc{;}(gF$zY&2>VS#0G6E|#i&e(cG$UL8=4&u8;|8ywecImfMP-7= z)QTJlvI26!qpjU8H}4G!A!iHTSlMz0!ft8~r%6+$PSsp8;_w-Dxm8yG&Uj2xADm~% z>RS;uv$(Rr?ZE#5&&E*pr-K^C0sW%WAA1?ouDAJg=X8AY)9Jr2!)Xu!^$R|2qvbqh zvI>FAA7v?;txU(-0eb@25r+sq#=1e{?ccZ=8u+9ra-ykB5y2g&q~~$Ni(CIj`R#<{>Hq%XXSG|`bFp_r>>-p+rus>i zMuFQC9>4h^dS#-9?UXcu)xLP#dc3r(IP9Ay#~I&Rd9HSh5Jqnu;VNDhiyq3T8cN|W zw5Cc-Tm*BJw!zPd-|3McOs>dlHhqytZ{y*4ZKU^GM}&)Tr}|4n{x~m2qY?LG!bxst zGS0SB2W)_U(8-`?KfNNee%PHK8`*`~N5*90HUj8jwq)tC8nZcJ_D_Y0?p z;;Mln^yf8HO`5`p2@fAjeLmvH{-oP`Dpm;DNS0sDV({-zVhE3YOiQ!#Y1>nZ_M9l=TH1fF8|mxBSdRuOC*{9s-OG z*jdMFn63f$LWdFyzM?MfSAHlJ(%}4(x~OSyBchL+6gLY&F{)szd)0)c<|Wo9tK;t? z1tiKT`5()^$q61zoy^hp&_i}uAgc#?t;to<+QJULn!WR*oE#jPy6)m&aI4$m!NzJ! zL=2=Ey@jfQx_^6z(>HijQYZkH)LJcTWkt93yx0;xWYYFi_{ceP&9(!HhT`UMnu(e~ z7EZ*&!-FxpGXJm}6$okrlaZBe1CTPUpd}{9B`2Tl&qjaagMx+>KeVPZCpY+SyuUzA z%wlz7RIRNJR5w$@rK_BSQtk*l>*^p8N9}UIZ;zo&P-*S%?=ta^E)esC`h5)6S=%0i zCKLs}YPwmMOa5ji(7pcKPAlkgpdj-itJQ2IP^k|PV<;BbiAnW?o#7Twhp&s7S4-;( zUHLfrkh~8go^U}{>1ATks4{r+eMQAA&SwA&D}I<7MXZ=rRMHZ5R2Xdul*lMulw3Hd zE85N;(=sW$f^U|t+(2R%4iKg^vLtmEh^f?HiH%f#Rf+>ZtcAI;=#T-nae z-YU6pgOf4*6)MYHKF1mK*AIduBFCn>B04Pb_T>>Z>m*(DZ&GBjM(A^Rdz%O!O2>_= ztE83^mDvrImY|J5kc%3@t))K@C+pN!N!&_TLa_O__Y#ake4BZE_!si#`}60|`)4I4 zs3n(AFhS^SVD*7t5~Si~N*qsUr;RXaCPgH|7&b((E6F1RK>Hxw*5eze)coLO=1NjxZMu>6+@q(FrGxlBemrYVV+& zI4H>g`3e9BL3E)(r2<1!-vtw>QlWl>DphNLOmAAuaM&RbP`Ep8V7LM;`VsV1?x!X* zcR6hEF!SKS(j4d4j;FT_Ytmq+}+ zmO|Nqb&hyxZ)^L9BkKEy8rK3U2o(XE_Vbg%zeN(oQFFy(s<=$QH(|_6v``~zE348o zkMiSMda~9uk@0>oa{Nd*C=-OoM z@7?p*x?$Hsdch!*(D)2dr>JWHC*tQP1Z)r!8EjP3Pf9bnWhyj@qkF+!SYGHPNepXQ z3S~E<&8$6ERyTwJ1?6^(2U%W`6_t@J6v=~$k@j=N72}HJ$Kdk|BS)w#v$ds6eoy02 zL46Ksl{GboEj@2%;9Fbm?d^g0fMusY1i>M@Zh$|8E__j<&WH0VlKV$lKSyY_*oXu7 z{j#a{-5hN#iFR)BciqqZgdYw1!n7g4GE2r**4Z78&4CzaPoR|%Q~vB!sC*DM;OyST z-QFMg?=J9Gx6`g#1A4R^In2~jgerjAx8*U|CwwCcyxH>@H1nRY*d+n9DaINzOG{h_ zFQRb&RMFyO6PX-Vf7n{`yEx_1=h+`tZp7eN&gu6_922|f2u5&8g>N|?W682wI~vBP zr>gsKx2i_OqT&F6STHKRPGkA@Xb6LDGr{Fs4xbw%K+k~osVOWh3~cFOx~Q>-Rv7E6 zt2dp;2^()dlan~l$=KJ=lWpgY!DXRxW@H7~-DheBwTK^JGw=f&^+C?9e<-MwXrw;U ztje&*S5zX0H4asRYokb9?A)yhd)8PRaA}TQ8|hHS?0lsk$Mf6(^|4Q`Z>6N9Or*1z zPDlkh@3Dibo(vA*xQ zKL?k)1PMk)^{Uy}ma65XsxAk;o}*p55lb#0fiI6|j=4C1Mn3U=ZyrAwm0VRiQ#Vsm zKovF*TQq!qdV4EK)L4l-`qgLeCCH?|zUq^JBIt^HzD#kegHWkl3yhkM?h97b?8-`p zEX;#D|DQj9l=SrU>?JB8%^Go34t>9GmL}n&BU-k6CA&B@}J{wmtCVXPKvC9>e|O z3@V2y#VJyoGO85${A5uXX=a?k!<+q2&~YO{ODBNuW#`V*NeD=5w~&oHdcwJX`Mm_+ zbd{pwVXXZ^Asnizt1GLmjT<2jK`LI&o5~Bt=D>Tronu`PHBnXX?nRk{K-zl0YUOb= z$u*+CSWlnpd0Y`xPuyTJ&G@_FsGSj*b!a=T1Sx_!P)AEfcHPv#I8#rj;%74J0H6Ap zh5=HI&p}_We;OJ4yM=@uKd6D$_D9yvhr)5&%tK5g;_fIYFv{lth?@+m&R|Q=3o)~+ zrb0!wKZ66n2+Nx%`#b+~pm`7`fX&brj>7m(!X@ZoI_#QzI`b>PygXc0Q2=v*nOPov zfH_{}K;9@80}a+oW!ixtGc%K#fkB`a0*Y1LICR9RLJR}OfRNtx|Fi)4c`X-mh!y3G z3fm8@&dBaKQt6Tt-CKaB*86}hkEi$rs8og$DP)mbN9F`X6YJ^6cULU(1ZrPi=LW-P`_JYR#s_bnI=#1Fn&$C$GOSkLONf(W4}8?iM<`N&Pzzt5z#ZJN zHUDme50=zOLn8Zu@$?|LP2jf?gyyGWH0Q?D%Wpo=cBizI1PzX($1mf+P^c?usRBD_ z8lX7<|Cj=jQS72eDIJVPU2u8pZUl?5gpIyhZ+Dg8mg|o+G!$%| z^^^eT^d&(rU44T2``y#u7W&1M)LmT4QYaHOZHfO5XQI%P#n@lofEc%-)|1rk+{5Xw zHW{+vfCiih5X_k&SD4&iPyg!s;cIhKi%DXNt2&Tm8TtS@+d1g~jm>Ct*dNlxKDi+s z1el)DW6AVIu95}cUCa|po4I>D%<%3e3rIhktUW>uwaIU|gJOVuQmgg2ThFf7Qwt%! zZNrk*)^vbq7RUEeom}|yC%j_7kA-xaI7z3E5A4-(Y>i8jMziA?9g@XP1$-JfZbf1XD?f>C??k zZaS;fzFaA|z865Y$;4x!WxwJdj~H^czMR02;NakJZx@KF$S4TC-3IQ`hN0ku*i#E& z20fUN28|hMsrHpEG^0GezrP*|9TLCq5X0j+4M)inPVMJ)5O|CEJv@7A^I0*BH!}_- zJRM4wQfwkK>ZLK-l`43ZEoIGP1ev)>hU(K4Eb}_iy-B=zp8n%Z z55&vHbtOAtXCw5EFLg#&vT{n#DLCHiTiiH;#xv&5K>Q@_?H%6>Km8Wx1pIIc-^uKa zjEvSt$3?O_nwl`znm>yI`BFe2-Qr@X3VhzQq_XmC@J-(h6c7*d=-nN$fNp-uU9%Ht zT-=L8!}*mcSyV{z2g>h)0f6f84EHQYpgGGIPoB_7!#IW`3z_hZQM#;iP}a^5Z6IDP z*`2Mrvr+6qm^DkPyJZ@cK9{FAEUjr4Cmih)h5P*a=&@DHK5j4+7?VByB}|dIuPSnJ zJ)!(c63`%>l_9Z&IKHPt%9+FIp8i4MkSAC}dNky{uDIvRxLn zfisOTY^kXxzpN_k?q;#UePp+8I?QW(gtEimsoCn7QKwS(CTaWMGl0|bW#vN8Zd>Rf z^SAC_Hhi2N?QWQh+M>eT1f>H4zoiLEk3Lx$eT$KYt3)B6f~mG$0cDY6~Bk$&m3(8_qf zdh9cvzr1~Y2RTtWjIk_a-&IJm9S%D@EpScm4H$YxKC#KPe(+gbot`jYzzW=ksCt9S zD#OK9O*Jr><Po6Jko5UHZu~OyvtBP#k4gH_1Isx>)Qycfbfmm)Cjk^# zgZU&aca^0f(%(&Z!~tq4YMf9JA0MB-oGyejk2;ql+23D{M9>%bwwYk|A?oG1!sN)%Aj)-W0}32m}3xRZELh z!6yN}d$#27tx3j(d!OQgTT<80vi{w{>xodG&K5&MY*JyF05bydB5|`a)Q#J<8)VcZ z>&gwSsXe^9~mMv*ibKpk$sAZWCeGy_q$AMlbx zJH4-((`XPpp2kvu@^!q-aX)4{ruztEA_uK#EYDot24S`yYVyHNjMzQMAeQ&IJX?tq z3H^%+st~V2FLCe1mFd^XF1!ZVFRvD4TwbJHoyIs-+6|@o#Xm@)>aAa)y8_ zuI%pB<0IrxP_6w=e~?oJQFX2Yv`vbQyQ+bC<21K(OFf!(Mq9|@mz;ia}Lb71+2?p?sE|22~t3jN%_l!1RZ%{4wE8d z0_WUdUIeab$uzYV2_q>gN}<$Rr=$P%xk7^(CD5S|BbgAT`vM*M52SA2CED(%3Uv2vrZvm;m9)eCdO~cD_(I7(&5s8c zgQtCoi7KSJN(R>BB3d!&S0d~+`|_>!tlTDCQ}3ti{i0p0kerIwr1Z?*t=Gke5*qRk zF*p$bOwH$d%DtUu2w)dy+>oIlPB*)Bg}{(e--GiiHdI-1#BhAWj5e!1oUz@K>Q$bL z|J#}6>Uz>)8U6fkVtG+$$RB>S#l?LbgPeUEl5YH51r-;CczluJ;uT60L^3#ZsdAx! zSm&d^e{q&Q-G0B33X+lOo#+#iUKoLqyOH0Z4uXek;y8}!jKvcD)w)RRf z%5hkYXJU+%j!1@Htuq5=K%Iqx{E_2DyJ=lfpWK(W(Q;JMUMqhOW~jhGBSMVH(eQ+p zU+{U4&X(oB#Vmh|`Ir*-qw2Y@%Khtl`RtIlY+iYO{Y*C`5NCVv6NPXnrlqG{GlLPz z`EHno29J+?%#`}u{WGcxhO<~~SsDDKU7||8)8SMgR2zt!ot>TFa{lK1p{FPRd^bTV z7?#N02;y1i2;T9S9^Z<&G9589IASG4R0cdQ!-XUzmXfy5QW`P|Dp@UE)e4t&nKb{7f z)X>eKmjFi>B~qf45Fr-%FyMWYk&sRZvpvel2J?{3J+?iNOp+u!ikIU_4&@gwwB)qD zAl>F_yAe{@U(_`m-Qe!=?F=~db0l?pH%4z3xOZqIOxq=6ntfarrA}q7C5!;;bf8Pp zqB9zaB?JmFI&2w=5&+^IYHXyHVVe(hi)jP_2b``Fwk;_X5Zw;@`k3Q$zq0N)&H(b1 zvtB#cxbS`O-AeL$vCVu#C)OAg!5}8njjbjksRa8Xf<&mmS6xxyed0dXwepp$dTte#1Vj{#l z59iu&2KQ@(cB94#&t0+qYa1hdG)7n2z2oz_B9R35x?bC5Y8eK8yx(~eJx;AOJ`8^T z(nZodzE8{>DLz+PXJ#b#rKV(BlHH6X*=RJIH#5z7zJXHCOP7x*F6`$|S}+|Qo$s3~ zh{pLu-@#z+KRU27v(*8Nb|Mg1d!?oE_Q z!D^gQc_HngJUgL0uWl#KN=_FQ)gucxU8rJ`6jytF=+Zqml!Y56Y=&Z288A&60!?L% z+SYr+v<7!uT-msvd-WggFlKZ(MfttCwn8y)SY@_4gKQ(eK!f;LSH~^KM{79dZXjFN zi{%`aQ;6Q#-ps1Y<*-MsIuqyurSI!CxX581MPPz$xzaYxCjLKMhl8gRRli?f##Xg{$A{aWj9`2I=?=SKM0L z0)vGD7*3Ua%44~muJdiQi9~i}&&hM=RHB=)^f>A=6nKXFr=Xe(aOUmvW*(GR-l$uT z1C2D5e~hp!zXNEIpD>sao6zc56+(4o3@H!|?#o8ou5*mAXF!dAKx97 zMbtASw0~1`TL{?Dxw@5Ji>3kDI@uKl*$~jdq|itVpXQp!=Ghy5ylI&mKi#OnwZqIg z->rm1q0&gLF-~`F^0N=ng2~qV>M8;X@%#oZ5eqwfu$}Ljz|*xrqxXUA>A+7-=diBV z9;(BuCIk|khdPFZF%LkJ@TK~7 zo^CvgF3D-Bz1x{lY{4IH;fbDoZTGrPb1^>jB2XDpT7*>P@*gu;MR;$IiaCT{X0d}8 zH9JS}iz23Y@C|AgwHLKxQce2wGaKVhRmRC|7o1FliLL*0Y^1o7XnbnoImFA_KmF6e z%no%}7$^TAQ2gxo&+G4-bhwK62#f2GAZ2*u67&Ip#BIL3?5Td-_x^>GRBFwwP448P zT*tQ8Y?*3ryZv&#@H!;??>`i;D|GRXPNyl2IwEmXh~AxVAjq!z*MAHFAx7P;5eHMw zQ=->8iEBcdk~Sq0q&pq}^*uIht?dTo*LH@u1dBs$nkKu;^V7TG!KETyCQngE;1aNv zCszp%vMT$kzQ~k+uP|57%FF9!X46G;|I1HJ7<%125<7Ba^<6kRsm1j9?ez&nTy~YB z3BOv+^};4fw5JFKCQlW3lp=Q&Y?Ri^5z2U{HDP>Ys%(;m>->jiqBf#tq2=L%49ddQ zRVBH(LIau5=_gG?lESK`pk#|6U_#6M0v+%h4!Itw4HjfI)Jk_f z`&aS`K?pw^3HY{-$W>M!H#zKs#Oh!k6=Y0WWc`41))sm1^bRdj{E7ZV(lWQSk= z?sF8W_6U)Kj(RDFB0O>zYRyd;S3x}!2ZJkb^egBlG7e~}qTI%{6(b|~UZQ_47-8l& zE`kjwMgF{?Po#Lfx1ZDo0?pEsiW-XzF%--{;u=`ompr}M!bSu>^naP09OxxHyTdGF z&=gl@9+E_7FrsGhALPq;@P)SHy1LgT>6b^@|>F}2h% zsg{tVOV13VDAMG@$-{x!0{?&eyH-P?EwpCO%W*cCgI?I(ual>>`<^=2=~V|OVfwrj zS(7C|c*(Pj6Zzgc#D-7j(dT%D!zk(xf=9EAIEjpYoFRZ$Pf%V6`7TZR__W>zZP#uw z5*v8;jTBspv6dRAxUkp0q?g;rv=~8to{+xB#I3(CGKh`7aZEW9uO?043aS7LwYvAN zjay<$5c>Ca_n5U4P0yB?bRCoC*l?z+PhD0>`@GdNcp4%+tYDIypKIFYTsr^z1#MSCW7*T~Cl!si` z031fe5OU5L8Cg}fSlt=PxeY3N!i+L0;4({R=#wNKu>bN#Bvtwqe~Z%q?OX{tJQ-qu z`LpPDJ7cWrA#zzF1ZfkpfJuh`1Gv#yjvE<5=SQYy_1>M=1p{@}XnyV2J4WWoLppcA zI0i;WIW;hvhZ*lgUx?Py|ReVd)UL-*P;g5W}Ciylg|_wNj}- z;L6+W@FDr^Ia3(*^KSn-?F6{r=pywzPfK2J=km~Qp%O$LpM26(xAgKVF5BWWlz1sR0zLosUer{bo+IN^!6MV98sqyI2cVJQJvb#v>S#r_*P6%p#TtFWWPTj&?( z^pEY*u8NWsC`j7xb7Twm{8#dJ+tk{Ba-cSwRfa*zQ5Vo}1uBHSr^CFmj*cj31T6buTnjR&;(1QN8j+QgPf8tk`S#+- z-vR-awPib(Ey0H`yFA{<^U^1E4@UMXOJioKI!mm^$?H#QRx+(&oK;hJitD&aEhELo z3G!B^!L}4|h=}sBaQQ+uU-)a^Z&4ThVKY)ep@tbUv!@Si3|NpYiA8alp)6Go{%pa5;eT7JsI{=22p~)g^p9 z=kiz_=c!%;{o8=c7T&hggtx(}v(=XJsnLyO$gSrN9WBD$_;54?L7_VwVcIY@iQ}s` z(I|@V+Soehj zGKmJS!yP*wgz_HS+|0??{b&L_;w=OD@mp>CUj;7%Q0xKdw+SBnJU1&6c3%Q^av^-V z26^#b%$nWd(MdJk?-%BT2Cf8TYIOr9*~lmdgiB$$J^t1(5$$@>`(Cs_3tm24MF?pl zH;!^J_UW)=!T5ntIOE=|3zQlD%EWC~;&%bxV7hyq&?U7=rphScxxO>`>p3g= zCkq`>Z7Z&7|Fb@lPc$z6ssBDY@a%?50^7IkUv1>K*TOt(G8U+cLlMPfAi+);T9|Nr zhq;)eFVEZrx6Ak?tSNSmq(VPtzGeEa6xS!{{dnZuddIFn4a8+LCkZxsxjzFWG4XPg z(&T8M>ceGDx3@h2Up_{o=dqsxKuYwM_8&oUhmG?Jp!b)H!vsm9odIm0ZMeF1why(m z;&hEfBTsF#nw>eqrsahbQ@5HpwK1>?5R1da^g_q>|*u{;Jqq*XT3 zm%ey|;8C&5uib0mQ!E~Y*_>qpw-O)upv4#7VRehl`!pb-;yKNuYNJr01{l>rruC~0%|s{>9dmtDyVkM9k? zcg5>BPsRSehKlfdd5K%%jXZ<~+G-{k@cz#)r@n%V9P!ME26Rj#7u*C}Yf)dInv*+K z3}P!VVo`Ah010Zjkjzc8xjy^9Sa4)X@x^CX%9is_73e_WsJqSpaHg4LXz0fa%gf

cPU}1%3JUO&#dzso05+(V26LnaWQzI{=nE!a zk=IJY5No>tU>#pRS26{-0MPi$RvuE;1xSJv!#;Z5g|IG;Kg#wSJo0}gYz}MY@8hrA zPt9R9R)tr$S1rx`+^}_D@?8n_p!9erP%(vfqb|>wkV7~U_bRQGWj*tIoRzjhDM~&Gy-FsI1{gnfkpPO zgq*!hrCjPd4PN zrPcAqgYT}!bs`(otc$z+MmRpxL@rHJIe<^!4XUZjaFGBqp@STS|s+T$6n(k2y za^e*+SsM#eaBHW7!*cS`$bLpY1Z|{MH>m=qg)F)P<78k*#Agw@dB`LP8H)f$a}xkC zowu6WpO{k*t?SxN?i(l0FXt2pHInQ|Uryb+r8jsWKh=5brTCyAa{a}jw&(wYgHnP? z!#30|eFBFaEaGYOiFB~M(f(P~w(GBkt=BoH&X-?6fB&TnV87V92Uvax#+06y1}$Oz zaBsR?xD_YY_@u5T4aTY#As+ZoefDp%fWb1wfdN|XCikZ+|A<*sck;);LsNPA_OF&r z;7x9d?}DE{>`1UU;Q86pz~f$?v3qaMnTu?sFM~H_*`}pL_NXhQ!)C3S#q3xuvpHHR zFbP@fYdgn)YwJT+#SlmLS=84LcBYuk08C*X{d%v+1B$M`qki}yIPo0jJ-ZT*4d zhYMkkFN43FQz(_9xR{>`kSD&i_h!L~iby%c(taBehgJWHtNWeRJWGVR5T@LP*%8mr zh3XH^@pze{AzUS5En;z=F$Z0s4;r`Bg)1U_1H<^L6h% z1=6;myDChyM}9qvyxWEfM10&UCsMaGa2psX*FvRfRch0`%s=0EjpxpQ{vlF|CAf%` z8iWf~?H}r%Jt-g%1TPxhXGnQn`4fgCB%4YokI8z2R;Yg<9Hx+6os*^6^0W>?xY5M) z?W;+5x^oAbXgJ6p#9RJ`9lg4wOX%|Vur-h=v|CCTTFa_Npxq-zWw|+K5pNvN_tAa= z5)}v(gGAx)1te&~?05i(6c`Z{f!i5rzaSt^63A&;X=M?ZDF;Bp_S|y2%%(WNS!MV% zrrCO8;pPMwS+2Vux=;p~Jv;PpGXl;^ErCoRa;;9!zY)$Hs#F-N>UhQV?7i7EB^a3Q z9$r59)r;$w3ApK@yZbHp9;WJSz{hnUOPnjB9YH+KT3!iP?`w7FpT&s0`bZxGQ`iTU zp0CQ>@Nhtn`Ttsgd_!Q!0yJjy4pn+Q9@)eYU9&?AFxHA@MPjh)n#{%6dCQvw?&a@1 zTb0(q54;}-pej*>VyQ4A>{mYC?G#cVWUyNJzI|1iRqksYb{O&IOjK>o4P@!dZ>#tC z4v8OvDoT*zZK@8U?PkA_)nhxvX7pUC2RZ!{YS`vD{{C~dRb>97Fj6j1d)UB&sy85@(vf`A|u z){C5pP6H;392czdG$tynC?JjxwkxpHhR^$H-0)zMYikc+h!j*+MbFKt0LwlIP;NWV zO33B#x;%1KgB%ZT9nIw#&90Y48M%u>o2R(x7BBW?g$zWc3*sWtgxq|51c_g!WZBQo z3VzWkNJ#sSPZ5qMx@Cq-0-Rx#XVb}{0&`-HWIVNeq^K^mDye5If| zfz#>@29a=JajHz)c*nwS`ugYj{XKk?Bc_I<>eqAmP`9>Q^Zk3|&cToGo*JCAZP@cz zGm*zlyySA_*_Gk&CkPpg9>Ek<^)ah-G{==DbEgAI^N0?!S9 zuaC+<3EyKn>n%+pHVdvo6aPX~;+dFa+qO`@I0@T|-P8;`YJpC%t@!K1gm<=+Ao(uE zPef(*fGPfUq2rs%gycKs~DrEK65UcMe@y~ldg7McT5`_-DCd(+vB5~ zA%xSm6IP@)xlM?uGAhdUy+ZDtts7U_X^Uvf=7%DR-fzv`=I_Ziz@m<{bZmo!UBkew zs@enY$sW(SU>sUQ1ttnxj=YTLy?ucECWtsCM!7h!eL3O7rAZQ^DEV!x9~!y3c?Q|Vk;fI zEL0G0pEX$35jmJW4QdV228ryr4lCPniLS4> z0e3ap!^N8Sz}(;@mjD#3`28pOZtrOg9o|#9qd^D#O?NBElk4bi(MT9G>SR{H)h?!S zHDpSQm)VXYh;&XO`&DP)r~Fy4mr6c{V9jXjvA1Xf`I86=`VRzFQ!%U>>)kC|Ph#LD z_t-6yWQ%W`)2$H}j!I6Z-$R@#+V~``F#EcJvT)T{mt7`eL`=r2IoQ+wI^O~%*(P8q zvk#rvQ)ve*iLn*TPNdIRC@WgL^?X4SmiSbrr4(df+!)WJd$3J}^JWEv#H{`iKPl>( zmlfN*=WncF@sdlk=d$_C*mB!=1sthyXIBh#TcV z?TpaS&^!<2JcZ!lHdg1)rs&{?p05F-#(a#rN!TFd<~V{gU$7kqp!rRu;U;#%r2<_Kf0z zlM?!jZ~7NMjxuDR*Tf_&pm#cJ5oG}BpfZ{_R42F~`-vEaJA!cEYH8D}&&`&y0Y|ZI z=jy`Co%yKGv)Gag>|2+ZkHSMWLt5n*JpWca4Z9QE?VQ!@X7nFHHZ}?12#{YK4ZpK) zc)&QJmnkp);@oM87zKRwlxc4eFmopQIEcJA6Cv z=GBgx^iSVxm7teCE*G4iQ9k<{we#P*5|+ipFdN(P2y{c4+t5^g@a0AJa3-|WBXkTF zpAGXwThf!=N+&&?l-&?VjnsD22aPW3ecrsz@Um3v+c!WwFs{|=T@&2hR?;Em^ ztpB>{s&^X1?!0nOQsh$dG|HCrbPkK8AnUPgurhYKap#<)DY5s)xxW#Td=?xj1>V$Q z<+0nPvT4J3X0^yppch9>zZI{`8^XN@x&9}(G|c{*ph_lfPT(ni>%mvV3EY!Z4)?y9UNC?eFPEzuv`um7o0%<{_fiQ`2R@S2KB- z6h#Uw(wRTk71^qO$R%4$B@JEZDwoUhaSk9JuIud+F*sqp@}E^EI%kb?B-@k5tat;= z<&~91&G(l`5V4?Ii|O16vSpbPx>VrEb#ZY4G5|XbF0Ods-j=@-#W!+X?QMT)P5uQ- zZ#|+4%VbZCLqe`%+v7mbh^p8Q43SwoWNe9a@!SYodj6jU+xE3(>T2c~3Z!%If?SD) z>(=iw;1OdA{TnpN8aOhlM6};ts?px9#AxW;FoHjZF`MLPMXZpEQ( z6Xf|5CYikhVm?b;o&=bbU^Kj0+Jg5SReTi{D{*H$T`pU(S5Mxh) z20Zvz=tl;>#RomT{wro_Djp+8{?f2~Dk--rKBsAH*vPB&+=&>x7LYhv!h|ovTgaoF z;lUPsNToinX-@0vq>NKR4{F?w^kK&+QQ_!!!+_yhIM1LXor405aKxxYVd)D?dByCc zYA^)!7Es1C##0n?a>A3IVtrWdH^oIeJsOFE8q}JjNdhNsQnvY=2^EFNJcGOW+rTx2 z#2cz?MSWw!7Qb6p@6UBSEO)xGg|a?A zNPJ9Tveoq3JzZ%FibkL^6krKHo}kQO*{)$n+#I##9Y`Yhi3jum(*>%<{!G^gNZ#fSx_~I3pr*KMAN3e%WX0i8j-Ml~=VfW|*fdvsmO$*AvXB z8#2|tJ&~}s*@|4cc6>;2bNaf0YrWZltJ$&s-Ve|jS2y(d4t4(hO0M^#uZ5%3gaU*a zrkP5d0bbSZVKo9z9M$O_&;k`i=YBWu-4I=YIXQeQtKBJEdJp)FPkw28u7WYQ57_H1 z{>Onoa^GL)*KY^qtJWJ5o4@NGJWf>$Wu+5>3JneZP3T&aw%@p?`O<{>QFBRteqn83 zM@?szA=3v(Nq`FQq!28*Ldf;#U%Zl=F9y z(Y|+>Rmp0C5gzM@s?SKmh++W+$XZ*X;ie-T7@=~06tT=-EVmyC`!}I(%;iTKpJnhQ zphXsc+4bKQ6Ax<)_OI;-{H51=xz3-2ev(57nN?$T`GHhD|G@m<$1=JVP1^OgY$Y=-fxQAnf~?XcCFA&4pOMRw83&f4z_Ta=Miamu39 ziSJcdzw5Kx)y9&#<<=^9hVV!rm8Q^W`+7D*lxeBXZs||CrNyj?}Wwz!|OQ}wGrGzKbW>)Q678tr=eKD5GWb) zdo_DEqAm`yuzi!x-MJWcu-c4=NXQ~dz0LeBK7?!b=X_+Wa}&uSa4mvv530xs^8^{S)^n(*>k7Elg!aF!UaBw$$7;96g5X{fv@#fL`Vo zsJ+mD%-akyqD)gfIM&uw-DB!^wK<5%auaq&)?D|4uI#DH^y5DyZaXaTa%A+-DjkwP z+!$=hdvd+t)q`y*@j#%vuQJ+64&iUx9Zj%tVK&-dOj*BNMq2+sd1$acWX;w7kn0fz zeY9Ee#?m zDJ4ircZalecWk6Pr8}g%kvfC#@4wbr>wGzT`N74r_dFB#+%vhU@1ea&Kp?UX&jt&=>7)d_qy?_e`ZkR8RWLCJJ7$Gp zVo0=U1OhsgU!JPn|FBlwjoVs8+>Y5ECS*Zyw;1IQV<71*3RJ(}2asPN@h9Cu;+fUR zqdfGD)F;Z4V&E#`uwg!L%y9Pjb2FdIX}1)luQSsvNk6^W?f$UW=tK@@wc0aoJHEQf z>T>Pa2KUp2+0)?7`l-N4C47?C%%_yjVQtdn_pAYb_!H5s7ZWjpOgkWu z$K(60rIiob8Y82g$f2sS1@HIbt~|W)=0DWBaw?cllGm$X(XM+WdPgGWz5Xlg6PI0B z`2BK!MpZ@Sxjq)6fsYSE;g8V>O9WG6U@5k-urKDp5t#1XccdGxcRc0RBnX=Voji&b zSd3L1654NVX91>#V5}A0e$ZK4UoXw}g$g~xjj~SkiQMqMj4B$hw1_SZ=ZfxyAdbAw zI|1zNI`qXJ8>N+mDRpbH{g?CEHhQk}&MkI}cMTiy{EqXkS>;ic5+;o&iMUnOpJ~bt z&*IM^9djF_LM#O9u9U(Ppi_F&Gl&qfVf?er2El7aU1C5!nG9v?ryHaM&Sgv_` z#+LlVm=B0}_UDw`=UYeIx0lY4d=}-T%jRs?1nzLf`so8#fjaB+{XdYcROGf~rrLv4 zOxv#>o_c?H6btzR^7s?b8t>Gn4@BS(a(I-1WfR{}dE71ZF*e>6KYG|{-K{15vedk3 zU@P=w?rE?>4u6g0a8SjjFpDs0bG*<3hrn``fBNHIOyx#dut@bRpkFffNWe83vktl zXd2-X?s2cu@7_6NeqQy z$AmOpX~tp4U3zK!Qr*wUrqb#NOhB!1sUs2Enq}T`*2m5KW8yOWsV6=xr>Q2T>Vvjn zcoC{*nj@Ug%_b?HCRaq|l*#RuVO#&uK(d~|?YjJl$_}&Xr(Laq^8rLR=rCq{C1)dN zYOM*K^95|kpWpMUw?o>Qzm89@OW#&K*}cQRUl)UXklIG`hMZkD!jsv4`LYf?0@gSY z1tV0>JWlPq+f#U-q44cxfGheI)H8dZETT8;wxsfy+~X%-2(><1owc|xdj1*)?lRp8 zHpT_)pM2~bN!dKD1H+*l_sBD?WfN_m$W6fxMeFV0!@-SH7Ke55T|bb|p)}f<4Yw>U zv?=c9{EK@N^x6rfi$@%_XpO)L*QxsBKoRA_r8t0sYnizqL4NWe4~0}ENofkEkhuQ% z+aoV;J}uhU5`WhMNL*hX6tw>)Qx};$>V0Hp1>bt&wTVe ztm3xHkTNaY?WgVAPgV=iNL;m_)`Lc$qTq$D-f6I01O&B3G4(e9*ul>YPJ4MncijS? zxO?H;iFxeK+!-4k5M^ipecl;xmwVY5ZN2x*y(>3yS37{?cz;F-*<4|Is+T8t2Z8iV4%a0DeiqD#+^4rXC5v9V|^99znmVRRb}j^49&BJ zW?PO|%KZqmWElwAW!N-M57VO^VaVs7Uc&YKNIS@!BRH{)t7iF=z9c)Z`AzfjH)PIk zP7Y^oRC=iX>p!oRy6=!~(B#X~DwFqKaIya|bhpy-@SVIKAYhPRpl?@=Kf+_$6|WA2tma zq>)A>C&b)$8N>4{zQ^bn$-Y9?oAzP6u;kjk**g%o9p6k=w5rk0JoVsM9;oP^cjW$f zlE0YE@+}!`_^;rU9bQTGceQO|6=TBo9X?vFGS zSwwS^m{HMVo~~wdL+jL0K3lbUuHQ0tgh#QBw`(HVj|Kn z@pn7OuL?(tw}WDRq}ehum{jQ`C{k9hhaubHzX?~{xy`-5@n-Pub|ZNlHy$1?G}dnz zbnwaRS414kKccnyGk=b>k~;5*z`Zfqi~M0E@(0?M=rd*d_{fkq#UZAxO>;afB8r*) z%I>vDTnY5$nc(1U=FZdVIvW>HLUX~2f@Tdvb+{Yt2(zwFKc5FN&q?g0k7%ua_~C4F z$4y`z>V46^roVI2sr$z5beG+lj62&>Nm_0vdvnIx=vC&g0P7z8&e}gSuR;RY?yRis zHm>RxcC&&glrUY>9N{yPES{tz@)i;ueeS)LSy+82Tt^#H>y&lP2`kI+T7W+?c&J!-KKx@;b*fN*NZzIlzI@c?bkeZM1y@F(=WwEs`UyJqixbNJq?Fs=P ztB0^oUSytY#a){dUQ>Cz`0k@s<_iKU1hKIV137?18 za=OV*rgaR8imi#>dfsr=AwBFuS~BtFNx;_hRs@}3M8HKKK%O)^`7wh?sk+t$@ME69gTlAz0AP@wT1 zo_)wSU4CM3x`w>4>kzZiyvDBiJa?@eVZE}S+zkycJxE^f(cZ#O$!c@CHo^Dvc=s_= z_+&mf_DV`Ii@UG(a{JV`XtKW^nifm8u+#hn)2;-IYFVeXjrEOKr!~iwQ&>og*z+FY z@xzyD>m9xDtMqhSlt|MSNX}J^c7!(!Eu5{tNzER6y}Gf%wE09>|8-_z=6vi(a}Lvn z+??hO9;RsUYue+JWD@G_nGuPA;WTzqvykvT~Lj>5aFV7}8e+Zfa1zsTEh3KrhG90{(UHpd0+ zDvCbq+E(Ku0Dpe?v|v~=;Ga#PYjmz>7U&NyP87Pu1>{J1Pp5d5(tei9chor*H?Y$09OZhH@q3;K z#pHq2{9VneAD&zygF(fdR|t`J4mF`UYjJ+3^2s{A*(r_p_77qgv)^+F{G~OX?W?7H zMH>8QQ!+BWxe48Q@b%KPC8L?6_GR+@jJ9Fkydeedn)r;aZXrs55guM(^bvZQ+H{9| zAe*w}U+2!9yC)doYCFO^JBEGs)^hy7=*Jeezb~svq9VTKn?Y=7^OPm8?go^#t(Fq6 zV1#39GhreuR_?TWt%=sPu7sB#BBcsTMla-~?Ea!Vk5sj-G`Bun=b*$3^}v*eJDhx` zR{TIP;2Cg#cQERxJ}hvbkE=;~+tO*0lx83heK*n&Epy8$bs3ch2ll39AKahn2D3e> zlT?HEN8|d*i_>VaXCH6C-T2cA~_Ju>BEEI}&j158CEc_^l@KY~@E2 za*QN4kWz;7hkn;tS{_@`%*Dc@V%qs@*0uQkxxj|2DV|$wvu=vS%3R-q;!uUu&)@8_PCfb7$H*?pqE1*$U~UF(TyeH0XkoItqp_mKnGQ}- z#QY^wDd*d0ocEPjP~>*OiQJH|-HX!jH9n>VzIrna>XjB7cAw!p2E^v_X4x8=5Ml)- zi*%0Ima04K5;d1`3y-_2p;MRJCbusOliHIT`RjBPYtrU8&6<1$ze04iCr`Kn31!|) zsAdMXkQ*$XQ5;+&*uO_>7HejoF$Wb?{;fvuUFgq~%PesVE&r|vW{?;xJ<^jbe49?G zs$x?Vyi{v^QW9qxqAT29m@EUUFAFFq#Q*Ouz=Q(v(vr18IwY)F=>b}J;V+J`8P{tnr@~u zF`X~T)!_iPX@CW9=}`^&q0(NjKZbD@-X=+Jwx zKglo}L1EC@`Bt?oi&~j}#>ziAF!@L}7CMIV#sYnvIRG}ECM)>0k(Zlbcwvw4F|w=w zakrOG>(T<-6SE}aG41K*=b5z8@Q_tX{^N=U=RHpOk#fQ8QG$PAshHXk^QLvIFb6%d zKY@b&kFav3#uQ!673SIkO3S09M$hY*<5$%tl<)0X;zY(?n}W*`^At9_^GDxK{@mS+ z!av3;^HAXe(fe{P@2C20D3djl{Q_S*R7fYVdm>$~@$jJ@Q`Wyqql%P1AiLeL-DY&D zr|NL<2MjH&pW|D}+!LFmMXvJ=HMdYn&xGPN$6D$SUmCr(Xl^D3p2%WCh@OF2R2GI$}@CH`1 z+Qs~%pf*j}3O!%d8@lk5lv}iWtWOu%{h26CjJWYW*8O7G!^Izrrto*w2c%rNdWZZf(=>%tpA7}x zTcxdI#XEQ+$^7DPSt9&gg=%STY$53(ImGI_I~D$Gsez(ac?lR`F?m8*h$p zqz`$2-^c~&ljbVp<({MQW7A{=$H-o75vP>cei6fF@zT?G&kB*1LFVfM6TytiGhIOb`2nFl#l&Pr|N0p{6a_;q(>MAo;bV#-L zDAM#NcNiZYdFKizttrz87hbSHljpr>w%1&UfQ5TKiFI`z=sj#tp7~!JsAKpjj>GXQ z9Ts1zmzmGuwC(q%7ul*Xs zYg|-@<1w8Lyby8k7okjdG)SIs+Q)bIw?mngQMc?k&?kv`Pv2~<+xI{I#%Q+B*9&4+ z)awgpI3gVPR-GVW2CXEia`?*noJK+H`03L8-kT3WB77Ql^1Xlc`nKeE)8KfA?w$*Y z_Pzf}UPf%)j0QAFh^9N^IG8AOwRE9P5wEiS)wiUbjoV&< zyc=6v7fXoPsyt6=N-Bb8LX~QxQCFGvs-B;Q;3%7L!u}a-Y46Eix_}LQS;Xacl??dn zO#$^ElEth8`p$7r(2*hiB(9dp(gU^HTF!xtwAB6>Qn7xMAvCCs$ff(2tald-bVz76 zMsr`e`wo&5U(u4X=hlW!n26+u?Wi67J1Y1AgMqGkzTa1i8z(v*v$MXT`7~HDxystg z$g@336*v}VXB{*dN;42^7`zcCv?wM;k1bgXqzQYuBdhCe*L&6?3xjtGDoYg;QQLzS zXC-pqo3Emv$;*vcOFs++*Zz>>0{33NsU0$B4e3UGMt#2+!D?~^kMKp(9DmJ$bJ77h zkPEF_jk*9UJl0mLw^EJ%4b2*r$--5ZFpJd4xTVJ0+w9-EQRmljfm%ka7K#F{;uEZu z#`HxWiFz=4UVT9crMs=AbWt?r$i@&w2@LFa=M+mp)qExDMbpN>77gS3(?%f>7=Qkkl&>7h8P@waBj%%1 z#|zFIY0)g&OPn7loVgBE@G(qUz*&>#e0;x1WSya3`aSh!0_$0_GzA{nwX~?}ABKcb zK))^JE2wi-g}c$x zPu=Ba3Ax4du|?`+aIg2&2w~tQnslm)-y)+;1j07Fr@F`ldpC*XFB%%EB z-;Af9Q)<$EOAjZEkcO%tcJ`&IEVmxL9*!zO{cLZCC@U&*rwJA+E0;Q-@)ogVXy%TY z|KwK?6KXa)W<^lSaC<#mL3-d@+u#?GrDDicT_S_aOyd%6Fft!xqvvaViQf_iB?emZ zhab1T5cA{`KZND)EKMFL7O*3MYxBzzmikkNAsYNGRhu(bVK{Z-MR2OI!Zq4fyyBoc ze}qdiM#=gd?$b9oT+fJs?D@3%N|$S-x)SdUS%%20A&g`8mis)1Mm0gyNsrRw(?rM|n*(I!v>) z*H;U6)y~%%F}5)ll%ua`6r&4*8H&o{Bps^uW?4%ke74&xL!iXeg7Ce zCx`quhB>4vuHV{Y#ytFq_6%1gLG!;17UFr&@J7FD#!wiX_Ec;7ky@On63eztqh$yY zU;*$*_IWHw((@Yl0crl!iCkm*m;XiSn3&dGr*g>QxcM^jarBX9XAMZBxwbgLQ#Q&c ziRRUF_bEh-5{|ZP3LLl@qPkI}e2^J@mB1(0fBuJ+nI+QB^McUvmt-Zl1DWBzg+I3X zS3hI^4oihNZ^E#@uwV6*=3e`48ADy!DoE78H~DJKv>f$OO+bZ2ZTGN@t|6BmJ5J^@ z1l732zJIxF8lrA@pYb+UaeIPS6a8=V@qt!Pglp!TAE-m*u%u8zlk3reZ?xkzH{^OY zI9ofLO8tR3@A8vDu3}qOSbKTZb%kb~s=n;W)$)jeE_6=GCps*?p5?7i_lZV=XIg>w zch8f+?PyJz7dC|C3u1fwYQVIsjFm3a9?S0XljN7wJEWLzx2o_Z(j{>N_aaRiEe!MB zx`WFx$Xv4B#;gfU`m*%#vWAn(PgnXYUNfA{WJNFAXd*Zeg#w2adGBj)ZCZN9_5k-e zY+TUq1lQY@0^6J{8^W~=ruXpjq5BPJ(Qe%7SvZdu6RiiH=X7Tkfmk<@iwT|tR{kC=eX-%0=e&uJlKZzJ!aNEzQw(y%8D~S zPZ$ov7Ybv?84AH>u(Y<0eeF=ivc}DCxcP0$VqIlAik_7N+>DS_RS3MqN{0cz9G=`=Qmj&)_FoAjezePO#%%Y-6hn>rJTqGZ~r+-5phk~ zd|MIegArH6$@4wx=?mJ+mERllRHac(h`wrsyfP%8O=;!VoGjAUYky}%`|Z7%FC;1& zPw`%H2B%KK^r|>kfi2n;0MO_;PpNmNu&IWXXlg(@Milt}q@Dh01R|!G*deo!f#MJ`Q6oIn!idgh8&~G|N_~p$`bQW}6dUUE?gnT+Y_d@mpYi%0O zfgZ*HT+wU41Kb#uren_|6e6RY(#cV2iGtt1%r{zNonU7~t!?yd7yTK#{$%rz3QI~u zo`;2siVA>%Bo%vTPyde0xV!atYUTfakr|d@=N_aG-Gk{t#E{2=`a06ehydp!tOXNt z-WRtjD|{IX$h0>hxcMly5nsn9OA=#BiYY1JA6|lt9(QHvoWgwgu1m=ZZjZ~0yuE*C+mFzB# zYr6mGxG6gIbh`?A0|io1l%->aecVji!JLl#2a!?0te-4>ls!$PCcYjH=83m zo`7)ra0wYS7*p#^em%1Jpt>T=20?Cw1Ch1;<4wOX+tc#xop9l9aM101^V9I?mS+Zq z63GaymhZgUOVmo2qxpN-rT(8-Zi>Hu__YJ>&p zxi6%<)RIE8uC%)T{;+&&8nOOWL;(9QWr~>9=qst&9b2khs@svUW91C`-f_kJit0Qs z|N6l zrDPa%@z+$Er7)VK{#^Aet+|RAsn|$kx=@E-4vTDSIDC}p$-DL&Sv=RSi%L1RR7YL6 z{Rf$=;a~0|G&_5)!UZth{qR`9I~Q5DUP@molrJt%jF>1y`ojM;*E>q(!WV4k6j+@{ zbxQAcc!1d=EcSk$N8eNT_#ERv(SU4z>_2sv5E|p&s&nkg&8>60qZwlDXNQQVV=E$c zaE9rI;JVGl&Ty^sYx=l`t?rXY8~+0zy=Oo#2jwX~36})H%}ltDscWvne4~;wfx=>V z?BdMde6n3?cDNgR3VA9_@_gg2CAX&4BH*TUD@3k{CoW0wVC1vT2Gb;GY-krcK~?3L z!Rx-)eJ2*6FH&8QJYs2)uS~QbR_SuALqw!JrZxo&&$!xFZAP>IWdeD_8Lruq>0>^_ zMEsm2Ta2m6f+W{mYP)aXz{~fAYSSq%n-^TF%RGo3*8Tm+2_C|==#Wb%86f<0K{js} zCaj!|QX9j~m(tJ@zFbd^&;4q;a;9>Q`GK8V*?`GlZiap4%5`iiXN*j{(|??mR}oNW zuWML=C0mhY12b&PK0Btj5Qg2KA?Yl&k2W1J6Fr95;l6ub^qMbLW1&u_araVaENX(z z*!J%{ED{XBT%*MvXeqCyUZbwk(Zig>E998zLo(sGbzsE{|qa9F?J z?-4#Tuyl3lR!1uHSaJ3|8&ZnUtH`O9M*^|%H*7`!qA^f%8ud0}jF&0ka>hz*tVtNMEX zL+*!%35O3l=mG{`<5D}Q1^X6E+)ikfAP-bd6q6s&ibNuly{!c+?fxI8D)L=O6_>~3L(ZmLk8<9OJGT#` zsumYio$3-wIb}Xa7vd3huAjC`GBYKlaw(ewM8w9+35ks0O#~^VybewI2|-8B1d!Fc$}oTYT@hzN zuspz+Knvb-bM9Nj-rzFiEbbQQn5))^3|drn=tSFIg8 zKWw>a8=$*Vwc@dF3kkbCPySt;SaZA4K&><1=|- zbGtFj$_6c6#&@j<-RfRan* znSZm_Wu@RWiD6Civ3FNiVn%|YH~QC;c!KPS(oi6N1-FSTndF@Yxrin}N0=K9b63`!k% z%KUFG6lo(iPG%sF@!&N8`!&#L_57CYSSm+l+4Ee1S|P>XDuuFYRGHR)It5fN_k`iY zyaNT2fHl@lAi`!rOXDf~G~ul-U?l-)v4DjFpw0I(LIJoIs2^=@8Q>RC06lN_;D8EX z>efoqaqg8voPNFfx0+H@*5$1|0MD_K5xH~6kVswD2(Kp?juZ+EpEmCqvxMLE$aJG8 z9DoMA*sUakfg_EwEu53GR$xmOFyAh%s(SSmOlPI@zJ|-uV*f}w|G3U;l$VjAt1!9G zhkIbTh$NPlmOd>taqic)vtcOZ{%9N!8MFGD6#IZ~sD4ja#gi6Yn zA;b6phS=NP$iH7wtyTLG8!V?3sTap*lzc145#DVg?(475I|DSn{&RPCHwyqjC~=+sYecy@MnGZrM& z1>mqPEiKKNC4I>p6Vo!|+ zFyzPAq?+4y_cy12sUlXIDq_Lp|F#?vC%mFrbNT+%_$vfh1X$?aOWuoHPwj74k=*}l zKA4h;ApBKmH$qTn@~;V5x-Vb8%r?6+LwQ3o@9*zlKr*GNENyJMH%^uH^#xb!uCYgu zk&ym+3BUSoY90a&_=I9&VqU=i68kb;qDCJ-chq(fC`x{Hw3M3cFaUm}rpB?#1=`%% z-NnD)Wm7nhVyHgmA9;w(`uTGwN`IPwkPy%ifqsM4s$O1bcC|B)YJG+qPTWOG1o7pX zeG+I^MwbO|XxRGw zHFvEfPW4@c`cIVF4{z0FW>)9xG}2aRrjV;l^Y%$fhG;OOhQ2Lqv!`T6r_dosI) zlTpvEQ^&5;(NDK5G6Yc2BktxHHtR56Z4eR5GGK$N)1!T`QW!$B#%Z&BSZ=T6bt)3sgG+*5lat zxUX)#?XN=@F?DqgZtl;p{=wkm63IAd&qD-cs3|(~w|(^AuPZAnnKsVhmo~P{@8^eeo6-KL`3j_RY8u@Z18WR8X~0$FrkPzQ3JipU8}9&7^4UX|B|)HD?Hl!)kalE> zi;F834x8HJH&5Ma&M$90XJ==>PVRP{oSY<33dhFACS_#w;2L_KF4n#Qn4;iUx`Eh` z6?Di-WOuhjX-8}4+6Q6`3hT|Dx zEA)Orx|B3#QC&_YuI3R=WbC6a)td!&KJW8e>N~&*Gh$?8AX_@UkQw`XPzOSVhbwYN>;D!*O! zW$TE)i}OH>ii1xxq0s#Ny#4kFaY<<@z}_+T^3o@S>mM5Wo&V0v+aB+%guA*pZDu5rG<6*PX z=EdXnmxBlmetKpGEH;XiW!PWohwHz8n~oM=e~10VMJP}G8rm0%B03nyMMFcQqN(X) z19jJ$ef##gRhFCzR$o;ZES_g56coamjJSx}{k>uX#aJ#G0t(sl$ZI@c zMc$d2y#YuDSw-|mOU*$Hnq}>uxWrVj0J83n+FErF=@rGx^WX5++6lnKg_o5v9nRbg z+W-jj^>$yFC@Ms-{6O2xIbN5Vj(0RQ@gewqUGaG$-q_gC`Cw^hr-Y2qA9STwIP(eE zp^brf28007c?&QGpo&z~)#F_}0pKe=0|R*o#nEzW=!*B;N!D5$9T%N8&H zlQ!pm!;Wxo%$1LV9p^ubLV(i%L+X9A_PPFCAu#6f;thh|+G$adC|0*kK(57Qyfl^l z!(Zp2p`mu;N=uuWsi`$UjQKRciKl`EhmVi{v%GvpJtdI}F`G8Kp#iVDx_aKJd>DB- z)~2MS#ByJ(s;X*iYAU|YV+PbKN5{tpw6(R>VDv!!lQ9oP0TQXtpV<{N1R5FQCtc+j zaXQN(k4WV+#mJ=aXhiQUEVj!o83x%5d>)*M_+3zd-xQcN2s&aUqOQJFrcrn;s4gk_ z^6GfSZgxAGQtDX&u(4$$lk52_J};$5NKw(w&Q3dE?6^|9i88~VDbw$~xTxVWrW|Ag z0t47$Txw}eD5Bq4TNfMMNYA(YSfu@WP~+FTUl@z{Qdf zF*GElrlxkXFC_*ZOI{eiE|NP5^tX##$a50oMLq)x@7;5oJ9%67JU?hdxI!vwYW(x^-ozy& zw4HpOJ1Wx!6li$(_~U{}xS(|7{`Oqw&$SxRd<{NNZ8-0M0}RmdzXPGbb=0q7?A$)z zmHa3`f{sQa@H3U#oGsbu2|Yj*Y<){BtMS_UmX@%L3=;6|TpOJB{}zF{w6c;lEFwZQ zhUzf!f<`H0bvx6iN7K-&cAwuUFQP^|2DW}7Tq=^}g`19ld0U&1joJrbTF`jGlKyX#{yED5mf*4B)`e)y4}|B^Zu(Hlals;b&F*7rYA={Py5(8z1f z$$=Rh9BgnoB$%F_UUh67`~AD7_`$-=Oax2fXAzQ2%rM;n15T4S^mt(`@FLrsM+kpu8XZyyXS2Ta?Z4LbiQN=S_TG>x(~wpPpJS(V>bd* z0uu}w{KE-_1B^1~=jZcxA=zX&q4;%gWMZ!7ehaH$fx=$^T@sy{8JV9?ZE9fw<_xT( zz%@bLmsus^RuwF>!CjMntM{OZ@bK`y(a~Oy<5u|UITV=~AX2&fZ8bGm>FMcxz|ypl zieS|2%VdmdP?G(pn=d*xx3)Tf0-!_!N#wnRUUO4ZOY~|mHDDcp)}6=t>F4K1d~Il8 zfTo=Q)c33JU7w|JK?r8QHH6<~%-UClBB4?PG376u>K-;{2WW&f@F$1&Z}wAwXi;Dx z{n~YUwsZ?+uI#D9`Ip4NgVENuay#NmRoJKG=AN-XM1LGS75Q3ih0)zy`a7xk;} zAB>Hmt*x!+b+C%3LV9{6!6X+DAD`c{aC2WTy1qq zW;VMXlN>EJ0%!SEnvmDLy(Wvg`YG5#>p@Ed9;)ZXa;%Z7~0 zBydguWF~lSP8)Fb0m)UtQc+g^3Xa}@O|xsemDE)AfAP;dGx+%J+c*2>C1AUO=UFlL z$?fUk+NgT&(N1)3s#tUF{I;U44Gr`dgu`iq9@8v~2Z11HTV6I88X2J=LIbi51L!So zR==Uw5%JF1HUqQw_CM8l_!fALx)*7&vFDK_A%>DwjFfmN6gZ*3OxehpnU4czUPPPW z1BWcQcjIZES3F0>^Loj(3992iHMo2+GW#Zh7%ad2eaVEi7v3 ztc!{$C@H`FH;7;tTN4vNnlo!}ZKR!Wx-(9>*yJqdVW@&N`-$s614=QU+}OyAf`+yZ zEL!04U{y(LB4Y(D0`zkL|9@Y>PaF*B?(ViAiEUy}qzwV*T^k7hv^?fvTED4ZHf`X- zQ~rB(?76u)pzwaJbp~a6C)7tiYuI3y_1Q`jmegV3=LTB8gBAG-+#?kgOc1y_hc3XW zs}H&%e}Dd-CC%2>*0Tb$Nh}_ILB-6B4k!s`>+Ld+TAyB+Ez-X_?Se{x0v{O(Me#Dx H4}Sj-fZ#Ly diff --git a/utils/devtool.rb b/utils/devtool.rb index d104f31c6..823cdcda7 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -146,7 +146,8 @@ def make(bsp = nil) def make_xtra return if @user_has_supplied_crates - puts 'Make Xtra crates'.light_blue + puts 'Make Xtra stuff'.light_blue + system('cd 07_uart_chainloader && bash update.sh') system('cd X1_JTAG_boot && bash update.sh') end diff --git a/utils/minipush.rb b/utils/minipush.rb index a6ccaa73f..710013437 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -17,7 +17,7 @@ class MiniPush < MiniTerm def initialize(serial_name, binary_image_path) super(serial_name) - @name_short = 'MP' + @name_short = 'MP' # override @binary_image_path = binary_image_path @binary_size = nil @binary_image = nil @@ -28,11 +28,13 @@ def initialize(serial_name, binary_image_path) # The three characters signaling the request token are expected to arrive as the last three # characters _at the end_ of a character stream (e.g. after a header print from Miniload). def wait_for_binary_request - Timeout.timeout(30) do - loop do - received = @target_serial.readpartial(4096) + puts "[#{@name_short}] 🔌 Please power the target now" - raise ConnectionError if received.nil? + # Timeout for the request token starts after the first sign of life was received. + received = @target_serial.readpartial(4096) + Timeout.timeout(10) do + loop do + raise ProtocolError if received.nil? if received.chars.last(3) == ["\u{3}", "\u{3}", "\u{3}"] # Print the last chunk minus the request token. @@ -41,6 +43,8 @@ def wait_for_binary_request end print received + + received = @target_serial.readpartial(4096) end end end @@ -70,18 +74,20 @@ def send_binary end end - # When the serial is still powered. - def handle_protocol_error + # override + def handle_reconnect connetion_reset puts puts "[#{@name_short}] ⚡ " \ - "#{'Protocol Error: Remove and insert the USB serial again'.light_red}" + "#{'Connection or protocol Error: '.light_red}" \ + "#{'Remove power and USB serial. Reinsert serial first, then power'.light_red}" sleep(1) while serial_connected? end public + # override def run open_serial wait_for_binary_request @@ -89,12 +95,9 @@ def run send_size send_binary terminal - rescue ConnectionError, EOFError, Errno::EIO + rescue ConnectionError, EOFError, Errno::EIO, ProtocolError, Timeout::Error handle_reconnect retry - rescue ProtocolError, Timeout::Error - handle_protocol_error - retry rescue StandardError => e handle_unexpected(e) ensure diff --git a/utils/miniterm.rb b/utils/miniterm.rb index fb05189ae..6b2c2f633 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -29,11 +29,12 @@ def serial_connected? end def wait_for_serial - loop do - break if serial_connected? + return if serial_connected? - print "\r[#{@name_short}] ⏳ Waiting for #{@target_serial_name}" + puts "[#{@name_short}] ⏳ Waiting for #{@target_serial_name}" + loop do sleep(1) + break if serial_connected? end end @@ -45,12 +46,10 @@ def open_serial # Ensure all output is immediately flushed to the device. @target_serial.sync = true rescue Errno::EACCES => e - puts puts "[#{@name_short}] 🚫 #{e.message} - Maybe try with 'sudo'" exit else - puts - puts "[#{@name_short}] ✅ Connected" + puts "[#{@name_short}] ✅ Serial connected" end def terminal From 53c1163c51931ac5c58da03cffeb46ada2486f22 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 27 Dec 2020 00:20:24 +0100 Subject: [PATCH 008/214] 01: Remove LTO to fix linking bug. For some reason, LTO caused "_start" to start at 0x00080020 instead of 0x00080000. --- 01_wait_forever/Cargo.toml | 3 -- 01_wait_forever/src/_arch/aarch64/cpu.rs | 18 -------- 01_wait_forever/src/panic_wait.rs | 4 +- 02_runtime_init/README.md | 57 ++++++++++++++++++++++++ 4 files changed, 59 insertions(+), 23 deletions(-) diff --git a/01_wait_forever/Cargo.toml b/01_wait_forever/Cargo.toml index 0f798fcbe..0595aa573 100644 --- a/01_wait_forever/Cargo.toml +++ b/01_wait_forever/Cargo.toml @@ -4,9 +4,6 @@ version = "0.1.0" authors = ["Andre Richter "] edition = "2018" -[profile.release] -lto = true - # The features section is used to select the target board. [features] default = [] diff --git a/01_wait_forever/src/_arch/aarch64/cpu.rs b/01_wait_forever/src/_arch/aarch64/cpu.rs index 1d3e871a6..12837a3b3 100644 --- a/01_wait_forever/src/_arch/aarch64/cpu.rs +++ b/01_wait_forever/src/_arch/aarch64/cpu.rs @@ -6,21 +6,3 @@ // Assembly counterpart to this file. global_asm!(include_str!("cpu.S")); - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Pause execution on the core. -#[inline(always)] -pub fn wait_forever() -> ! { - unsafe { - loop { - #[rustfmt::skip] - asm!( - "wfe", - options(nomem, nostack, preserves_flags) - ); - } - } -} diff --git a/01_wait_forever/src/panic_wait.rs b/01_wait_forever/src/panic_wait.rs index 560a1724e..0a70c7215 100644 --- a/01_wait_forever/src/panic_wait.rs +++ b/01_wait_forever/src/panic_wait.rs @@ -4,10 +4,10 @@ //! A panic handler that infinitely waits. -use crate::cpu; +// use crate::cpu; use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { - cpu::wait_forever() + unimplemented!() } diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index e54520cd3..75a6fa1b8 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -23,6 +23,46 @@ ## Diff to previous ```diff +diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml +--- 01_wait_forever/Cargo.toml ++++ 02_runtime_init/Cargo.toml +@@ -4,6 +4,9 @@ + authors = ["Andre Richter "] + edition = "2018" + ++[profile.release] ++lto = true ++ + # The features section is used to select the target board. + [features] + default = [] + +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs +--- 01_wait_forever/src/_arch/aarch64/cpu.rs ++++ 02_runtime_init/src/_arch/aarch64/cpu.rs +@@ -6,3 +6,21 @@ + + // Assembly counterpart to this file. + global_asm!(include_str!("cpu.S")); ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Pause execution on the core. ++#[inline(always)] ++pub fn wait_forever() -> ! { ++ unsafe { ++ loop { ++ #[rustfmt::skip] ++ asm!( ++ "wfe", ++ options(nomem, nostack, preserves_flags) ++ ); ++ } ++ } ++} + diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.S 02_runtime_init/src/_arch/aarch64/cpu.S --- 01_wait_forever/src/_arch/aarch64/cpu.S +++ 02_runtime_init/src/_arch/aarch64/cpu.S @@ -191,6 +231,23 @@ diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs + } +} +diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs +--- 01_wait_forever/src/panic_wait.rs ++++ 02_runtime_init/src/panic_wait.rs +@@ -4,10 +4,10 @@ + + //! A panic handler that infinitely waits. + +-// use crate::cpu; ++use crate::cpu; + use core::panic::PanicInfo; + + #[panic_handler] + fn panic(_info: &PanicInfo) -> ! { +- unimplemented!() ++ cpu::wait_forever() + } + diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.rs --- 01_wait_forever/src/runtime_init.rs +++ 02_runtime_init/src/runtime_init.rs From 4caf880e0c32b34c1c5276734ad2188ab1a342fa Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 27 Dec 2020 00:35:47 +0100 Subject: [PATCH 009/214] Update README --- 01_wait_forever/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/01_wait_forever/README.md b/01_wait_forever/README.md index b1927c9bb..0bd3e7429 100644 --- a/01_wait_forever/README.md +++ b/01_wait_forever/README.md @@ -25,8 +25,8 @@ - `#![no_std]`, `#![no_main]` - `cpu.S`: Assembly `_start()` function that executes `wfe` (Wait For Event), halting all cores that are executing `_start()`. -- We (have to) define a `#[panic_handler]` function. - - Just waits infinitely for a cpu event. +- We (have to) define a `#[panic_handler]` function to make the compiler happy. + - Make it `unimplemented!()` because it will be stripped out since it is not used. [inner attributes]: https://doc.rust-lang.org/reference/attributes.html From 4618858f4549fcbb80e3790d33d9b8745c8f3dea Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 27 Dec 2020 22:58:10 +0100 Subject: [PATCH 010/214] Update panic_wait.rs --- 01_wait_forever/src/panic_wait.rs | 1 - 1 file changed, 1 deletion(-) diff --git a/01_wait_forever/src/panic_wait.rs b/01_wait_forever/src/panic_wait.rs index 0a70c7215..1a9bec2a1 100644 --- a/01_wait_forever/src/panic_wait.rs +++ b/01_wait_forever/src/panic_wait.rs @@ -4,7 +4,6 @@ //! A panic handler that infinitely waits. -// use crate::cpu; use core::panic::PanicInfo; #[panic_handler] From cf34736f555c99c180bf5107751786a66deedda0 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 28 Dec 2020 15:23:36 +0100 Subject: [PATCH 011/214] update deps --- 13_integrated_testing/Cargo.lock | 8 ++++---- 14_exceptions_part2_peripheral_IRQs/Cargo.lock | 8 ++++---- 15_virtual_mem_part2_mmio_remap/Cargo.lock | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/13_integrated_testing/Cargo.lock b/13_integrated_testing/Cargo.lock index 0334159de..5ce481b37 100644 --- a/13_integrated_testing/Cargo.lock +++ b/13_integrated_testing/Cargo.lock @@ -31,9 +31,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" +checksum = "0e64f0ef443037525a562c4fa6ad3460a351ec10c7ebb0b6c49a87a576d0d1b5" [[package]] name = "quote" @@ -55,9 +55,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.55" +version = "1.0.56" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" +checksum = "a9802ddde94170d186eeee5005b798d9c159fa970403f1be19976d0cfb939b72" dependencies = [ "proc-macro2", "quote", diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.lock b/14_exceptions_part2_peripheral_IRQs/Cargo.lock index 0334159de..5ce481b37 100644 --- a/14_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/14_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -31,9 +31,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" +checksum = "0e64f0ef443037525a562c4fa6ad3460a351ec10c7ebb0b6c49a87a576d0d1b5" [[package]] name = "quote" @@ -55,9 +55,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.55" +version = "1.0.56" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" +checksum = "a9802ddde94170d186eeee5005b798d9c159fa970403f1be19976d0cfb939b72" dependencies = [ "proc-macro2", "quote", diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.lock b/15_virtual_mem_part2_mmio_remap/Cargo.lock index 0334159de..5ce481b37 100644 --- a/15_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/15_virtual_mem_part2_mmio_remap/Cargo.lock @@ -31,9 +31,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b73ae13954572c7ca0ec48ba9fe6a59c0392066eba62f8cb384ffd5addf538c5" +checksum = "0e64f0ef443037525a562c4fa6ad3460a351ec10c7ebb0b6c49a87a576d0d1b5" [[package]] name = "quote" @@ -55,9 +55,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.55" +version = "1.0.56" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a571a711dddd09019ccc628e1b17fe87c59b09d513c06c026877aa708334f37a" +checksum = "a9802ddde94170d186eeee5005b798d9c159fa970403f1be19976d0cfb939b72" dependencies = [ "proc-macro2", "quote", From 44bb3f89429e28a5855e205ee1472784c88d580d Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 1 Jan 2021 11:28:32 +0100 Subject: [PATCH 012/214] Update copyright year --- .githooks/pre-commit | 2 +- 01_wait_forever/Makefile | 2 +- 01_wait_forever/src/_arch/aarch64/cpu.S | 2 +- 01_wait_forever/src/_arch/aarch64/cpu.rs | 2 +- 01_wait_forever/src/bsp.rs | 2 +- 01_wait_forever/src/bsp/raspberrypi.rs | 2 +- 01_wait_forever/src/bsp/raspberrypi/link.ld | 2 +- 01_wait_forever/src/cpu.rs | 2 +- 01_wait_forever/src/main.rs | 2 +- 01_wait_forever/src/panic_wait.rs | 2 +- 02_runtime_init/Makefile | 2 +- 02_runtime_init/README.CN.md | 4 ++-- 02_runtime_init/README.md | 9 ++++----- 02_runtime_init/src/_arch/aarch64/cpu.S | 2 +- 02_runtime_init/src/_arch/aarch64/cpu.rs | 2 +- 02_runtime_init/src/bsp.rs | 2 +- 02_runtime_init/src/bsp/raspberrypi.rs | 2 +- 02_runtime_init/src/bsp/raspberrypi/link.ld | 2 +- 02_runtime_init/src/bsp/raspberrypi/memory.rs | 2 +- 02_runtime_init/src/cpu.rs | 2 +- 02_runtime_init/src/main.rs | 2 +- 02_runtime_init/src/memory.rs | 2 +- 02_runtime_init/src/panic_wait.rs | 2 +- 02_runtime_init/src/runtime_init.rs | 2 +- 03_hacky_hello_world/Makefile | 2 +- 03_hacky_hello_world/README.md | 6 +++--- 03_hacky_hello_world/src/_arch/aarch64/cpu.S | 2 +- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs | 2 +- 03_hacky_hello_world/src/bsp.rs | 2 +- 03_hacky_hello_world/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 03_hacky_hello_world/src/console.rs | 2 +- 03_hacky_hello_world/src/cpu.rs | 2 +- 03_hacky_hello_world/src/main.rs | 2 +- 03_hacky_hello_world/src/memory.rs | 2 +- 03_hacky_hello_world/src/panic_wait.rs | 2 +- 03_hacky_hello_world/src/print.rs | 2 +- 03_hacky_hello_world/src/runtime_init.rs | 2 +- 04_zero_overhead_abstraction/Makefile | 2 +- 04_zero_overhead_abstraction/README.md | 8 ++++---- .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- 04_zero_overhead_abstraction/src/bsp.rs | 2 +- .../src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 04_zero_overhead_abstraction/src/console.rs | 2 +- 04_zero_overhead_abstraction/src/cpu.rs | 2 +- 04_zero_overhead_abstraction/src/cpu/smp.rs | 2 +- 04_zero_overhead_abstraction/src/main.rs | 2 +- 04_zero_overhead_abstraction/src/memory.rs | 2 +- .../src/panic_wait.rs | 2 +- 04_zero_overhead_abstraction/src/print.rs | 2 +- .../src/runtime_init.rs | 2 +- 05_safe_globals/Makefile | 2 +- 05_safe_globals/README.md | 2 +- 05_safe_globals/src/_arch/aarch64/cpu.rs | 2 +- 05_safe_globals/src/_arch/aarch64/cpu/smp.rs | 2 +- 05_safe_globals/src/bsp.rs | 2 +- 05_safe_globals/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- 05_safe_globals/src/bsp/raspberrypi/cpu.rs | 2 +- 05_safe_globals/src/bsp/raspberrypi/link.ld | 2 +- 05_safe_globals/src/bsp/raspberrypi/memory.rs | 2 +- 05_safe_globals/src/console.rs | 2 +- 05_safe_globals/src/cpu.rs | 2 +- 05_safe_globals/src/cpu/smp.rs | 2 +- 05_safe_globals/src/main.rs | 2 +- 05_safe_globals/src/memory.rs | 2 +- 05_safe_globals/src/panic_wait.rs | 2 +- 05_safe_globals/src/print.rs | 2 +- 05_safe_globals/src/runtime_init.rs | 2 +- 05_safe_globals/src/synchronization.rs | 2 +- 06_drivers_gpio_uart/Makefile | 2 +- 06_drivers_gpio_uart/README.md | 14 ++++++------- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- 06_drivers_gpio_uart/src/bsp.rs | 2 +- 06_drivers_gpio_uart/src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- 06_drivers_gpio_uart/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 06_drivers_gpio_uart/src/console.rs | 2 +- 06_drivers_gpio_uart/src/cpu.rs | 2 +- 06_drivers_gpio_uart/src/cpu/smp.rs | 2 +- 06_drivers_gpio_uart/src/driver.rs | 2 +- 06_drivers_gpio_uart/src/main.rs | 2 +- 06_drivers_gpio_uart/src/memory.rs | 2 +- 06_drivers_gpio_uart/src/panic_wait.rs | 2 +- 06_drivers_gpio_uart/src/print.rs | 2 +- 06_drivers_gpio_uart/src/runtime_init.rs | 2 +- 06_drivers_gpio_uart/src/synchronization.rs | 2 +- 07_uart_chainloader/Makefile | 2 +- 07_uart_chainloader/README.md | 2 +- 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- 07_uart_chainloader/src/bsp.rs | 2 +- 07_uart_chainloader/src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- 07_uart_chainloader/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 07_uart_chainloader/src/console.rs | 2 +- 07_uart_chainloader/src/cpu.rs | 2 +- 07_uart_chainloader/src/cpu/smp.rs | 2 +- 07_uart_chainloader/src/driver.rs | 2 +- 07_uart_chainloader/src/main.rs | 2 +- 07_uart_chainloader/src/memory.rs | 2 +- 07_uart_chainloader/src/panic_wait.rs | 2 +- 07_uart_chainloader/src/print.rs | 2 +- 07_uart_chainloader/src/relocate.rs | 2 +- 07_uart_chainloader/src/runtime_init.rs | 2 +- 07_uart_chainloader/src/synchronization.rs | 2 +- 08_timestamps/Makefile | 2 +- 08_timestamps/README.md | 6 +++--- 08_timestamps/src/_arch/aarch64/cpu.rs | 2 +- 08_timestamps/src/_arch/aarch64/cpu/smp.rs | 2 +- 08_timestamps/src/_arch/aarch64/time.rs | 2 +- 08_timestamps/src/bsp.rs | 2 +- 08_timestamps/src/bsp/device_driver.rs | 2 +- 08_timestamps/src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- 08_timestamps/src/bsp/device_driver/common.rs | 2 +- 08_timestamps/src/bsp/raspberrypi.rs | 2 +- 08_timestamps/src/bsp/raspberrypi/console.rs | 2 +- 08_timestamps/src/bsp/raspberrypi/cpu.rs | 2 +- 08_timestamps/src/bsp/raspberrypi/driver.rs | 2 +- 08_timestamps/src/bsp/raspberrypi/link.ld | 2 +- 08_timestamps/src/bsp/raspberrypi/memory.rs | 2 +- 08_timestamps/src/console.rs | 2 +- 08_timestamps/src/cpu.rs | 2 +- 08_timestamps/src/cpu/smp.rs | 2 +- 08_timestamps/src/driver.rs | 2 +- 08_timestamps/src/main.rs | 2 +- 08_timestamps/src/memory.rs | 2 +- 08_timestamps/src/panic_wait.rs | 2 +- 08_timestamps/src/print.rs | 2 +- 08_timestamps/src/runtime_init.rs | 2 +- 08_timestamps/src/synchronization.rs | 2 +- 08_timestamps/src/time.rs | 2 +- 09_hw_debug_JTAG/Makefile | 2 +- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs | 2 +- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs | 2 +- 09_hw_debug_JTAG/src/_arch/aarch64/time.rs | 2 +- 09_hw_debug_JTAG/src/bsp.rs | 2 +- 09_hw_debug_JTAG/src/bsp/device_driver.rs | 2 +- 09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- 09_hw_debug_JTAG/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- 09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- 09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 09_hw_debug_JTAG/src/console.rs | 2 +- 09_hw_debug_JTAG/src/cpu.rs | 2 +- 09_hw_debug_JTAG/src/cpu/smp.rs | 2 +- 09_hw_debug_JTAG/src/driver.rs | 2 +- 09_hw_debug_JTAG/src/main.rs | 2 +- 09_hw_debug_JTAG/src/memory.rs | 2 +- 09_hw_debug_JTAG/src/panic_wait.rs | 2 +- 09_hw_debug_JTAG/src/print.rs | 2 +- 09_hw_debug_JTAG/src/runtime_init.rs | 2 +- 09_hw_debug_JTAG/src/synchronization.rs | 2 +- 09_hw_debug_JTAG/src/time.rs | 2 +- 10_privilege_level/Makefile | 2 +- 10_privilege_level/README.md | 8 ++++---- 10_privilege_level/src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- 10_privilege_level/src/_arch/aarch64/time.rs | 2 +- 10_privilege_level/src/bsp.rs | 2 +- 10_privilege_level/src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- 10_privilege_level/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- 10_privilege_level/src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- 10_privilege_level/src/console.rs | 2 +- 10_privilege_level/src/cpu.rs | 2 +- 10_privilege_level/src/cpu/smp.rs | 2 +- 10_privilege_level/src/driver.rs | 2 +- 10_privilege_level/src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- 10_privilege_level/src/main.rs | 2 +- 10_privilege_level/src/memory.rs | 2 +- 10_privilege_level/src/panic_wait.rs | 2 +- 10_privilege_level/src/print.rs | 2 +- 10_privilege_level/src/runtime_init.rs | 2 +- 10_privilege_level/src/synchronization.rs | 2 +- 10_privilege_level/src/time.rs | 2 +- .../Makefile | 2 +- .../README.md | 6 +++--- .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/_arch/aarch64/time.rs | 2 +- .../src/bsp.rs | 2 +- .../src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- .../src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 2 +- .../src/console.rs | 2 +- .../src/cpu.rs | 2 +- .../src/cpu/smp.rs | 2 +- .../src/driver.rs | 2 +- .../src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- .../src/main.rs | 2 +- .../src/memory.rs | 2 +- .../src/memory/mmu.rs | 2 +- .../src/panic_wait.rs | 2 +- .../src/print.rs | 2 +- .../src/runtime_init.rs | 2 +- .../src/synchronization.rs | 2 +- .../src/time.rs | 2 +- 12_exceptions_part1_groundwork/Makefile | 2 +- 12_exceptions_part1_groundwork/README.md | 2 +- .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/_arch/aarch64/time.rs | 2 +- 12_exceptions_part1_groundwork/src/bsp.rs | 2 +- .../src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- .../src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 2 +- 12_exceptions_part1_groundwork/src/console.rs | 2 +- 12_exceptions_part1_groundwork/src/cpu.rs | 2 +- 12_exceptions_part1_groundwork/src/cpu/smp.rs | 2 +- 12_exceptions_part1_groundwork/src/driver.rs | 2 +- .../src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- 12_exceptions_part1_groundwork/src/main.rs | 2 +- 12_exceptions_part1_groundwork/src/memory.rs | 2 +- .../src/memory/mmu.rs | 2 +- .../src/panic_wait.rs | 2 +- 12_exceptions_part1_groundwork/src/print.rs | 2 +- .../src/runtime_init.rs | 2 +- .../src/synchronization.rs | 2 +- 12_exceptions_part1_groundwork/src/time.rs | 2 +- 13_integrated_testing/Makefile | 2 +- 13_integrated_testing/README.md | 20 +++++++++---------- .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/_arch/aarch64/time.rs | 2 +- 13_integrated_testing/src/bsp.rs | 2 +- .../src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- 13_integrated_testing/src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 2 +- 13_integrated_testing/src/console.rs | 2 +- 13_integrated_testing/src/cpu.rs | 2 +- 13_integrated_testing/src/cpu/smp.rs | 2 +- 13_integrated_testing/src/driver.rs | 2 +- 13_integrated_testing/src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- 13_integrated_testing/src/lib.rs | 2 +- 13_integrated_testing/src/main.rs | 2 +- 13_integrated_testing/src/memory.rs | 2 +- 13_integrated_testing/src/memory/mmu.rs | 2 +- 13_integrated_testing/src/panic_wait.rs | 2 +- 13_integrated_testing/src/print.rs | 2 +- 13_integrated_testing/src/runtime_init.rs | 2 +- 13_integrated_testing/src/synchronization.rs | 2 +- 13_integrated_testing/src/time.rs | 2 +- 13_integrated_testing/test-macros/src/lib.rs | 2 +- 13_integrated_testing/test-types/src/lib.rs | 2 +- .../tests/00_console_sanity.rb | 2 +- .../tests/00_console_sanity.rs | 2 +- .../tests/01_timer_sanity.rs | 2 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/panic_exit_failure/mod.rs | 2 +- .../tests/panic_exit_success/mod.rs | 2 +- 13_integrated_testing/tests/runner.rb | 2 +- 14_exceptions_part2_peripheral_IRQs/Makefile | 2 +- 14_exceptions_part2_peripheral_IRQs/README.md | 20 +++++++++---------- .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/_arch/aarch64/time.rs | 2 +- .../src/bsp.rs | 2 +- .../src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/arm.rs | 2 +- .../src/bsp/device_driver/arm/gicv2.rs | 2 +- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 2 +- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../bcm/bcm2xxx_interrupt_controller.rs | 2 +- .../peripheral_ic.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- .../src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/exception.rs | 2 +- .../bsp/raspberrypi/exception/asynchronous.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 2 +- .../src/console.rs | 2 +- .../src/cpu.rs | 2 +- .../src/cpu/smp.rs | 2 +- .../src/driver.rs | 2 +- .../src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- .../src/lib.rs | 2 +- .../src/main.rs | 2 +- .../src/memory.rs | 2 +- .../src/memory/mmu.rs | 2 +- .../src/panic_wait.rs | 2 +- .../src/print.rs | 2 +- .../src/runtime_init.rs | 2 +- .../src/state.rs | 2 +- .../src/synchronization.rs | 2 +- .../src/time.rs | 2 +- .../test-macros/src/lib.rs | 2 +- .../test-types/src/lib.rs | 2 +- .../tests/00_console_sanity.rb | 2 +- .../tests/00_console_sanity.rs | 2 +- .../tests/01_timer_sanity.rs | 2 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/03_exception_irq_sanity.rs | 2 +- .../tests/panic_exit_failure/mod.rs | 2 +- .../tests/panic_exit_success/mod.rs | 2 +- .../tests/runner.rb | 2 +- 15_virtual_mem_part2_mmio_remap/Makefile | 2 +- 15_virtual_mem_part2_mmio_remap/README.md | 12 +++++------ .../src/_arch/aarch64/cpu.rs | 2 +- .../src/_arch/aarch64/cpu/smp.rs | 2 +- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 2 +- .../_arch/aarch64/exception/asynchronous.rs | 2 +- .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/_arch/aarch64/time.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/bsp.rs | 2 +- .../src/bsp/device_driver.rs | 2 +- .../src/bsp/device_driver/arm.rs | 2 +- .../src/bsp/device_driver/arm/gicv2.rs | 2 +- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 2 +- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 2 +- .../src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../bcm/bcm2xxx_interrupt_controller.rs | 2 +- .../peripheral_ic.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/device_driver/common.rs | 2 +- .../src/bsp/raspberrypi.rs | 2 +- .../src/bsp/raspberrypi/console.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/driver.rs | 2 +- .../src/bsp/raspberrypi/exception.rs | 2 +- .../bsp/raspberrypi/exception/asynchronous.rs | 2 +- .../src/bsp/raspberrypi/link.ld | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/common.rs | 2 +- .../src/console.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/cpu.rs | 2 +- .../src/cpu/smp.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/driver.rs | 2 +- .../src/exception.rs | 2 +- .../src/exception/asynchronous.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/main.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/memory.rs | 2 +- .../src/memory/mmu.rs | 2 +- .../src/memory/mmu/mapping_record.rs | 2 +- .../src/memory/mmu/types.rs | 2 +- .../src/panic_wait.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/print.rs | 2 +- .../src/runtime_init.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/state.rs | 2 +- .../src/synchronization.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/time.rs | 2 +- .../test-macros/src/lib.rs | 2 +- .../test-types/src/lib.rs | 2 +- .../tests/00_console_sanity.rb | 2 +- .../tests/00_console_sanity.rs | 2 +- .../tests/01_timer_sanity.rs | 2 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/03_exception_irq_sanity.rs | 2 +- .../tests/panic_exit_failure/mod.rs | 2 +- .../tests/panic_exit_success/mod.rs | 2 +- .../tests/runner.rb | 2 +- X1_JTAG_boot/Makefile | 2 +- X1_JTAG_boot/src/_arch/aarch64/cpu.rs | 2 +- X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs | 2 +- X1_JTAG_boot/src/_arch/aarch64/time.rs | 2 +- X1_JTAG_boot/src/bsp.rs | 2 +- X1_JTAG_boot/src/bsp/device_driver.rs | 2 +- X1_JTAG_boot/src/bsp/device_driver/bcm.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- X1_JTAG_boot/src/bsp/device_driver/common.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/console.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/driver.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/link.ld | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/memory.rs | 2 +- X1_JTAG_boot/src/console.rs | 2 +- X1_JTAG_boot/src/cpu.rs | 2 +- X1_JTAG_boot/src/cpu/smp.rs | 2 +- X1_JTAG_boot/src/driver.rs | 2 +- X1_JTAG_boot/src/main.rs | 2 +- X1_JTAG_boot/src/memory.rs | 2 +- X1_JTAG_boot/src/panic_wait.rs | 2 +- X1_JTAG_boot/src/print.rs | 2 +- X1_JTAG_boot/src/runtime_init.rs | 2 +- X1_JTAG_boot/src/synchronization.rs | 2 +- X1_JTAG_boot/src/time.rs | 2 +- docker/rustembedded-osdev-utils/Dockerfile | 4 ++-- docker/rustembedded-osdev-utils/Makefile | 2 +- utils/devtool.rb | 2 +- utils/devtool/copyright.rb | 2 +- utils/diff_tut_folders.bash | 4 ++-- utils/minipush.rb | 2 +- utils/minipush/progressbar_patch.rb | 2 +- utils/miniterm.rb | 2 +- utils/update_copyright.rb | 20 +++++++++++++++++++ 485 files changed, 551 insertions(+), 532 deletions(-) create mode 100755 utils/update_copyright.rb diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 9696481b9..6636f0075 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2018-2020 Andre Richter +# Copyright (c) 2018-2021 Andre Richter require 'rubygems' require 'bundler/setup' diff --git a/01_wait_forever/Makefile b/01_wait_forever/Makefile index 1481f8a45..ef6395f1b 100644 --- a/01_wait_forever/Makefile +++ b/01_wait_forever/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/01_wait_forever/src/_arch/aarch64/cpu.S b/01_wait_forever/src/_arch/aarch64/cpu.S index bd80689c7..2de0c61c5 100644 --- a/01_wait_forever/src/_arch/aarch64/cpu.S +++ b/01_wait_forever/src/_arch/aarch64/cpu.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter .section ".text._start" diff --git a/01_wait_forever/src/_arch/aarch64/cpu.rs b/01_wait_forever/src/_arch/aarch64/cpu.rs index 12837a3b3..6d24fe289 100644 --- a/01_wait_forever/src/_arch/aarch64/cpu.rs +++ b/01_wait_forever/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/01_wait_forever/src/bsp.rs b/01_wait_forever/src/bsp.rs index db48385eb..8f0d27c87 100644 --- a/01_wait_forever/src/bsp.rs +++ b/01_wait_forever/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/01_wait_forever/src/bsp/raspberrypi.rs b/01_wait_forever/src/bsp/raspberrypi.rs index cdaac7405..e7a4b6af2 100644 --- a/01_wait_forever/src/bsp/raspberrypi.rs +++ b/01_wait_forever/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/01_wait_forever/src/bsp/raspberrypi/link.ld b/01_wait_forever/src/bsp/raspberrypi/link.ld index c95db067b..81f01c11d 100644 --- a/01_wait_forever/src/bsp/raspberrypi/link.ld +++ b/01_wait_forever/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/01_wait_forever/src/cpu.rs b/01_wait_forever/src/cpu.rs index fd5c9794e..27aea2041 100644 --- a/01_wait_forever/src/cpu.rs +++ b/01_wait_forever/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/01_wait_forever/src/main.rs b/01_wait_forever/src/main.rs index 28b68b59f..883453535 100644 --- a/01_wait_forever/src/main.rs +++ b/01_wait_forever/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/01_wait_forever/src/panic_wait.rs b/01_wait_forever/src/panic_wait.rs index 1a9bec2a1..2887ed170 100644 --- a/01_wait_forever/src/panic_wait.rs +++ b/01_wait_forever/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/02_runtime_init/Makefile b/02_runtime_init/Makefile index 1481f8a45..ef6395f1b 100644 --- a/02_runtime_init/Makefile +++ b/02_runtime_init/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/02_runtime_init/README.CN.md b/02_runtime_init/README.CN.md index eb932091d..1369031d6 100644 --- a/02_runtime_init/README.CN.md +++ b/02_runtime_init/README.CN.md @@ -105,7 +105,7 @@ diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management. + @@ -139,7 +139,7 @@ diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.r @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Rust runtime initialization code. + diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 75a6fa1b8..206424f9e 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -123,7 +123,7 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/memory.rs 02_runtime_init/src/bsp/ @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management. + @@ -202,7 +202,7 @@ diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management. + @@ -234,11 +234,10 @@ diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs --- 01_wait_forever/src/panic_wait.rs +++ 02_runtime_init/src/panic_wait.rs -@@ -4,10 +4,10 @@ +@@ -4,9 +4,10 @@ //! A panic handler that infinitely waits. --// use crate::cpu; +use crate::cpu; use core::panic::PanicInfo; @@ -254,7 +253,7 @@ diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.r @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Rust runtime initialization code. + diff --git a/02_runtime_init/src/_arch/aarch64/cpu.S b/02_runtime_init/src/_arch/aarch64/cpu.S index 3ed5c0613..449cf5027 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu.S +++ b/02_runtime_init/src/_arch/aarch64/cpu.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter .section ".text._start" diff --git a/02_runtime_init/src/_arch/aarch64/cpu.rs b/02_runtime_init/src/_arch/aarch64/cpu.rs index 1d3e871a6..6ca5986aa 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu.rs +++ b/02_runtime_init/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/02_runtime_init/src/bsp.rs b/02_runtime_init/src/bsp.rs index db48385eb..8f0d27c87 100644 --- a/02_runtime_init/src/bsp.rs +++ b/02_runtime_init/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/02_runtime_init/src/bsp/raspberrypi.rs b/02_runtime_init/src/bsp/raspberrypi.rs index e4ecbd1d8..f9bd5b888 100644 --- a/02_runtime_init/src/bsp/raspberrypi.rs +++ b/02_runtime_init/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/02_runtime_init/src/bsp/raspberrypi/link.ld b/02_runtime_init/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/02_runtime_init/src/bsp/raspberrypi/link.ld +++ b/02_runtime_init/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/02_runtime_init/src/bsp/raspberrypi/memory.rs b/02_runtime_init/src/bsp/raspberrypi/memory.rs index 25ba9f156..5c6b1ea5b 100644 --- a/02_runtime_init/src/bsp/raspberrypi/memory.rs +++ b/02_runtime_init/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/02_runtime_init/src/cpu.rs b/02_runtime_init/src/cpu.rs index fd5c9794e..27aea2041 100644 --- a/02_runtime_init/src/cpu.rs +++ b/02_runtime_init/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/02_runtime_init/src/main.rs b/02_runtime_init/src/main.rs index 336824a16..76e7a772f 100644 --- a/02_runtime_init/src/main.rs +++ b/02_runtime_init/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/02_runtime_init/src/memory.rs b/02_runtime_init/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/02_runtime_init/src/memory.rs +++ b/02_runtime_init/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/02_runtime_init/src/panic_wait.rs b/02_runtime_init/src/panic_wait.rs index 560a1724e..a1705688d 100644 --- a/02_runtime_init/src/panic_wait.rs +++ b/02_runtime_init/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/02_runtime_init/src/runtime_init.rs b/02_runtime_init/src/runtime_init.rs index 0e3702b47..068dde843 100644 --- a/02_runtime_init/src/runtime_init.rs +++ b/02_runtime_init/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/03_hacky_hello_world/Makefile b/03_hacky_hello_world/Makefile index a9b0f82ce..90238d091 100644 --- a/03_hacky_hello_world/Makefile +++ b/03_hacky_hello_world/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 4fac48da4..95760f02d 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -56,7 +56,7 @@ diff -uNr 02_runtime_init/src/bsp/raspberrypi/console.rs 03_hacky_hello_world/sr @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BSP console facilities. + @@ -118,7 +118,7 @@ diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! System console. + @@ -200,7 +200,7 @@ diff -uNr 02_runtime_init/src/print.rs 03_hacky_hello_world/src/print.rs @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Printing facilities. + diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu.S b/03_hacky_hello_world/src/_arch/aarch64/cpu.S index 3ed5c0613..449cf5027 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu.S +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter .section ".text._start" diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs index 1d3e871a6..6ca5986aa 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/03_hacky_hello_world/src/bsp.rs b/03_hacky_hello_world/src/bsp.rs index db48385eb..8f0d27c87 100644 --- a/03_hacky_hello_world/src/bsp.rs +++ b/03_hacky_hello_world/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/03_hacky_hello_world/src/bsp/raspberrypi.rs b/03_hacky_hello_world/src/bsp/raspberrypi.rs index fba4f0a26..ac9d53e95 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/console.rs b/03_hacky_hello_world/src/bsp/raspberrypi/console.rs index 0b699e62b..fca7042e6 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/console.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld +++ b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs b/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs index 25ba9f156..5c6b1ea5b 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/03_hacky_hello_world/src/console.rs b/03_hacky_hello_world/src/console.rs index 3a3b638bb..27b79f7dd 100644 --- a/03_hacky_hello_world/src/console.rs +++ b/03_hacky_hello_world/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/03_hacky_hello_world/src/cpu.rs b/03_hacky_hello_world/src/cpu.rs index fd5c9794e..27aea2041 100644 --- a/03_hacky_hello_world/src/cpu.rs +++ b/03_hacky_hello_world/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/03_hacky_hello_world/src/main.rs b/03_hacky_hello_world/src/main.rs index 8b6f4b252..3a4cf64c4 100644 --- a/03_hacky_hello_world/src/main.rs +++ b/03_hacky_hello_world/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/03_hacky_hello_world/src/memory.rs b/03_hacky_hello_world/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/03_hacky_hello_world/src/memory.rs +++ b/03_hacky_hello_world/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/03_hacky_hello_world/src/panic_wait.rs b/03_hacky_hello_world/src/panic_wait.rs index b4d5bc1ae..255eedeeb 100644 --- a/03_hacky_hello_world/src/panic_wait.rs +++ b/03_hacky_hello_world/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/03_hacky_hello_world/src/print.rs b/03_hacky_hello_world/src/print.rs index 77ebd8b40..59ec9e8ee 100644 --- a/03_hacky_hello_world/src/print.rs +++ b/03_hacky_hello_world/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/03_hacky_hello_world/src/runtime_init.rs b/03_hacky_hello_world/src/runtime_init.rs index 0e3702b47..068dde843 100644 --- a/03_hacky_hello_world/src/runtime_init.rs +++ b/03_hacky_hello_world/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/04_zero_overhead_abstraction/Makefile b/04_zero_overhead_abstraction/Makefile index a9b0f82ce..90238d091 100644 --- a/04_zero_overhead_abstraction/Makefile +++ b/04_zero_overhead_abstraction/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index ca679c52a..003108219 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -29,7 +29,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abs @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Architectural symmetric multiprocessing. + @@ -114,7 +114,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.S 04_zero_overhead_abstract @@ -1,21 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// --// Copyright (c) 2018-2020 Andre Richter +-// Copyright (c) 2018-2021 Andre Richter - -.section ".text._start" - @@ -140,7 +140,7 @@ diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs 04_zero_overhead_abstr @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BSP Processor code. + @@ -197,7 +197,7 @@ diff -uNr 03_hacky_hello_world/src/cpu/smp.rs 04_zero_overhead_abstraction/src/c @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Symmetric multiprocessing. + diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs index 64c5397fb..61a68a18e 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/04_zero_overhead_abstraction/src/bsp.rs b/04_zero_overhead_abstraction/src/bsp.rs index db48385eb..8f0d27c87 100644 --- a/04_zero_overhead_abstraction/src/bsp.rs +++ b/04_zero_overhead_abstraction/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs index 0ad1b2aed..fdb7551a1 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs index 0b699e62b..fca7042e6 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld b/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs index 51cc958ad..77fdafd7a 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/04_zero_overhead_abstraction/src/console.rs b/04_zero_overhead_abstraction/src/console.rs index 3a3b638bb..27b79f7dd 100644 --- a/04_zero_overhead_abstraction/src/console.rs +++ b/04_zero_overhead_abstraction/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/04_zero_overhead_abstraction/src/cpu.rs b/04_zero_overhead_abstraction/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/04_zero_overhead_abstraction/src/cpu.rs +++ b/04_zero_overhead_abstraction/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/04_zero_overhead_abstraction/src/cpu/smp.rs b/04_zero_overhead_abstraction/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/04_zero_overhead_abstraction/src/cpu/smp.rs +++ b/04_zero_overhead_abstraction/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/04_zero_overhead_abstraction/src/main.rs b/04_zero_overhead_abstraction/src/main.rs index da1a257da..22e179229 100644 --- a/04_zero_overhead_abstraction/src/main.rs +++ b/04_zero_overhead_abstraction/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/04_zero_overhead_abstraction/src/memory.rs b/04_zero_overhead_abstraction/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/04_zero_overhead_abstraction/src/memory.rs +++ b/04_zero_overhead_abstraction/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/04_zero_overhead_abstraction/src/panic_wait.rs b/04_zero_overhead_abstraction/src/panic_wait.rs index b4d5bc1ae..255eedeeb 100644 --- a/04_zero_overhead_abstraction/src/panic_wait.rs +++ b/04_zero_overhead_abstraction/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/04_zero_overhead_abstraction/src/print.rs b/04_zero_overhead_abstraction/src/print.rs index 77ebd8b40..59ec9e8ee 100644 --- a/04_zero_overhead_abstraction/src/print.rs +++ b/04_zero_overhead_abstraction/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/04_zero_overhead_abstraction/src/runtime_init.rs b/04_zero_overhead_abstraction/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/04_zero_overhead_abstraction/src/runtime_init.rs +++ b/04_zero_overhead_abstraction/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/05_safe_globals/Makefile b/05_safe_globals/Makefile index a9b0f82ce..90238d091 100644 --- a/05_safe_globals/Makefile +++ b/05_safe_globals/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/05_safe_globals/README.md b/05_safe_globals/README.md index 1cbe90b8e..a66e7f2b4 100644 --- a/05_safe_globals/README.md +++ b/05_safe_globals/README.md @@ -253,7 +253,7 @@ diff -uNr 04_zero_overhead_abstraction/src/synchronization.rs 05_safe_globals/sr @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Synchronization primitives. +//! diff --git a/05_safe_globals/src/_arch/aarch64/cpu.rs b/05_safe_globals/src/_arch/aarch64/cpu.rs index 64c5397fb..61a68a18e 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs b/05_safe_globals/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/05_safe_globals/src/bsp.rs b/05_safe_globals/src/bsp.rs index db48385eb..8f0d27c87 100644 --- a/05_safe_globals/src/bsp.rs +++ b/05_safe_globals/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/05_safe_globals/src/bsp/raspberrypi.rs b/05_safe_globals/src/bsp/raspberrypi.rs index 0ad1b2aed..fdb7551a1 100644 --- a/05_safe_globals/src/bsp/raspberrypi.rs +++ b/05_safe_globals/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/05_safe_globals/src/bsp/raspberrypi/console.rs b/05_safe_globals/src/bsp/raspberrypi/console.rs index 074f4fa17..f340d940f 100644 --- a/05_safe_globals/src/bsp/raspberrypi/console.rs +++ b/05_safe_globals/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/05_safe_globals/src/bsp/raspberrypi/cpu.rs b/05_safe_globals/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/05_safe_globals/src/bsp/raspberrypi/cpu.rs +++ b/05_safe_globals/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/05_safe_globals/src/bsp/raspberrypi/link.ld b/05_safe_globals/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/05_safe_globals/src/bsp/raspberrypi/link.ld +++ b/05_safe_globals/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/05_safe_globals/src/bsp/raspberrypi/memory.rs b/05_safe_globals/src/bsp/raspberrypi/memory.rs index 51cc958ad..77fdafd7a 100644 --- a/05_safe_globals/src/bsp/raspberrypi/memory.rs +++ b/05_safe_globals/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/05_safe_globals/src/console.rs b/05_safe_globals/src/console.rs index 12b31c636..658cf66df 100644 --- a/05_safe_globals/src/console.rs +++ b/05_safe_globals/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/05_safe_globals/src/cpu.rs b/05_safe_globals/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/05_safe_globals/src/cpu.rs +++ b/05_safe_globals/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/05_safe_globals/src/cpu/smp.rs b/05_safe_globals/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/05_safe_globals/src/cpu/smp.rs +++ b/05_safe_globals/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/05_safe_globals/src/main.rs b/05_safe_globals/src/main.rs index 6a57de237..2c29444fd 100644 --- a/05_safe_globals/src/main.rs +++ b/05_safe_globals/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/05_safe_globals/src/memory.rs b/05_safe_globals/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/05_safe_globals/src/memory.rs +++ b/05_safe_globals/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/05_safe_globals/src/panic_wait.rs b/05_safe_globals/src/panic_wait.rs index b4d5bc1ae..255eedeeb 100644 --- a/05_safe_globals/src/panic_wait.rs +++ b/05_safe_globals/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/05_safe_globals/src/print.rs b/05_safe_globals/src/print.rs index 77ebd8b40..59ec9e8ee 100644 --- a/05_safe_globals/src/print.rs +++ b/05_safe_globals/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/05_safe_globals/src/runtime_init.rs b/05_safe_globals/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/05_safe_globals/src/runtime_init.rs +++ b/05_safe_globals/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/05_safe_globals/src/synchronization.rs b/05_safe_globals/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/05_safe_globals/src/synchronization.rs +++ b/05_safe_globals/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/06_drivers_gpio_uart/Makefile b/06_drivers_gpio_uart/Makefile index 06c268de7..0fde2eb8d 100644 --- a/06_drivers_gpio_uart/Makefile +++ b/06_drivers_gpio_uart/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index e3d502b60..33df63e87 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -210,7 +210,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_g @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! GPIO Driver. + @@ -436,7 +436,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri @@ -0,0 +1,305 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! PL011 UART driver. + @@ -746,7 +746,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm.rs 06_drivers_gpio_uart/src/ @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BCM driver top level. + @@ -762,7 +762,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/common.rs 06_drivers_gpio_uart/s @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Common device driver code. + @@ -805,7 +805,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver.rs 06_drivers_gpio_uart/src/bsp/ @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Device driver. + @@ -958,7 +958,7 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/driver.rs 06_drivers_gpio_uart/src @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BSP driver support. + @@ -1137,7 +1137,7 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Driver support. + diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index 1d5190ee1..0f545a008 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/06_drivers_gpio_uart/src/bsp.rs b/06_drivers_gpio_uart/src/bsp.rs index 3a5657ade..257502491 100644 --- a/06_drivers_gpio_uart/src/bsp.rs +++ b/06_drivers_gpio_uart/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/06_drivers_gpio_uart/src/bsp/device_driver.rs b/06_drivers_gpio_uart/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index f5abbee8a..771151e76 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f17a51ece..de0bf813b 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/common.rs b/06_drivers_gpio_uart/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/common.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs index fae051257..fb47b1418 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/06_drivers_gpio_uart/src/console.rs b/06_drivers_gpio_uart/src/console.rs index 2644e12c2..0a2385947 100644 --- a/06_drivers_gpio_uart/src/console.rs +++ b/06_drivers_gpio_uart/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/06_drivers_gpio_uart/src/cpu.rs b/06_drivers_gpio_uart/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/06_drivers_gpio_uart/src/cpu.rs +++ b/06_drivers_gpio_uart/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/06_drivers_gpio_uart/src/cpu/smp.rs b/06_drivers_gpio_uart/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/06_drivers_gpio_uart/src/cpu/smp.rs +++ b/06_drivers_gpio_uart/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/06_drivers_gpio_uart/src/driver.rs b/06_drivers_gpio_uart/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/06_drivers_gpio_uart/src/driver.rs +++ b/06_drivers_gpio_uart/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index 9e58e80b7..c45f7f785 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/06_drivers_gpio_uart/src/memory.rs b/06_drivers_gpio_uart/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/06_drivers_gpio_uart/src/memory.rs +++ b/06_drivers_gpio_uart/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/06_drivers_gpio_uart/src/panic_wait.rs b/06_drivers_gpio_uart/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/06_drivers_gpio_uart/src/panic_wait.rs +++ b/06_drivers_gpio_uart/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/06_drivers_gpio_uart/src/print.rs b/06_drivers_gpio_uart/src/print.rs index 77ebd8b40..59ec9e8ee 100644 --- a/06_drivers_gpio_uart/src/print.rs +++ b/06_drivers_gpio_uart/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/06_drivers_gpio_uart/src/runtime_init.rs b/06_drivers_gpio_uart/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/06_drivers_gpio_uart/src/runtime_init.rs +++ b/06_drivers_gpio_uart/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/06_drivers_gpio_uart/src/synchronization.rs b/06_drivers_gpio_uart/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/06_drivers_gpio_uart/src/synchronization.rs +++ b/06_drivers_gpio_uart/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/07_uart_chainloader/Makefile b/07_uart_chainloader/Makefile index 884a29f73..1fc61b6b2 100644 --- a/07_uart_chainloader/Makefile +++ b/07_uart_chainloader/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 8872bc18c..1491c84b8 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -512,7 +512,7 @@ diff -uNr 06_drivers_gpio_uart/src/relocate.rs 07_uart_chainloader/src/relocate. @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Relocation code. + diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/07_uart_chainloader/src/_arch/aarch64/cpu.rs index bd974d4d5..ccafe9ff6 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs b/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/07_uart_chainloader/src/bsp.rs b/07_uart_chainloader/src/bsp.rs index 3a5657ade..257502491 100644 --- a/07_uart_chainloader/src/bsp.rs +++ b/07_uart_chainloader/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/07_uart_chainloader/src/bsp/device_driver.rs b/07_uart_chainloader/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/07_uart_chainloader/src/bsp/device_driver.rs +++ b/07_uart_chainloader/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm.rs b/07_uart_chainloader/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index f5abbee8a..771151e76 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index a17d63ee7..1eb98fc4f 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/07_uart_chainloader/src/bsp/device_driver/common.rs b/07_uart_chainloader/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/07_uart_chainloader/src/bsp/device_driver/common.rs +++ b/07_uart_chainloader/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/07_uart_chainloader/src/bsp/raspberrypi.rs b/07_uart_chainloader/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/07_uart_chainloader/src/bsp/raspberrypi/console.rs b/07_uart_chainloader/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/console.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/07_uart_chainloader/src/bsp/raspberrypi/driver.rs b/07_uart_chainloader/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/driver.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/07_uart_chainloader/src/bsp/raspberrypi/link.ld b/07_uart_chainloader/src/bsp/raspberrypi/link.ld index a6dc653ab..721856cbc 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/link.ld +++ b/07_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs index 0af09903d..4b2642e36 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/07_uart_chainloader/src/console.rs b/07_uart_chainloader/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/07_uart_chainloader/src/console.rs +++ b/07_uart_chainloader/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/07_uart_chainloader/src/cpu.rs b/07_uart_chainloader/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/07_uart_chainloader/src/cpu.rs +++ b/07_uart_chainloader/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/07_uart_chainloader/src/cpu/smp.rs b/07_uart_chainloader/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/07_uart_chainloader/src/cpu/smp.rs +++ b/07_uart_chainloader/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/07_uart_chainloader/src/driver.rs b/07_uart_chainloader/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/07_uart_chainloader/src/driver.rs +++ b/07_uart_chainloader/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index 6640c754b..e71ab16cd 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/07_uart_chainloader/src/memory.rs b/07_uart_chainloader/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/07_uart_chainloader/src/memory.rs +++ b/07_uart_chainloader/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/07_uart_chainloader/src/panic_wait.rs b/07_uart_chainloader/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/07_uart_chainloader/src/panic_wait.rs +++ b/07_uart_chainloader/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/07_uart_chainloader/src/print.rs b/07_uart_chainloader/src/print.rs index 77ebd8b40..59ec9e8ee 100644 --- a/07_uart_chainloader/src/print.rs +++ b/07_uart_chainloader/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/07_uart_chainloader/src/relocate.rs b/07_uart_chainloader/src/relocate.rs index bbedfa475..85ddab672 100644 --- a/07_uart_chainloader/src/relocate.rs +++ b/07_uart_chainloader/src/relocate.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Relocation code. diff --git a/07_uart_chainloader/src/runtime_init.rs b/07_uart_chainloader/src/runtime_init.rs index 8ed768d4e..5e9b74974 100644 --- a/07_uart_chainloader/src/runtime_init.rs +++ b/07_uart_chainloader/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/07_uart_chainloader/src/synchronization.rs b/07_uart_chainloader/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/07_uart_chainloader/src/synchronization.rs +++ b/07_uart_chainloader/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/08_timestamps/Makefile b/08_timestamps/Makefile index 2f04337ed..071167b9e 100644 --- a/08_timestamps/Makefile +++ b/08_timestamps/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 9ec2e42fc..cf3173bdc 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -173,7 +173,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ @@ -0,0 +1,98 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Architectural timer primitives. + @@ -643,7 +643,7 @@ diff -uNr 07_uart_chainloader/src/relocate.rs 08_timestamps/src/relocate.rs @@ -1,51 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// --// Copyright (c) 2018-2020 Andre Richter +-// Copyright (c) 2018-2021 Andre Richter - -//! Relocation code. - @@ -720,7 +720,7 @@ diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Timer primitives. + diff --git a/08_timestamps/src/_arch/aarch64/cpu.rs b/08_timestamps/src/_arch/aarch64/cpu.rs index eac29d8d7..6072751db 100644 --- a/08_timestamps/src/_arch/aarch64/cpu.rs +++ b/08_timestamps/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/08_timestamps/src/_arch/aarch64/cpu/smp.rs b/08_timestamps/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/08_timestamps/src/_arch/aarch64/cpu/smp.rs +++ b/08_timestamps/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/08_timestamps/src/_arch/aarch64/time.rs b/08_timestamps/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/08_timestamps/src/_arch/aarch64/time.rs +++ b/08_timestamps/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/08_timestamps/src/bsp.rs b/08_timestamps/src/bsp.rs index 3a5657ade..257502491 100644 --- a/08_timestamps/src/bsp.rs +++ b/08_timestamps/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/08_timestamps/src/bsp/device_driver.rs b/08_timestamps/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/08_timestamps/src/bsp/device_driver.rs +++ b/08_timestamps/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/08_timestamps/src/bsp/device_driver/bcm.rs b/08_timestamps/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/08_timestamps/src/bsp/device_driver/bcm.rs +++ b/08_timestamps/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/08_timestamps/src/bsp/device_driver/common.rs b/08_timestamps/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/08_timestamps/src/bsp/device_driver/common.rs +++ b/08_timestamps/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/08_timestamps/src/bsp/raspberrypi.rs b/08_timestamps/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/08_timestamps/src/bsp/raspberrypi.rs +++ b/08_timestamps/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/08_timestamps/src/bsp/raspberrypi/console.rs b/08_timestamps/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/08_timestamps/src/bsp/raspberrypi/console.rs +++ b/08_timestamps/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/08_timestamps/src/bsp/raspberrypi/cpu.rs b/08_timestamps/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/08_timestamps/src/bsp/raspberrypi/cpu.rs +++ b/08_timestamps/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/08_timestamps/src/bsp/raspberrypi/driver.rs b/08_timestamps/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/08_timestamps/src/bsp/raspberrypi/driver.rs +++ b/08_timestamps/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/08_timestamps/src/bsp/raspberrypi/link.ld b/08_timestamps/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/08_timestamps/src/bsp/raspberrypi/link.ld +++ b/08_timestamps/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/08_timestamps/src/bsp/raspberrypi/memory.rs b/08_timestamps/src/bsp/raspberrypi/memory.rs index fae051257..fb47b1418 100644 --- a/08_timestamps/src/bsp/raspberrypi/memory.rs +++ b/08_timestamps/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/08_timestamps/src/console.rs b/08_timestamps/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/08_timestamps/src/console.rs +++ b/08_timestamps/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/08_timestamps/src/cpu.rs b/08_timestamps/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/08_timestamps/src/cpu.rs +++ b/08_timestamps/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/08_timestamps/src/cpu/smp.rs b/08_timestamps/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/08_timestamps/src/cpu/smp.rs +++ b/08_timestamps/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/08_timestamps/src/driver.rs b/08_timestamps/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/08_timestamps/src/driver.rs +++ b/08_timestamps/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index 2c9f1fb7f..4dad25714 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/08_timestamps/src/memory.rs b/08_timestamps/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/08_timestamps/src/memory.rs +++ b/08_timestamps/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/08_timestamps/src/panic_wait.rs b/08_timestamps/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/08_timestamps/src/panic_wait.rs +++ b/08_timestamps/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/08_timestamps/src/print.rs b/08_timestamps/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/08_timestamps/src/print.rs +++ b/08_timestamps/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/08_timestamps/src/runtime_init.rs b/08_timestamps/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/08_timestamps/src/runtime_init.rs +++ b/08_timestamps/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/08_timestamps/src/synchronization.rs b/08_timestamps/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/08_timestamps/src/synchronization.rs +++ b/08_timestamps/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/08_timestamps/src/time.rs b/08_timestamps/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/08_timestamps/src/time.rs +++ b/08_timestamps/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/09_hw_debug_JTAG/Makefile b/09_hw_debug_JTAG/Makefile index 5f49b1af5..9254018cf 100644 --- a/09_hw_debug_JTAG/Makefile +++ b/09_hw_debug_JTAG/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs index eac29d8d7..6072751db 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/09_hw_debug_JTAG/src/bsp.rs b/09_hw_debug_JTAG/src/bsp.rs index 3a5657ade..257502491 100644 --- a/09_hw_debug_JTAG/src/bsp.rs +++ b/09_hw_debug_JTAG/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver.rs b/09_hw_debug_JTAG/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/common.rs b/09_hw_debug_JTAG/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/common.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs index fae051257..fb47b1418 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/09_hw_debug_JTAG/src/console.rs b/09_hw_debug_JTAG/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/09_hw_debug_JTAG/src/console.rs +++ b/09_hw_debug_JTAG/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/09_hw_debug_JTAG/src/cpu.rs b/09_hw_debug_JTAG/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/09_hw_debug_JTAG/src/cpu.rs +++ b/09_hw_debug_JTAG/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/09_hw_debug_JTAG/src/cpu/smp.rs b/09_hw_debug_JTAG/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/09_hw_debug_JTAG/src/cpu/smp.rs +++ b/09_hw_debug_JTAG/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/09_hw_debug_JTAG/src/driver.rs b/09_hw_debug_JTAG/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/09_hw_debug_JTAG/src/driver.rs +++ b/09_hw_debug_JTAG/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index 2c9f1fb7f..4dad25714 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/09_hw_debug_JTAG/src/memory.rs b/09_hw_debug_JTAG/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/09_hw_debug_JTAG/src/memory.rs +++ b/09_hw_debug_JTAG/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/09_hw_debug_JTAG/src/panic_wait.rs b/09_hw_debug_JTAG/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/09_hw_debug_JTAG/src/panic_wait.rs +++ b/09_hw_debug_JTAG/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/09_hw_debug_JTAG/src/print.rs b/09_hw_debug_JTAG/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/09_hw_debug_JTAG/src/print.rs +++ b/09_hw_debug_JTAG/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/09_hw_debug_JTAG/src/runtime_init.rs b/09_hw_debug_JTAG/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/09_hw_debug_JTAG/src/runtime_init.rs +++ b/09_hw_debug_JTAG/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/09_hw_debug_JTAG/src/synchronization.rs b/09_hw_debug_JTAG/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/09_hw_debug_JTAG/src/synchronization.rs +++ b/09_hw_debug_JTAG/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/09_hw_debug_JTAG/src/time.rs b/09_hw_debug_JTAG/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/09_hw_debug_JTAG/src/time.rs +++ b/09_hw_debug_JTAG/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/10_privilege_level/Makefile b/10_privilege_level/Makefile index 5f49b1af5..9254018cf 100644 --- a/10_privilege_level/Makefile +++ b/10_privilege_level/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 84ccf9248..fab885983 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -306,7 +306,7 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privil @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Architectural asynchronous exception handling. + @@ -385,7 +385,7 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs 10_privilege_level/src @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Architectural synchronous and asynchronous exception handling. + @@ -413,7 +413,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception/asynchronous.rs 10_privilege_level/src/ @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Asynchronous exception handling. + @@ -428,7 +428,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Synchronous and asynchronous exception handling. + diff --git a/10_privilege_level/src/_arch/aarch64/cpu.rs b/10_privilege_level/src/_arch/aarch64/cpu.rs index e546dd606..3846968b3 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs b/10_privilege_level/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/10_privilege_level/src/_arch/aarch64/exception.rs b/10_privilege_level/src/_arch/aarch64/exception.rs index ebfef8c82..b6ee28b8c 100644 --- a/10_privilege_level/src/_arch/aarch64/exception.rs +++ b/10_privilege_level/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs b/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs index f660544c1..445f490eb 100644 --- a/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs +++ b/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/10_privilege_level/src/_arch/aarch64/time.rs b/10_privilege_level/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/10_privilege_level/src/_arch/aarch64/time.rs +++ b/10_privilege_level/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/10_privilege_level/src/bsp.rs b/10_privilege_level/src/bsp.rs index 3a5657ade..257502491 100644 --- a/10_privilege_level/src/bsp.rs +++ b/10_privilege_level/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/10_privilege_level/src/bsp/device_driver.rs b/10_privilege_level/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/10_privilege_level/src/bsp/device_driver.rs +++ b/10_privilege_level/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/10_privilege_level/src/bsp/device_driver/bcm.rs b/10_privilege_level/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/10_privilege_level/src/bsp/device_driver/common.rs b/10_privilege_level/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/10_privilege_level/src/bsp/device_driver/common.rs +++ b/10_privilege_level/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/10_privilege_level/src/bsp/raspberrypi.rs b/10_privilege_level/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/10_privilege_level/src/bsp/raspberrypi.rs +++ b/10_privilege_level/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/10_privilege_level/src/bsp/raspberrypi/console.rs b/10_privilege_level/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/10_privilege_level/src/bsp/raspberrypi/console.rs +++ b/10_privilege_level/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/10_privilege_level/src/bsp/raspberrypi/cpu.rs b/10_privilege_level/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/10_privilege_level/src/bsp/raspberrypi/cpu.rs +++ b/10_privilege_level/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/10_privilege_level/src/bsp/raspberrypi/driver.rs b/10_privilege_level/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/10_privilege_level/src/bsp/raspberrypi/driver.rs +++ b/10_privilege_level/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/10_privilege_level/src/bsp/raspberrypi/link.ld b/10_privilege_level/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/10_privilege_level/src/bsp/raspberrypi/link.ld +++ b/10_privilege_level/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/10_privilege_level/src/bsp/raspberrypi/memory.rs b/10_privilege_level/src/bsp/raspberrypi/memory.rs index fae051257..fb47b1418 100644 --- a/10_privilege_level/src/bsp/raspberrypi/memory.rs +++ b/10_privilege_level/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/10_privilege_level/src/console.rs b/10_privilege_level/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/10_privilege_level/src/console.rs +++ b/10_privilege_level/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/10_privilege_level/src/cpu.rs b/10_privilege_level/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/10_privilege_level/src/cpu.rs +++ b/10_privilege_level/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/10_privilege_level/src/cpu/smp.rs b/10_privilege_level/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/10_privilege_level/src/cpu/smp.rs +++ b/10_privilege_level/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/10_privilege_level/src/driver.rs b/10_privilege_level/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/10_privilege_level/src/driver.rs +++ b/10_privilege_level/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/10_privilege_level/src/exception.rs b/10_privilege_level/src/exception.rs index d2e621795..432c606b9 100644 --- a/10_privilege_level/src/exception.rs +++ b/10_privilege_level/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/10_privilege_level/src/exception/asynchronous.rs b/10_privilege_level/src/exception/asynchronous.rs index 3c75f90a2..fbdba9579 100644 --- a/10_privilege_level/src/exception/asynchronous.rs +++ b/10_privilege_level/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index 1cc057ac8..faa42f902 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/10_privilege_level/src/memory.rs b/10_privilege_level/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/10_privilege_level/src/memory.rs +++ b/10_privilege_level/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/10_privilege_level/src/panic_wait.rs b/10_privilege_level/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/10_privilege_level/src/panic_wait.rs +++ b/10_privilege_level/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/10_privilege_level/src/print.rs b/10_privilege_level/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/10_privilege_level/src/print.rs +++ b/10_privilege_level/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/10_privilege_level/src/runtime_init.rs b/10_privilege_level/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/10_privilege_level/src/runtime_init.rs +++ b/10_privilege_level/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/10_privilege_level/src/synchronization.rs b/10_privilege_level/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/10_privilege_level/src/synchronization.rs +++ b/10_privilege_level/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/10_privilege_level/src/time.rs b/10_privilege_level/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/10_privilege_level/src/time.rs +++ b/10_privilege_level/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/11_virtual_mem_part1_identity_mapping/Makefile b/11_virtual_mem_part1_identity_mapping/Makefile index 5f49b1af5..9254018cf 100644 --- a/11_virtual_mem_part1_identity_mapping/Makefile +++ b/11_virtual_mem_part1_identity_mapping/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 0533361b1..c8c35972e 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -306,7 +306,7 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part @@ -0,0 +1,333 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management Unit Driver. +//! @@ -665,7 +665,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management Unit. + @@ -918,7 +918,7 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map @@ -0,0 +1,199 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit. +//! diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs index e546dd606..3846968b3 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs index ebfef8c82..b6ee28b8c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs index f660544c1..445f490eb 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs index ff653a091..aeacbe8ec 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management Unit Driver. //! diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp.rs b/11_virtual_mem_part1_identity_mapping/src/bsp.rs index 5add2e382..3d7587673 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld index 5b5adbaa6..aec8e79b7 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs index 810acd8af..9ab302324 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs index 00990c7ef..6930b6755 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management Unit. diff --git a/11_virtual_mem_part1_identity_mapping/src/console.rs b/11_virtual_mem_part1_identity_mapping/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/11_virtual_mem_part1_identity_mapping/src/console.rs +++ b/11_virtual_mem_part1_identity_mapping/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/11_virtual_mem_part1_identity_mapping/src/driver.rs b/11_virtual_mem_part1_identity_mapping/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/11_virtual_mem_part1_identity_mapping/src/driver.rs +++ b/11_virtual_mem_part1_identity_mapping/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/11_virtual_mem_part1_identity_mapping/src/exception.rs b/11_virtual_mem_part1_identity_mapping/src/exception.rs index d2e621795..432c606b9 100644 --- a/11_virtual_mem_part1_identity_mapping/src/exception.rs +++ b/11_virtual_mem_part1_identity_mapping/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs b/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs index 3c75f90a2..fbdba9579 100644 --- a/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs +++ b/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 680e28194..eed8b35c6 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/11_virtual_mem_part1_identity_mapping/src/memory.rs b/11_virtual_mem_part1_identity_mapping/src/memory.rs index 59236a120..e8cc752f5 100644 --- a/11_virtual_mem_part1_identity_mapping/src/memory.rs +++ b/11_virtual_mem_part1_identity_mapping/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs index 9675b8ab6..4880ce913 100644 --- a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. //! diff --git a/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs b/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs +++ b/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/11_virtual_mem_part1_identity_mapping/src/print.rs b/11_virtual_mem_part1_identity_mapping/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/11_virtual_mem_part1_identity_mapping/src/print.rs +++ b/11_virtual_mem_part1_identity_mapping/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/11_virtual_mem_part1_identity_mapping/src/runtime_init.rs b/11_virtual_mem_part1_identity_mapping/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/11_virtual_mem_part1_identity_mapping/src/runtime_init.rs +++ b/11_virtual_mem_part1_identity_mapping/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/11_virtual_mem_part1_identity_mapping/src/synchronization.rs b/11_virtual_mem_part1_identity_mapping/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/11_virtual_mem_part1_identity_mapping/src/synchronization.rs +++ b/11_virtual_mem_part1_identity_mapping/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/11_virtual_mem_part1_identity_mapping/src/time.rs b/11_virtual_mem_part1_identity_mapping/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/11_virtual_mem_part1_identity_mapping/src/time.rs +++ b/11_virtual_mem_part1_identity_mapping/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/12_exceptions_part1_groundwork/Makefile b/12_exceptions_part1_groundwork/Makefile index 5f49b1af5..9254018cf 100644 --- a/12_exceptions_part1_groundwork/Makefile +++ b/12_exceptions_part1_groundwork/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 21c0f0eef..f70a33c54 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -745,7 +745,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S 12 @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs index e546dd606..3846968b3 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S index ee3b5497e..4d125334c 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter /// Call the function provided by parameter `\handler` after saving the exception context. Provide /// the context as the first parameter to '\handler'. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs index d932445c3..10d003542 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs index f660544c1..445f490eb 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs index ff653a091..aeacbe8ec 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management Unit Driver. //! diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/12_exceptions_part1_groundwork/src/bsp.rs b/12_exceptions_part1_groundwork/src/bsp.rs index 3a5657ade..257502491 100644 --- a/12_exceptions_part1_groundwork/src/bsp.rs +++ b/12_exceptions_part1_groundwork/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld index 90d096ffe..4dfe60c7f 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs index 810acd8af..9ab302324 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs index 5658ddb6d..1114bcce2 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management Unit. diff --git a/12_exceptions_part1_groundwork/src/console.rs b/12_exceptions_part1_groundwork/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/12_exceptions_part1_groundwork/src/console.rs +++ b/12_exceptions_part1_groundwork/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/12_exceptions_part1_groundwork/src/cpu.rs b/12_exceptions_part1_groundwork/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/12_exceptions_part1_groundwork/src/cpu.rs +++ b/12_exceptions_part1_groundwork/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/12_exceptions_part1_groundwork/src/cpu/smp.rs b/12_exceptions_part1_groundwork/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/12_exceptions_part1_groundwork/src/cpu/smp.rs +++ b/12_exceptions_part1_groundwork/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/12_exceptions_part1_groundwork/src/driver.rs b/12_exceptions_part1_groundwork/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/12_exceptions_part1_groundwork/src/driver.rs +++ b/12_exceptions_part1_groundwork/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/12_exceptions_part1_groundwork/src/exception.rs b/12_exceptions_part1_groundwork/src/exception.rs index d2e621795..432c606b9 100644 --- a/12_exceptions_part1_groundwork/src/exception.rs +++ b/12_exceptions_part1_groundwork/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/12_exceptions_part1_groundwork/src/exception/asynchronous.rs b/12_exceptions_part1_groundwork/src/exception/asynchronous.rs index 3c75f90a2..fbdba9579 100644 --- a/12_exceptions_part1_groundwork/src/exception/asynchronous.rs +++ b/12_exceptions_part1_groundwork/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 6f59108aa..66a32d085 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/12_exceptions_part1_groundwork/src/memory.rs b/12_exceptions_part1_groundwork/src/memory.rs index 59236a120..e8cc752f5 100644 --- a/12_exceptions_part1_groundwork/src/memory.rs +++ b/12_exceptions_part1_groundwork/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/12_exceptions_part1_groundwork/src/memory/mmu.rs b/12_exceptions_part1_groundwork/src/memory/mmu.rs index 25a18bf99..826cda6c9 100644 --- a/12_exceptions_part1_groundwork/src/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. //! diff --git a/12_exceptions_part1_groundwork/src/panic_wait.rs b/12_exceptions_part1_groundwork/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/12_exceptions_part1_groundwork/src/panic_wait.rs +++ b/12_exceptions_part1_groundwork/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/12_exceptions_part1_groundwork/src/print.rs b/12_exceptions_part1_groundwork/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/12_exceptions_part1_groundwork/src/print.rs +++ b/12_exceptions_part1_groundwork/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/12_exceptions_part1_groundwork/src/runtime_init.rs b/12_exceptions_part1_groundwork/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/12_exceptions_part1_groundwork/src/runtime_init.rs +++ b/12_exceptions_part1_groundwork/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/12_exceptions_part1_groundwork/src/synchronization.rs b/12_exceptions_part1_groundwork/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/12_exceptions_part1_groundwork/src/synchronization.rs +++ b/12_exceptions_part1_groundwork/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/12_exceptions_part1_groundwork/src/time.rs b/12_exceptions_part1_groundwork/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/12_exceptions_part1_groundwork/src/time.rs +++ b/12_exceptions_part1_groundwork/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/13_integrated_testing/Makefile b/13_integrated_testing/Makefile index ad567bb5c..9f7ea9a39 100644 --- a/13_integrated_testing/Makefile +++ b/13_integrated_testing/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index e9509fd7e..3ecdc0396 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1133,7 +1133,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2020 Andre Richter ++// Copyright (c) 2018-2021 Andre Richter + +// Rust embedded logo for `make doc`. +#![doc(html_logo_url = "https://git.io/JeGIp")] @@ -1644,7 +1644,7 @@ diff -uNr 12_exceptions_part1_groundwork/test-macros/src/lib.rs 13_integrated_te @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +use proc_macro::TokenStream; +use proc_macro2::Span; @@ -1680,7 +1680,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rb 13_integrate + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# -+# Copyright (c) 2019-2020 Andre Richter ++# Copyright (c) 2019-2021 Andre Richter + +require 'expect' + @@ -1733,7 +1733,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +//! Console sanity tests - RX, TX and statistics. + @@ -1774,7 +1774,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_ @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +//! Timer sanity tests. + @@ -1829,7 +1829,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +//! Page faults must result in synchronous exceptions. + @@ -1878,7 +1878,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_failure/mod.rs 13_inte @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +#[no_mangle] @@ -1892,7 +1892,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 13_inte @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +#[no_mangle] @@ -1909,7 +1909,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/runner.rb 13_integrated_testing/t + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# -+# Copyright (c) 2019-2020 Andre Richter ++# Copyright (c) 2019-2021 Andre Richter + +require 'English' +require 'pty' @@ -2064,7 +2064,7 @@ diff -uNr 12_exceptions_part1_groundwork/test-types/src/lib.rs 13_integrated_tes @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2019-2020 Andre Richter ++// Copyright (c) 2019-2021 Andre Richter + +//! Types for the `custom_test_frameworks` implementation. + diff --git a/13_integrated_testing/src/_arch/aarch64/cpu.rs b/13_integrated_testing/src/_arch/aarch64/cpu.rs index 3e3ac706f..491f17ca8 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs b/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/13_integrated_testing/src/_arch/aarch64/exception.S b/13_integrated_testing/src/_arch/aarch64/exception.S index ee3b5497e..4d125334c 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.S +++ b/13_integrated_testing/src/_arch/aarch64/exception.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter /// Call the function provided by parameter `\handler` after saving the exception context. Provide /// the context as the first parameter to '\handler'. diff --git a/13_integrated_testing/src/_arch/aarch64/exception.rs b/13_integrated_testing/src/_arch/aarch64/exception.rs index 56f3e6074..7070d8140 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs b/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs index f660544c1..445f490eb 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs index 1ccb3fd71..5061f6007 100644 --- a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management Unit Driver. //! diff --git a/13_integrated_testing/src/_arch/aarch64/time.rs b/13_integrated_testing/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/13_integrated_testing/src/_arch/aarch64/time.rs +++ b/13_integrated_testing/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/13_integrated_testing/src/bsp.rs b/13_integrated_testing/src/bsp.rs index 3a5657ade..257502491 100644 --- a/13_integrated_testing/src/bsp.rs +++ b/13_integrated_testing/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/13_integrated_testing/src/bsp/device_driver.rs b/13_integrated_testing/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/13_integrated_testing/src/bsp/device_driver.rs +++ b/13_integrated_testing/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/13_integrated_testing/src/bsp/device_driver/bcm.rs b/13_integrated_testing/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/13_integrated_testing/src/bsp/device_driver/common.rs b/13_integrated_testing/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/13_integrated_testing/src/bsp/device_driver/common.rs +++ b/13_integrated_testing/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/13_integrated_testing/src/bsp/raspberrypi.rs b/13_integrated_testing/src/bsp/raspberrypi.rs index 10888d37f..c0b639c97 100644 --- a/13_integrated_testing/src/bsp/raspberrypi.rs +++ b/13_integrated_testing/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/13_integrated_testing/src/bsp/raspberrypi/console.rs b/13_integrated_testing/src/bsp/raspberrypi/console.rs index 16b2f59bc..d6f95e4e9 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/console.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/13_integrated_testing/src/bsp/raspberrypi/driver.rs b/13_integrated_testing/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/driver.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/13_integrated_testing/src/bsp/raspberrypi/link.ld b/13_integrated_testing/src/bsp/raspberrypi/link.ld index 90d096ffe..4dfe60c7f 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/link.ld +++ b/13_integrated_testing/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory.rs b/13_integrated_testing/src/bsp/raspberrypi/memory.rs index 810acd8af..9ab302324 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs index d5078b5a0..d41de813c 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management Unit. diff --git a/13_integrated_testing/src/console.rs b/13_integrated_testing/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/13_integrated_testing/src/console.rs +++ b/13_integrated_testing/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/13_integrated_testing/src/cpu.rs b/13_integrated_testing/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/13_integrated_testing/src/cpu.rs +++ b/13_integrated_testing/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/13_integrated_testing/src/cpu/smp.rs b/13_integrated_testing/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/13_integrated_testing/src/cpu/smp.rs +++ b/13_integrated_testing/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/13_integrated_testing/src/driver.rs b/13_integrated_testing/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/13_integrated_testing/src/driver.rs +++ b/13_integrated_testing/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/13_integrated_testing/src/exception.rs b/13_integrated_testing/src/exception.rs index 81ea2b26e..dfa852a87 100644 --- a/13_integrated_testing/src/exception.rs +++ b/13_integrated_testing/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/13_integrated_testing/src/exception/asynchronous.rs b/13_integrated_testing/src/exception/asynchronous.rs index 3c75f90a2..fbdba9579 100644 --- a/13_integrated_testing/src/exception/asynchronous.rs +++ b/13_integrated_testing/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index 19e6f24fc..8e60240f2 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/13_integrated_testing/src/main.rs b/13_integrated_testing/src/main.rs index 250172057..989bd045c 100644 --- a/13_integrated_testing/src/main.rs +++ b/13_integrated_testing/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/13_integrated_testing/src/memory.rs b/13_integrated_testing/src/memory.rs index 32353d4f6..1ef0285a6 100644 --- a/13_integrated_testing/src/memory.rs +++ b/13_integrated_testing/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/13_integrated_testing/src/memory/mmu.rs b/13_integrated_testing/src/memory/mmu.rs index 39f3ef3cd..163419ec0 100644 --- a/13_integrated_testing/src/memory/mmu.rs +++ b/13_integrated_testing/src/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. //! diff --git a/13_integrated_testing/src/panic_wait.rs b/13_integrated_testing/src/panic_wait.rs index 218c0a884..76438a9bf 100644 --- a/13_integrated_testing/src/panic_wait.rs +++ b/13_integrated_testing/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/13_integrated_testing/src/print.rs b/13_integrated_testing/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/13_integrated_testing/src/print.rs +++ b/13_integrated_testing/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/13_integrated_testing/src/runtime_init.rs b/13_integrated_testing/src/runtime_init.rs index 1306578dd..0a1c685cd 100644 --- a/13_integrated_testing/src/runtime_init.rs +++ b/13_integrated_testing/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/13_integrated_testing/src/synchronization.rs b/13_integrated_testing/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/13_integrated_testing/src/synchronization.rs +++ b/13_integrated_testing/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/13_integrated_testing/src/time.rs b/13_integrated_testing/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/13_integrated_testing/src/time.rs +++ b/13_integrated_testing/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/13_integrated_testing/test-macros/src/lib.rs b/13_integrated_testing/test-macros/src/lib.rs index 092c4806e..36c95e8a6 100644 --- a/13_integrated_testing/test-macros/src/lib.rs +++ b/13_integrated_testing/test-macros/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter use proc_macro::TokenStream; use proc_macro2::Span; diff --git a/13_integrated_testing/test-types/src/lib.rs b/13_integrated_testing/test-types/src/lib.rs index 371bb5574..fe7d918f5 100644 --- a/13_integrated_testing/test-types/src/lib.rs +++ b/13_integrated_testing/test-types/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Types for the `custom_test_frameworks` implementation. diff --git a/13_integrated_testing/tests/00_console_sanity.rb b/13_integrated_testing/tests/00_console_sanity.rb index a6b549b2a..dfd6b16ea 100644 --- a/13_integrated_testing/tests/00_console_sanity.rb +++ b/13_integrated_testing/tests/00_console_sanity.rb @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'expect' diff --git a/13_integrated_testing/tests/00_console_sanity.rs b/13_integrated_testing/tests/00_console_sanity.rs index 5aa38f098..08b654aef 100644 --- a/13_integrated_testing/tests/00_console_sanity.rs +++ b/13_integrated_testing/tests/00_console_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Console sanity tests - RX, TX and statistics. diff --git a/13_integrated_testing/tests/01_timer_sanity.rs b/13_integrated_testing/tests/01_timer_sanity.rs index e0b3c1622..52dca0fca 100644 --- a/13_integrated_testing/tests/01_timer_sanity.rs +++ b/13_integrated_testing/tests/01_timer_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Timer sanity tests. diff --git a/13_integrated_testing/tests/02_exception_sync_page_fault.rs b/13_integrated_testing/tests/02_exception_sync_page_fault.rs index 64fc5486e..bf2ba29f1 100644 --- a/13_integrated_testing/tests/02_exception_sync_page_fault.rs +++ b/13_integrated_testing/tests/02_exception_sync_page_fault.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Page faults must result in synchronous exceptions. diff --git a/13_integrated_testing/tests/panic_exit_failure/mod.rs b/13_integrated_testing/tests/panic_exit_failure/mod.rs index b4ac73d1e..af2ba3781 100644 --- a/13_integrated_testing/tests/panic_exit_failure/mod.rs +++ b/13_integrated_testing/tests/panic_exit_failure/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/13_integrated_testing/tests/panic_exit_success/mod.rs b/13_integrated_testing/tests/panic_exit_success/mod.rs index 54bb072d7..29d1f9750 100644 --- a/13_integrated_testing/tests/panic_exit_success/mod.rs +++ b/13_integrated_testing/tests/panic_exit_success/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/13_integrated_testing/tests/runner.rb b/13_integrated_testing/tests/runner.rb index 2a0aa393e..53116e088 100755 --- a/13_integrated_testing/tests/runner.rb +++ b/13_integrated_testing/tests/runner.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'English' require 'pty' diff --git a/14_exceptions_part2_peripheral_IRQs/Makefile b/14_exceptions_part2_peripheral_IRQs/Makefile index 58361af8a..4f7561dbd 100644 --- a/14_exceptions_part2_peripheral_IRQs/Makefile +++ b/14_exceptions_part2_peripheral_IRQs/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index e3fa7e512..58e1bf22e 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -878,7 +878,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 14_excep @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! GICC Driver - GIC CPU interface. + @@ -1020,7 +1020,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 14_excep @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! GICD Driver - GIC Distributor. +//! @@ -1220,7 +1220,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2.rs 14_exceptions @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! GICv2 Driver - ARM Generic Interrupt Controller v2. +//! @@ -1444,7 +1444,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm.rs 14_exceptions_part2 @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! ARM driver top level. + @@ -1489,7 +1489,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Peripheral Interrupt regsler Driver. + @@ -1657,7 +1657,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Interrupt Controller Driver. + @@ -2095,7 +2095,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs 14 @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! BSP asynchronous exception handling. + @@ -2136,7 +2136,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception.rs 14_exceptions_p @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! BSP synchronous and asynchronous exception handling. + @@ -2511,7 +2511,7 @@ diff -uNr 13_integrated_testing/src/state.rs 14_exceptions_part2_peripheral_IRQs @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! State information about the kernel itself. + @@ -2725,7 +2725,7 @@ diff -uNr 13_integrated_testing/tests/03_exception_irq_sanity.rs 14_exceptions_p @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! IRQ handling sanity tests. + diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs index 3e3ac706f..491f17ca8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S index ee3b5497e..4d125334c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter /// Call the function provided by parameter `\handler` after saving the exception context. Provide /// the context as the first parameter to '\handler'. diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs index a4451cde6..99fbd85bd 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs index df6a5c3f7..968eedf34 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 1ccb3fd71..5061f6007 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management Unit Driver. //! diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp.rs index 3a5657ade..257502491 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs index 3fe1fa555..eac6bdd16 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs index 6d9d3bdf0..0bb1bd4b6 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! ARM driver top level. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs index 9981b9794..edbc41660 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICv2 Driver - ARM Generic Interrupt Controller v2. //! diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs index c6708cdcd..424bafba3 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICC Driver - GIC CPU interface. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs index 35f7eecd0..32c428089 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICD Driver - GIC Distributor. //! diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs index 5e4c9e705..01bcaa895 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 88e68be13..aaf9e1caf 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index 6b11224b6..dbe9988e8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Interrupt Controller Driver. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 94af8d746..79814d8c3 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Peripheral Interrupt regsler Driver. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 3207539c0..29bf95c13 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs index 25af58d6d..a2e2fa9a6 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs index 16b2f59bc..d6f95e4e9 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs index 0b2348fbc..58949a815 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs index 55e821198..94d0728ff 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! BSP synchronous and asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs index 787d59269..a253daa94 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! BSP asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld index 90d096ffe..4dfe60c7f 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs index 8cb07905b..44c20e2c8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs index d5078b5a0..d41de813c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management Unit. diff --git a/14_exceptions_part2_peripheral_IRQs/src/console.rs b/14_exceptions_part2_peripheral_IRQs/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/console.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/14_exceptions_part2_peripheral_IRQs/src/driver.rs b/14_exceptions_part2_peripheral_IRQs/src/driver.rs index b7d44dad5..380eace51 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/driver.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/exception.rs index 81ea2b26e..dfa852a87 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs index 1a3902a3b..c16ce0077 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index 100ed3f13..141adc3ab 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/14_exceptions_part2_peripheral_IRQs/src/main.rs b/14_exceptions_part2_peripheral_IRQs/src/main.rs index 737796134..6a21c8d41 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/main.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory.rs b/14_exceptions_part2_peripheral_IRQs/src/memory.rs index 32353d4f6..1ef0285a6 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/memory.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs index 39f3ef3cd..163419ec0 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. //! diff --git a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs index 52a89d172..0b724abd7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/14_exceptions_part2_peripheral_IRQs/src/print.rs b/14_exceptions_part2_peripheral_IRQs/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/print.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs b/14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs index 1306578dd..0a1c685cd 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/14_exceptions_part2_peripheral_IRQs/src/state.rs b/14_exceptions_part2_peripheral_IRQs/src/state.rs index af1d93482..d08e67d6e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/state.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/state.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! State information about the kernel itself. diff --git a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs index 039df47a4..6133826eb 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/14_exceptions_part2_peripheral_IRQs/src/time.rs b/14_exceptions_part2_peripheral_IRQs/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/time.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs index 092c4806e..36c95e8a6 100644 --- a/14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter use proc_macro::TokenStream; use proc_macro2::Span; diff --git a/14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs index 371bb5574..fe7d918f5 100644 --- a/14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Types for the `custom_test_frameworks` implementation. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb index a6b549b2a..dfd6b16ea 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb +++ b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'expect' diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs index 5aa38f098..08b654aef 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Console sanity tests - RX, TX and statistics. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs index e0b3c1622..52dca0fca 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Timer sanity tests. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs index 64fc5486e..bf2ba29f1 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Page faults must result in synchronous exceptions. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs index 6e2ae8f7c..1f3bb7702 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! IRQ handling sanity tests. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs b/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs index b4ac73d1e..af2ba3781 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs b/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs index 54bb072d7..29d1f9750 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/14_exceptions_part2_peripheral_IRQs/tests/runner.rb b/14_exceptions_part2_peripheral_IRQs/tests/runner.rb index 2a0aa393e..53116e088 100755 --- a/14_exceptions_part2_peripheral_IRQs/tests/runner.rb +++ b/14_exceptions_part2_peripheral_IRQs/tests/runner.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'English' require 'pty' diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 58361af8a..4f7561dbd 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 312bb5f19..f7a9b1921 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1140,7 +1140,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -2,12 +2,14 @@ // - // Copyright (c) 2020 Andre Richter + // Copyright (c) 2020-2021 Andre Richter -//! Peripheral Interrupt regsler Driver. +//! Peripheral Interrupt Controller Driver. @@ -1695,7 +1695,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -3,9 +3,41 @@ - // Copyright (c) 2018-2020 Andre Richter + // Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. +//! @@ -1943,7 +1943,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/common.rs 15_virtual_mem_part2 @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! General purpose code. + @@ -2089,7 +2089,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! A record of mapped pages. + @@ -2315,7 +2315,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual @@ -0,0 +1,283 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2020 Andre Richter ++// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit Types. + @@ -2601,7 +2601,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p --- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -3,23 +3,18 @@ - // Copyright (c) 2020 Andre Richter + // Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. -//! diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs index d8bcc895c..6ecea1825 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S index ee3b5497e..4d125334c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter /// Call the function provided by parameter `\handler` after saving the exception context. Provide /// the context as the first parameter to '\handler'. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs index a4451cde6..99fbd85bd 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs index df6a5c3f7..968eedf34 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs index cf0989bef..0e45185e3 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management Unit Driver. //! diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs index e9b0234db..a2c57eb09 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp.rs b/15_virtual_mem_part2_mmio_remap/src/bsp.rs index 3a5657ade..257502491 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs index 3fe1fa555..eac6bdd16 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs index 6d9d3bdf0..0bb1bd4b6 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! ARM driver top level. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs index 9b23af353..e7a37697e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICv2 Driver - ARM Generic Interrupt Controller v2. //! diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs index ee1b64087..a1279f956 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICC Driver - GIC CPU interface. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs index f221e7246..5f56e54a1 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! GICD Driver - GIC Distributor. //! diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs index 5e4c9e705..01bcaa895 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 680ad4cb9..c1fa1d14c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index d6ecf677a..dfc5b66ed 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Interrupt Controller Driver. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 35cf7c1db..8fe915b34 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Peripheral Interrupt Controller Driver. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 374331741..c2354d430 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs index 7ffb72499..10d482781 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs index c5d0e05a5..f63a9e9bb 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs index 313b957e0..4d5007e4e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs index 55e821198..94d0728ff 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! BSP synchronous and asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs index 787d59269..a253daa94 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! BSP asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld index ae5c28196..e5975256f 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs index 5c05e2080..9d359cb7c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. //! diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs index 41f55941d..fc16c87c2 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management Unit. diff --git a/15_virtual_mem_part2_mmio_remap/src/common.rs b/15_virtual_mem_part2_mmio_remap/src/common.rs index 58209804b..71443cfbd 100644 --- a/15_virtual_mem_part2_mmio_remap/src/common.rs +++ b/15_virtual_mem_part2_mmio_remap/src/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! General purpose code. diff --git a/15_virtual_mem_part2_mmio_remap/src/console.rs b/15_virtual_mem_part2_mmio_remap/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/15_virtual_mem_part2_mmio_remap/src/console.rs +++ b/15_virtual_mem_part2_mmio_remap/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs b/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/15_virtual_mem_part2_mmio_remap/src/driver.rs b/15_virtual_mem_part2_mmio_remap/src/driver.rs index b6875b7f7..83fac4a7c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/driver.rs +++ b/15_virtual_mem_part2_mmio_remap/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/15_virtual_mem_part2_mmio_remap/src/exception.rs b/15_virtual_mem_part2_mmio_remap/src/exception.rs index 81ea2b26e..dfa852a87 100644 --- a/15_virtual_mem_part2_mmio_remap/src/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/exception.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronous and asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs index 1a3902a3b..c16ce0077 100644 --- a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Asynchronous exception handling. diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index 50789867e..9ff2a50c7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/15_virtual_mem_part2_mmio_remap/src/main.rs b/15_virtual_mem_part2_mmio_remap/src/main.rs index d693bee68..11bb6902d 100644 --- a/15_virtual_mem_part2_mmio_remap/src/main.rs +++ b/15_virtual_mem_part2_mmio_remap/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/15_virtual_mem_part2_mmio_remap/src/memory.rs b/15_virtual_mem_part2_mmio_remap/src/memory.rs index 32353d4f6..1ef0285a6 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index 778fd646f..9204874a9 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs index f46e629ed..cd3378089 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! A record of mapped pages. diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs index d0098f6f2..cea7bb39c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit Types. diff --git a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs index 52a89d172..0b724abd7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs +++ b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/15_virtual_mem_part2_mmio_remap/src/print.rs b/15_virtual_mem_part2_mmio_remap/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/15_virtual_mem_part2_mmio_remap/src/print.rs +++ b/15_virtual_mem_part2_mmio_remap/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/15_virtual_mem_part2_mmio_remap/src/runtime_init.rs b/15_virtual_mem_part2_mmio_remap/src/runtime_init.rs index 1306578dd..0a1c685cd 100644 --- a/15_virtual_mem_part2_mmio_remap/src/runtime_init.rs +++ b/15_virtual_mem_part2_mmio_remap/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/15_virtual_mem_part2_mmio_remap/src/state.rs b/15_virtual_mem_part2_mmio_remap/src/state.rs index af1d93482..d08e67d6e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/state.rs +++ b/15_virtual_mem_part2_mmio_remap/src/state.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! State information about the kernel itself. diff --git a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs index 039df47a4..6133826eb 100644 --- a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs +++ b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/15_virtual_mem_part2_mmio_remap/src/time.rs b/15_virtual_mem_part2_mmio_remap/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/15_virtual_mem_part2_mmio_remap/src/time.rs +++ b/15_virtual_mem_part2_mmio_remap/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs b/15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs index 092c4806e..36c95e8a6 100644 --- a/15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter use proc_macro::TokenStream; use proc_macro2::Span; diff --git a/15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs b/15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs index 371bb5574..fe7d918f5 100644 --- a/15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Types for the `custom_test_frameworks` implementation. diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb index a6b549b2a..dfd6b16ea 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb +++ b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'expect' diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs index 5aa38f098..08b654aef 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Console sanity tests - RX, TX and statistics. diff --git a/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs index e0b3c1622..52dca0fca 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Timer sanity tests. diff --git a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs index 0e219a8fb..858357a4d 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter //! Page faults must result in synchronous exceptions. diff --git a/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs index 6e2ae8f7c..1f3bb7702 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! IRQ handling sanity tests. diff --git a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs b/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs index b4ac73d1e..af2ba3781 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs b/15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs index 54bb072d7..29d1f9750 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2019-2020 Andre Richter +// Copyright (c) 2019-2021 Andre Richter /// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. #[no_mangle] diff --git a/15_virtual_mem_part2_mmio_remap/tests/runner.rb b/15_virtual_mem_part2_mmio_remap/tests/runner.rb index 2a0aa393e..53116e088 100755 --- a/15_virtual_mem_part2_mmio_remap/tests/runner.rb +++ b/15_virtual_mem_part2_mmio_remap/tests/runner.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2019-2020 Andre Richter +# Copyright (c) 2019-2021 Andre Richter require 'English' require 'pty' diff --git a/X1_JTAG_boot/Makefile b/X1_JTAG_boot/Makefile index 2f04337ed..071167b9e 100644 --- a/X1_JTAG_boot/Makefile +++ b/X1_JTAG_boot/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2018-2020 Andre Richter +## Copyright (c) 2018-2021 Andre Richter # Default to the RPi3 BSP ?= rpi3 diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu.rs index eac29d8d7..6072751db 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs index 8429e1d2b..c80f7e780 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. diff --git a/X1_JTAG_boot/src/_arch/aarch64/time.rs b/X1_JTAG_boot/src/_arch/aarch64/time.rs index af98ddd01..7f1bc6963 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/time.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. diff --git a/X1_JTAG_boot/src/bsp.rs b/X1_JTAG_boot/src/bsp.rs index 3a5657ade..257502491 100644 --- a/X1_JTAG_boot/src/bsp.rs +++ b/X1_JTAG_boot/src/bsp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Conditional re-exporting of Board Support Packages. diff --git a/X1_JTAG_boot/src/bsp/device_driver.rs b/X1_JTAG_boot/src/bsp/device_driver.rs index ce7396f22..2fcbc83e7 100644 --- a/X1_JTAG_boot/src/bsp/device_driver.rs +++ b/X1_JTAG_boot/src/bsp/device_driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Device driver. diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm.rs index 59071d5d8..77613d261 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BCM driver top level. diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 15bd3b78b..41d8b861c 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! GPIO Driver. diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e39bc7cdb..66b734638 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. diff --git a/X1_JTAG_boot/src/bsp/device_driver/common.rs b/X1_JTAG_boot/src/bsp/device_driver/common.rs index cd2c07601..41553b7a5 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/common.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/common.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Common device driver code. diff --git a/X1_JTAG_boot/src/bsp/raspberrypi.rs b/X1_JTAG_boot/src/bsp/raspberrypi.rs index a4a812327..ecc6fd108 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Top-level BSP file for the Raspberry Pi 3 and 4. diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/console.rs b/X1_JTAG_boot/src/bsp/raspberrypi/console.rs index d1b60dd82..c6bb5263c 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/console.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP console facilities. diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs index 8410a9615..3951618d7 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Processor code. diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs b/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs index fe24dd71b..ab910185c 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP driver support. diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld index c03bb5e54..573abc5f0 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld +++ b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT OR Apache-2.0 * - * Copyright (c) 2018-2020 Andre Richter + * Copyright (c) 2018-2021 Andre Richter */ SECTIONS diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs index fae051257..fb47b1418 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. diff --git a/X1_JTAG_boot/src/console.rs b/X1_JTAG_boot/src/console.rs index e6323a20d..2d38cc1d1 100644 --- a/X1_JTAG_boot/src/console.rs +++ b/X1_JTAG_boot/src/console.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! System console. diff --git a/X1_JTAG_boot/src/cpu.rs b/X1_JTAG_boot/src/cpu.rs index 9c67c0e7d..c9e5af72c 100644 --- a/X1_JTAG_boot/src/cpu.rs +++ b/X1_JTAG_boot/src/cpu.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Processor code. diff --git a/X1_JTAG_boot/src/cpu/smp.rs b/X1_JTAG_boot/src/cpu/smp.rs index b1428884d..90ecbdf30 100644 --- a/X1_JTAG_boot/src/cpu/smp.rs +++ b/X1_JTAG_boot/src/cpu/smp.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Symmetric multiprocessing. diff --git a/X1_JTAG_boot/src/driver.rs b/X1_JTAG_boot/src/driver.rs index e2875b876..6e8c5533a 100644 --- a/X1_JTAG_boot/src/driver.rs +++ b/X1_JTAG_boot/src/driver.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Driver support. diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index 85db786de..5e54c329c 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] diff --git a/X1_JTAG_boot/src/memory.rs b/X1_JTAG_boot/src/memory.rs index 827b8b850..1f79e0c95 100644 --- a/X1_JTAG_boot/src/memory.rs +++ b/X1_JTAG_boot/src/memory.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Memory Management. diff --git a/X1_JTAG_boot/src/panic_wait.rs b/X1_JTAG_boot/src/panic_wait.rs index 1386e1e23..7980f5de5 100644 --- a/X1_JTAG_boot/src/panic_wait.rs +++ b/X1_JTAG_boot/src/panic_wait.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! A panic handler that infinitely waits. diff --git a/X1_JTAG_boot/src/print.rs b/X1_JTAG_boot/src/print.rs index 8b6f3f988..1ea96b6ad 100644 --- a/X1_JTAG_boot/src/print.rs +++ b/X1_JTAG_boot/src/print.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Printing facilities. diff --git a/X1_JTAG_boot/src/runtime_init.rs b/X1_JTAG_boot/src/runtime_init.rs index 9de285958..ee0946863 100644 --- a/X1_JTAG_boot/src/runtime_init.rs +++ b/X1_JTAG_boot/src/runtime_init.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2018-2020 Andre Richter +// Copyright (c) 2018-2021 Andre Richter //! Rust runtime initialization code. diff --git a/X1_JTAG_boot/src/synchronization.rs b/X1_JTAG_boot/src/synchronization.rs index 6de32807c..f956b5cdd 100644 --- a/X1_JTAG_boot/src/synchronization.rs +++ b/X1_JTAG_boot/src/synchronization.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Synchronization primitives. //! diff --git a/X1_JTAG_boot/src/time.rs b/X1_JTAG_boot/src/time.rs index cd3ceec33..4f2f4e38f 100644 --- a/X1_JTAG_boot/src/time.rs +++ b/X1_JTAG_boot/src/time.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT OR Apache-2.0 // -// Copyright (c) 2020 Andre Richter +// Copyright (c) 2020-2021 Andre Richter //! Timer primitives. diff --git a/docker/rustembedded-osdev-utils/Dockerfile b/docker/rustembedded-osdev-utils/Dockerfile index 0ee3d9886..7127bd14b 100644 --- a/docker/rustembedded-osdev-utils/Dockerfile +++ b/docker/rustembedded-osdev-utils/Dockerfile @@ -1,7 +1,7 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2017-2020 Andre Richter -## Copyright (c) 2019-2020 Nao Taco +## Copyright (c) 2017-2021 Andre Richter +## Copyright (c) 2019-2021 Nao Taco FROM ubuntu:20.04 ARG VCS_REF diff --git a/docker/rustembedded-osdev-utils/Makefile b/docker/rustembedded-osdev-utils/Makefile index 4b9f70f5e..e40b5ac4f 100644 --- a/docker/rustembedded-osdev-utils/Makefile +++ b/docker/rustembedded-osdev-utils/Makefile @@ -1,6 +1,6 @@ ## SPDX-License-Identifier: MIT OR Apache-2.0 ## -## Copyright (c) 2019-2020 Andre Richter +## Copyright (c) 2019-2021 Andre Richter default: docker_build diff --git a/utils/devtool.rb b/utils/devtool.rb index 823cdcda7..47e3404af 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2020 Andre Richter +# Copyright (c) 2020-2021 Andre Richter require 'rubygems' require 'bundler/setup' diff --git a/utils/devtool/copyright.rb b/utils/devtool/copyright.rb index 5da012562..956efce10 100644 --- a/utils/devtool/copyright.rb +++ b/utils/devtool/copyright.rb @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2018-2020 Andre Richter +# Copyright (c) 2018-2021 Andre Richter require 'colorize' diff --git a/utils/diff_tut_folders.bash b/utils/diff_tut_folders.bash index 6952f8e8b..65a0e7fa6 100755 --- a/utils/diff_tut_folders.bash +++ b/utils/diff_tut_folders.bash @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2018-2020 Andre Richter +# Copyright (c) 2018-2021 Andre Richter DIFF=$( diff -uNr \ @@ -13,7 +13,7 @@ DIFF=$( -x Cargo.lock \ -x target \ $1 $2 \ - | sed -r "s/[12][90][127][90]-[0-9][0-9]-[0-9][0-9] .*//g" \ + | sed -r "s/[12][90][127][0-9]-[0-9][0-9]-[0-9][0-9] .*//g" \ | sed -r "s/[[:space:]]*$//g" \ | sed -r "s/%/modulo/g" \ | sed -r "s/diff -uNr -x README.md -x README.CN.md -x kernel -x kernel8.img -x Cargo.lock -x target/\ndiff -uNr/g" diff --git a/utils/minipush.rb b/utils/minipush.rb index 710013437..e06433a8a 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2020 Andre Richter +# Copyright (c) 2020-2021 Andre Richter require_relative 'miniterm' require 'ruby-progressbar' diff --git a/utils/minipush/progressbar_patch.rb b/utils/minipush/progressbar_patch.rb index 734540646..2be5ad5a3 100644 --- a/utils/minipush/progressbar_patch.rb +++ b/utils/minipush/progressbar_patch.rb @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2020 Andre Richter +# Copyright (c) 2020-2021 Andre Richter # Monkey-patch ruby-progressbar so that it supports reporting the progress in KiB instead of Byte. diff --git a/utils/miniterm.rb b/utils/miniterm.rb index 6b2c2f633..0875ada55 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Copyright (c) 2020 Andre Richter +# Copyright (c) 2020-2021 Andre Richter require 'rubygems' require 'bundler/setup' diff --git a/utils/update_copyright.rb b/utils/update_copyright.rb new file mode 100755 index 000000000..2d56fd816 --- /dev/null +++ b/utils/update_copyright.rb @@ -0,0 +1,20 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +require 'date' + +files = `git ls-files`.split("\n") +files = files.delete_if { |f| File.symlink?(f) } +files = files.join(' ') + +year = Date.today.year + +# Update "Copyright (c) 20..-20.." +`sed -i -- 's,\\(Copyright .c. 20..\\)-20..,\\1-#{year},g' #{files}` + +# Update "Copyright (c) 20.. Name" -> "Copyright (c) 20..-20.. Name" +`sed -i -- 's,\\(Copyright .c. 20..\\) ,\\1-#{year} ,g' #{files}` From 9b89f297d70f6df1fe7656947f6c8da3c56ecc4a Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 1 Jan 2021 12:04:01 +0100 Subject: [PATCH 013/214] Update dependencies --- 04_zero_overhead_abstraction/Cargo.lock | 8 ++++---- 05_safe_globals/Cargo.lock | 8 ++++---- 06_drivers_gpio_uart/Cargo.lock | 8 ++++---- 07_uart_chainloader/Cargo.lock | 8 ++++---- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6712 -> 6712 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6592 -> 6592 bytes 08_timestamps/Cargo.lock | 8 ++++---- 09_hw_debug_JTAG/Cargo.lock | 8 ++++---- 10_privilege_level/Cargo.lock | 8 ++++---- .../Cargo.lock | 8 ++++---- 12_exceptions_part1_groundwork/Cargo.lock | 8 ++++---- 13_integrated_testing/Cargo.lock | 12 ++++++------ .../Cargo.lock | 12 ++++++------ 15_virtual_mem_part2_mmio_remap/Cargo.lock | 12 ++++++------ 14 files changed, 54 insertions(+), 54 deletions(-) diff --git a/04_zero_overhead_abstraction/Cargo.lock b/04_zero_overhead_abstraction/Cargo.lock index 235957ef8..dd0cd5773 100644 --- a/04_zero_overhead_abstraction/Cargo.lock +++ b/04_zero_overhead_abstraction/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.0" +version = "5.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" +checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" dependencies = [ "register", ] @@ -18,9 +18,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.1" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" dependencies = [ "tock-registers", ] diff --git a/05_safe_globals/Cargo.lock b/05_safe_globals/Cargo.lock index 235957ef8..dd0cd5773 100644 --- a/05_safe_globals/Cargo.lock +++ b/05_safe_globals/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.0" +version = "5.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" +checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" dependencies = [ "register", ] @@ -18,9 +18,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.1" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" dependencies = [ "tock-registers", ] diff --git a/06_drivers_gpio_uart/Cargo.lock b/06_drivers_gpio_uart/Cargo.lock index 115e07626..e2d680a76 100644 --- a/06_drivers_gpio_uart/Cargo.lock +++ b/06_drivers_gpio_uart/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.0" +version = "5.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" +checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.1" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" dependencies = [ "tock-registers", ] diff --git a/07_uart_chainloader/Cargo.lock b/07_uart_chainloader/Cargo.lock index 115e07626..e2d680a76 100644 --- a/07_uart_chainloader/Cargo.lock +++ b/07_uart_chainloader/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.0" +version = "5.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc37a4862a4f5b40df96f5e0f3bd4ce6a1bd8bb59a965cd476844673faf83896" +checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" dependencies = [ "register", ] @@ -19,9 +19,9 @@ dependencies = [ [[package]] name = "register" -version = "1.0.1" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "faf386c0c48fea132b7fbbf71c32d1b1d62e7c59043bbabdd15c9ed9f254517e" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" dependencies = [ "tock-registers", ] diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index cb83285f1b2ca41d0175a18604b3e5cca9882987..7b314d3c9d3ff8f3360db8b5c7c4e5f4b0989ed9 100755 GIT binary patch delta 115 zcmdmCvcqJ2tLa+5bm MRDcv{N%AoQ00LwqssI20 delta 120 zcmdmCvcqJE|+VCGo$kePE;-Q-YFW2PNKo4Z7}F#+kxP7-P$Dhons RNy<&$AW;EQpe4!22mrnyC++|M diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index 9abdf41e85b09f016e2b760668f110cab1de31e2..a7c0fdc068b7a1db1a3e23a3c4c940d9805fde3b 100755 GIT binary patch delta 359 zcmX?Le86~u3d;fJ_-h+gSeO{sZPsMk%*uFW^IdiWR>sGh`FNKz3Mm>* z;gvJP#B4^J$;bIp8FMG=@~1Oin>>}jn{nl2MS=T#9~l^?JZNWH`ThU@58s70^9gQa zoP0vag;94huduYB*#H026&Mt*{6EZi@t-ln&##j$g_kqxO@1gmpOI~Hqlmg}<3EUs zN6d_?I9ME3K49ipwd~)2aRvs5pHn6u74dddVc=i@0SJkp1DGHp8c-PtDBsE1S0TX1 zz|c@3)G;W8f#JpEKv8kVH=8p>+qCdT!fb(uD^GG5*Mkllcl@$zON-sOx!f<_a$85kz;FfeR*<;*ZK z`=j;b(|oCnd6NzK(;2T%p2^?MxN5Shz Date: Fri, 1 Jan 2021 21:03:18 +0100 Subject: [PATCH 014/214] Memory Mapping: Improve various aspects --- .../README.md | 49 ++++++++--- .../src/_arch/aarch64/memory/mmu.rs | 16 +++- .../src/bsp/raspberrypi/memory.rs | 13 +++ .../src/bsp/raspberrypi/memory/mmu.rs | 7 +- 12_exceptions_part1_groundwork/README.md | 80 ++++++++--------- .../src/_arch/aarch64/memory/mmu.rs | 16 +++- .../src/bsp/raspberrypi/memory.rs | 13 +++ .../src/bsp/raspberrypi/memory/mmu.rs | 7 +- 12_exceptions_part1_groundwork/src/main.rs | 4 +- 13_integrated_testing/README.md | 8 +- .../src/_arch/aarch64/memory/mmu.rs | 16 +++- .../src/bsp/raspberrypi/memory.rs | 13 +++ .../src/bsp/raspberrypi/memory/mmu.rs | 7 +- 14_exceptions_part2_peripheral_IRQs/README.md | 4 +- .../src/_arch/aarch64/memory/mmu.rs | 16 +++- .../src/bsp/raspberrypi/memory.rs | 13 +++ .../src/bsp/raspberrypi/memory/mmu.rs | 7 +- 15_virtual_mem_part2_mmio_remap/README.md | 86 ++++++++++++------- .../src/_arch/aarch64/memory/mmu.rs | 12 ++- 19 files changed, 278 insertions(+), 109 deletions(-) diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index c8c35972e..df84c9669 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -140,7 +140,6 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; type ArchTranslationTable = FixedSizeTranslationTable; @@ -205,7 +204,7 @@ virtual addresses: - Since we identity map the whole `Device MMIO` region, it is accessible by asserting its physical base address (`0x3F20_1000` or `0xFA20_1000` depending on which RPi you use) after the `MMU` is turned on. -- Additionally, it is also mapped into the last `64 KiB` entry of the `lvl3` table, making it +- Additionally, it is also mapped into the last `64 KiB` slot in the first `512 MiB`, making it accessible through base address `0x1FFF_1000`. The following block diagram visualizes the underlying translation for the second mapping. @@ -303,7 +302,7 @@ Minipush 1.0 diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs --- 10_privilege_level/src/_arch/aarch64/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs -@@ -0,0 +1,333 @@ +@@ -0,0 +1,343 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -343,6 +342,12 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ ++ /// Unprivileged execute-never. ++ UXN OFFSET(54) NUMBITS(1) [ ++ False = 0, ++ True = 1 ++ ], ++ + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, @@ -416,7 +421,6 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + lvl2: [TableDescriptor; NUM_TABLES], +} + -+/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; +type ArchTranslationTable = FixedSizeTranslationTable; + @@ -500,13 +504,16 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + -+ // Execute Never. ++ // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + ++ // Always set unprivileged exectue-never as long as userspace is not implemented yet. ++ desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; ++ + desc + } +} @@ -579,16 +586,18 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part +/// Configure various settings of stage 1 of the EL1 translation regime. +fn configure_translation_control() { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); ++ let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) ++ + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks -+ + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. ++ + TCR_EL1::T0SZ.val(t0sz), + ); +} + @@ -662,7 +671,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_id diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs --- 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs -@@ -0,0 +1,88 @@ +@@ -0,0 +1,93 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -743,8 +752,13 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa +//-------------------------------------------------------------------------------------------------- + +/// Return the address space size in bytes. ++/// ++/// Guarantees size to be a power of two. +pub const fn addr_space_size() -> usize { -+ memory_map::END_INCLUSIVE + 1 ++ let size = memory_map::END_INCLUSIVE + 1; ++ assert!(size.is_power_of_two()); ++ ++ size +} + +/// Return a reference to the virtual memory layout. @@ -773,16 +787,29 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } //-------------------------------------------------------------------------------------------------- -@@ -23,6 +27,8 @@ +@@ -23,6 +27,21 @@ /// The board's memory map. #[rustfmt::skip] pub(super) mod map { ++ /// The inclusive end address of the memory map. ++ /// ++ /// End address + 1 must be power of two. ++ /// ++ /// # Note ++ /// ++ /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for ++ /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. ++ /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. ++ /// ++ /// However, making this trade-off has the downside of making it possible for the CPU to assert a ++ /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on ++ /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. + pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; + pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; -@@ -36,6 +42,7 @@ +@@ -36,6 +55,7 @@ pub const START: usize = 0x3F00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -790,7 +817,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } /// Physical devices. -@@ -46,10 +53,35 @@ +@@ -46,10 +66,35 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs index aeacbe8ec..69023cef1 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -37,6 +37,12 @@ register_bitfields! {u64, // A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + /// Privileged execute-never. PXN OFFSET(53) NUMBITS(1) [ False = 0, @@ -110,7 +116,6 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; type ArchTranslationTable = FixedSizeTranslationTable; @@ -194,13 +199,16 @@ impl convert::From AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, }; - // Execute Never. + // The execute-never attribute is mapped to PXN in AArch64. desc += if attribute_fields.execute_never { STAGE1_PAGE_DESCRIPTOR::PXN::True } else { STAGE1_PAGE_DESCRIPTOR::PXN::False }; + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + desc } } @@ -273,16 +281,18 @@ unsafe fn populate_tt_entries() -> Result<(), &'static str> { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control() { let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); TCR_EL1.write( TCR_EL1::TBI0::Ignored + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. + + TCR_EL1::T0SZ.val(t0sz), ); } diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs index 9ab302324..d4bcf0a3b 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -27,6 +27,19 @@ extern "Rust" { /// The board's memory map. #[rustfmt::skip] pub(super) mod map { + /// The inclusive end address of the memory map. + /// + /// End address + 1 must be power of two. + /// + /// # Note + /// + /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for + /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. + /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. + /// + /// However, making this trade-off has the downside of making it possible for the CPU to assert a + /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on + /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; pub const BOOT_CORE_STACK_END: usize = 0x8_0000; diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs index 6930b6755..982fc0655 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -78,8 +78,13 @@ fn mmio_range_inclusive() -> RangeInclusive { //-------------------------------------------------------------------------------------------------- /// Return the address space size in bytes. +/// +/// Guarantees size to be a power of two. pub const fn addr_space_size() -> usize { - memory_map::END_INCLUSIVE + 1 + let size = memory_map::END_INCLUSIVE + 1; + assert!(size.is_power_of_two()); + + size } /// Return a reference to the virtual memory layout. diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index f70a33c54..979636866 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -350,14 +350,14 @@ we cause a data abort exception by reading from memory address `8 GiB`: // For demo purposes, the exception handler will catch the faulting 8 GiB address and allow // execution to continue. info!(""); -info!("Trying to write to address 8 GiB..."); +info!("Trying to read from address 8 GiB..."); let mut big_addr: u64 = 8 * 1024 * 1024 * 1024; unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; ``` This triggers our exception code, because we try to read from a virtual address for which no mapping -has been installed. Remember, we only installed identity-mapped page tables for the first `1 GiB` -(RPi3) or `4 GiB` (RPi4) of address space in the previous tutorial. +has been installed. Remember, we only installed up to `4 GiB` of address space in the previous +tutorial. To survive this exception, the respective handler has a special demo case: @@ -412,29 +412,29 @@ Minipush 1.0 [MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 3.090618] Booting on: Raspberry Pi 3 -[ 3.091701] MMU online. Special regions: -[ 3.093610] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.097688] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.101246] Current privilege level: EL1 -[ 3.103155] Exception handling state: -[ 3.104934] Debug: Masked -[ 3.106496] SError: Masked -[ 3.108058] IRQ: Masked -[ 3.109619] FIQ: Masked -[ 3.111181] Architectural timer resolution: 52 ns -[ 3.113481] Drivers loaded: -[ 3.114826] 1. BCM GPIO -[ 3.116257] 2. BCM PL011 UART -[ 3.117950] Timer test, spinning for 1 second -[ 4.120076] -[ 4.120079] Trying to write to address 8 GiB... -[ 4.122242] ************************************************ -[ 4.125018] Whoa! We recovered from a synchronous exception! -[ 4.127795] ************************************************ -[ 4.130571] -[ 4.131266] Let's try again -[ 4.132611] Trying to write to address 9 GiB... +[ 3.091032] Booting on: Raspberry Pi 3 +[ 3.092116] MMU online. Special regions: +[ 3.094025] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.098103] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 3.101661] Current privilege level: EL1 +[ 3.103570] Exception handling state: +[ 3.105348] Debug: Masked +[ 3.106910] SError: Masked +[ 3.108472] IRQ: Masked +[ 3.110034] FIQ: Masked +[ 3.111596] Architectural timer resolution: 52 ns +[ 3.113895] Drivers loaded: +[ 3.115240] 1. BCM GPIO +[ 3.116672] 2. BCM PL011 UART +[ 3.118364] Timer test, spinning for 1 second +[ 4.120490] +[ 4.120494] Trying to read from address 8 GiB... +[ 4.122700] ************************************************ +[ 4.125476] Whoa! We recovered from a synchronous exception! +[ 4.128253] ************************************************ +[ 4.131030] +[ 4.131724] Let's try again +[ 4.133069] Trying to read from address 9 GiB... Kernel panic: @@ -443,7 +443,7 @@ FAR_EL1: 0x0000000240000000 ESR_EL1: 0x96000004 Exception Class (EC) : 0x25 - Data Abort, current EL Instr Specific Syndrome (ISS): 0x4 -ELR_EL1: 0x0000000000081454 +ELR_EL1: 0x0000000000081458 SPSR_EL1: 0x600003c5 Flags: Negative (N): Not set @@ -458,22 +458,22 @@ SPSR_EL1: 0x600003c5 Illegal Execution State (IL): Not set General purpose register: - x0 : 0x0000000000000000 x1 : 0x0000000000085726 - x2 : 0x0000000000000026 x3 : 0x0000000000083bf0 - x4 : 0x0000000000000003 x5 : 0xfb4f101900000000 - x6 : 0x0000000000000000 x7 : 0x7e9198052b2b0200 + x0 : 0x0000000000000000 x1 : 0x0000000000085727 + x2 : 0x0000000000000027 x3 : 0x0000000000000000 + x4 : 0x0000000000000002 x5 : 0x3f27329c00000000 + x6 : 0x0000000000000000 x7 : 0xdbd1b90800000000 x8 : 0x0000000240000000 x9 : 0x000000003f201000 - x10: 0x0000000000000019 x11: 0x0000000000000000 - x12: 0x0000000000000006 x13: 0x0000000000000031 + x10: 0x0000000000000019 x11: 0x00000000000819d0 + x12: 0x0000000000000000 x13: 0x0000000000000033 x14: 0x000000000007fc2d x15: 0x0000000000000000 - x16: 0x0000000000000040 x17: 0xb557f006f276cfb6 + x16: 0x0000000000000040 x17: 0xfd7f702255f847d0 x18: 0x0000000000000003 x19: 0x0000000000090008 x20: 0x0000000000085510 x21: 0x000000003b9aca00 - x22: 0x00000000000003e8 x23: 0x000000000008160c - x24: 0x0000000000082264 x25: 0x00000000000f4240 + x22: 0x00000000000003e8 x23: 0x0000000000081610 + x24: 0x0000000000082268 x25: 0x00000000000f4240 x26: 0xffffffffc4653600 x27: 0x00000000000855f0 - x28: 0x0000000000083f84 x29: 0x0000000000086810 - lr : 0x0000000000081448 + x28: 0x0000000000083f80 x29: 0x0000000000086810 + lr : 0x000000000008144c ``` ## Diff to previous @@ -989,7 +989,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ + // For demo purposes, the exception handler will catch the faulting 8 GiB address and allow + // execution to continue. + info!(""); -+ info!("Trying to write to address 8 GiB..."); ++ info!("Trying to read from address 8 GiB..."); + let mut big_addr: u64 = 8 * 1024 * 1024 * 1024; + unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; + @@ -1000,7 +1000,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ + info!("Let's try again"); + + // Now use address 9 GiB. The exception handler won't forgive us this time. -+ info!("Trying to write to address 9 GiB..."); ++ info!("Trying to read from address 9 GiB..."); + big_addr = 9 * 1024 * 1024 * 1024; + unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs index aeacbe8ec..69023cef1 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs @@ -37,6 +37,12 @@ register_bitfields! {u64, // A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + /// Privileged execute-never. PXN OFFSET(53) NUMBITS(1) [ False = 0, @@ -110,7 +116,6 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; type ArchTranslationTable = FixedSizeTranslationTable; @@ -194,13 +199,16 @@ impl convert::From AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, }; - // Execute Never. + // The execute-never attribute is mapped to PXN in AArch64. desc += if attribute_fields.execute_never { STAGE1_PAGE_DESCRIPTOR::PXN::True } else { STAGE1_PAGE_DESCRIPTOR::PXN::False }; + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + desc } } @@ -273,16 +281,18 @@ unsafe fn populate_tt_entries() -> Result<(), &'static str> { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control() { let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); TCR_EL1.write( TCR_EL1::TBI0::Ignored + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. + + TCR_EL1::T0SZ.val(t0sz), ); } diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs index 9ab302324..d4bcf0a3b 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs @@ -27,6 +27,19 @@ extern "Rust" { /// The board's memory map. #[rustfmt::skip] pub(super) mod map { + /// The inclusive end address of the memory map. + /// + /// End address + 1 must be power of two. + /// + /// # Note + /// + /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for + /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. + /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. + /// + /// However, making this trade-off has the downside of making it possible for the CPU to assert a + /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on + /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; pub const BOOT_CORE_STACK_END: usize = 0x8_0000; diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs index 1114bcce2..59d736a7c 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -63,8 +63,13 @@ fn mmio_range_inclusive() -> RangeInclusive { //-------------------------------------------------------------------------------------------------- /// Return the address space size in bytes. +/// +/// Guarantees size to be a power of two. pub const fn addr_space_size() -> usize { - memory_map::END_INCLUSIVE + 1 + let size = memory_map::END_INCLUSIVE + 1; + assert!(size.is_power_of_two()); + + size } /// Return a reference to the virtual memory layout. diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 66a32d085..4d8f5d7c3 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -204,7 +204,7 @@ fn kernel_main() -> ! { // For demo purposes, the exception handler will catch the faulting 8 GiB address and allow // execution to continue. info!(""); - info!("Trying to write to address 8 GiB..."); + info!("Trying to read from address 8 GiB..."); let mut big_addr: u64 = 8 * 1024 * 1024 * 1024; unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; @@ -215,7 +215,7 @@ fn kernel_main() -> ! { info!("Let's try again"); // Now use address 9 GiB. The exception handler won't forgive us this time. - info!("Trying to write to address 9 GiB..."); + info!("Trying to read from address 9 GiB..."); big_addr = 9 * 1024 * 1024 * 1024; unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 3ecdc0396..5f8baf23a 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -990,7 +990,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs -@@ -331,3 +331,40 @@ +@@ -341,3 +341,40 @@ Ok(()) } } @@ -1053,7 +1053,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 13_integ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs -@@ -71,3 +71,46 @@ +@@ -76,3 +76,46 @@ pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { &LAYOUT } @@ -1469,7 +1469,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m - // For demo purposes, the exception handler will catch the faulting 8 GiB address and allow - // execution to continue. - info!(""); -- info!("Trying to write to address 8 GiB..."); +- info!("Trying to read from address 8 GiB..."); - let mut big_addr: u64 = 8 * 1024 * 1024 * 1024; - unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; - @@ -1480,7 +1480,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m - info!("Let's try again"); - - // Now use address 9 GiB. The exception handler won't forgive us this time. -- info!("Trying to write to address 9 GiB..."); +- info!("Trying to read from address 9 GiB..."); - big_addr = 9 * 1024 * 1024 * 1024; - unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; - diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs index 5061f6007..9b658e864 100644 --- a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -37,6 +37,12 @@ register_bitfields! {u64, // A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + /// Privileged execute-never. PXN OFFSET(53) NUMBITS(1) [ False = 0, @@ -110,7 +116,6 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; type ArchTranslationTable = FixedSizeTranslationTable; @@ -194,13 +199,16 @@ impl convert::From AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, }; - // Execute Never. + // The execute-never attribute is mapped to PXN in AArch64. desc += if attribute_fields.execute_never { STAGE1_PAGE_DESCRIPTOR::PXN::True } else { STAGE1_PAGE_DESCRIPTOR::PXN::False }; + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + desc } } @@ -273,16 +281,18 @@ unsafe fn populate_tt_entries() -> Result<(), &'static str> { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control() { let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); TCR_EL1.write( TCR_EL1::TBI0::Ignored + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. + + TCR_EL1::T0SZ.val(t0sz), ); } diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory.rs b/13_integrated_testing/src/bsp/raspberrypi/memory.rs index 9ab302324..d4bcf0a3b 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory.rs @@ -27,6 +27,19 @@ extern "Rust" { /// The board's memory map. #[rustfmt::skip] pub(super) mod map { + /// The inclusive end address of the memory map. + /// + /// End address + 1 must be power of two. + /// + /// # Note + /// + /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for + /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. + /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. + /// + /// However, making this trade-off has the downside of making it possible for the CPU to assert a + /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on + /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; pub const BOOT_CORE_STACK_END: usize = 0x8_0000; diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs index d41de813c..485b8a46a 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -63,8 +63,13 @@ fn mmio_range_inclusive() -> RangeInclusive { //-------------------------------------------------------------------------------------------------- /// Return the address space size in bytes. +/// +/// Guarantees size to be a power of two. pub const fn addr_space_size() -> usize { - memory_map::END_INCLUSIVE + 1 + let size = memory_map::END_INCLUSIVE + 1; + assert!(size.is_power_of_two()); + + size } /// Return a reference to the virtual memory layout. diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 58e1bf22e..042c3f601 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2145,7 +2145,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception.rs 14_exceptions_p diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs --- 13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs -@@ -39,10 +39,12 @@ +@@ -52,10 +52,12 @@ pub mod mmio { use super::*; @@ -2162,7 +2162,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part } /// Physical devices. -@@ -53,6 +55,8 @@ +@@ -66,6 +68,8 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 5061f6007..9b658e864 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -37,6 +37,12 @@ register_bitfields! {u64, // A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + /// Privileged execute-never. PXN OFFSET(53) NUMBITS(1) [ False = 0, @@ -110,7 +116,6 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; type ArchTranslationTable = FixedSizeTranslationTable; @@ -194,13 +199,16 @@ impl convert::From AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, }; - // Execute Never. + // The execute-never attribute is mapped to PXN in AArch64. desc += if attribute_fields.execute_never { STAGE1_PAGE_DESCRIPTOR::PXN::True } else { STAGE1_PAGE_DESCRIPTOR::PXN::False }; + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + desc } } @@ -273,16 +281,18 @@ unsafe fn populate_tt_entries() -> Result<(), &'static str> { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control() { let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); TCR_EL1.write( TCR_EL1::TBI0::Ignored + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. + + TCR_EL1::T0SZ.val(t0sz), ); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs index 44c20e2c8..4ac3b57b9 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -27,6 +27,19 @@ extern "Rust" { /// The board's memory map. #[rustfmt::skip] pub(super) mod map { + /// The inclusive end address of the memory map. + /// + /// End address + 1 must be power of two. + /// + /// # Note + /// + /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for + /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. + /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. + /// + /// However, making this trade-off has the downside of making it possible for the CPU to assert a + /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on + /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; pub const BOOT_CORE_STACK_END: usize = 0x8_0000; diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs index d41de813c..485b8a46a 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs @@ -63,8 +63,13 @@ fn mmio_range_inclusive() -> RangeInclusive { //-------------------------------------------------------------------------------------------------- /// Return the address space size in bytes. +/// +/// Guarantees size to be a power of two. pub const fn addr_space_size() -> usize { - memory_map::END_INCLUSIVE + 1 + let size = memory_map::END_INCLUSIVE + 1; + assert!(size.is_power_of_two()); + + size } /// Return a reference to the virtual memory layout. diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index f7a9b1921..ab11fbfa6 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -357,7 +357,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 // A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. register_bitfields! {u64, -@@ -81,9 +91,6 @@ +@@ -87,9 +97,6 @@ ] } @@ -367,7 +367,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 /// A table descriptor for 64 KiB aperture. /// /// The output points to the next table. -@@ -98,36 +105,65 @@ +@@ -104,35 +111,65 @@ #[repr(transparent)] struct PageDescriptor(u64); @@ -417,7 +417,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 lvl2: [TableDescriptor; NUM_TABLES], -} --/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; + /// Index of the next free MMIO page. @@ -449,7 +448,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 //-------------------------------------------------------------------------------------------------- // Global instances -@@ -138,7 +174,8 @@ +@@ -143,7 +180,8 @@ /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". @@ -459,7 +458,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 static MMU: MemoryManagementUnit = MemoryManagementUnit; -@@ -146,13 +183,15 @@ +@@ -151,13 +189,15 @@ // Private Code //-------------------------------------------------------------------------------------------------- @@ -481,7 +480,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 } } -@@ -160,7 +199,7 @@ +@@ -165,7 +205,7 @@ fn from(next_lvl_table_addr: usize) -> Self { let val = InMemoryRegister::::new(0); @@ -490,7 +489,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 val.write( STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::TYPE::Table -@@ -207,23 +246,33 @@ +@@ -215,23 +255,33 @@ impl PageDescriptor { /// Create an instance. @@ -527,7 +526,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 /// Create an instance. pub const fn new() -> Self { assert!(NUM_TABLES > 0); -@@ -231,7 +280,55 @@ +@@ -239,7 +289,55 @@ Self { lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], lvl2: [TableDescriptor(0); NUM_TABLES], @@ -583,7 +582,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 } } -@@ -248,28 +345,6 @@ +@@ -256,32 +354,9 @@ ); } @@ -612,16 +611,20 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control() { let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -@@ -282,7 +357,7 @@ +- let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored +@@ -292,7 +367,7 @@ + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks -- + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. +- + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::T0SZ.val(T0SZ), ); } -@@ -290,17 +365,126 @@ +@@ -300,17 +375,126 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -751,7 +754,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 // Fail early if translation granule is not supported. Both RPis support it, though. if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { return Err("Translation granule not supported in HW"); -@@ -309,11 +493,8 @@ +@@ -319,11 +503,8 @@ // Prepare the memory attribute indirection register. set_up_mair(); @@ -764,7 +767,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 configure_translation_control(); -@@ -337,6 +518,9 @@ +@@ -347,6 +528,9 @@ //-------------------------------------------------------------------------------------------------- #[cfg(test)] @@ -774,7 +777,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 mod tests { use super::*; use test_macros::kernel_test; -@@ -363,7 +547,7 @@ +@@ -373,7 +557,7 @@ #[kernel_test] fn kernel_tables_in_bss() { let bss_range = bsp::memory::bss_range_inclusive(); @@ -1465,7 +1468,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs -@@ -4,72 +4,128 @@ +@@ -4,77 +4,128 @@ //! BSP Memory Management Unit. @@ -1584,23 +1587,22 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs //-------------------------------------------------------------------------------------------------- -/// Return the address space size in bytes. --pub const fn addr_space_size() -> usize { -- memory_map::END_INCLUSIVE + 1 +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( + super::phys_addr_space_end().into_usize(), + KernelGranule::SIZE, + ) as *const Page<_> - } - --/// Return a reference to the virtual memory layout. --pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { -- &LAYOUT ++} ++ +/// Map the kernel binary. +/// +/// # Safety -+/// + /// +-/// Guarantees size to be a power of two. +-pub const fn addr_space_size() -> usize { +- let size = memory_map::END_INCLUSIVE + 1; +- assert!(size.is_power_of_two()); +/// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. +pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { + kernel_mmu::kernel_map_pages_at( @@ -1613,7 +1615,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + execute_never: true, + }, + )?; -+ + +- size +-} + kernel_mmu::kernel_map_pages_at( + "Kernel code and RO data", + &phys_ro_page_desc(), @@ -1624,7 +1628,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + execute_never: false, + }, + )?; -+ + +-/// Return a reference to the virtual memory layout. +-pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +- &LAYOUT + kernel_mmu::kernel_map_pages_at( + "Kernel data and bss", + &phys_data_page_desc(), @@ -1640,7 +1647,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs } //-------------------------------------------------------------------------------------------------- -@@ -84,14 +140,12 @@ +@@ -89,14 +140,12 @@ /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { @@ -1660,7 +1667,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs assert!(end >= start); } } -@@ -99,17 +153,18 @@ +@@ -104,17 +153,18 @@ /// Ensure the kernel's virtual memory layout is free of overlaps. #[kernel_test] fn virt_mem_layout_has_no_overlaps() { @@ -1736,7 +1743,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- -@@ -17,34 +49,39 @@ +@@ -17,47 +49,39 @@ static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; static __ro_start: UnsafeCell<()>; @@ -1753,6 +1760,19 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { +- /// The inclusive end address of the memory map. +- /// +- /// End address + 1 must be power of two. +- /// +- /// # Note +- /// +- /// RPi3 and RPi4 boards can have different amounts of RAM. To make our code lean for +- /// educational purposes, we set the max size of the address space to 4 GiB regardless of board. +- /// This way, we can map the entire range that we need (end of MMIO for RPi4) in one take. +- /// +- /// However, making this trade-off has the downside of making it possible for the CPU to assert a +- /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on +- /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. - pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; + use super::*; @@ -1789,7 +1809,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } /// Physical devices. -@@ -52,13 +89,22 @@ +@@ -65,13 +89,22 @@ pub mod mmio { use super::*; @@ -1818,7 +1838,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } //-------------------------------------------------------------------------------------------------- -@@ -71,8 +117,8 @@ +@@ -84,8 +117,8 @@ /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] @@ -1829,7 +1849,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } /// Size of the Read-Only (RO) range of the kernel binary. -@@ -81,8 +127,42 @@ +@@ -94,8 +127,42 @@ /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] @@ -1874,7 +1894,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } //-------------------------------------------------------------------------------------------------- -@@ -91,8 +171,10 @@ +@@ -104,8 +171,10 @@ /// Exclusive end address of the boot core's stack. #[inline(always)] diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs index 0e45185e3..8abb997e2 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -47,6 +47,12 @@ register_bitfields! {u64, // A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + /// Privileged execute-never. PXN OFFSET(53) NUMBITS(1) [ False = 0, @@ -233,13 +239,16 @@ impl convert::From AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, }; - // Execute Never. + // The execute-never attribute is mapped to PXN in AArch64. desc += if attribute_fields.execute_never { STAGE1_PAGE_DESCRIPTOR::PXN::True } else { STAGE1_PAGE_DESCRIPTOR::PXN::False }; + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + desc } } @@ -352,6 +361,7 @@ fn configure_translation_control() { TCR_EL1.write( TCR_EL1::TBI0::Ignored + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable From fb33dddacf3834a52df597eb3f469649227fbd8e Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 1 Jan 2021 21:07:58 +0100 Subject: [PATCH 015/214] Copyright update, again --- LICENSE-MIT | 2 +- utils/update_copyright.rb | 7 ++----- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/LICENSE-MIT b/LICENSE-MIT index e91db966e..cfd323793 100644 --- a/LICENSE-MIT +++ b/LICENSE-MIT @@ -1,6 +1,6 @@ MIT License -Copyright (C) 2018-2020 by the respective authors +Copyright (C) 2018-2021 by the respective authors Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/utils/update_copyright.rb b/utils/update_copyright.rb index 2d56fd816..c86d8ac30 100755 --- a/utils/update_copyright.rb +++ b/utils/update_copyright.rb @@ -13,8 +13,5 @@ year = Date.today.year -# Update "Copyright (c) 20..-20.." -`sed -i -- 's,\\(Copyright .c. 20..\\)-20..,\\1-#{year},g' #{files}` - -# Update "Copyright (c) 20.. Name" -> "Copyright (c) 20..-20.. Name" -`sed -i -- 's,\\(Copyright .c. 20..\\) ,\\1-#{year} ,g' #{files}` +`sed -i -- 's,\\(Copyright .* 20..\\)-20..,\\1-#{year},g' #{files}` +`sed -i -- 's,\\(Copyright .* #{year - 1}\\) ,\\1-#{year} ,g' #{files}` From 269df3e25d6c72ddf9d995787691b2e855a34065 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 1 Jan 2021 21:13:09 +0100 Subject: [PATCH 016/214] Update toolchain --- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6712 -> 6712 bytes X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8064 -> 8064 bytes rust-toolchain | 2 +- 3 files changed, 1 insertion(+), 1 deletion(-) diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 7b314d3c9d3ff8f3360db8b5c7c4e5f4b0989ed9..a002d0525555b040896b5c76cbfb2571010361bd 100755 GIT binary patch delta 61 zcmV-D0K)&cG`KXd(GLO<6_e8smI4wLvuh9r0|N0BvvU*40T}=QK>2h2{}%`(K&j#b T0jl5w2&&-&395dxRTbj|nyeNw delta 61 zcmV-D0K)&cG`KXd(GLRg6qC~rmICq=vuh9r0|G%6vvU*40T~D+K&j#b0jl5w2&&-& T395kp{}%uNK>2R7RTbj|&juGh diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 6d23c37d397a23916acbf64b4df727d723bf9647..e2653eb1b0788a031a8c78cf2eee9dee4b4b351f 100755 GIT binary patch delta 59 zcmZp$Z?NAmQ;2cl=2=4KOhD?A$azM_kDC{XwJ}O^s5q>A#LT$r0W-&{hs>O-)c^e# PXJBynDZ1HF!k!%fqRbV@ delta 59 zcmZp$Z?NAmQ;6~4=2=4KOhD?A$azM_g_{?NwJ}OEFgW~_`uAU)L&ag`BWA``512Vt PJ!IxwrMlTs!k!%frrH&| diff --git a/rust-toolchain b/rust-toolchain index 23eb56819..b50e4d140 100644 --- a/rust-toolchain +++ b/rust-toolchain @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2020-12-09" +channel = "nightly-2021-01-01" components = ["llvm-tools-preview"] targets = ["aarch64-unknown-none-softfloat"] From ee52e8e288ef6f00416eed94921c2b0e1acfab73 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 1 Jan 2021 23:12:42 +0100 Subject: [PATCH 017/214] Bump UART to 576000 baud --- 06_drivers_gpio_uart/README.md | 18 ++++++++++-------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- 07_uart_chainloader/README.md | 4 ++-- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6712 -> 6720 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6592 -> 6600 bytes .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- 08_timestamps/README.md | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- 14_exceptions_part2_peripheral_IRQs/README.md | 12 ++++++------ .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- 15_virtual_mem_part2_mmio_remap/README.md | 16 ++++++++-------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 16 +++++++++------- utils/miniterm.rb | 2 +- 19 files changed, 127 insertions(+), 103 deletions(-) diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 33df63e87..0588109df 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -433,7 +433,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_g diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -0,0 +1,305 @@ +@@ -0,0 +1,307 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -610,19 +610,21 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + + /// Set up baud rate and characteristics. + /// -+ /// The calculation for the BRD given a target rate of 2300400 and a clock set to 48 MHz is: -+ /// `(48_000_000/16)/230400 = 13,02083`. `13` goes to the `IBRD` (integer field). The `FBRD` -+ /// (fractional field) is only 6 bits so `0,0208*64 = 1,3312 rounded to 1` will give the best -+ /// approximation we can get. A 5 modulo error margin is acceptable for UART and we're now at 0,01 modulo. ++ /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// -+ /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). ++ /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: ++ /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). ++ /// ++ /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will ++ /// give the best approximation we can get. A 5 modulo error margin is acceptable for UART and we're ++ /// now at 0.01 modulo. + pub fn init(&mut self) { + // Turn it off temporarily. + self.registers.CR.set(0); + + self.registers.ICR.write(ICR::ALL::CLEAR); -+ self.registers.IBRD.write(IBRD::IBRD.val(13)); -+ self.registers.FBRD.write(FBRD::FBRD.val(1)); ++ self.registers.IBRD.write(IBRD::IBRD.val(5)); ++ self.registers.FBRD.write(FBRD::FBRD.val(13)); + self.registers + .LCRH + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index de0bf813b..fdd92f65e 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -174,19 +174,21 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// The calculation for the BRD given a target rate of 2300400 and a clock set to 48 MHz is: - /// `(48_000_000/16)/230400 = 13,02083`. `13` goes to the `IBRD` (integer field). The `FBRD` - /// (fractional field) is only 6 bits so `0,0208*64 = 1,3312 rounded to 1` will give the best - /// approximation we can get. A 5 % error margin is acceptable for UART and we're now at 0,01 %. + /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). /// - /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). + /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: + /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// + /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will + /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're + /// now at 0.01 %. pub fn init(&mut self) { // Turn it off temporarily. self.registers.CR.set(0); self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(13)); - self.registers.FBRD.write(FBRD::FBRD.val(1)); + self.registers.IBRD.write(IBRD::IBRD.val(5)); + self.registers.FBRD.write(FBRD::FBRD.val(13)); self.registers .LCRH .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 1491c84b8..2b7d797b7 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -222,7 +222,7 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -268,6 +268,15 @@ +@@ -270,6 +270,15 @@ // readability. self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) } @@ -238,7 +238,7 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 } impl console::interface::Read for PL011Uart { -@@ -278,18 +287,20 @@ +@@ -280,18 +289,20 @@ cpu::nop(); } diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index a002d0525555b040896b5c76cbfb2571010361bd..983c85c293b8c3a8cf564f8951f2ae20016e22c5 100755 GIT binary patch delta 1105 zcmaJ@g6q<^N z;DDQ&b{Z|>8_cs zd-YP|O9xzK0Qbzdz~e?CBK5Hb2=W5d<|qX zmslH*-^ADQAS)G8j_pihZFpMQD5(J)bw+VkxgoJYVbr;cDO9TW7M|68lUcm*ba_=` zZ}4g3es%?QkHLm;#FJpR@rtKU@6dQf25#0l^LAr#Si#Lc;N^Blek+Mq+LgSUaD4*5 zdbYU765IzcdGa<6c$?iU%IDT=GzcE|rri@!{!pS84SeM7tUsINrtF_8`@m|z&)#Ii z*%n@|)<~%&r4BL!PpO>^r6?!i$pI3xe^oaty)8D$7qydVm}+X)qW#w0C^S7+fIr>< zW9!)E$hIi%ZfZUh?zb)xdNiz9^;*AmIcyY^dZGI4xR(tT!Eg`5CEC%tQX#WIlQG+cABG^>K zi>>W2f+xY?X$yi7mF8qhp%f3Lp!6u43KC;I^q_RBb^SKGLa7c6?Dx%^_ujnyiXC$u zFYB24_crtxBlJ@Jgh)ipH!(@`N?9OU#!&` zJWm(v2k|70xn(TV5x0cz(MRrN%*&Gsi-=ldR}5njHAK`TBIP!lasosvZJgqSer$kl zxsN-~NTdOwnX!Ac#}jgv_L5Z}4-cU?JYCL#FnPTO7Df8T(_S|iB`WKmEjJ*omVWm{ zU4zY}ted0&iF;3Bk>2vQyK-Ry!ZSS}W?c0i<08!l$q#Qk_R_d7#0x3yMHu;(IYizb zK{pTLsixBcJ?{$*@F{H)y4QG4tMjI`DM2n=-)`&trt1L&oGfe`Y@|wLNZv;VxNQ1< zaUWS@G1>mDKM|4m!2#zFA!Vm4rNue9Fv9!W;~XkDx3X!6i>~^PHc`o193E9$pG~h1A-IkT#zVV!REUU; zB--6F7wrekM9ZA|9FxR6)PON)jpSn!`^kDI()4kg@|pH*20oH=Lj<}dOW@X5^}sP9;Z z17^)wz0~MvW)!|dR=?5e@5Xs34s?yXEH;mYnifnpNT|9ri^Xr%pRF23ySjGcS9Yl` zS+(85FI%{oP}4#?@g=Y=Wc9&a(}(s&lePv#gx!F$Dn3QUA)4{rnH#a*mvhh0J>Pe~ zb7!jkfA!Drkhab%aG3zin}zZhQV;1(glUbGa#cCMqCwU%`k><3{l2xKl+(M!p0Mxa zePOeUjw_|y<3*r7n7w2ssaa(Y<%6VA-ldEXVzP5;hV-#(>UG>-Q$HzRZrM##tV@rU z|JkxGlY#QB4_rwiadx|PluWR`wyar-(^+gMf0T6!V63?Nq~(o0;5-}AY(TzHWixGu zgN=6Dv=qB31KofN&m*HG%fJ*5rFBHRjj`!*&Vn)eRCVt&}t?- z6G)K)`ytS2Ug@N1L=9v7k7Uy8?G{!0d9lStm8Q)CTM1a&>^54LLvw3P4Q?m1?9-ru z#}5S$@+Lv+t41ds#wN{QW&5$K*zUioy`B3RaQr4ATK^g7s!ZtB4fp$!3Yj$-9FNQ> z+XzOnPozCX?y@i2Exj@5+){xaYOs^-oyGfu&L#9t-B+9s?+!ZO#H;;fIh0hd+QchBhf|5fmDx zfRt_jg(Rs*1z*D30U|9UzijZS zWt|z%;`E@ad^}+Ij@yDotIv}cy zxbNd#ymdk3L&85JeEdqh^?rRp{CNK2vDPSkX^%#JPr*1#R43FFcxA1J>70zX`Z646#$STa|Jhtt5SB zMFd)n*^UDHUaJvckw7}tr_(W9w~_?69sw8_zlZ&=1T%>DgBKQ2KSn=chXM&=uycWW zy--h6I5mv<+d|yswsVsCEmvitN&~vV#sf)z{~|gehOSOANogegtXa|U^Uca8heS!P zE!WcyB&l2$%}Lv5+D|02(ZOl3JdzNdxCr#ENa(d0`%GSf^cN89giMYqL(ojMBcJ$@C}NR#BD#fH)N~?#eb$ zAI2jS4ep#CDW93fSq;$e)R`kcYO|UoqjU_N+Y=78G)d?w=47~5ed@!xv`jzCE& z<2Nack#)rXu@_+5(jOj~S77sk@tPqCZaGHqR7Y@47Uf4bN?I67a;rA%NGlh#;p&=v zrn;;RH)Szmy|T--Z=mCRf;uX-W8Y*#H0>N(n*g*y;_^Gn5&HH%&}Qe~un$2)!Xfw+ zn?sS_cI0eC&i21K%Q~&h0&6lUnx5Tp>hhwkf84+s4ri6aIc=}_N6C@6r=*c^dmJST z>}Dt!yfouzZdikEMC~UQ6rYEBiIR`vpubLuZHcJ*#fX>1cj%rH#{$P^+0n?l+-cS8 z3h=BN^DwUJo&v9jINr}OzG(OS-@lXBe^2e=xQh30=a_FHfXX(iyJJIq>?ki^;rKHf xQ#E-s char { @@ -2003,7 +2003,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } fn clear(&self) { -@@ -321,3 +410,24 @@ +@@ -323,3 +412,24 @@ self.inner.lock(|inner| inner.chars_read) } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 29bf95c13..9504cfc31 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -226,19 +226,21 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// The calculation for the BRD given a target rate of 2300400 and a clock set to 48 MHz is: - /// `(48_000_000/16)/230400 = 13,02083`. `13` goes to the `IBRD` (integer field). The `FBRD` - /// (fractional field) is only 6 bits so `0,0208*64 = 1,3312 rounded to 1` will give the best - /// approximation we can get. A 5 % error margin is acceptable for UART and we're now at 0,01 %. + /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). /// - /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). + /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: + /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// + /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will + /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're + /// now at 0.01 %. pub fn init(&mut self) { // Turn it off temporarily. self.registers.CR.set(0); self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(13)); - self.registers.FBRD.write(FBRD::FBRD.val(1)); + self.registers.IBRD.write(IBRD::IBRD.val(5)); + self.registers.FBRD.write(FBRD::FBRD.val(13)); self.registers .LCRH .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index ab11fbfa6..77f6ad626 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1304,10 +1304,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ inner: IRQSafeNullLock, irq_number: bsp::device_driver::IRQNumber, } -@@ -232,7 +237,15 @@ - /// approximation we can get. A 5 modulo error margin is acceptable for UART and we're now at 0,01 modulo. - /// - /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). +@@ -234,7 +239,15 @@ + /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will + /// give the best approximation we can get. A 5 modulo error margin is acceptable for UART and we're + /// now at 0.01 modulo. - pub fn init(&mut self) { + /// + /// # Safety @@ -1321,7 +1321,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ // Turn it off temporarily. self.registers.CR.set(0); -@@ -249,6 +262,8 @@ +@@ -251,6 +264,8 @@ self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -1330,7 +1330,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } /// Send a character. -@@ -318,13 +333,18 @@ +@@ -320,13 +335,18 @@ /// /// # Safety /// @@ -1352,7 +1352,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ irq_number, } } -@@ -341,7 +361,14 @@ +@@ -343,7 +363,14 @@ } unsafe fn init(&self) -> Result<(), &'static str> { @@ -1368,7 +1368,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ Ok(()) } -@@ -360,6 +387,16 @@ +@@ -362,6 +389,16 @@ Ok(()) } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index c2354d430..b2fc254bd 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -231,12 +231,14 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// The calculation for the BRD given a target rate of 2300400 and a clock set to 48 MHz is: - /// `(48_000_000/16)/230400 = 13,02083`. `13` goes to the `IBRD` (integer field). The `FBRD` - /// (fractional field) is only 6 bits so `0,0208*64 = 1,3312 rounded to 1` will give the best - /// approximation we can get. A 5 % error margin is acceptable for UART and we're now at 0,01 %. + /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). /// - /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). + /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: + /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// + /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will + /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're + /// now at 0.01 %. /// /// # Safety /// @@ -250,8 +252,8 @@ impl PL011UartInner { self.registers.CR.set(0); self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(13)); - self.registers.FBRD.write(FBRD::FBRD.val(1)); + self.registers.IBRD.write(IBRD::IBRD.val(5)); + self.registers.FBRD.write(FBRD::FBRD.val(13)); self.registers .LCRH .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 66b734638..46cceff92 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -174,19 +174,21 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// The calculation for the BRD given a target rate of 2300400 and a clock set to 48 MHz is: - /// `(48_000_000/16)/230400 = 13,02083`. `13` goes to the `IBRD` (integer field). The `FBRD` - /// (fractional field) is only 6 bits so `0,0208*64 = 1,3312 rounded to 1` will give the best - /// approximation we can get. A 5 % error margin is acceptable for UART and we're now at 0,01 %. + /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). /// - /// This results in 8N1 and 230400 baud (we set the clock to 48 MHz in config.txt). + /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: + /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// + /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will + /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're + /// now at 0.01 %. pub fn init(&mut self) { // Turn it off temporarily. self.registers.CR.set(0); self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(13)); - self.registers.FBRD.write(FBRD::FBRD.val(1)); + self.registers.IBRD.write(IBRD::IBRD.val(5)); + self.registers.FBRD.write(FBRD::FBRD.val(13)); self.registers .LCRH .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on diff --git a/utils/miniterm.rb b/utils/miniterm.rb index 0875ada55..010761cd8 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -41,7 +41,7 @@ def wait_for_serial def open_serial wait_for_serial - @target_serial = SerialPort.new(@target_serial_name, 230_400, 8, 1, SerialPort::NONE) + @target_serial = SerialPort.new(@target_serial_name, 576_000, 8, 1, SerialPort::NONE) # Ensure all output is immediately flushed to the device. @target_serial.sync = true From f2a891236e24289e9f4eb9c6c31ee01f67a4a036 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 2 Jan 2021 16:00:16 +0100 Subject: [PATCH 018/214] Remove unused feature --- 04_zero_overhead_abstraction/README.md | 5 ++--- 04_zero_overhead_abstraction/src/main.rs | 1 - 05_safe_globals/README.md | 8 ++++---- 05_safe_globals/src/main.rs | 1 - 06_drivers_gpio_uart/README.md | 6 +++--- 06_drivers_gpio_uart/src/main.rs | 1 - 07_uart_chainloader/README.md | 8 ++++---- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6720 -> 6720 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6600 -> 6600 bytes 07_uart_chainloader/src/main.rs | 1 - 08_timestamps/README.md | 8 ++++---- 08_timestamps/src/main.rs | 1 - 09_hw_debug_JTAG/src/main.rs | 1 - 10_privilege_level/README.md | 6 +++--- 10_privilege_level/src/main.rs | 1 - .../README.md | 8 ++++---- .../src/main.rs | 1 - 12_exceptions_part1_groundwork/README.md | 6 +++--- 12_exceptions_part1_groundwork/src/main.rs | 1 - 13_integrated_testing/README.md | 12 +++++------- 13_integrated_testing/src/lib.rs | 1 - 14_exceptions_part2_peripheral_IRQs/README.md | 2 +- .../src/lib.rs | 1 - 15_virtual_mem_part2_mmio_remap/README.md | 2 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 1 - X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8064 -> 8080 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6800 -> 6800 bytes X1_JTAG_boot/src/main.rs | 1 - 28 files changed, 34 insertions(+), 50 deletions(-) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index 003108219..1e8d4c6d9 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -219,18 +219,17 @@ diff -uNr 03_hacky_hello_world/src/cpu.rs 04_zero_overhead_abstraction/src/cpu.r diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main.rs --- 03_hacky_hello_world/src/main.rs +++ 04_zero_overhead_abstraction/src/main.rs -@@ -92,9 +92,8 @@ +@@ -92,9 +92,7 @@ //! - `crate::memory::*` //! - `crate::bsp::memory::*` -#![feature(asm)] #![feature(format_args_nl)] -#![feature(global_asm)] -+#![feature(naked_functions)] #![feature(panic_info_message)] #![no_main] #![no_std] -@@ -116,7 +115,8 @@ +@@ -116,7 +114,8 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/04_zero_overhead_abstraction/src/main.rs b/04_zero_overhead_abstraction/src/main.rs index 22e179229..cd4c99e58 100644 --- a/04_zero_overhead_abstraction/src/main.rs +++ b/04_zero_overhead_abstraction/src/main.rs @@ -93,7 +93,6 @@ //! - `crate::bsp::memory::*` #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![no_main] #![no_std] diff --git a/05_safe_globals/README.md b/05_safe_globals/README.md index a66e7f2b4..537b0ba3b 100644 --- a/05_safe_globals/README.md +++ b/05_safe_globals/README.md @@ -213,15 +213,15 @@ diff -uNr 04_zero_overhead_abstraction/src/console.rs 05_safe_globals/src/consol diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs --- 04_zero_overhead_abstraction/src/main.rs +++ 05_safe_globals/src/main.rs -@@ -95,6 +95,7 @@ +@@ -94,6 +94,7 @@ + #![feature(format_args_nl)] - #![feature(naked_functions)] #![feature(panic_info_message)] +#![feature(trait_alias)] #![no_main] #![no_std] -@@ -108,6 +109,7 @@ +@@ -107,6 +108,7 @@ mod panic_wait; mod print; mod runtime_init; @@ -229,7 +229,7 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs /// Early init code. /// -@@ -115,8 +117,15 @@ +@@ -114,8 +116,15 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/05_safe_globals/src/main.rs b/05_safe_globals/src/main.rs index 2c29444fd..64d4c5fd6 100644 --- a/05_safe_globals/src/main.rs +++ b/05_safe_globals/src/main.rs @@ -93,7 +93,6 @@ //! - `crate::bsp::memory::*` #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 0588109df..4c030e744 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -1206,9 +1206,9 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] - #![feature(naked_functions)] #![feature(panic_info_message)] -@@ -105,6 +114,7 @@ + #![feature(trait_alias)] +@@ -104,6 +113,7 @@ mod bsp; mod console; mod cpu; @@ -1216,7 +1216,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -116,16 +126,46 @@ +@@ -115,16 +125,46 @@ /// # Safety /// /// - Only a single core must be active and running this function. diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index c45f7f785..e0bf47a93 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -102,7 +102,6 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 2b7d797b7..b87b4bed0 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -413,9 +413,9 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs #![feature(const_fn_fn_ptr_basics)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] - #![feature(naked_functions)] #![feature(panic_info_message)] -@@ -109,7 +111,8 @@ + #![feature(trait_alias)] +@@ -108,7 +110,8 @@ #![no_std] // `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls @@ -425,7 +425,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod bsp; mod console; -@@ -118,6 +121,7 @@ +@@ -117,6 +120,7 @@ mod memory; mod panic_wait; mod print; @@ -433,7 +433,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -144,28 +148,52 @@ +@@ -143,28 +147,52 @@ /// The main function running after the early init. fn kernel_main() -> ! { diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 983c85c293b8c3a8cf564f8951f2ae20016e22c5..34c3214cfbe8d37cc863e86b50c5d3630ce0304e 100755 GIT binary patch delta 14 VcmX?La=>JRfH-5@WJRfH-6OW ! { diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index 4dad25714..b9db5fef4 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -104,7 +104,6 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index 4dad25714..b9db5fef4 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -104,7 +104,6 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index fab885983..8716099e8 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -456,7 +456,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs --- 09_hw_debug_JTAG/src/main.rs +++ 10_privilege_level/src/main.rs -@@ -117,6 +117,7 @@ +@@ -116,6 +116,7 @@ mod console; mod cpu; mod driver; @@ -464,7 +464,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -147,12 +148,19 @@ +@@ -146,12 +147,19 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -484,7 +484,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -167,11 +175,12 @@ +@@ -166,11 +174,12 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index faa42f902..0c730ff96 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -104,7 +104,6 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index df84c9669..6216fd65e 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -892,9 +892,9 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s +#![feature(const_generics)] +#![feature(const_panic)] #![feature(format_args_nl)] - #![feature(naked_functions)] #![feature(panic_info_message)] -@@ -130,9 +135,18 @@ + #![feature(trait_alias)] +@@ -129,9 +134,18 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -914,7 +914,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -155,6 +169,9 @@ +@@ -154,6 +168,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -924,7 +924,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -178,6 +195,13 @@ +@@ -177,6 +194,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index eed8b35c6..41ca22e33 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -109,7 +109,6 @@ #![feature(const_generics)] #![feature(const_panic)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 979636866..c6264088f 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -961,10 +961,10 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ #![feature(const_panic)] #![feature(format_args_nl)] +#![feature(global_asm)] - #![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -144,6 +145,8 @@ + #![no_main] +@@ -143,6 +144,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -973,7 +973,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } -@@ -195,13 +198,28 @@ +@@ -194,13 +197,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 4d8f5d7c3..495840e87 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -110,7 +110,6 @@ #![feature(const_panic)] #![feature(format_args_nl)] #![feature(global_asm)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 5f8baf23a..a282e6593 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1130,7 +1130,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,170 @@ +@@ -0,0 +1,169 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1246,7 +1246,6 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +#![feature(format_args_nl)] +#![feature(global_asm)] +#![feature(linkage)] -+#![feature(naked_functions)] +#![feature(panic_info_message)] +#![feature(trait_alias)] +#![no_std] @@ -1305,7 +1304,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,130 +6,12 @@ +@@ -6,129 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1414,7 +1413,6 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m + #![feature(format_args_nl)] -#![feature(global_asm)] --#![feature(naked_functions)] -#![feature(panic_info_message)] -#![feature(trait_alias)] #![no_main] @@ -1438,7 +1436,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -141,6 +23,7 @@ +@@ -140,6 +23,7 @@ /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -1446,7 +1444,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -166,9 +49,7 @@ +@@ -165,9 +49,7 @@ /// The main function running after the early init. fn kernel_main() -> ! { use console::interface::All; @@ -1456,7 +1454,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -195,31 +76,6 @@ +@@ -194,31 +76,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index 8e60240f2..f958ae5d5 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -113,7 +113,6 @@ #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_std] diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index d9baef354..dc18b75f4 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2407,7 +2407,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -@@ -137,6 +144,7 @@ +@@ -136,6 +143,7 @@ pub mod exception; pub mod memory; pub mod print; diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index 141adc3ab..635732bd8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -120,7 +120,6 @@ #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_std] diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 77f6ad626..fb574abf4 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -2038,7 +2038,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -138,6 +139,7 @@ +@@ -137,6 +138,7 @@ mod synchronization; pub mod bsp; diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index 9ff2a50c7..e66dd3e86 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -121,7 +121,6 @@ #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_std] diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index e2653eb1b0788a031a8c78cf2eee9dee4b4b351f..443facda4297a662552100554a1bd6de4427e137 100755 GIT binary patch delta 1309 zcmb7DUu;uV82`?_z2#oZx^`XHwH>|fhADTL;5Hd-plwGCFdiVqL}KFTm~lKUW-t0; z+My4!7k2oNX?#(M8CRW*RLw5DtS>(Bq@tMk)J?^>Iq{j^TFqN32x5;f^Y7wRqcOOY8%@?VVsV=yhGh za_iUZSG?*=OYtP$^G&f;JmpVP%B+7UrF`kvCFsT<{A0|*!M2HDDbDX)k4)O0qzR1Ow+6}4uNj%p+(o4*1GVt;iw<$O!3In`+0C*+oG7k~AT2e6^Wo!T+ zv`>?cQ^5gJ@WWuLV`euWC26WrsE>Z(jaI2mLxy1gvk&Pe`6*|(04CFR(T2at)X&#s z#v_|ekRp$sv{Q4x#4ppAOb|-=Y{D?4{*buz)Ky&d2*}d%AW@-e-W&YsmkLm zWCahMfkj}r9t2BVIxV9w8@WREu^D~2_6BfINna+pEYVi*G}t~~TaZnI&K28bRt^BK z?t!{}NihYw7dGtqvI4tL#^vvA+YSjxcp|htpCi~R{jbvm;6-tEpUIjOd?a8ruxJ;D*tMn!+dA9)8%by@*xj*fv{-tkxxOolVA8zQofQinK_V}h9 z0(x)(%yjD1tGSA~&b`ckBKFYHaEH+h?~&99u7wBtE=QX|`h|_u^fLj!loDvDTeh zk7J+3)vjGbszc*ZX>E1#`|)bz3a)QUOH9Y%*qHIV6Yx9j#Wzj6?D$(9|6MG_CfPAG H^ymKq)e~>A delta 1314 zcmb7DO>7%Q6n?Yob$7Ai*h!opJ9zE3!g4AR#i3~(3SOszw&768AfXD$O&UaUicl_y z1M;?TK#>p`9yG0bNm`+>fzW~hWpD9?Lr;aaAaSWvpmIWll=eVPNR~J21@SAvNKZ5G z&3p5{@0*#KoSC~4lA7O8;B5wQrTQ)KaE#?K z=sSzm=4saaIBXvzN>|>F(eoX;;f_(iQ zu=9>HE7lsED`@Aze$4a5o=*PG@ydHbv4(0eBcQ76ah@j2G4%(H!81THi45QnAN5hk zxkLP$EoU|Xi5ntcAT z_-&c-=uQKq$Yabbz%ov@$2$x^e1Zf-fN1@0VXC$VczQE|m)ZwQl%-Or^cL8qvtaj7 zS|Df^HeLm;Gq438Jpwj?;#yu`PLG&{2TtqD;t>Wa`ZCF73y$LG^#f2(R|a{_@maOQ zz&93T%ekaj0@-uh&g_Z;`%fn1n~vjj2sn--Svb;>D2)%oRk#N6@M;+uFPs^|Z2)djCh$UF7@vK>9nx<>Ryv(E0uJmL8UCc9r^ zb9lZx)pyU^Yxhbu4B^<1{33<&6-RhjJymDL7b`#J2@B}ey(KP+{LWM%cwAUqres>L6Obrf|5*p z2Ur=lykKUS$jQJkLG$C}K4EKbpdNWfhMn@F3_Cd;<#()P1e(z(zk`zxNFS8np_u@r zf$}es8796kXZXnh)XCAPI?)rT?}a+^%K!B&D_B^<1{33f&6-RhjJ%8uL7b`#J2@B}erhso?qXKsWt5w|QD6(xN8!!sf|5*p z3s@PpykKUS$jQJkLG$9|K4I$wpdNWfhMn@F3_Cd;<#%W@0L^HW-@(P#5X5y*euq{9 zkOs=XNM@MG1=Qu$s5185Nk&;n1OMIJy)G=P?PAX~CfdEyIphM%t{ zuM|;c{rwkY%taAp*)ouU3<_5mfF6AQ@4xsXX2w+JXuiGo`nGjHv5Q9 aVPx#u{7|f&k&{Q110=>Ev$;@0o(TXLG-Ztd diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index 5e54c329c..340acd4eb 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -104,7 +104,6 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] -#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] From ff382c3fafa6257e37c208015cbea3aa3d1b2515 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 2 Jan 2021 16:04:27 +0100 Subject: [PATCH 019/214] Remove comment --- 04_zero_overhead_abstraction/README.md | 5 ++--- 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs | 1 - 05_safe_globals/src/_arch/aarch64/cpu.rs | 1 - 06_drivers_gpio_uart/README.md | 2 +- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 1 - 07_uart_chainloader/README.md | 5 ++--- 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 1 - 08_timestamps/README.md | 7 +++---- 08_timestamps/src/_arch/aarch64/cpu.rs | 1 - 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs | 1 - 10_privilege_level/README.md | 4 ++-- 11 files changed, 10 insertions(+), 19 deletions(-) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index 1e8d4c6d9..ce452fb79 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -53,7 +53,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abs diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs --- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs +++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs -@@ -4,8 +4,35 @@ +@@ -4,8 +4,34 @@ //! Architectural processor code. @@ -79,7 +79,6 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstrac +pub unsafe fn _start() -> ! { + use crate::runtime_init; + -+ // Expect the boot core to start in EL2. + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() @@ -91,7 +90,7 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstrac //-------------------------------------------------------------------------------------------------- // Public Code -@@ -14,13 +41,7 @@ +@@ -14,13 +40,7 @@ /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs index 61a68a18e..b02903c90 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); runtime_init::runtime_init() diff --git a/05_safe_globals/src/_arch/aarch64/cpu.rs b/05_safe_globals/src/_arch/aarch64/cpu.rs index 61a68a18e..b02903c90 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); runtime_init::runtime_init() diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 4c030e744..7e3da3ba5 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -185,7 +185,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 05_safe_globals/src/_arch/aarch64/cpu.rs +++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -@@ -38,6 +38,17 @@ +@@ -37,6 +37,17 @@ // Public Code //-------------------------------------------------------------------------------------------------- diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index 0f545a008..dfc6be130 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); runtime_init::runtime_init() diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index b87b4bed0..ef90e02ee 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -183,14 +183,13 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs --- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs -@@ -22,12 +22,12 @@ +@@ -22,11 +22,11 @@ /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { - use crate::runtime_init; + use crate::relocate; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() @@ -198,7 +197,7 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ } else { // If not core0, infinitely wait for events. wait_forever() -@@ -56,3 +56,19 @@ +@@ -55,3 +55,19 @@ asm::wfe() } } diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/07_uart_chainloader/src/_arch/aarch64/cpu.rs index ccafe9ff6..1895b6f16 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::relocate; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); relocate::relocate_self() diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 0bd342521..45a5c270e 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -115,14 +115,13 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ 08_timestamps/src/_arch/aarch64/cpu.rs -@@ -22,12 +22,12 @@ +@@ -22,11 +22,11 @@ /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { - use crate::relocate; + use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); - relocate::relocate_self() @@ -130,7 +129,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a } else { // If not core0, infinitely wait for events. wait_forever() -@@ -40,15 +40,6 @@ +@@ -39,15 +39,6 @@ pub use asm::nop; @@ -146,7 +145,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -56,19 +47,3 @@ +@@ -55,19 +46,3 @@ asm::wfe() } } diff --git a/08_timestamps/src/_arch/aarch64/cpu.rs b/08_timestamps/src/_arch/aarch64/cpu.rs index 6072751db..a65a06827 100644 --- a/08_timestamps/src/_arch/aarch64/cpu.rs +++ b/08_timestamps/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); runtime_init::runtime_init() diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs index 6072751db..a65a06827 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs @@ -24,7 +24,6 @@ use cortex_a::{asm, regs::*}; pub unsafe fn _start() -> ! { use crate::runtime_init; - // Expect the boot core to start in EL2. if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { SP.set(bsp::memory::boot_core_stack_end() as u64); runtime_init::runtime_init() diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 8716099e8..3ffa1bbe4 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -226,7 +226,7 @@ Minipush 1.0 diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch/aarch64/cpu.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ 10_privilege_level/src/_arch/aarch64/cpu.rs -@@ -18,22 +18,65 @@ +@@ -18,21 +18,65 @@ /// # Safety /// /// - Linker script must ensure to place this function where it is expected by the target machine. @@ -238,10 +238,10 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch pub unsafe fn _start() -> ! { - use crate::runtime_init; - - // Expect the boot core to start in EL2. - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() ++ // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { From c35a30cd0bce2725c3a2586309b7edd3875fb117 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 4 Jan 2021 14:37:17 +0100 Subject: [PATCH 020/214] Bump UART to 921_600 baud + other bugfixes Fixes #95 Fixes #98 Co-authored-by: Takumasa Sakao --- 06_drivers_gpio_uart/README.md | 158 +++++++++++++----- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 1 - .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 130 ++++++++++---- 06_drivers_gpio_uart/src/console.rs | 7 + 06_drivers_gpio_uart/src/main.rs | 3 + 07_uart_chainloader/README.md | 158 ++++++++---------- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6720 -> 6856 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6600 -> 6728 bytes 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 1 - .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 126 ++++++++++---- 07_uart_chainloader/src/console.rs | 6 +- 07_uart_chainloader/src/main.rs | 5 +- 08_timestamps/README.md | 91 +++++++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- 08_timestamps/src/console.rs | 6 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- 09_hw_debug_JTAG/src/console.rs | 6 +- 10_privilege_level/README.md | 10 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- 10_privilege_level/src/console.rs | 6 +- 10_privilege_level/src/main.rs | 4 + .../README.md | 8 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- .../src/console.rs | 6 +- .../src/main.rs | 4 + 12_exceptions_part1_groundwork/README.md | 6 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- 12_exceptions_part1_groundwork/src/console.rs | 6 +- 12_exceptions_part1_groundwork/src/main.rs | 4 + 13_integrated_testing/README.md | 10 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- 13_integrated_testing/src/console.rs | 6 +- 13_integrated_testing/src/main.rs | 4 + 14_exceptions_part2_peripheral_IRQs/README.md | 118 +++---------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 85 +++++++--- .../src/console.rs | 6 +- 15_virtual_mem_part2_mmio_remap/README.md | 20 +-- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 85 +++++++--- .../src/console.rs | 6 +- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8080 -> 8240 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6800 -> 7968 bytes .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 135 ++++++++++----- X1_JTAG_boot/src/console.rs | 6 +- utils/minipush.rb | 21 ++- utils/miniterm.rb | 4 +- 46 files changed, 1356 insertions(+), 714 deletions(-) diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 7e3da3ba5..6ca14cfe0 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -185,14 +185,13 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 05_safe_globals/src/_arch/aarch64/cpu.rs +++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -@@ -37,6 +37,17 @@ +@@ -37,6 +37,16 @@ // Public Code //-------------------------------------------------------------------------------------------------- +pub use asm::nop; + +/// Spin for `n` cycles. -+#[cfg(feature = "bsp_rpi3")] +#[inline(always)] +pub fn spin_for_cycles(n: usize) { + for _ in 0..n { @@ -433,7 +432,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_g diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -0,0 +1,307 @@ +@@ -0,0 +1,381 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -572,6 +571,12 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + ++#[derive(PartialEq)] ++enum BlockingMode { ++ Blocking, ++ NonBlocking, ++} ++ +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- @@ -610,24 +615,41 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + + /// Set up baud rate and characteristics. + /// -+ /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). ++ /// This results in 8N1 and 921_600 baud. + /// -+ /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: -+ /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). ++ /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): ++ /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// -+ /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will -+ /// give the best approximation we can get. A 5 modulo error margin is acceptable for UART and we're -+ /// now at 0.01 modulo. ++ /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will ++ /// give the best approximation we can get. A 5modulo error margin is acceptable for UART and we're ++ /// now at 0.02modulo. + pub fn init(&mut self) { -+ // Turn it off temporarily. ++ // Execution can arrive here while there are still characters queued in the TX FIFO and ++ // actively being sent out by the UART hardware. If the UART is turned off in this case, ++ // those queued characters would be lost. ++ // ++ // For example, this can happen during runtime on a call to panic!(), because panic!() ++ // initializes its own UART instance and calls init(). ++ // ++ // Hence, flush first to ensure all pending characters are transmitted. ++ self.flush(); ++ ++ // Turn the UART off temporarily. + self.registers.CR.set(0); + ++ // Clear all pending interrupts. + self.registers.ICR.write(ICR::ALL::CLEAR); -+ self.registers.IBRD.write(IBRD::IBRD.val(5)); -+ self.registers.FBRD.write(FBRD::FBRD.val(13)); ++ ++ // Set the baud rate. ++ self.registers.IBRD.write(IBRD::IBRD.val(3)); ++ self.registers.FBRD.write(FBRD::FBRD.val(16)); ++ ++ // Set 8N1 + FIFO on. + self.registers + .LCRH -+ .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on ++ .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); ++ ++ // Turn the UART on. + self.registers + .CR + .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -645,6 +667,58 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + + self.chars_written += 1; + } ++ ++ /// Block execution until the last buffered character has been physically put on the TX wire. ++ fn flush(&self) { ++ // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per ++ // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = ++ // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. ++ // ++ // Now make an educated guess for a good delay value. According to Wikipedia, the fastest ++ // RPi4 clocks around 1.5 GHz. ++ // ++ // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 ++ // ns would the CPU be clocked at 2 GHz. ++ const CHAR_TIME_SAFE: usize = 24_000; ++ ++ // Spin until TX FIFO empty is set. ++ while !self.registers.FR.matches_all(FR::TXFE::SET) { ++ cpu::nop(); ++ } ++ ++ // After the last character has been queued for transmission, wait for the time of one ++ // character + some extra time for safety. ++ cpu::spin_for_cycles(CHAR_TIME_SAFE); ++ } ++ ++ /// Retrieve a character. ++ fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { ++ // If RX FIFO is empty, ++ if self.registers.FR.matches_all(FR::RXFE::SET) { ++ // immediately return in non-blocking mode. ++ if blocking_mode == BlockingMode::NonBlocking { ++ return None; ++ } ++ ++ // Otherwise, wait until a char was received. ++ while self.registers.FR.matches_all(FR::RXFE::SET) { ++ cpu::nop(); ++ } ++ } ++ ++ // Read one character. ++ let mut ret = self.registers.DR.get() as u8 as char; ++ ++ // Convert carrige return to newline. ++ if ret == '\r' { ++ ret = '\n' ++ } ++ ++ // Update statistics. ++ self.chars_read += 1; ++ ++ Some(ret) ++ } +} + +/// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -667,6 +741,8 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri +} + +impl PL011Uart { ++ /// Create an instance. ++ /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. @@ -706,29 +782,26 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + // readability. + self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) + } ++ ++ fn flush(&self) { ++ // Spin until TX FIFO empty is set. ++ self.inner.lock(|inner| inner.flush()); ++ } +} + +impl console::interface::Read for PL011Uart { + fn read_char(&self) -> char { -+ self.inner.lock(|inner| { -+ // Spin while RX FIFO empty is set. -+ while inner.registers.FR.matches_all(FR::RXFE::SET) { -+ cpu::nop(); -+ } -+ -+ // Read one character. -+ let mut ret = inner.registers.DR.get() as u8 as char; -+ -+ // Convert carrige return to newline. -+ if ret == '\r' { -+ ret = '\n' -+ } -+ -+ // Update statistics. -+ inner.chars_read += 1; ++ self.inner ++ .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) ++ } + -+ ret -+ }) ++ fn clear_rx(&self) { ++ // Read from the RX FIFO until it is indicating empty. ++ while self ++ .inner ++ .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) ++ .is_some() ++ {} + } +} + @@ -1096,7 +1169,7 @@ diff -uNr 05_safe_globals/src/bsp.rs 06_drivers_gpio_uart/src/bsp.rs diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs --- 05_safe_globals/src/console.rs +++ 06_drivers_gpio_uart/src/console.rs -@@ -14,18 +14,34 @@ +@@ -14,8 +14,26 @@ /// Console write functions. pub trait Write { @@ -1105,19 +1178,25 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs + /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - } - ++ ++ /// Block execution until the last buffered character has been physically put on the TX ++ /// wire. ++ fn flush(&self); ++ } ++ + /// Console read functions. + pub trait Read { + /// Read a single character. + fn read_char(&self) -> char { + ' ' + } -+ } + ++ /// Clear RX buffers, if any. ++ fn clear_rx(&self); + } + /// Console statistics. - pub trait Statistics { - /// Return the number of characters written. +@@ -24,8 +42,13 @@ fn chars_written(&self) -> usize { 0 } @@ -1216,7 +1295,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -115,16 +125,46 @@ +@@ -115,16 +125,49 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1240,6 +1319,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs - println!("[0] Hello from pure Rust!"); +/// The main function running after the early init. +fn kernel_main() -> ! { ++ use bsp::console::console; + use console::interface::All; + use driver::interface::DriverManager; + @@ -1263,6 +1343,8 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs - println!("[2] Stopping here."); - cpu::wait_forever() ++ // Discard any spurious received characters before going into echo mode. ++ console().clear_rx(); + loop { + let c = bsp::console::console().read_char(); + bsp::console::console().write_char(c); diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index dfc6be130..2d5d0b04a 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -40,7 +40,6 @@ pub unsafe fn _start() -> ! { pub use asm::nop; /// Spin for `n` cycles. -#[cfg(feature = "bsp_rpi3")] #[inline(always)] pub fn spin_for_cycles(n: usize) { for _ in 0..n { diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index fdd92f65e..39834723b 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,58 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + // + // Now make an educated guess for a good delay value. According to Wikipedia, the fastest + // RPi4 clocks around 1.5 GHz. + // + // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 + // ns would the CPU be clocked at 2 GHz. + const CHAR_TIME_SAFE: usize = 24_000; + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + cpu::spin_for_cycles(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +306,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -270,29 +347,26 @@ impl console::interface::Write for PL011Uart { // readability. self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) } + + fn flush(&self) { + // Spin until TX FIFO empty is set. + self.inner.lock(|inner| inner.flush()); + } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) + } - ret - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/06_drivers_gpio_uart/src/console.rs b/06_drivers_gpio_uart/src/console.rs index 0a2385947..3552823cc 100644 --- a/06_drivers_gpio_uart/src/console.rs +++ b/06_drivers_gpio_uart/src/console.rs @@ -19,6 +19,10 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; + + /// Block execution until the last buffered character has been physically put on the TX + /// wire. + fn flush(&self); } /// Console read functions. @@ -27,6 +31,9 @@ pub mod interface { fn read_char(&self) -> char { ' ' } + + /// Clear RX buffers, if any. + fn clear_rx(&self); } /// Console statistics. diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index e0bf47a93..1674b9e42 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -143,6 +143,7 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; use driver::interface::DriverManager; @@ -163,6 +164,8 @@ fn kernel_main() -> ! { ); println!("[3] Echoing input now"); + // Discard any spurious received characters before going into echo mode. + console().clear_rx(); loop { let c = bsp::console::console().read_char(); bsp::console::console().write_char(c); diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index ef90e02ee..fe6251109 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -197,7 +197,7 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ } else { // If not core0, infinitely wait for events. wait_forever() -@@ -55,3 +55,19 @@ +@@ -54,3 +54,19 @@ asm::wfe() } } @@ -218,55 +218,62 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ + core::intrinsics::unreachable() +} +diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +@@ -144,7 +144,7 @@ + // Make an educated guess for a good delay value (Sequence described in the BCM2837 + // peripherals PDF). + // +- // - According to Wikipedia, the fastest Pi3 clocks around 1.4 GHz. ++ // - According to Wikipedia, the fastest RPi4 clocks around 1.5 GHz. + // - The Linux 2837 GPIO driver waits 1 µs between the steps. + // + // So lets try to be on the safe side and default to 2000 cycles, which would equal 1 µs + diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -270,6 +270,15 @@ - // readability. - self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) +@@ -257,7 +257,7 @@ } -+ -+ fn flush(&self) { -+ // Spin until TX FIFO empty is set. -+ self.inner.lock(|inner| { -+ while !inner.registers.FR.matches_all(FR::TXFE::SET) { -+ cpu::nop(); -+ } -+ }); -+ } - } - impl console::interface::Read for PL011Uart { -@@ -280,18 +289,20 @@ - cpu::nop(); - } + /// Retrieve a character. +- fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { ++ fn read_char(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. +@@ -272,12 +272,7 @@ + } -- // Read one character. -- let mut ret = inner.registers.DR.get() as u8 as char; -- -- // Convert carrige return to newline. -- if ret == '\r' { -- ret = '\n' -- } + // Read one character. +- let mut ret = self.registers.DR.get() as u8 as char; - - // Update statistics. - inner.chars_read += 1; +- // Convert carrige return to newline. +- if ret == '\r' { +- ret = '\n' +- } ++ let ret = self.registers.DR.get() as u8 as char; + + // Update statistics. + self.chars_read += 1; +@@ -357,14 +352,14 @@ + impl console::interface::Read for PL011Uart { + fn read_char(&self) -> char { + self.inner +- .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) ++ .lock(|inner| inner.read_char(BlockingMode::Blocking).unwrap()) + } -- ret -+ // Read one character. -+ inner.registers.DR.get() as u8 as char -+ }) -+ } -+ -+ fn clear(&self) { -+ self.inner.lock(|inner| { -+ // Read from the RX FIFO until it is indicating empty. -+ while !inner.registers.FR.matches_all(FR::RXFE::SET) { -+ inner.registers.DR.get(); -+ } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner +- .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) ++ .lock(|inner| inner.read_char(BlockingMode::NonBlocking)) + .is_some() + {} } - } diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 07_uart_chainloader/src/bsp/raspberrypi/link.ld --- 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -376,31 +383,6 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader unsafe { range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); -diff -uNr 06_drivers_gpio_uart/src/console.rs 07_uart_chainloader/src/console.rs ---- 06_drivers_gpio_uart/src/console.rs -+++ 07_uart_chainloader/src/console.rs -@@ -19,6 +19,10 @@ - - /// Write a Rust format string. - fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; -+ -+ /// Block execution until the last character has been physically put on the TX wire -+ /// (draining TX buffers/FIFOs, if any). -+ fn flush(&self); - } - - /// Console read functions. -@@ -27,6 +31,9 @@ - fn read_char(&self) -> char { - ' ' - } -+ -+ /// Clear RX buffers, if any. -+ fn clear(&self); - } - - /// Console statistics. - diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs +++ 07_uart_chainloader/src/main.rs @@ -432,23 +414,13 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -143,28 +147,52 @@ - - /// The main function running after the early init. +@@ -145,29 +149,49 @@ fn kernel_main() -> ! { -+ use bsp::console::console; + use bsp::console::console; use console::interface::All; - use driver::interface::DriverManager; -- -- println!("[0] Booting on: {}", bsp::board_name()); -- println!("[1] Drivers loaded:"); -- for (i, driver) in bsp::driver::driver_manager() -- .all_device_drivers() -- .iter() -- .enumerate() -- { -- println!(" {}. {}", i + 1, driver.compatible()); +- println!("[0] Booting on: {}", bsp::board_name()); + println!(" __ __ _ _ _ _ "); + println!("| \\/ (_)_ _ (_) | ___ __ _ __| |"); + println!("| |\\/| | | ' \\| | |__/ _ \\/ _` / _` |"); @@ -458,22 +430,30 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + println!(); + println!("[ML] Requesting binary"); + console().flush(); -+ -+ // Clear the RX FIFOs, if any, of spurious received characters before starting with the loader -+ // protocol. -+ console().clear(); -+ -+ // Notify `Minipush` to send the binary. -+ for _ in 0..3 { -+ console().write_char(3 as char); - } + +- println!("[1] Drivers loaded:"); +- for (i, driver) in bsp::driver::driver_manager() +- .all_device_drivers() +- .iter() +- .enumerate() +- { +- println!(" {}. {}", i + 1, driver.compatible()); +- } ++ // Discard any spurious received characters before starting with the loader protocol. ++ console().clear_rx(); - println!( - "[2] Chars written: {}", - bsp::console::console().chars_written() - ); - println!("[3] Echoing input now"); -- ++ // Notify `Minipush` to send the binary. ++ for _ in 0..3 { ++ console().write_char(3 as char); ++ } + +- // Discard any spurious received characters before going into echo mode. +- console().clear_rx(); - loop { - let c = bsp::console::console().read_char(); - bsp::console::console().write_char(c); diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 34c3214cfbe8d37cc863e86b50c5d3630ce0304e..47a0ae1037b18fa72ffdd9262c664c64bfc89289 100755 GIT binary patch delta 1724 zcmcgsZ)jUp6hHU9{BPGJYtok&7n8J0)0b?PZfjCOme?7btqRLJoeW(dQV@i8A8{>7Nf>QE&q-`0aHnq)~*5(q?mgo}0Xh%fynF8N z{Q13e&p8W+uJzCCA*(d7LO%HP=P2=RnP8Rx%xgaa4d;CHrU-P7g@rD1xSSDu z7sS(RHb~d)a=J1F1Ib`W{GOjz^;E+C#dMa-iWxQ+b$uk*2bC9~nC1HGVz3mqF9U z>;P>6m}$k}wIe{s`W~X^NPw-h<;fMMIMYHgz{Z@N!cr%D&p8-7Cea!?)2tSirzae` zpK54UjhfV>$hj6Zt9Ep!%mvtOr%XO!Ue`VcjnHG1|cPYW5 z3;MZ@7dQHx=|MlrT#$a=+1=2&>Wav@Ht6>l*zBbk=EYO0wnII}{&a;p23_>lWuUW{ zLbR$kiE!s}ci62tAbAQMYGQloHX6Lno^|^K1YdJU?#tOAdB%bdh@O@8Am!1AP?;NF zbt_WZ0!fbj!zv$YFxqkl3Kvb#Z$r%fOs3AiF?{$Mgpw(et~m0U`j8o@BX6$r`xo_S zVu%#8W&lr|U?Dt^R>u$@vp_wL_}3(qjH6vE+KmNZ_c_E0{cv`eO#55W$3E0~e5W_T z3yPn=(R)uNf75k|8xQY98^9fu0f~sXO}HV{O-GA+Y2IASz6}wqU_!Tc^z$JIea8t; zn~GWciA+6aMuJ|?!I&8<@fKF-3AVQT(Ly**72GLe`^8RRgnJemUKN^0%fSVO?)j_f zt&u*i`wnb#{Z*`YqBk*pncUbp2z3o3Phx6qKYcU-HuwySeP#m-X{b-4Et@H-7X($q z0-r+PwO*pK$@T{e;#=7uMOJ7Nd><-3n4(Fe&-nQo_L(8XD~)ehKfVL?|!JX zv=y_B=i7z$xnefvhDvP|Xxs}G-q$^-%lRKgJ7qCSYf0pKnIIpN>fh+XZtQAyV-+j= z<}y$d{z%cv5}xD#C%qm{Z#_aaWK*+3ozp9Ddi*7A*zMIqliunb{rtkVe&|1ypmjGZ zAs40mHy$x6f$I@0O#r1a&orD(d_M8n1~{umc67rAE;CG{*G-QjR|C}KkY()tA*^=~p_+umPbpxL_ zqZj$~@?vk72rpuBvtx2jf*RIWqXwr^g*N1{}xCbBBG7zUzjK0Y* zX;ql?k2#!5MSQ3$sJ9>}QCRjcTlz3XoGZ8@4CHohl4Y9@K6$6z^*i_84Yvou1BY|J z@B4k<@4LVA=i9-x{ng#%9F3kQvtM3I5&4o6774%?+OI$p`5?XG0QwP22uU(fD+pnd zfe>AD9NqFjURR+~+;+rPWBZ`a?O4HDTY90c61M6X8uVbPcAUJVhotCnXQkNO0QxYd z&XFFPSWZz5DU32v8!3m{+zaYWCOQuY>!DiCIY9){!`>5z1S!bQi)RJGmuPL-HB17S zo*4kRa)&Z2(o?Nb?}jm$o0LbK4Pi zxpRsvuqV8EVX}*rz1_m;I9u_K_U)5t6N_nXm&*H2z;3ExS|jCD$FT2aN>p3WpPBAr zUY|l#mhnB(T1rwsa=d;z&5rsatt3jf`eiO5d&`%Jt$AS(bE+~HUel4H| zi22^vt1NfGV8D(o&ZJS7z-q$}dIxKf;=w<>^!5tS#hE>{VOWju#GI6n;;oQ<4hw1y z|LR|u@FpusVF81)Qg7F?8?rCB@;YbFtVbzlA4jmsDrG_y5wbjXud9(gW=CziqGgnn z?_l*1sT5mrHAR#Q`n-~_&qI7D7I$3Y-#-|s>^#ZWC!z)Y^C;}PAOihn5K51d`N%|6 z)E5x50BIA?zb_bj5fAkfNp2dfSwb>al*p~oh>>6BP0 zzO8lRsU&pgjwp^fk=|UvU|(`e0*-h*|t z5yM*Tu&y@cUO%nc-Ce3NqleFmu-6%WN>q&>oHT2+p zYLrFw$_CKic8n!WH}>Nun0qjCZxhyftWC)GQVyK5H-Q5Q{x@h3na_iU&jNoM_)`J+ zG?dt;>?bGLHFNM~B(dp->_jvb;eXJ8av6 zZHL*rpsyc=-85 zJYw}RX7i diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index c7919ed6517de38049e1f9fd7eb05966c58a4063..14e578c1784483707bb257c94ea90a0cb9d7cb97 100755 GIT binary patch delta 1870 zcmb_cTWlLe6uq-+JG*%~#;-W3V>=HV+msN}#JohDRzYeCYOpH!=tmL?f;K``-H(8T zWJ5uara~D_qzWOCKBS@zA5aA(tX7aTd^CV)DEigs^J_z-DXzmiw1rg}mlU+iH^m^$u$t0)5 zzBvgJs_aW-hvB|>ASiyz`?FeShy3wKl8+-+xH7G2I|8kc<=8;CVRu6&DK{B?SzvF9 z9ZjW}D_0^a`5B-QboYcBX%2JFi^7DKCQ3dGN`a~3{lq)jX4y_iBRgP?6AwFOJ*EHs zlVi3Ah_jJK2D~&M{zS=@5QRthX>z{gH<5TI&y{T^!hZXtv*HBt?7Nqi0gRSo+QJr~ zgO5K%50P4Su5y4JWUcnDa4kKCG{lO8+7E#_ZHsWB_#mi%ZI=|lH82&VBm?vZ_LluI zhYJ&0lsqQGZH)FDtz}F0fPDkaVdTP!b~p&ZZ{BB3Rgp*ooyFSm&2Y2_(Q^y9`g=if zVt@acW-6hTLVh+>)sS8-J)tW_?Qo_F+OLYB>WHI$y-aRIe^lS7=hxC!w>bRX@4v|RYjK%0A2R< zfubG6}xUIJ4+gr4T< zC5S&~OQ?$&`KlLG87V~O{j`TudRJ5mnnlsRwkkTsIH;>z$sr5Ah4(NOlyp>l%>cvIuBVH=ht2c?EmWv;e%{oBbJ{BuD~?{S_j%nv~I{o`q3gL zzYAy{g`wYp=_b0B3n$FZIIFn^juW~t1JuV$X@mAYJQ58{e=FO9Mie{@^?xUu{x7yA zCfoT2{l@0QqN20M2Tn`ltAV$6P38?KV%)$l{LNU_Hr=L~T%)0x{w5v6HS@Ho8N3=3 z?6gaA4i~x566%m{QE(7u-??6{xmkiKZ)up&Xc9}gzv_{Ex4XN%#u(+Jx9(JGZwh-c za~|ITNwfw=V5Pb8Iv5Kbm8c zt^ng~tM}QO<{E=WubJ*1bNoUz``lYs+Uqvl{ciTB_YwIeyD_}Tx~k*kBzv*?$=21m eBjyGiWG$Lc5{B55#L@xs$nVL6MqA{Zas?t delta 1698 zcmZ8hZ%kWN6hF7GrGFT-(3VpE?De%_T~V?iW6g|lWx^~Z&ZHrZWQjobL8dcAO#Ech z;-8E;SbDugK4~^JhR3o5tr{AQ1M$Q7AYvRCW0n;7QjtX+VdD+qdG33!#Br1MoO{o| z-~F9)->sv!n$k7o>(6VQChw#fMhU>A)Z?MKRiG;Z&<`>#CW$=mr86Xzm=?O1Y!Fk# zM#yTiO_wsH681VnC+rL1ShEfD9KavXXOrsad2=eUHVgDIbRCw0RGO}%In1vniJT7$ z^4bxQ*I9}AIPqqlH9t>Ckd0ZQ#KV5Fe53u>X2y%2B<>)M^m=JN{GFV$5}9YXo!PYB z6^JLZT6~h2R!cMClFvwm^aL#iNOK5l&j5XeJVl3zpRJbml780aY(f2^Q?wqWBn5Pd z-EclvR*q@SavoDUFskF4pB1}AXBEw1{MwE>7J$HQnH_gUA`+cLHu7Q6yeqKI(h5fZ zZj#-ozyAFpiq}uuq-u8A6-pM!By?w29h-JS^M(M5=5ei>MvxFMdVduRYW3BmMczyn z$w*w9H)0#{YzagoYX#*x=D1n{&4)2A?*_$#exn~BLc3>T17I|F(Ih`m)xRH0RBFKF_iE5VBzbtZ`>rqe&h4l#_bO8$n91uN=_H8Vf z6WkMTvca;3h=<-^1!GbO{gwl>LbR_arrgJx8$M9%2rjZ*&3-KtISq({U*S4>rYsQM zqoKD(qk+@E|N6YcJmw>eL7mT4x*f^K7NxZY|JJIE_+!}-2pkgw!XnpyRKw8`63gDG zh4P!t@LB^IsXCopZdK+jKu<55l`H6U1rBQgT%T8uU;tz<+-X&Y(dWV)vvR;l{OV3P zdRJ9_fCV8CqyoxwsQ@*jd-odK873=(OVHItyTiHGDe8*7bqvn|keQz?{+6D&t2T;Fmoa2{6 zTl@X4RxpR}>Ewq#J=|ruiGF=wsCvn;UTMu{YqF zKxA{im#W5%vI~ZJ(%5hLSV!eS$AU*kmGiKH%2W0}r@rlCGnG-&$dt-6jfJy-5m{yw Z7zk***UY{{esD4Q&7b diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/07_uart_chainloader/src/_arch/aarch64/cpu.rs index 1895b6f16..184347d78 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -40,7 +40,6 @@ pub unsafe fn _start() -> ! { pub use asm::nop; /// Spin for `n` cycles. -#[cfg(feature = "bsp_rpi3")] #[inline(always)] pub fn spin_for_cycles(n: usize) { for _ in 0..n { diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 771151e76..9d41900d7 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -144,7 +144,7 @@ impl GPIOInner { // Make an educated guess for a good delay value (Sequence described in the BCM2837 // peripherals PDF). // - // - According to Wikipedia, the fastest Pi3 clocks around 1.4 GHz. + // - According to Wikipedia, the fastest RPi4 clocks around 1.5 GHz. // - The Linux 2837 GPIO driver waits 1 µs between the steps. // // So lets try to be on the safe side and default to 2000 cycles, which would equal 1 µs diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 1678cf00f..9a92c9d21 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,53 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + // + // Now make an educated guess for a good delay value. According to Wikipedia, the fastest + // RPi4 clocks around 1.5 GHz. + // + // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 + // ns would the CPU be clocked at 2 GHz. + const CHAR_TIME_SAFE: usize = 24_000; + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + cpu::spin_for_cycles(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let ret = self.registers.DR.get() as u8 as char; + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +301,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,37 +345,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Update statistics. - inner.chars_read += 1; - - // Read one character. - inner.registers.DR.get() as u8 as char - }) + self.inner + .lock(|inner| inner.read_char(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/07_uart_chainloader/src/console.rs b/07_uart_chainloader/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/07_uart_chainloader/src/console.rs +++ b/07_uart_chainloader/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index 2e436374c..c8f2a91f6 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -160,9 +160,8 @@ fn kernel_main() -> ! { println!("[ML] Requesting binary"); console().flush(); - // Clear the RX FIFOs, if any, of spurious received characters before starting with the loader - // protocol. - console().clear(); + // Discard any spurious received characters before starting with the loader protocol. + console().clear_rx(); // Notify `Minipush` to send the binary. for _ in 0..3 { diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 45a5c270e..111b1168c 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -129,12 +129,11 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a } else { // If not core0, infinitely wait for events. wait_forever() -@@ -39,15 +39,6 @@ +@@ -39,14 +39,6 @@ pub use asm::nop; -/// Spin for `n` cycles. --#[cfg(feature = "bsp_rpi3")] -#[inline(always)] -pub fn spin_for_cycles(n: usize) { - for _ in 0..n { @@ -145,7 +144,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -55,19 +46,3 @@ +@@ -54,19 +46,3 @@ asm::wfe() } } @@ -283,7 +282,7 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 08_times - // Make an educated guess for a good delay value (Sequence described in the BCM2837 - // peripherals PDF). - // -- // - According to Wikipedia, the fastest Pi3 clocks around 1.4 GHz. +- // - According to Wikipedia, the fastest RPi4 clocks around 1.5 GHz. - // - The Linux 2837 GPIO driver waits 1 µs between the steps. - // - // So lets try to be on the safe side and default to 2000 cycles, which would equal 1 µs @@ -308,27 +307,72 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 08_times diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -289,11 +289,18 @@ - cpu::nop(); - } +@@ -235,16 +235,13 @@ -+ // Read one character. -+ let mut ret = inner.registers.DR.get() as u8 as char; -+ -+ // Convert carrige return to newline. -+ if ret == '\r' { -+ ret = '\n' -+ } + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { ++ use crate::{time, time::interface::TimeManager}; ++ use core::time::Duration; + - // Update statistics. - inner.chars_read += 1; + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. +- // +- // Now make an educated guess for a good delay value. According to Wikipedia, the fastest +- // RPi4 clocks around 1.5 GHz. +- // +- // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 +- // ns would the CPU be clocked at 2 GHz. +- const CHAR_TIME_SAFE: usize = 24_000; ++ const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { +@@ -253,11 +250,11 @@ + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. +- cpu::spin_for_cycles(CHAR_TIME_SAFE); ++ time::time_manager().spin_for(CHAR_TIME_SAFE); + } -- // Read one character. -- inner.registers.DR.get() as u8 as char -+ ret - }) + /// Retrieve a character. +- fn read_char(&mut self, blocking_mode: BlockingMode) -> Option { ++ fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. +@@ -272,7 +269,12 @@ + } + + // Read one character. +- let ret = self.registers.DR.get() as u8 as char; ++ let mut ret = self.registers.DR.get() as u8 as char; ++ ++ // Convert carrige return to newline. ++ if ret == '\r' { ++ ret = '\n' ++ } + + // Update statistics. + self.chars_read += 1; +@@ -352,14 +354,14 @@ + impl console::interface::Read for PL011Uart { + fn read_char(&self) -> char { + self.inner +- .lock(|inner| inner.read_char(BlockingMode::Blocking).unwrap()) ++ .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner +- .lock(|inner| inner.read_char(BlockingMode::NonBlocking)) ++ .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} + } diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/link.ld 08_timestamps/src/bsp/raspberrypi/link.ld --- 07_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -484,7 +528,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs /// Early init code. /// -@@ -147,52 +146,31 @@ +@@ -147,51 +146,31 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -504,9 +548,8 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs - println!("[ML] Requesting binary"); - console().flush(); - -- // Clear the RX FIFOs, if any, of spurious received characters before starting with the loader -- // protocol. -- console().clear(); +- // Discard any spurious received characters before starting with the loader protocol. +- console().clear_rx(); - - // Notify `Minipush` to send the binary. - for _ in 0..3 { diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/08_timestamps/src/console.rs b/08_timestamps/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/08_timestamps/src/console.rs +++ b/08_timestamps/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/09_hw_debug_JTAG/src/console.rs b/09_hw_debug_JTAG/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/09_hw_debug_JTAG/src/console.rs +++ b/09_hw_debug_JTAG/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 3ffa1bbe4..8cd313810 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -464,10 +464,11 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -146,12 +147,19 @@ +@@ -146,12 +147,20 @@ /// The main function running after the early init. fn kernel_main() -> ! { ++ use bsp::console::console; + use console::interface::All; use core::time::Duration; use driver::interface::DriverManager; @@ -484,7 +485,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -166,11 +174,12 @@ +@@ -166,11 +175,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -492,8 +493,11 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs - time::time_manager().spin_for(Duration::from_nanos(1)); + info!("Timer test, spinning for 1 second"); + time::time_manager().spin_for(Duration::from_secs(1)); - ++ + info!("Echoing input now"); + ++ // Discard any spurious received characters before going into echo mode. ++ console().clear_rx(); loop { - info!("Spinning for 1 second"); - time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/10_privilege_level/src/console.rs b/10_privilege_level/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/10_privilege_level/src/console.rs +++ b/10_privilege_level/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index 0c730ff96..fc2bd6912 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -147,6 +147,7 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; use core::time::Duration; use driver::interface::DriverManager; @@ -178,6 +179,9 @@ fn kernel_main() -> ! { time::time_manager().spin_for(Duration::from_secs(1)); info!("Echoing input now"); + + // Discard any spurious received characters before going into echo mode. + console().clear_rx(); loop { let c = bsp::console::console().read_char(); bsp::console::console().write_char(c); diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 6216fd65e..c64442967 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -914,7 +914,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -154,6 +168,9 @@ +@@ -155,6 +169,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -924,7 +924,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -177,6 +194,13 @@ +@@ -178,6 +195,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -936,8 +936,8 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s + .unwrap(); + info!("Echoing input now"); - loop { - let c = bsp::console::console().read_char(); + + // Discard any spurious received characters before going into echo mode. diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs --- 10_privilege_level/src/memory/mmu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/11_virtual_mem_part1_identity_mapping/src/console.rs b/11_virtual_mem_part1_identity_mapping/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/11_virtual_mem_part1_identity_mapping/src/console.rs +++ b/11_virtual_mem_part1_identity_mapping/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 41ca22e33..259715354 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -161,6 +161,7 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; use core::time::Duration; use driver::interface::DriverManager; @@ -202,6 +203,9 @@ fn kernel_main() -> ! { .unwrap(); info!("Echoing input now"); + + // Discard any spurious received characters before going into echo mode. + console().clear_rx(); loop { let c = bsp::console::console().read_char(); bsp::console::console().write_char(c); diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index c6264088f..be16673f5 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -973,7 +973,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } -@@ -194,13 +197,28 @@ +@@ -195,13 +198,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -1006,8 +1006,8 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ + // Will never reach here in this tutorial. info!("Echoing input now"); - loop { - let c = bsp::console::console().read_char(); + + // Discard any spurious received characters before going into echo mode. diff -uNr 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs 12_exceptions_part1_groundwork/src/memory/mmu.rs --- 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/12_exceptions_part1_groundwork/src/console.rs b/12_exceptions_part1_groundwork/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/12_exceptions_part1_groundwork/src/console.rs +++ b/12_exceptions_part1_groundwork/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 495840e87..a2cf77527 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -164,6 +164,7 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; use core::time::Duration; use driver::interface::DriverManager; @@ -220,6 +221,9 @@ fn kernel_main() -> ! { // Will never reach here in this tutorial. info!("Echoing input now"); + + // Discard any spurious received characters before going into echo mode. + console().clear_rx(); loop { let c = bsp::console::console().read_char(); bsp::console::console().write_char(c); diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index a282e6593..849cbbfb2 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1444,9 +1444,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -165,9 +49,7 @@ - /// The main function running after the early init. +@@ -166,9 +50,7 @@ fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; - use core::time::Duration; use driver::interface::DriverManager; @@ -1454,7 +1454,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -194,31 +76,6 @@ +@@ -195,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1484,8 +1484,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m - - // Will never reach here in this tutorial. info!("Echoing input now"); - loop { - let c = bsp::console::console().read_char(); + + // Discard any spurious received characters before going into echo mode. diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing/src/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/memory/mmu.rs diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/13_integrated_testing/src/console.rs b/13_integrated_testing/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/13_integrated_testing/src/console.rs +++ b/13_integrated_testing/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/13_integrated_testing/src/main.rs b/13_integrated_testing/src/main.rs index 989bd045c..769f9e8f0 100644 --- a/13_integrated_testing/src/main.rs +++ b/13_integrated_testing/src/main.rs @@ -48,6 +48,7 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use bsp::console::console; use console::interface::All; use driver::interface::DriverManager; @@ -77,6 +78,9 @@ fn kernel_main() -> ! { } info!("Echoing input now"); + + // Discard any spurious received characters before going into echo mode. + console().clear_rx(); loop { let c = bsp::console::console().read_char(); bsp::console::console().write_char(c); diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index dc18b75f4..8e68edcb8 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -1862,20 +1862,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (0x44 => ICR: WriteOnly), (0x48 => @END), } -@@ -136,6 +181,12 @@ - /// Abstraction for the associated MMIO registers. - type Registers = MMIODerefWrapper; - -+#[derive(PartialEq)] -+enum BlockingMode { -+ Blocking, -+ NonBlocking, -+} -+ - //-------------------------------------------------------------------------------------------------- - // Public Definitions - //-------------------------------------------------------------------------------------------------- -@@ -151,7 +202,8 @@ +@@ -157,7 +202,8 @@ /// Representation of the UART. pub struct PL011Uart { @@ -1885,59 +1872,22 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } //-------------------------------------------------------------------------------------------------- -@@ -192,6 +244,10 @@ - self.registers +@@ -214,6 +260,14 @@ .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on -+ self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); // RX FIFO fill level at 1/8 + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + ++ // Set RX FIFO fill level at 1/8. ++ self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); ++ ++ // Enable RX IRQ + RX timeout IRQ. + self.registers + .IMSC -+ .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); // RX IRQ + RX timeout IRQ ++ .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); ++ + // Turn the UART on. self.registers .CR - .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); -@@ -209,6 +265,35 @@ - - self.chars_written += 1; - } -+ -+ /// Retrieve a character. -+ fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { -+ // If RX FIFO is empty, -+ if self.registers.FR.matches_all(FR::RXFE::SET) { -+ // immediately return in non-blocking mode. -+ if blocking_mode == BlockingMode::NonBlocking { -+ return None; -+ } -+ -+ // Otherwise, wait until a char was received. -+ while self.registers.FR.matches_all(FR::RXFE::SET) { -+ cpu::nop(); -+ } -+ } -+ -+ // Read one character. -+ let mut ret = self.registers.DR.get() as u8 as char; -+ -+ // Convert carrige return to newline. -+ if ret == '\r' { -+ ret = '\n' -+ } -+ -+ // Update statistics. -+ self.chars_read += 1; -+ -+ Some(ret) -+ } - } - - /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are -@@ -231,12 +316,18 @@ - } - - impl PL011Uart { -+ /// Create an instance. -+ /// +@@ -308,9 +362,13 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -1953,7 +1903,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } } } -@@ -256,6 +347,21 @@ +@@ -330,6 +388,21 @@ Ok(()) } @@ -1975,35 +1925,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } impl console::interface::Write for PL011Uart { -@@ -283,25 +389,8 @@ - - impl console::interface::Read for PL011Uart { - fn read_char(&self) -> char { -- self.inner.lock(|inner| { -- // Spin while RX FIFO empty is set. -- while inner.registers.FR.matches_all(FR::RXFE::SET) { -- cpu::nop(); -- } -- -- // Read one character. -- let mut ret = inner.registers.DR.get() as u8 as char; -- -- // Convert carrige return to newline. -- if ret == '\r' { -- ret = '\n' -- } -- -- // Update statistics. -- inner.chars_read += 1; -- -- ret -- }) -+ self.inner -+ .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) - } - - fn clear(&self) { -@@ -323,3 +412,24 @@ +@@ -376,3 +449,24 @@ self.inner.lock(|inner| inner.chars_read) } } @@ -2439,7 +2361,7 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; -@@ -42,14 +42,27 @@ +@@ -42,15 +42,27 @@ bsp::driver::driver_manager().post_device_driver_init(); // println! is usable from here on. @@ -2462,24 +2384,28 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ /// The main function running after the early init. fn kernel_main() -> ! { +- use bsp::console::console; - use console::interface::All; use driver::interface::DriverManager; + use exception::asynchronous::interface::IRQManager; info!("Booting on: {}", bsp::board_name()); -@@ -76,9 +89,9 @@ +@@ -77,12 +89,9 @@ info!(" {}. {}", i + 1, driver.compatible()); } +- info!("Echoing input now"); + info!("Registered IRQ handlers:"); + bsp::exception::asynchronous::irq_manager().print_handler(); -+ - info!("Echoing input now"); + +- // Discard any spurious received characters before going into echo mode. +- console().clear_rx(); - loop { - let c = bsp::console::console().read_char(); - bsp::console::console().write_char(c); - } ++ info!("Echoing input now"); + cpu::wait_forever(); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 9504cfc31..7b3a85cf0 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -226,28 +226,49 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on - self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); // RX FIFO fill level at 1/8 + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Set RX FIFO fill level at 1/8. + self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); + + // Enable RX IRQ + RX timeout IRQ. self.registers .IMSC - .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); // RX IRQ + RX timeout IRQ + .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -266,6 +287,26 @@ impl PL011UartInner { self.chars_written += 1; } + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + /// Retrieve a character. fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { // If RX FIFO is empty, @@ -379,11 +420,7 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } @@ -393,13 +430,13 @@ impl console::interface::Read for PL011Uart { .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/console.rs b/14_exceptions_part2_peripheral_IRQs/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/console.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index fb574abf4..acb48ded6 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1305,9 +1305,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ irq_number: bsp::device_driver::IRQNumber, } @@ -234,7 +239,15 @@ - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 modulo error margin is acceptable for UART and we're - /// now at 0.01 modulo. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5modulo error margin is acceptable for UART and we're + /// now at 0.02modulo. - pub fn init(&mut self) { + /// + /// # Safety @@ -1318,10 +1318,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + self.registers = Registers::new(addr); + } + - // Turn it off temporarily. - self.registers.CR.set(0); - -@@ -251,6 +264,8 @@ + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. +@@ -272,6 +285,8 @@ self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -1330,7 +1330,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } /// Send a character. -@@ -320,13 +335,18 @@ +@@ -361,13 +376,18 @@ /// /// # Safety /// @@ -1352,7 +1352,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ irq_number, } } -@@ -343,7 +363,14 @@ +@@ -384,7 +404,14 @@ } unsafe fn init(&self) -> Result<(), &'static str> { @@ -1368,7 +1368,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ Ok(()) } -@@ -362,6 +389,16 @@ +@@ -403,6 +430,16 @@ Ok(()) } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index b2fc254bd..e404f11bf 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -231,14 +231,14 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. /// /// # Safety /// @@ -248,19 +248,40 @@ impl PL011UartInner { self.registers = Registers::new(addr); } - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on - self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); // RX FIFO fill level at 1/8 + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Set RX FIFO fill level at 1/8. + self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); + + // Enable RX IRQ + RX timeout IRQ. self.registers .IMSC - .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); // RX IRQ + RX timeout IRQ + .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -281,6 +302,26 @@ impl PL011UartInner { self.chars_written += 1; } + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + /// Retrieve a character. fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { // If RX FIFO is empty, @@ -416,11 +457,7 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } @@ -430,13 +467,13 @@ impl console::interface::Read for PL011Uart { .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/15_virtual_mem_part2_mmio_remap/src/console.rs b/15_virtual_mem_part2_mmio_remap/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/15_virtual_mem_part2_mmio_remap/src/console.rs +++ b/15_virtual_mem_part2_mmio_remap/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 443facda4297a662552100554a1bd6de4427e137..4248095748295740449cb7c013902e8a4ea778fb 100755 GIT binary patch delta 2340 zcmb7_YfKbZ6vxlZukJGLoMlNLf^t*A+peps-HyQpQWrn+fidhYCysuskSupL85e+jPIGu0*W6phCvz@) z4jB+OU!Sh-S46M}?+MT)rP$SGSXOM!6t1!QuvWGde%og4oo0uB%CE}q9SNUf;%l>eZ$_4L z{N9+}f~e29#6^oqBtY9NB%i1NvMsKZv}VclX7m=^g~o&(qTZOxyui~hV#|t7$mAp< z2~qZ74Op*h7x|y3*F(@JK)&QHxt8BJ)UF8;w=zZ8AYxm_;+ahgExB)Ya>TJCmh`lK z-H;~@9g}c7xfaOpnijZIU{>C}(Cl=~I@xfkwv_nI5R|5>M&-tl_Q_tJ1nM~bi1O5$ z(QR1;Brj`L>h{a+&dCx{*+=Ql-Q%$V$Il_g=ZWCE47t%pEe zp=pQpF*u^zNk>5~Hu~7O72Ef#S1Hrx@`o&@SP^k;b{)k*R0s?6wS z3vKGz0Wb+4p95+HCg9^QfJa1Xvo~@x1##GJ4?}> zCWXBkep{I0t3V1I_vMBwOybHZyf9`c+KexRrfErxn87h(9SC7(ZgmLEg;K;rkdZ!0 z=y-x~^>YxGKArH{fG#A=V}y|gVMmF6kr^c;6p>oQhsWm%5ympY;8U{1i4~zNky)x= zw*AZwgImnDGiAEGf{5>;)Fn-8Nr3O^gDEZPdw@Q=HaU}SPu{?J=wNaZ*F%3v{weNZ z25$Svr&2Yg7%l!tDK4x`3yool(!XxQ7Tu_SPtM(!M&Fo|QT~6YtueO2w61@f(t0n< zd7Vp%#n+B~L&kWr(GJk8G^^B;7#jVSR;9ffkq{SZtxc!L(&o+WOb;~^^#uS_qm5Zd z|Bj^7@wAnTJOW9H8C&NM8N^mF*aus07Uy`w?zVuX&le?r8FvdEN(NDs{svhhosik_anKY$zfooJ~A=pE7q8W^RB`NxL<*JnlD5nQ+NS7b4SXZRCf- zxY4x`nW_;cjV@hG1r{daTw7v4!pZH0M2k>o3^4IkBGS@ON;e;*prnh#l9EpL5Yy{EUW zRW~v}t+_HEJNAavV$(W1B=`Ja$7q{hRp(}UFliZIMylj54o3qB%j5!TpUla-yELKfeY+pJ7sJ*kJJD<#zOU$xZJ<$o;R^`O%cb(X0eLUuaHqPdH zxy*K7bXp&>4LV!e*OA?geQndHF+0DwW1!_m%#cIN$)?6 zi8~vSsYXOhi)|CW^(f$XiOf5M6wk9r8CbHbT}LFeZs|#LKs_x4lU!hvoHDSk0>*V% z_D_K#9EcANYf((ZKY&;?sIwAgvy4VsY_f9(n8tuA*MQ}DP~j-3(k!1n%$1p?pbV&R z?IdQL7*}Wp6yXGC?Lz`v2AjvjEHK6BrvdHkcqW+{0+g#zKn8y`Fprh9h@`~u8eOdQuagKd~X!v3VAqtP-O0t zNM?6K-h4GEFa=WidL*`U2^3jS*CJRb!i<24>WfIDdo^j$IIOSN7m}$SflNVMCd&Lq zJNEg3>OEKI>#)}E#UZ86a;3WORLb|edYtbm@cqnr#3(wf6=QjEB<|19Momy;RrE96 zW_!&krh};c5e{q9iA@1nV^*q?64%HYq;T7C5Gfl~wJZ1yy>FEdA!S3@=HCopFV&3q z@e3Qew5XHIC~YhwCYa?|^2>nq7MfDh0B2RWOUd_ts9jo4i!Xp+I*?c(WIar;Bvw2M zS@DU8xT7K&SJmiPL)?`JE^Gwb41|4o8WvoI*AUe%FmEB|Y(&pT7 zedNn$4hI*&P2u%(Z6#vOP#+eki}S;!^juPg(7u-Xlgg5z>@AR7uO0;~$K&)~fB}i5 z<;f}bzD2dmjM2KfO!2a+s>^S&`bT539awQv|Cv*Oni|e2kHgT)(2YleH-fh<-txGb zswiAqs)=HG5L+LK>?0Joa~iszd)qwOaL*EqqPxH;npTLqt1n7=^ql91Ks036~IGugFaB?olI{rxFn_Gyt_n!=`}kR#48ZxuqN@ z!a;K)LBK%5qeFqaf!E&#e4G7VkpqYj@_z@O&w(3@h))YZ?HZ_}2qpm`34RYAdt?y! z0|31KX6TBR-El`Vi$izxNHP4vFzJOro_FbfGhf*67A44|+@G_=>%<(;Ezw_B6tW;zQQ4G+LaV znB2F?%#hY7_PH!$jgL1cn|{Lu{wZO277qcpM|-*YJrOw^Nn{l4!# z@AY|~^H!R7xJoKuP3AKWD?53U84fT2r=4z*L~Z~X ziwAc?$a9DHz=DcGk-Mgz`;o0Mp*{zt0=Lk=R+8C0C=Y7s+ z2-=^%JTQ$6olKGMo#f@FD`#*kdfxJWwC|ih6Yux1N})KzK2*lYP3QM`J!wb%sVLv*EFpIhiM!>T zjMb9c6(+eEt8d2Y@6EKxr!XP4@@N?(c+}_XU1UbbhoSvg`96Girzk}(K_vLm_5A{( z?-~uP?=Lx$Y9B62bw#n_+eN&5dl@7iBC&%ZgNV-Ch{Rzt%ms7`!3x6R-XP8*Cfs)( zo<#;5SW<5edd(ah5JCLG+Vjdo zHdnD(BE4jpLZ@0jU>ULdk|p2FI>qi=sq>i3`-<|*9~Xdht^tyJF~7Sk{e7k|hQcT| z-1T}DK_W=VDk+Zr+PH!z`v#`cTN!mG$<^zmnWlGWB=ZWBM$b%e2kMH+Q7o2AcPHwg z@Vc5a(n1opKbI&qyTf};L2pyer1DZTgn5(+(B&0wLRi&H>hjx>ifH4)UFE%i-9#KD z2rg6`Ce?_Qq#peodY~LsJO5C37u=tr8uTg3Dpb^Hp)AG$QVEIQmh-aar0yxg1cBT{ z5NSbHZ8m-f(di~xEmiwSs%O)!S&oT4vq=Q2{mzJSPE|jVRm+~S(~+!Vbx26skhlTr z_a+%bLb}{Lhn8oTvc2VWb@u!;ZK5HcIfksIN3tgcf*R+fBNrf1<9H96;B2%AG^k3CI7s9e$h#4G{>v}?!mo=+{5T^k3M&hYJN!1n zRg16(5X-QJ`2d7Rb^wXsD*jVBsP>3=K22)T!N@UAjwuMqV(dQNm1AZ(R-t{N(%&VngFD-sF}e z9N%TetaVZ>r1qS7lkp5Lm6)CX{B1l!H15HCwC7Epp3k{l9GUXy zYq>zbBMr4@iQnJir=RB)F^A|t-mHKjUSQYN^iTjH!%for<8HSY`bCEQ8ee1MI)B@y z`sU_!ew6)Re@??5Dh-Dd48bzvUP^0iA2S?XVYAVfY!d>a@jjX8%P)$riYr@vo5i(T z{Ppd=md&ErzM+1zSnqGx;A>yk(2nBf<~PK)Ev>CB{`PfiXNYYZeXVHGXnowWy`@EL zZrR*8P6Y~eV9x3J1b{aL5hK9|6Lkgs7a+O!*!0sCV^8Yk8U-Zvc3bq*A!ASK8ETO1 zz|UG7WR<&sUdWyH1T`9~aRrqhZ}jd0dUbNuoI^$V=nc z?m6e)^Sk$)oB33EtUMsCxULR)B;Se)&J%zuQA4M-b)dh>Krc#_loTSz0qv0X(Pd)f zm*lA%Dx_QCN1D`Me(Y0sRI10=H|{4S z;;(dep0`!BI zfd;F9)Bw-DW4Wv($_R>z&A3*x%IM7#T4$*Vb5L@q=c3nvjKkMvxpZ((fmPFWR zz3GSML-a#DgZyB+H4Wh#*0^+i`$=eRKC7B|{xU2@C^3MZ0XWYt^yS8T`e-BWGI=Hs z`X`q|I9JRp?$L}9(<(~lEt$|;G1El1P+w1yPF-L>W-5c%ObzR$3pla{;LDSf2B zX<5HdUOhx_uLDKm^0|6~cA>nM8DN+7q?8$CKkJVb?dlM$4xwkat+_OTv8jmsqh(os zn-bzw!c|WhaT=A4d`e`qeYF^s(Kl=CrY@9cA!hG0qZs)=aW78gR2RzH!HHH z8TpfX-@c@R6+~N6gN3|y6RJG09js=OlwW!RS|!)F_d%|TEned{?`?C~Yw0mA{2^gye_O5b_gT!@;J|K82uJ1Xc0zIo7dki4U~jNA z%GUg2-UN)o0oV_2z}CXC>1EzPD)feeH1`aA0roHyb~12naw~AZTk|_}Y+ZXy6bh#b zsQuw{>^@=N8Q1W!!a(ZamRRl}0NfGtcGjKg?h5~2b~JRET^M}4_*%$GbVPui7W30$ zzUg)R&M!b)_UVo`EMz+l|CsF2jyo|K z;s1*+2c1toERti4C=7-|akr4ZO==pyDWF@i? diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 46cceff92..876aa461d 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -136,6 +136,12 @@ register_structs! { /// Abstraction for the associated MMIO registers. type Registers = MMIODerefWrapper; +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -174,24 +180,41 @@ impl PL011UartInner { /// Set up baud rate and characteristics. /// - /// This results in 8N1 and 576000 baud (we set the clock to 48 MHz in config.txt). + /// This results in 8N1 and 921_600 baud. /// - /// The calculation for the BRD given a target rate of 576000 and a clock set to 48 MHz is: - /// `(48_000_000 / 16) / 576000 = 5.2083`. `5` goes to the `IBRD` (integer field). + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2083 * 64 = 13.3 rounded to 13` will - /// give the best approximation we can get. A 5 % error margin is acceptable for UART and we're - /// now at 0.01 %. + /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will + /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're + /// now at 0.02%. pub fn init(&mut self) { - // Turn it off temporarily. + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. self.registers.CR.set(0); + // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - self.registers.IBRD.write(IBRD::IBRD.val(5)); - self.registers.FBRD.write(FBRD::FBRD.val(13)); + + // Set the baud rate. + self.registers.IBRD.write(IBRD::IBRD.val(3)); + self.registers.FBRD.write(FBRD::FBRD.val(16)); + + // Set 8N1 + FIFO on. self.registers .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); // 8N1 + Fifo on + .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + + // Turn the UART on. self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -209,6 +232,55 @@ impl PL011UartInner { self.chars_written += 1; } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per + // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = + // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. + const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); + + // Spin until TX FIFO empty is set. + while !self.registers.FR.matches_all(FR::TXFE::SET) { + cpu::nop(); + } + + // After the last character has been queued for transmission, wait for the time of one + // character + some extra time for safety. + time::time_manager().spin_for(CHAR_TIME_SAFE); + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } } /// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are @@ -231,6 +303,8 @@ impl fmt::Write for PL011UartInner { } impl PL011Uart { + /// Create an instance. + /// /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -273,44 +347,23 @@ impl console::interface::Write for PL011Uart { fn flush(&self) { // Spin until TX FIFO empty is set. - self.inner.lock(|inner| { - while !inner.registers.FR.matches_all(FR::TXFE::SET) { - cpu::nop(); - } - }); + self.inner.lock(|inner| inner.flush()); } } impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { - self.inner.lock(|inner| { - // Spin while RX FIFO empty is set. - while inner.registers.FR.matches_all(FR::RXFE::SET) { - cpu::nop(); - } - - // Read one character. - let mut ret = inner.registers.DR.get() as u8 as char; - - // Convert carrige return to newline. - if ret == '\r' { - ret = '\n' - } - - // Update statistics. - inner.chars_read += 1; - - ret - }) + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) } - fn clear(&self) { - self.inner.lock(|inner| { - // Read from the RX FIFO until it is indicating empty. - while !inner.registers.FR.matches_all(FR::RXFE::SET) { - inner.registers.DR.get(); - } - }) + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} } } diff --git a/X1_JTAG_boot/src/console.rs b/X1_JTAG_boot/src/console.rs index 2d38cc1d1..3552823cc 100644 --- a/X1_JTAG_boot/src/console.rs +++ b/X1_JTAG_boot/src/console.rs @@ -20,8 +20,8 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). + /// Block execution until the last buffered character has been physically put on the TX + /// wire. fn flush(&self); } @@ -33,7 +33,7 @@ pub mod interface { } /// Clear RX buffers, if any. - fn clear(&self); + fn clear_rx(&self); } /// Console statistics. diff --git a/utils/minipush.rb b/utils/minipush.rb index e06433a8a..bda97bb1f 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -25,24 +25,29 @@ def initialize(serial_name, binary_image_path) private - # The three characters signaling the request token are expected to arrive as the last three - # characters _at the end_ of a character stream (e.g. after a header print from Miniload). + # The three characters signaling the request token form the consecutive sequence "\x03\x03\x03". def wait_for_binary_request puts "[#{@name_short}] 🔌 Please power the target now" # Timeout for the request token starts after the first sign of life was received. received = @target_serial.readpartial(4096) Timeout.timeout(10) do + count = 0 + loop do raise ProtocolError if received.nil? - if received.chars.last(3) == ["\u{3}", "\u{3}", "\u{3}"] - # Print the last chunk minus the request token. - print received[0..-4] - return - end + received.chars.each do |c| + if c == "\u{3}" + count += 1 + return true if count == 3 + else + # A normal character resets token counting. + count = 0 - print received + print c + end + end received = @target_serial.readpartial(4096) end diff --git a/utils/miniterm.rb b/utils/miniterm.rb index 010761cd8..091566697 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -11,6 +11,8 @@ require 'colorize' require 'serialport' +SERIAL_BAUD = 921_600 + class ConnectionError < StandardError; end # The main class @@ -41,7 +43,7 @@ def wait_for_serial def open_serial wait_for_serial - @target_serial = SerialPort.new(@target_serial_name, 576_000, 8, 1, SerialPort::NONE) + @target_serial = SerialPort.new(@target_serial_name, SERIAL_BAUD, 8, 1, SerialPort::NONE) # Ensure all output is immediately flushed to the device. @target_serial.sync = true From 4d1f84bbfffb322b724a1fa178b74d0e315fcbee Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 4 Jan 2021 17:51:38 +0100 Subject: [PATCH 021/214] Add pull request template --- .github/pull_request_template.md | 13 +++++++++++++ .github/workflows/sanity.yml | 2 +- contributor_setup.sh | 28 ++++++++++++++++++++++++++++ utils/devtool.rb | 2 +- 4 files changed, 43 insertions(+), 2 deletions(-) create mode 100644 .github/pull_request_template.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..983399703 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,13 @@ +### Description + + + +Related Issue: + +### Pre-commit steps + + - [ ] Tested on QEMU and real HW Rasperry Pi. + - Not needed if it is just a README change or similar. + - [ ] Ran `./contributor_setup.sh` followed by `./devtool ready_for_publish` + - You'll need `Ruby` with `Bundler` and `NPM` installed locally. + - This step is optional, but much appreciated if done. diff --git a/.github/workflows/sanity.yml b/.github/workflows/sanity.yml index 46c559a8e..2c4551f2d 100644 --- a/.github/workflows/sanity.yml +++ b/.github/workflows/sanity.yml @@ -41,7 +41,7 @@ jobs: - name: Setup misspell run: | curl -L -o ./install-misspell.sh https://git.io/misspell - sh ./install-misspell.sh -b ~/bin + sh ./install-misspell.sh -b .vendor - name: Run checks run: | BSP=rpi3 bundle exec ruby utils/devtool.rb clippy diff --git a/contributor_setup.sh b/contributor_setup.sh index d379dba7d..8bad881fe 100755 --- a/contributor_setup.sh +++ b/contributor_setup.sh @@ -2,7 +2,35 @@ git config core.hooksPath .githooks +# +# Ruby and Bundler +# +if ! command -v bundle &> /dev/null +then + echo "'bundle' could not be found. Please install Ruby and Bundler." + exit +fi bundle config set path '.vendor/bundle' bundle install +# +# NPM +# +if ! command -v npm &> /dev/null +then + echo "'npm' could not be found. Please install it." + exit +fi npm install --save-dev --save-exact prettier + +# +# Misspell +# +if ! command -v curl &> /dev/null +then + echo "'curl' could not be found. Please install it." + exit +fi +curl -L -o ./install-misspell.sh https://git.io/misspell +sh ./install-misspell.sh -b .vendor +rm install-misspell.sh diff --git a/utils/devtool.rb b/utils/devtool.rb index 47e3404af..66271202f 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -165,7 +165,7 @@ def copyright def misspell puts 'Misspell'.light_blue - exit(1) unless system("~/bin/misspell -error #{tracked_files.join(' ')}") + exit(1) unless system(".vendor/misspell -error #{tracked_files.join(' ')}") end def rubocop From e815b346032ed70db95d611a1ee81c1f8c55e34b Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 4 Jan 2021 22:13:40 +0100 Subject: [PATCH 022/214] Update READMEs --- 08_timestamps/README.md | 5 +++-- README.md | 11 ++++++----- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 111b1168c..b79121c0f 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -2,8 +2,9 @@ ## tl;dr -- We add abstractions for the architectural timer, implement it for `aarch64` and use it to annotate - prints with timestamps. +- We add abstractions for the architectural timer and implement them for `_arch/aarch64`. +- The new timer functions are used to annotate UART prints with timestamps, and to get rid of the + cycle-based delays in the `GPIO` and `UART` device drivers, which boosts accuracy. - A `warn!()` macro is added. ## Test it diff --git a/README.md b/README.md index 00cbe66ef..daf3b7d74 100644 --- a/README.md +++ b/README.md @@ -92,15 +92,15 @@ other Unix flavors such as **macOS**, but this is only _experimental_. [docker group]: https://docs.docker.com/engine/install/linux-postinstall/ [Rust Analyzer extension]: https://marketplace.visualstudio.com/items?itemName=matklad.rust-analyzer -### 🧰 The Long Version: Eliminating Toolchain Hassle +### 🧰 More Details: Eliminating Toolchain Hassle This series tries to put a strong focus on user friendliness. Therefore, efforts were made to -eliminate the biggest painpoint in embedded development as much as possible: Toolchain hassle. +eliminate the biggest painpoint in embedded development as much as possible: `Toolchain hassle`. Rust itself is already helping a lot in that regard, because it has built-in support for cross-compilation. All that we need for cross-compiling from an `x86` host to the Raspberry Pi's -`AArch64` architecture is to install the respective target through `rustup`. However, besides the -Rust compiler, we will use some more tools. Among others: +`AArch64` architecture will be automatically installed by `rustup`. However, besides the Rust +compiler, we will use some more tools. Among others: - `QEMU` to emulate our kernel on the host system. - A self-made tool called `Minipush` to load a kernel onto the Raspberry Pi on-demand over `UART`. @@ -122,7 +122,8 @@ provided container, please refer to the repository's [docker](docker) folder. Since the kernel developed in the tutorials runs on the real hardware, it is highly recommended to get a USB serial cable to get the full experience. -- You can find USB-to-serial cables that should work right away at [\[1\]] [\[2\]]. +- You can find USB-to-serial cables that should work right away at [\[1\]] [\[2\]], but many others + will work too. Ideally, your cable is based on the `CP2102` chip. - You connect it to `GND` and GPIO pins `14/15` as shown below. - [Tutorial 6](06_drivers_gpio_uart) is the first where you can use it. Check it out for instructions on how to prepare the SD card to boot your self-made kernel from it. From e7cb61b3899cde97ab253543a9eb102e8d1511eb Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 23 Jan 2021 22:43:59 +0100 Subject: [PATCH 023/214] Refactoring - Don't wildcard-import from arch modules. Make it explicit. - Put translation table code into its own module. - Put boot code in boot.rs instead of cpu.rs - Other minor changes, most memory subsystem. --- 00_before_we_start/README.md | 37 +- 01_wait_forever/Makefile | 3 +- 01_wait_forever/README.md | 4 +- 01_wait_forever/src/_arch/aarch64/cpu.rs | 8 - .../src/_arch/aarch64/{cpu.S => cpu/boot.S} | 0 01_wait_forever/src/_arch/aarch64/cpu/boot.rs | 15 + 01_wait_forever/src/bsp.rs | 2 +- 01_wait_forever/src/cpu.rs | 5 +- 01_wait_forever/src/cpu/boot.rs | 9 + 01_wait_forever/src/main.rs | 35 +- 02_runtime_init/Makefile | 3 +- 02_runtime_init/README.CN.md | 205 ++- 02_runtime_init/README.md | 95 +- 02_runtime_init/src/_arch/aarch64/cpu.rs | 10 +- .../src/_arch/aarch64/{cpu.S => cpu/boot.S} | 0 02_runtime_init/src/_arch/aarch64/cpu/boot.rs | 15 + 02_runtime_init/src/bsp.rs | 2 +- 02_runtime_init/src/cpu.rs | 8 +- 02_runtime_init/src/cpu/boot.rs | 9 + 02_runtime_init/src/main.rs | 37 +- 03_hacky_hello_world/Makefile | 3 +- 03_hacky_hello_world/README.md | 20 +- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs | 10 +- .../src/_arch/aarch64/{cpu.S => cpu/boot.S} | 0 .../src/_arch/aarch64/cpu/boot.rs | 15 + 03_hacky_hello_world/src/bsp.rs | 2 +- 03_hacky_hello_world/src/cpu.rs | 8 +- 03_hacky_hello_world/src/cpu/boot.rs | 9 + 03_hacky_hello_world/src/main.rs | 39 +- 03_hacky_hello_world/src/print.rs | 2 +- 04_zero_overhead_abstraction/Makefile | 3 +- 04_zero_overhead_abstraction/README.md | 168 ++- .../src/_arch/aarch64/cpu.rs | 36 +- .../src/_arch/aarch64/cpu/boot.rs | 41 + .../src/_arch/aarch64/cpu/smp.rs | 7 + 04_zero_overhead_abstraction/src/bsp.rs | 2 +- 04_zero_overhead_abstraction/src/cpu.rs | 8 +- 04_zero_overhead_abstraction/src/cpu/boot.rs | 9 + 04_zero_overhead_abstraction/src/cpu/smp.rs | 8 +- 04_zero_overhead_abstraction/src/main.rs | 39 +- 04_zero_overhead_abstraction/src/print.rs | 2 +- 05_safe_globals/Makefile | 3 +- 05_safe_globals/README.md | 14 +- 05_safe_globals/src/_arch/aarch64/cpu.rs | 36 +- 05_safe_globals/src/_arch/aarch64/cpu/boot.rs | 41 + 05_safe_globals/src/_arch/aarch64/cpu/smp.rs | 7 + 05_safe_globals/src/bsp.rs | 2 +- 05_safe_globals/src/cpu.rs | 8 +- 05_safe_globals/src/cpu/boot.rs | 9 + 05_safe_globals/src/cpu/smp.rs | 8 +- 05_safe_globals/src/main.rs | 39 +- 05_safe_globals/src/print.rs | 2 +- 06_drivers_gpio_uart/Makefile | 3 +- 06_drivers_gpio_uart/README.md | 63 +- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 38 +- .../src/_arch/aarch64/cpu/boot.rs | 41 + .../src/_arch/aarch64/cpu/smp.rs | 7 + 06_drivers_gpio_uart/src/bsp.rs | 2 +- 06_drivers_gpio_uart/src/console.rs | 3 +- 06_drivers_gpio_uart/src/cpu.rs | 8 +- 06_drivers_gpio_uart/src/cpu/boot.rs | 9 + 06_drivers_gpio_uart/src/cpu/smp.rs | 8 +- 06_drivers_gpio_uart/src/main.rs | 47 +- 06_drivers_gpio_uart/src/print.rs | 2 +- 07_uart_chainloader/Makefile | 3 +- 07_uart_chainloader/README.md | 77 +- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6856 -> 6856 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6728 -> 6728 bytes 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 38 +- .../src/_arch/aarch64/cpu/boot.rs | 41 + .../src/_arch/aarch64/cpu/smp.rs | 7 + 07_uart_chainloader/src/bsp.rs | 2 +- 07_uart_chainloader/src/console.rs | 3 +- 07_uart_chainloader/src/cpu.rs | 8 +- 07_uart_chainloader/src/cpu/boot.rs | 9 + 07_uart_chainloader/src/cpu/smp.rs | 8 +- 07_uart_chainloader/src/main.rs | 51 +- 07_uart_chainloader/src/print.rs | 2 +- 08_timestamps/Makefile | 3 +- 08_timestamps/README.md | 94 +- 08_timestamps/src/_arch/aarch64/cpu.rs | 38 +- 08_timestamps/src/_arch/aarch64/cpu/boot.rs | 41 + 08_timestamps/src/_arch/aarch64/cpu/smp.rs | 7 + 08_timestamps/src/_arch/aarch64/time.rs | 9 +- 08_timestamps/src/bsp.rs | 2 +- 08_timestamps/src/console.rs | 3 +- 08_timestamps/src/cpu.rs | 8 +- 08_timestamps/src/cpu/boot.rs | 9 + 08_timestamps/src/cpu/smp.rs | 8 +- 08_timestamps/src/main.rs | 49 +- 08_timestamps/src/print.rs | 2 +- 08_timestamps/src/time.rs | 8 +- 09_hw_debug_JTAG/Makefile | 3 +- 09_hw_debug_JTAG/README.md | 10 +- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs | 38 +- .../src/_arch/aarch64/cpu/boot.rs | 41 + 09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs | 7 + 09_hw_debug_JTAG/src/_arch/aarch64/time.rs | 9 +- 09_hw_debug_JTAG/src/bsp.rs | 2 +- 09_hw_debug_JTAG/src/console.rs | 3 +- 09_hw_debug_JTAG/src/cpu.rs | 8 +- 09_hw_debug_JTAG/src/cpu/boot.rs | 9 + 09_hw_debug_JTAG/src/cpu/smp.rs | 8 +- 09_hw_debug_JTAG/src/main.rs | 49 +- 09_hw_debug_JTAG/src/print.rs | 2 +- 09_hw_debug_JTAG/src/time.rs | 8 +- 10_privilege_level/Makefile | 3 +- 10_privilege_level/README.md | 156 ++- 10_privilege_level/src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + 10_privilege_level/src/_arch/aarch64/time.rs | 9 +- 10_privilege_level/src/bsp.rs | 2 +- 10_privilege_level/src/console.rs | 3 +- 10_privilege_level/src/cpu.rs | 8 +- 10_privilege_level/src/cpu/boot.rs | 9 + 10_privilege_level/src/cpu/smp.rs | 8 +- 10_privilege_level/src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 8 +- 10_privilege_level/src/main.rs | 49 +- 10_privilege_level/src/print.rs | 2 +- 10_privilege_level/src/time.rs | 8 +- .../Makefile | 3 +- .../README.md | 572 +++++--- .../src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + .../src/_arch/aarch64/memory/mmu.rs | 311 +---- .../aarch64/memory/mmu/translation_table.rs | 288 ++++ .../src/_arch/aarch64/time.rs | 9 +- .../src/bsp.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 17 +- .../src/console.rs | 3 +- .../src/cpu.rs | 8 +- .../src/cpu/boot.rs | 9 + .../src/cpu/smp.rs | 8 +- .../src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 8 +- .../src/main.rs | 51 +- .../src/memory/mmu.rs | 54 +- .../src/memory/mmu/translation_table.rs | 14 + .../src/print.rs | 2 +- .../src/time.rs | 8 +- 12_exceptions_part1_groundwork/Makefile | 3 +- 12_exceptions_part1_groundwork/README.md | 69 +- .../src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + .../src/_arch/aarch64/memory/mmu.rs | 311 +---- .../aarch64/memory/mmu/translation_table.rs | 288 ++++ .../src/_arch/aarch64/time.rs | 9 +- 12_exceptions_part1_groundwork/src/bsp.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 17 +- 12_exceptions_part1_groundwork/src/console.rs | 3 +- 12_exceptions_part1_groundwork/src/cpu.rs | 8 +- .../src/cpu/boot.rs | 9 + 12_exceptions_part1_groundwork/src/cpu/smp.rs | 8 +- .../src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 8 +- 12_exceptions_part1_groundwork/src/main.rs | 51 +- .../src/memory/mmu.rs | 53 +- .../src/memory/mmu/translation_table.rs | 14 + 12_exceptions_part1_groundwork/src/print.rs | 2 +- 12_exceptions_part1_groundwork/src/time.rs | 8 +- 13_integrated_testing/Makefile | 3 +- 13_integrated_testing/README.md | 183 +-- .../src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + .../src/_arch/aarch64/memory/mmu.rs | 329 +---- .../aarch64/memory/mmu/translation_table.rs | 316 +++++ .../src/_arch/aarch64/time.rs | 9 +- 13_integrated_testing/src/bsp.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 17 +- 13_integrated_testing/src/console.rs | 3 +- 13_integrated_testing/src/cpu.rs | 8 +- 13_integrated_testing/src/cpu/boot.rs | 9 + 13_integrated_testing/src/cpu/smp.rs | 8 +- 13_integrated_testing/src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 8 +- 13_integrated_testing/src/lib.rs | 53 +- 13_integrated_testing/src/memory/mmu.rs | 53 +- .../src/memory/mmu/translation_table.rs | 14 + 13_integrated_testing/src/print.rs | 2 +- 13_integrated_testing/src/time.rs | 8 +- 14_exceptions_part2_peripheral_IRQs/README.md | 71 +- .../src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + .../src/_arch/aarch64/memory/mmu.rs | 329 +---- .../aarch64/memory/mmu/translation_table.rs | 316 +++++ .../src/_arch/aarch64/time.rs | 9 +- .../src/bsp.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 17 +- .../src/console.rs | 3 +- .../src/cpu.rs | 8 +- .../src/cpu/boot.rs | 9 + .../src/cpu/smp.rs | 8 +- .../src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 13 +- .../src/lib.rs | 58 +- .../src/memory/mmu.rs | 53 +- .../src/memory/mmu/translation_table.rs | 14 + .../src/print.rs | 2 +- .../src/state.rs | 2 +- .../src/time.rs | 8 +- 15_virtual_mem_part2_mmio_remap/README.md | 1197 ++++++++--------- .../src/_arch/aarch64/cpu.rs | 82 +- .../src/_arch/aarch64/cpu/boot.rs | 89 ++ .../src/_arch/aarch64/cpu/smp.rs | 7 + .../src/_arch/aarch64/exception.rs | 7 + .../_arch/aarch64/exception/asynchronous.rs | 7 + .../src/_arch/aarch64/memory/mmu.rs | 507 +------ .../aarch64/memory/mmu/translation_table.rs | 462 +++++++ .../src/_arch/aarch64/time.rs | 7 + 15_virtual_mem_part2_mmio_remap/src/bsp.rs | 2 +- .../src/bsp/device_driver/arm/gicv2.rs | 2 +- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 2 +- .../bcm/bcm2xxx_interrupt_controller.rs | 2 +- .../peripheral_ic.rs | 2 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 2 +- .../src/bsp/raspberrypi/memory.rs | 2 +- .../src/bsp/raspberrypi/memory/mmu.rs | 17 +- .../src/console.rs | 3 +- 15_virtual_mem_part2_mmio_remap/src/cpu.rs | 8 +- .../src/cpu/boot.rs | 9 + .../src/cpu/smp.rs | 8 +- .../src/exception.rs | 6 +- .../src/exception/asynchronous.rs | 13 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 58 +- 15_virtual_mem_part2_mmio_remap/src/memory.rs | 75 +- .../src/memory/mmu.rs | 160 +-- .../src/memory/mmu/mapping_record.rs | 10 +- .../src/memory/mmu/translation_table.rs | 111 ++ .../src/memory/mmu/types.rs | 80 +- 15_virtual_mem_part2_mmio_remap/src/print.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/state.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/time.rs | 8 +- X1_JTAG_boot/Cargo.lock | 12 +- X1_JTAG_boot/Cargo.toml | 4 +- X1_JTAG_boot/Makefile | 3 +- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8240 -> 8240 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 7968 -> 7968 bytes X1_JTAG_boot/src/_arch/aarch64/cpu.rs | 39 +- X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs | 41 + X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs | 7 + X1_JTAG_boot/src/_arch/aarch64/time.rs | 9 +- X1_JTAG_boot/src/bsp.rs | 2 +- X1_JTAG_boot/src/console.rs | 3 +- X1_JTAG_boot/src/cpu.rs | 8 +- X1_JTAG_boot/src/cpu/boot.rs | 9 + X1_JTAG_boot/src/cpu/smp.rs | 8 +- X1_JTAG_boot/src/main.rs | 49 +- X1_JTAG_boot/src/print.rs | 2 +- X1_JTAG_boot/src/time.rs | 8 +- rust-toolchain | 2 +- 266 files changed, 6393 insertions(+), 4254 deletions(-) delete mode 100644 01_wait_forever/src/_arch/aarch64/cpu.rs rename 01_wait_forever/src/_arch/aarch64/{cpu.S => cpu/boot.S} (100%) create mode 100644 01_wait_forever/src/_arch/aarch64/cpu/boot.rs create mode 100644 01_wait_forever/src/cpu/boot.rs rename 02_runtime_init/src/_arch/aarch64/{cpu.S => cpu/boot.S} (100%) create mode 100644 02_runtime_init/src/_arch/aarch64/cpu/boot.rs create mode 100644 02_runtime_init/src/cpu/boot.rs rename 03_hacky_hello_world/src/_arch/aarch64/{cpu.S => cpu/boot.S} (100%) create mode 100644 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs create mode 100644 03_hacky_hello_world/src/cpu/boot.rs create mode 100644 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs create mode 100644 04_zero_overhead_abstraction/src/cpu/boot.rs create mode 100644 05_safe_globals/src/_arch/aarch64/cpu/boot.rs create mode 100644 05_safe_globals/src/cpu/boot.rs create mode 100644 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs create mode 100644 06_drivers_gpio_uart/src/cpu/boot.rs create mode 100644 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs create mode 100644 07_uart_chainloader/src/cpu/boot.rs create mode 100644 08_timestamps/src/_arch/aarch64/cpu/boot.rs create mode 100644 08_timestamps/src/cpu/boot.rs create mode 100644 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs create mode 100644 09_hw_debug_JTAG/src/cpu/boot.rs create mode 100644 10_privilege_level/src/_arch/aarch64/cpu/boot.rs create mode 100644 10_privilege_level/src/cpu/boot.rs create mode 100644 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs create mode 100644 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs create mode 100644 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs create mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs create mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 12_exceptions_part1_groundwork/src/cpu/boot.rs create mode 100644 12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs create mode 100644 13_integrated_testing/src/_arch/aarch64/cpu/boot.rs create mode 100644 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 13_integrated_testing/src/cpu/boot.rs create mode 100644 13_integrated_testing/src/memory/mmu/translation_table.rs create mode 100644 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs create mode 100644 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs create mode 100644 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs create mode 100644 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs create mode 100644 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs create mode 100644 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs create mode 100644 X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs create mode 100644 X1_JTAG_boot/src/cpu/boot.rs diff --git a/00_before_we_start/README.md b/00_before_we_start/README.md index 02c25118e..60e060bcc 100644 --- a/00_before_we_start/README.md +++ b/00_before_we_start/README.md @@ -12,7 +12,7 @@ the tutorials advance. Have fun! -## Code organization and architecture +# Code organization and architecture The code is divided into different *modules*, each representing a typical **subsystem** of the `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example, @@ -25,15 +25,22 @@ architecture. For each supported processor architecture, there exists a subfolde for example, `src/_arch/aarch64`. The architecture folders mirror the subsystem modules laid out in `src`. For example, architectural -code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go into -`src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -`src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -module organization. That means a public function `foo()` defined in `src/_arch/aarch64/memory.rs` -would be reachable as `crate::memory::foo()` only. +code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go into +`src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in `src/memory/mmu.rs` +using the `path attribute`. Usually, the chosen module name is the generic module's name prefixed +with `arch_`. -The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. Rather, -it's contents are conditionally pulled into respective files using the `#[path = -"_arch/xxx/yyy.rs"]` attribute. +For example, this is the top of `src/memory/mmu.rs`: + +``` +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/memory/mmu.rs"] +mod arch_mmu; +``` + +Often times, items from the `arch_ module` will be publicly reexported by the parent module. This +way, each architecture specific module can provide its implementation of an item, while the caller +must not be concerned which architecture has been conditionally compiled. ## BSP code @@ -42,9 +49,8 @@ target board specific definitions and functions. These are things such as the bo instances of drivers for devices that are featured on the respective board. Just like processor architecture code, the `BSP` code's module structure tries to mirror the -`kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -whatever is provided must be called starting from the `bsp` namespace, e.g. -`bsp::driver::driver_manager()`. +`kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is provided +must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. ## Kernel interfaces @@ -96,3 +102,10 @@ From a namespace perspective, **memory** subsystem code lives in: - `crate::memory::*` - `crate::bsp::memory::*` + +# Boot flow + +1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. + - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. + + diff --git a/01_wait_forever/Makefile b/01_wait_forever/Makefile index ef6395f1b..abecc406b 100644 --- a/01_wait_forever/Makefile +++ b/01_wait_forever/Makefile @@ -34,8 +34,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/01_wait_forever/README.md b/01_wait_forever/README.md index 0bd3e7429..785e9817e 100644 --- a/01_wait_forever/README.md +++ b/01_wait_forever/README.md @@ -23,8 +23,8 @@ - Only `.text` section. - `main.rs`: Important [inner attributes]: - `#![no_std]`, `#![no_main]` -- `cpu.S`: Assembly `_start()` function that executes `wfe` (Wait For Event), halting all cores that - are executing `_start()`. +- `boot.S`: Assembly `_start()` function that executes `wfe` (Wait For Event), halting all cores + that are executing `_start()`. - We (have to) define a `#[panic_handler]` function to make the compiler happy. - Make it `unimplemented!()` because it will be stripped out since it is not used. diff --git a/01_wait_forever/src/_arch/aarch64/cpu.rs b/01_wait_forever/src/_arch/aarch64/cpu.rs deleted file mode 100644 index 6d24fe289..000000000 --- a/01_wait_forever/src/_arch/aarch64/cpu.rs +++ /dev/null @@ -1,8 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural processor code. - -// Assembly counterpart to this file. -global_asm!(include_str!("cpu.S")); diff --git a/01_wait_forever/src/_arch/aarch64/cpu.S b/01_wait_forever/src/_arch/aarch64/cpu/boot.S similarity index 100% rename from 01_wait_forever/src/_arch/aarch64/cpu.S rename to 01_wait_forever/src/_arch/aarch64/cpu/boot.S diff --git a/01_wait_forever/src/_arch/aarch64/cpu/boot.rs b/01_wait_forever/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c3c325d3a --- /dev/null +++ b/01_wait_forever/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +// Assembly counterpart to this file. Includes function _start(). +global_asm!(include_str!("boot.S")); diff --git a/01_wait_forever/src/bsp.rs b/01_wait_forever/src/bsp.rs index 8f0d27c87..2b92251f5 100644 --- a/01_wait_forever/src/bsp.rs +++ b/01_wait_forever/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; diff --git a/01_wait_forever/src/cpu.rs b/01_wait_forever/src/cpu.rs index 27aea2041..c3962ff8e 100644 --- a/01_wait_forever/src/cpu.rs +++ b/01_wait_forever/src/cpu.rs @@ -4,7 +4,4 @@ //! Processor code. -#[cfg(target_arch = "aarch64")] -#[path = "_arch/aarch64/cpu.rs"] -mod arch_cpu; -pub use arch_cpu::*; +mod boot; diff --git a/01_wait_forever/src/cpu/boot.rs b/01_wait_forever/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/01_wait_forever/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/01_wait_forever/src/main.rs b/01_wait_forever/src/main.rs index 883453535..10cf6bb42 100644 --- a/01_wait_forever/src/main.rs +++ b/01_wait_forever/src/main.rs @@ -20,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -37,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -91,14 +97,17 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. #![feature(asm)] #![feature(global_asm)] #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. - mod bsp; mod cpu; mod panic_wait; diff --git a/02_runtime_init/Makefile b/02_runtime_init/Makefile index ef6395f1b..abecc406b 100644 --- a/02_runtime_init/Makefile +++ b/02_runtime_init/Makefile @@ -34,8 +34,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/02_runtime_init/README.CN.md b/02_runtime_init/README.CN.md index 1369031d6..0ffecb496 100644 --- a/02_runtime_init/README.CN.md +++ b/02_runtime_init/README.CN.md @@ -2,7 +2,7 @@ ## tl;dr -我们拓展了`cpu.S`,在第一次启动的时候调用Rust代码。在Rust的代码中先清零了[bss] section,然后通过调用`panic()`挂起CPU。再次运行`make qemu`看看新增加的代码是怎么运行的。 +我们拓展了`boot.S`,在第一次启动的时候调用Rust代码。在Rust的代码中先清零了[bss] section,然后通过调用`panic()`挂起CPU。再次运行`make qemu`看看新增加的代码是怎么运行的。 ## 值得注意的变化 @@ -21,9 +21,23 @@ ## 相比之前的变化(diff) ```diff -diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.S 02_runtime_init/src/_arch/aarch64/cpu.S ---- 01_wait_forever/src/_arch/aarch64/cpu.S -+++ 02_runtime_init/src/_arch/aarch64/cpu.S +diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml +--- 01_wait_forever/Cargo.toml ++++ 02_runtime_init/Cargo.toml +@@ -4,6 +4,9 @@ + authors = ["Andre Richter "] + edition = "2018" + ++[profile.release] ++lto = true ++ + # The features section is used to select the target board. + [features] + default = [] + +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.S 02_runtime_init/src/_arch/aarch64/cpu/boot.S +--- 01_wait_forever/src/_arch/aarch64/cpu/boot.S ++++ 02_runtime_init/src/_arch/aarch64/cpu/boot.S @@ -7,5 +7,15 @@ .global _start @@ -43,10 +57,45 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.S 02_runtime_init/src/_arch/aarc + b 1b // We should never reach here. But just in case, + // park this core aswell +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs +--- 01_wait_forever/src/_arch/aarch64/cpu.rs ++++ 02_runtime_init/src/_arch/aarch64/cpu.rs +@@ -0,0 +1,30 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! Architectural processor code. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::cpu::arch_cpu ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Pause execution on the core. ++#[inline(always)] ++pub fn wait_forever() -> ! { ++ unsafe { ++ loop { ++ #[rustfmt::skip] ++ asm!( ++ "wfe", ++ options(nomem, nostack, preserves_flags) ++ ); ++ } ++ } ++} + diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld -@@ -13,5 +13,24 @@ +@@ -13,5 +13,27 @@ *(.text._start) *(.text*) } @@ -66,22 +115,97 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/ra + __bss_start = .; + *(.bss*); + . = ALIGN(8); -+ __bss_end = .; ++ ++ /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ ++ . += 8; ++ __bss_end_inclusive = . - 8; + } + /DISCARD/ : { *(.comment*) } } +diff -uNr 01_wait_forever/src/bsp/raspberrypi/memory.rs 02_runtime_init/src/bsp/raspberrypi/memory.rs +--- 01_wait_forever/src/bsp/raspberrypi/memory.rs ++++ 02_runtime_init/src/bsp/raspberrypi/memory.rs +@@ -0,0 +1,37 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! BSP Memory Management. ++ ++use core::{cell::UnsafeCell, ops::RangeInclusive}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++// Symbols from the linker script. ++extern "Rust" { ++ static __bss_start: UnsafeCell; ++ static __bss_end_inclusive: UnsafeCell; ++} ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Return the inclusive range spanning the .bss section. ++/// ++/// # Safety ++/// ++/// - Values are provided by the linker script and must be trusted as-is. ++/// - The linker-provided addresses must be u64 aligned. ++pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { ++ let range; ++ unsafe { ++ range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); ++ } ++ assert!(!range.is_empty()); ++ ++ range ++} + +diff -uNr 01_wait_forever/src/bsp/raspberrypi.rs 02_runtime_init/src/bsp/raspberrypi.rs +--- 01_wait_forever/src/bsp/raspberrypi.rs ++++ 02_runtime_init/src/bsp/raspberrypi.rs +@@ -4,4 +4,4 @@ + + //! Top-level BSP file for the Raspberry Pi 3 and 4. + +-// Coming soon. ++pub mod memory; + +diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs +--- 01_wait_forever/src/cpu.rs ++++ 02_runtime_init/src/cpu.rs +@@ -4,4 +4,13 @@ + + //! Processor code. + ++#[cfg(target_arch = "aarch64")] ++#[path = "_arch/aarch64/cpu.rs"] ++mod arch_cpu; ++ + mod boot; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_cpu::wait_forever; + diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs --- 01_wait_forever/src/main.rs +++ 02_runtime_init/src/main.rs -@@ -97,10 +97,20 @@ - #![no_main] - #![no_std] +@@ -102,6 +102,7 @@ + //! + //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. ++//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. + //! + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html --// `mod cpu` provides the `_start()` function, the first function to run. -+// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -+// `runtime_init()`, which jumps to `kernel_init()`. +@@ -112,6 +113,15 @@ mod bsp; mod cpu; @@ -102,73 +226,70 @@ diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs --- 01_wait_forever/src/memory.rs +++ 02_runtime_init/src/memory.rs -@@ -0,0 +1,29 @@ +@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management. + -+use core::ops::Range; ++use core::ops::RangeInclusive; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + -+/// Zero out a memory region. ++/// Zero out an inclusive memory range. +/// +/// # Safety +/// +/// - `range.start` and `range.end` must be valid. +/// - `range.start` and `range.end` must be `T` aligned. -+pub unsafe fn zero_volatile(range: Range<*mut T>) ++pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) +where + T: From, +{ -+ let mut ptr = range.start; ++ let mut ptr = *range.start(); ++ let end_inclusive = *range.end(); + -+ while ptr < range.end { ++ while ptr <= end_inclusive { + core::ptr::write_volatile(ptr, T::from(0)); + ptr = ptr.offset(1); + } +} +diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs +--- 01_wait_forever/src/panic_wait.rs ++++ 02_runtime_init/src/panic_wait.rs +@@ -4,9 +4,10 @@ + + //! A panic handler that infinitely waits. + ++use crate::cpu; + use core::panic::PanicInfo; + + #[panic_handler] + fn panic(_info: &PanicInfo) -> ! { +- unimplemented!() ++ cpu::wait_forever() + } + diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.rs --- 01_wait_forever/src/runtime_init.rs +++ 02_runtime_init/src/runtime_init.rs -@@ -0,0 +1,58 @@ +@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Rust runtime initialization code. + -+use crate::memory; -+use core::ops::Range; ++use crate::{bsp, memory}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + -+/// Return the range spanning the .bss section. -+/// -+/// # Safety -+/// -+/// - The symbol-provided addresses must be valid. -+/// - The symbol-provided addresses must be usize aligned. -+unsafe fn bss_range() -> Range<*mut usize> { -+ extern "C" { -+ // Boundaries of the .bss section, provided by linker script symbols. -+ static mut __bss_start: usize; -+ static mut __bss_end: usize; -+ } -+ -+ Range { -+ start: &mut __bss_start, -+ end: &mut __bss_end, -+ } -+} -+ +/// Zero out the .bss section. +/// +/// # Safety @@ -176,7 +297,7 @@ diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.r +/// - Must only be called pre `kernel_init()`. +#[inline(always)] +unsafe fn zero_bss() { -+ memory::zero_volatile(bss_range()); ++ memory::zero_volatile(bsp::memory::bss_range_inclusive()); +} + +//-------------------------------------------------------------------------------------------------- @@ -190,7 +311,7 @@ diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.r +/// +/// - Only a single core must be active and running this function. +#[no_mangle] -+pub unsafe extern "C" fn runtime_init() -> ! { ++pub unsafe fn runtime_init() -> ! { + zero_bss(); + + crate::kernel_init() diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 206424f9e..7948aae4a 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -2,7 +2,7 @@ ## tl;dr -- We extend `cpu.S` to call into Rust code for the first time. There, we zero the [bss] section +- We extend `boot.S` to call into Rust code for the first time. There, we zero the [bss] section before execution is halted with a call to `panic()`. - Check out `make qemu` again to see the additional code run. @@ -37,13 +37,44 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml [features] default = [] +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.S 02_runtime_init/src/_arch/aarch64/cpu/boot.S +--- 01_wait_forever/src/_arch/aarch64/cpu/boot.S ++++ 02_runtime_init/src/_arch/aarch64/cpu/boot.S +@@ -7,5 +7,15 @@ + .global _start + + _start: +-1: wfe // Wait for event +- b 1b // In case an event happened, jump back to 1 ++ mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register ++ and x1, x1, #3 // Clear all bits except [1:0], which hold core id ++ cbz x1, 2f // Jump to label 2 if we are core 0 ++1: wfe // Wait for event ++ b 1b // In case an event happened, jump back to 1 ++2: // If we are here, we are core0 ++ ldr x1, =_start // Load address of function "_start()" ++ mov sp, x1 // Set start of stack to before our code, aka first ++ // address before "_start()" ++ bl runtime_init // Jump to the "runtime_init()" kernel function ++ b 1b // We should never reach here. But just in case, ++ // park this core aswell + diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs --- 01_wait_forever/src/_arch/aarch64/cpu.rs +++ 02_runtime_init/src/_arch/aarch64/cpu.rs -@@ -6,3 +6,21 @@ - - // Assembly counterpart to this file. - global_asm!(include_str!("cpu.S")); +@@ -0,0 +1,30 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! Architectural processor code. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::cpu::arch_cpu + +//-------------------------------------------------------------------------------------------------- +// Public Code @@ -63,28 +94,6 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aar + } +} -diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.S 02_runtime_init/src/_arch/aarch64/cpu.S ---- 01_wait_forever/src/_arch/aarch64/cpu.S -+++ 02_runtime_init/src/_arch/aarch64/cpu.S -@@ -7,5 +7,15 @@ - .global _start - - _start: --1: wfe // Wait for event -- b 1b // In case an event happened, jump back to 1 -+ mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register -+ and x1, x1, #3 // Clear all bits except [1:0], which hold core id -+ cbz x1, 2f // Jump to label 2 if we are core 0 -+1: wfe // Wait for event -+ b 1b // In case an event happened, jump back to 1 -+2: // If we are here, we are core0 -+ ldr x1, =_start // Load address of function "_start()" -+ mov sp, x1 // Set start of stack to before our code, aka first -+ // address before "_start()" -+ bl runtime_init // Jump to the "runtime_init()" kernel function -+ b 1b // We should never reach here. But just in case, -+ // park this core aswell - diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld @@ -169,16 +178,36 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi.rs 02_runtime_init/src/bsp/raspber -// Coming soon. +pub mod memory; +diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs +--- 01_wait_forever/src/cpu.rs ++++ 02_runtime_init/src/cpu.rs +@@ -4,4 +4,13 @@ + + //! Processor code. + ++#[cfg(target_arch = "aarch64")] ++#[path = "_arch/aarch64/cpu.rs"] ++mod arch_cpu; ++ + mod boot; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_cpu::wait_forever; + diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs --- 01_wait_forever/src/main.rs +++ 02_runtime_init/src/main.rs -@@ -97,10 +97,20 @@ - #![no_main] - #![no_std] +@@ -102,6 +102,7 @@ + //! + //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. ++//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. --// `mod cpu` provides the `_start()` function, the first function to run. -+// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -+// `runtime_init()`, which jumps to `kernel_init()`. + #![feature(asm)] + #![feature(global_asm)] +@@ -110,6 +111,15 @@ mod bsp; mod cpu; diff --git a/02_runtime_init/src/_arch/aarch64/cpu.rs b/02_runtime_init/src/_arch/aarch64/cpu.rs index 6ca5986aa..4fd7e3132 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu.rs +++ b/02_runtime_init/src/_arch/aarch64/cpu.rs @@ -3,9 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -// Assembly counterpart to this file. -global_asm!(include_str!("cpu.S")); +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/02_runtime_init/src/_arch/aarch64/cpu.S b/02_runtime_init/src/_arch/aarch64/cpu/boot.S similarity index 100% rename from 02_runtime_init/src/_arch/aarch64/cpu.S rename to 02_runtime_init/src/_arch/aarch64/cpu/boot.S diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.rs b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c3c325d3a --- /dev/null +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +// Assembly counterpart to this file. Includes function _start(). +global_asm!(include_str!("boot.S")); diff --git a/02_runtime_init/src/bsp.rs b/02_runtime_init/src/bsp.rs index 8f0d27c87..2b92251f5 100644 --- a/02_runtime_init/src/bsp.rs +++ b/02_runtime_init/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; diff --git a/02_runtime_init/src/cpu.rs b/02_runtime_init/src/cpu.rs index 27aea2041..d8f780823 100644 --- a/02_runtime_init/src/cpu.rs +++ b/02_runtime_init/src/cpu.rs @@ -7,4 +7,10 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::wait_forever; diff --git a/02_runtime_init/src/cpu/boot.rs b/02_runtime_init/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/02_runtime_init/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/02_runtime_init/src/main.rs b/02_runtime_init/src/main.rs index 76e7a772f..ba5a877c7 100644 --- a/02_runtime_init/src/main.rs +++ b/02_runtime_init/src/main.rs @@ -20,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -37,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -91,15 +97,18 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. #![feature(asm)] #![feature(global_asm)] #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod cpu; mod memory; diff --git a/03_hacky_hello_world/Makefile b/03_hacky_hello_world/Makefile index 90238d091..9d5a8f78b 100644 --- a/03_hacky_hello_world/Makefile +++ b/03_hacky_hello_world/Makefile @@ -34,8 +34,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 95760f02d..11c804bff 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -139,8 +139,17 @@ diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs --- 02_runtime_init/src/main.rs +++ 03_hacky_hello_world/src/main.rs -@@ -93,7 +93,9 @@ - //! - `crate::bsp::memory::*` +@@ -100,19 +100,25 @@ + //! + //! # Boot flow + //! +-//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +-//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. ++//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. ++//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. + //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. ++//! ++//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(asm)] +#![feature(format_args_nl)] @@ -149,9 +158,6 @@ diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs #![no_main] #![no_std] -@@ -101,9 +103,11 @@ - // `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; +mod console; mod cpu; @@ -161,7 +167,7 @@ diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs mod runtime_init; /// Early init code. -@@ -112,5 +116,7 @@ +@@ -121,5 +127,7 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { @@ -202,7 +208,7 @@ diff -uNr 02_runtime_init/src/print.rs 03_hacky_hello_world/src/print.rs +// +// Copyright (c) 2018-2021 Andre Richter + -+//! Printing facilities. ++//! Printing. + +use crate::{bsp, console}; +use core::fmt; diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs index 6ca5986aa..4fd7e3132 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs @@ -3,9 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -// Assembly counterpart to this file. -global_asm!(include_str!("cpu.S")); +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu.S b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S similarity index 100% rename from 03_hacky_hello_world/src/_arch/aarch64/cpu.S rename to 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c3c325d3a --- /dev/null +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +// Assembly counterpart to this file. Includes function _start(). +global_asm!(include_str!("boot.S")); diff --git a/03_hacky_hello_world/src/bsp.rs b/03_hacky_hello_world/src/bsp.rs index 8f0d27c87..2b92251f5 100644 --- a/03_hacky_hello_world/src/bsp.rs +++ b/03_hacky_hello_world/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; diff --git a/03_hacky_hello_world/src/cpu.rs b/03_hacky_hello_world/src/cpu.rs index 27aea2041..d8f780823 100644 --- a/03_hacky_hello_world/src/cpu.rs +++ b/03_hacky_hello_world/src/cpu.rs @@ -7,4 +7,10 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::wait_forever; diff --git a/03_hacky_hello_world/src/cpu/boot.rs b/03_hacky_hello_world/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/03_hacky_hello_world/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/03_hacky_hello_world/src/main.rs b/03_hacky_hello_world/src/main.rs index 3a4cf64c4..378e0bafb 100644 --- a/03_hacky_hello_world/src/main.rs +++ b/03_hacky_hello_world/src/main.rs @@ -20,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -37,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -91,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(asm)] #![feature(format_args_nl)] @@ -99,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/03_hacky_hello_world/src/print.rs b/03_hacky_hello_world/src/print.rs index 59ec9e8ee..fa6451c75 100644 --- a/03_hacky_hello_world/src/print.rs +++ b/03_hacky_hello_world/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/04_zero_overhead_abstraction/Makefile b/04_zero_overhead_abstraction/Makefile index 90238d091..9d5a8f78b 100644 --- a/04_zero_overhead_abstraction/Makefile +++ b/04_zero_overhead_abstraction/Makefile @@ -34,8 +34,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index ce452fb79..6c9ad70c4 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -23,47 +23,20 @@ diff -uNr 03_hacky_hello_world/Cargo.toml 04_zero_overhead_abstraction/Cargo.tom +cortex-a = { version = "5.x.x" } + -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs ---- 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs -@@ -0,0 +1,22 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Architectural symmetric multiprocessing. -+ -+use cortex_a::regs::*; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Return the executing core's id. -+#[inline(always)] -+pub fn core_id() -> T -+where -+ T: From, -+{ -+ const CORE_MASK: u64 = 0b11; -+ -+ T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -+} +diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs +--- 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs ++++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs +@@ -11,5 +11,31 @@ + //! + //! crate::cpu::boot::arch_boot -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs ---- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs -@@ -4,8 +4,34 @@ - - //! Architectural processor code. - --// Assembly counterpart to this file. --global_asm!(include_str!("cpu.S")); +-// Assembly counterpart to this file. Includes function _start(). +-global_asm!(include_str!("boot.S")); +use crate::{bsp, cpu}; -+use cortex_a::{asm, regs::*}; ++use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- -+// Boot Code ++// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. @@ -84,32 +57,13 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstrac + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. -+ wait_forever() ++ cpu::wait_forever() + } +} - //-------------------------------------------------------------------------------------------------- - // Public Code -@@ -14,13 +40,7 @@ - /// Pause execution on the core. - #[inline(always)] - pub fn wait_forever() -> ! { -- unsafe { -- loop { -- #[rustfmt::skip] -- asm!( -- "wfe", -- options(nomem, nostack, preserves_flags) -- ); -- } -+ loop { -+ asm::wfe() - } - } - -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.S 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.S ---- 03_hacky_hello_world/src/_arch/aarch64/cpu.S -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.S +diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.S +--- 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S ++++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.S @@ -1,21 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// @@ -133,6 +87,69 @@ diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.S 04_zero_overhead_abstract - b 1b // We should never reach here. But just in case, - // park this core aswell +diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs +--- 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs ++++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs +@@ -0,0 +1,29 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! Architectural symmetric multiprocessing. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::cpu::smp::arch_smp ++ ++use cortex_a::regs::*; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Return the executing core's id. ++#[inline(always)] ++pub fn core_id() -> T ++where ++ T: From, ++{ ++ const CORE_MASK: u64 = 0b11; ++ ++ T::from((MPIDR_EL1.get() & CORE_MASK) as u8) ++} + +diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +--- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs ++++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +@@ -11,6 +11,8 @@ + //! + //! crate::cpu::arch_cpu + ++use cortex_a::asm; ++ + //-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- +@@ -18,13 +20,7 @@ + /// Pause execution on the core. + #[inline(always)] + pub fn wait_forever() -> ! { +- unsafe { +- loop { +- #[rustfmt::skip] +- asm!( +- "wfe", +- options(nomem, nostack, preserves_flags) +- ); +- } ++ loop { ++ asm::wfe() + } + } + diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs --- 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs +++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs @@ -193,7 +210,7 @@ diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi.rs 04_zero_overhead_abstracti diff -uNr 03_hacky_hello_world/src/cpu/smp.rs 04_zero_overhead_abstraction/src/cpu/smp.rs --- 03_hacky_hello_world/src/cpu/smp.rs +++ 04_zero_overhead_abstraction/src/cpu/smp.rs -@@ -0,0 +1,10 @@ +@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -202,25 +219,32 @@ diff -uNr 03_hacky_hello_world/src/cpu/smp.rs 04_zero_overhead_abstraction/src/c + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/smp.rs"] -+mod arch_cpu_smp; -+pub use arch_cpu_smp::*; ++mod arch_smp; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_smp::core_id; diff -uNr 03_hacky_hello_world/src/cpu.rs 04_zero_overhead_abstraction/src/cpu.rs --- 03_hacky_hello_world/src/cpu.rs +++ 04_zero_overhead_abstraction/src/cpu.rs -@@ -8,3 +8,5 @@ - #[path = "_arch/aarch64/cpu.rs"] - mod arch_cpu; - pub use arch_cpu::*; -+ +@@ -10,6 +10,8 @@ + + mod boot; + +pub mod smp; ++ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main.rs --- 03_hacky_hello_world/src/main.rs +++ 04_zero_overhead_abstraction/src/main.rs -@@ -92,9 +92,7 @@ - //! - `crate::memory::*` - //! - `crate::bsp::memory::*` +@@ -106,9 +106,7 @@ + //! + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -#![feature(asm)] #![feature(format_args_nl)] @@ -228,7 +252,7 @@ diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main #![feature(panic_info_message)] #![no_main] #![no_std] -@@ -116,7 +114,8 @@ +@@ -127,7 +125,8 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs index b02903c90..5f97ea410 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs @@ -3,35 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs +++ b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/04_zero_overhead_abstraction/src/bsp.rs b/04_zero_overhead_abstraction/src/bsp.rs index 8f0d27c87..2b92251f5 100644 --- a/04_zero_overhead_abstraction/src/bsp.rs +++ b/04_zero_overhead_abstraction/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; diff --git a/04_zero_overhead_abstraction/src/cpu.rs b/04_zero_overhead_abstraction/src/cpu.rs index c9e5af72c..6f326b325 100644 --- a/04_zero_overhead_abstraction/src/cpu.rs +++ b/04_zero_overhead_abstraction/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::wait_forever; diff --git a/04_zero_overhead_abstraction/src/cpu/boot.rs b/04_zero_overhead_abstraction/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/04_zero_overhead_abstraction/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/04_zero_overhead_abstraction/src/cpu/smp.rs b/04_zero_overhead_abstraction/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/04_zero_overhead_abstraction/src/cpu/smp.rs +++ b/04_zero_overhead_abstraction/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/04_zero_overhead_abstraction/src/main.rs b/04_zero_overhead_abstraction/src/main.rs index cd4c99e58..7de836fe5 100644 --- a/04_zero_overhead_abstraction/src/main.rs +++ b/04_zero_overhead_abstraction/src/main.rs @@ -20,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -37,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -91,15 +97,20 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(format_args_nl)] #![feature(panic_info_message)] #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/04_zero_overhead_abstraction/src/print.rs b/04_zero_overhead_abstraction/src/print.rs index 59ec9e8ee..fa6451c75 100644 --- a/04_zero_overhead_abstraction/src/print.rs +++ b/04_zero_overhead_abstraction/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/05_safe_globals/Makefile b/05_safe_globals/Makefile index 90238d091..9d5a8f78b 100644 --- a/05_safe_globals/Makefile +++ b/05_safe_globals/Makefile @@ -34,8 +34,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/05_safe_globals/README.md b/05_safe_globals/README.md index 537b0ba3b..3db54d6be 100644 --- a/05_safe_globals/README.md +++ b/05_safe_globals/README.md @@ -3,8 +3,8 @@ ## tl;dr - A pseudo-lock is introduced. -- It is a first showcase of OS synchronization primitives and enables safe access to a global data - structure. +- It is a first showcase of OS synchronization primitives and enables safe access to a global data + structure. ## Mutable globals in Rust @@ -26,9 +26,7 @@ variant of a *MUTual EXclusion* primitive. `Mutex` is introduced as a trait in ` and implemented by the `NullLock` in the same file. In order to make the code lean for teaching purposes, it leaves out the actual architecture-specific logic for protection against concurrent access, since we don't need it as long as the kernel only executes on a single core with interrupts -disabled. That is also why it is implemented in the same file as the interface itself. In later -tutorials, an implementation might move to the `_arch` once it pulls in arch-specific code that can -not be further abstracted. +disabled. The `NullLock` focuses on showcasing the Rust core concept of [interior mutability]. Make sure to read up on it. I also recommend to read this article about an [accurate mental model for Rust's @@ -213,7 +211,7 @@ diff -uNr 04_zero_overhead_abstraction/src/console.rs 05_safe_globals/src/consol diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs --- 04_zero_overhead_abstraction/src/main.rs +++ 05_safe_globals/src/main.rs -@@ -94,6 +94,7 @@ +@@ -108,6 +108,7 @@ #![feature(format_args_nl)] #![feature(panic_info_message)] @@ -221,7 +219,7 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs #![no_main] #![no_std] -@@ -107,6 +108,7 @@ +@@ -118,6 +119,7 @@ mod panic_wait; mod print; mod runtime_init; @@ -229,7 +227,7 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs /// Early init code. /// -@@ -114,8 +116,15 @@ +@@ -125,8 +127,15 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/05_safe_globals/src/_arch/aarch64/cpu.rs b/05_safe_globals/src/_arch/aarch64/cpu.rs index b02903c90..5f97ea410 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu.rs @@ -3,35 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/05_safe_globals/src/_arch/aarch64/cpu/boot.rs b/05_safe_globals/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/05_safe_globals/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs b/05_safe_globals/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/05_safe_globals/src/bsp.rs b/05_safe_globals/src/bsp.rs index 8f0d27c87..2b92251f5 100644 --- a/05_safe_globals/src/bsp.rs +++ b/05_safe_globals/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; diff --git a/05_safe_globals/src/cpu.rs b/05_safe_globals/src/cpu.rs index c9e5af72c..6f326b325 100644 --- a/05_safe_globals/src/cpu.rs +++ b/05_safe_globals/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::wait_forever; diff --git a/05_safe_globals/src/cpu/boot.rs b/05_safe_globals/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/05_safe_globals/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/05_safe_globals/src/cpu/smp.rs b/05_safe_globals/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/05_safe_globals/src/cpu/smp.rs +++ b/05_safe_globals/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/05_safe_globals/src/main.rs b/05_safe_globals/src/main.rs index 64d4c5fd6..bb6abcb82 100644 --- a/05_safe_globals/src/main.rs +++ b/05_safe_globals/src/main.rs @@ -20,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -37,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -91,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(format_args_nl)] #![feature(panic_info_message)] @@ -98,9 +112,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/05_safe_globals/src/print.rs b/05_safe_globals/src/print.rs index 59ec9e8ee..fa6451c75 100644 --- a/05_safe_globals/src/print.rs +++ b/05_safe_globals/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/06_drivers_gpio_uart/Makefile b/06_drivers_gpio_uart/Makefile index 0fde2eb8d..ee7ebe389 100644 --- a/06_drivers_gpio_uart/Makefile +++ b/06_drivers_gpio_uart/Makefile @@ -40,8 +40,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 6ca14cfe0..2f23d1b5b 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -19,13 +19,16 @@ from kernel code. - Drivers are stored in `src/bsp/device_driver`, and can be reused between `BSP`s. - We introduce the `GPIO` driver, which pinmuxes the RPi's PL011 UART. + - Note how this driver differentiates between **RPi 3** and **RPi4**. Their HW is different, + so we have to account for it in SW. - Most importantly, the `PL011Uart` driver: It implements the `console::interface::*` traits and is from now on used as the main system console output. -- `BSP`s now contain a memory map in `src/bsp/raspberrypi/memory.rs`. In the specific case, they contain the - Raspberry's `MMIO` addresses which are used to instantiate the respectivedevice drivers. +- `BSP`s now contain a memory map in `src/bsp/raspberrypi/memory.rs`. In the specific case, they + contain the Raspberry's `MMIO` addresses which are used to instantiate the respective device + drivers. - We also modify the `panic!` handler, so that it does not anymore rely on `println!`, which uses the globally-shared instance of the `UART` that might be locked when an error is encountered (for - now this can't happen due to the `NullLock`, but with a real lock it becomes an issue). + now, this can't happen due to the `NullLock`, but with a real lock it becomes an issue). - Instead, it creates a new UART driver instance, re-initializes the device and uses that one to print. This increases the chances that the system is able to print a final important message before it suspends itself. @@ -145,7 +148,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile # BSP-specific arguments ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat -@@ -51,13 +57,23 @@ +@@ -52,13 +58,23 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t @@ -171,7 +174,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile all: $(KERNEL_BIN) -@@ -78,6 +94,9 @@ +@@ -79,6 +95,9 @@ @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -185,7 +188,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 05_safe_globals/src/_arch/aarch64/cpu.rs +++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -@@ -37,6 +37,16 @@ +@@ -17,6 +17,16 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -1158,7 +1161,7 @@ diff -uNr 05_safe_globals/src/bsp.rs 06_drivers_gpio_uart/src/bsp.rs +++ 06_drivers_gpio_uart/src/bsp.rs @@ -4,6 +4,8 @@ - //! Conditional re-exporting of Board Support Packages. + //! Conditional reexporting of Board Support Packages. +mod device_driver; + @@ -1169,7 +1172,7 @@ diff -uNr 05_safe_globals/src/bsp.rs 06_drivers_gpio_uart/src/bsp.rs diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs --- 05_safe_globals/src/console.rs +++ 06_drivers_gpio_uart/src/console.rs -@@ -14,8 +14,26 @@ +@@ -14,8 +14,25 @@ /// Console write functions. pub trait Write { @@ -1179,8 +1182,7 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; + -+ /// Block execution until the last buffered character has been physically put on the TX -+ /// wire. ++ /// Block until the last buffered character has been physically put on the TX wire. + fn flush(&self); + } + @@ -1196,7 +1198,7 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs } /// Console statistics. -@@ -24,8 +42,13 @@ +@@ -24,8 +41,13 @@ fn chars_written(&self) -> usize { 0 } @@ -1212,6 +1214,16 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs + pub trait All = Write + Read + Statistics; } +diff -uNr 05_safe_globals/src/cpu.rs 06_drivers_gpio_uart/src/cpu.rs +--- 05_safe_globals/src/cpu.rs ++++ 06_drivers_gpio_uart/src/cpu.rs +@@ -15,4 +15,4 @@ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_cpu::wait_forever; ++pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; + diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs --- 05_safe_globals/src/driver.rs +++ 06_drivers_gpio_uart/src/driver.rs @@ -1264,30 +1276,15 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs --- 05_safe_globals/src/main.rs +++ 06_drivers_gpio_uart/src/main.rs -@@ -7,6 +7,14 @@ - - //! The `kernel` binary. - //! -+//! # TL;DR - Overview of important Kernel entities -+//! -+//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -+//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -+//! -+//! [console interface]: ../libkernel/console/interface/index.html -+//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -+//! - //! # Code organization and architecture +@@ -106,6 +106,7 @@ //! - //! The code is divided into different *modules*, each representing a typical **subsystem** of the -@@ -92,6 +100,7 @@ - //! - `crate::memory::*` - //! - `crate::bsp::memory::*` + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -104,6 +113,7 @@ +@@ -115,6 +116,7 @@ mod bsp; mod console; mod cpu; @@ -1295,7 +1292,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -115,16 +125,49 @@ +@@ -126,16 +128,49 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1303,7 +1300,8 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs unsafe fn kernel_init() -> ! { - use console::interface::Statistics; + use driver::interface::DriverManager; -+ + +- println!("[0] Hello from pure Rust!"); + for i in bsp::driver::driver_manager().all_device_drivers().iter() { + if let Err(x) = i.init() { + panic!("Error loading driver: {}: {}", i.compatible(), x); @@ -1315,8 +1313,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs + // Transition from unsafe to safe. + kernel_main() +} - -- println!("[0] Hello from pure Rust!"); ++ +/// The main function running after the early init. +fn kernel_main() -> ! { + use bsp::console::console; diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index 2d5d0b04a..e16fdecc8 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -3,35 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/06_drivers_gpio_uart/src/bsp.rs b/06_drivers_gpio_uart/src/bsp.rs index 257502491..c558922f7 100644 --- a/06_drivers_gpio_uart/src/bsp.rs +++ b/06_drivers_gpio_uart/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/06_drivers_gpio_uart/src/console.rs b/06_drivers_gpio_uart/src/console.rs index 3552823cc..c3154ba23 100644 --- a/06_drivers_gpio_uart/src/console.rs +++ b/06_drivers_gpio_uart/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/06_drivers_gpio_uart/src/cpu.rs b/06_drivers_gpio_uart/src/cpu.rs index c9e5af72c..103f00dba 100644 --- a/06_drivers_gpio_uart/src/cpu.rs +++ b/06_drivers_gpio_uart/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; diff --git a/06_drivers_gpio_uart/src/cpu/boot.rs b/06_drivers_gpio_uart/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/06_drivers_gpio_uart/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/06_drivers_gpio_uart/src/cpu/smp.rs b/06_drivers_gpio_uart/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/06_drivers_gpio_uart/src/cpu/smp.rs +++ b/06_drivers_gpio_uart/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index 1674b9e42..a16c0de7b 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -7,14 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -28,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -45,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -99,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] @@ -107,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/06_drivers_gpio_uart/src/print.rs b/06_drivers_gpio_uart/src/print.rs index 59ec9e8ee..fa6451c75 100644 --- a/06_drivers_gpio_uart/src/print.rs +++ b/06_drivers_gpio_uart/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/07_uart_chainloader/Makefile b/07_uart_chainloader/Makefile index 1fc61b6b2..d32147d65 100644 --- a/07_uart_chainloader/Makefile +++ b/07_uart_chainloader/Makefile @@ -42,8 +42,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index fe6251109..0bd4bce20 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -24,20 +24,23 @@ tutorials in a quick manner. Our chainloader is called `MiniLoad` and is inspired by [raspbootin]. You can try it with this tutorial already: -1. Depending on your target hardware:`make` or `BSP=rpi4 make`. +1. Depending on your target hardware, run:`make` or `BSP=rpi4 make`. 1. Copy `kernel8.img` to the SD card. -1. Execute `make chainboot` or `BSP=rpi4 make chainboot`. +1. Run `make chainboot` or `BSP=rpi4 make chainboot`. 1. Connect the USB serial to your host PC. - Wiring diagram at [top-level README](../README.md#-usb-serial-output). - Make sure that you **DID NOT** connect the power pin of the USB serial. Only RX/TX and GND. 1. Connect the RPi to the (USB) power cable. 1. Observe the loader fetching a kernel over `UART`: -> ❗ **NOTE**: By default, `make chainboot` tries to connect to `/dev/ttyUSB0`. -> Should the USB serial on your system have a different name, you have to provide it explicitly. For -> example: -> -> `DEV_SERIAL=/dev/tty.usbserial-0001 make chainboot` +> ❗ **NOTE**: `make chainboot` assumes a default serial device name of `/dev/ttyUSB0`. Depending on +> your host operating system, the device name might differ. For example, on `macOS`, it might be +> something like `/dev/tty.usbserial-0001`. In this case, please give the name explicitly: + + +```console +$ DEV_SERIAL=/dev/tty.usbserial-0001 make chainboot +``` [raspbootin]: https://github.com/mrvn/raspbootin @@ -127,7 +130,7 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile endif # Export for build.rs -@@ -67,13 +69,14 @@ +@@ -68,13 +70,14 @@ ifeq ($(UNAME_S),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) @@ -145,7 +148,7 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile all: $(KERNEL_BIN) -@@ -87,15 +90,18 @@ +@@ -88,15 +91,18 @@ $(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) @@ -167,7 +170,7 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile clippy: RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -107,7 +113,10 @@ +@@ -108,7 +114,10 @@ readelf --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) @@ -180,10 +183,10 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile nm: $(KERNEL_ELF) @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs ---- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -+++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs -@@ -22,11 +22,11 @@ +diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs +--- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs ++++ 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs +@@ -29,11 +29,11 @@ /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { @@ -196,8 +199,12 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/ + relocate::relocate_self() } else { // If not core0, infinitely wait for events. - wait_forever() -@@ -54,3 +54,19 @@ + cpu::wait_forever() + +diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs +--- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs ++++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs +@@ -34,3 +34,19 @@ asm::wfe() } } @@ -383,12 +390,28 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader unsafe { range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); +diff -uNr 06_drivers_gpio_uart/src/cpu.rs 07_uart_chainloader/src/cpu.rs +--- 06_drivers_gpio_uart/src/cpu.rs ++++ 07_uart_chainloader/src/cpu.rs +@@ -15,4 +15,4 @@ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; ++pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; + diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs +++ 07_uart_chainloader/src/main.rs -@@ -100,7 +100,9 @@ - //! - `crate::memory::*` - //! - `crate::bsp::memory::*` +@@ -102,11 +102,14 @@ + //! + //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +-//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. ++//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. ++//! 3. Finally, [`runtime_init::runtime_init()`] is called. + //! + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -396,17 +419,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -108,7 +110,8 @@ - #![no_std] - - // `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls --// `runtime_init()`, which jumps to `kernel_init()`. -+// `relocate::relocate_self()`. `relocate::relocate_self()` calls `runtime_init()`, which jumps to -+// `kernel_init()`. - - mod bsp; - mod console; -@@ -117,6 +120,7 @@ +@@ -120,6 +123,7 @@ mod memory; mod panic_wait; mod print; @@ -414,7 +427,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -145,29 +149,49 @@ +@@ -148,29 +152,49 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -478,7 +491,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + println!("[ML] Loaded! Executing the payload now\n"); + console().flush(); + -+ // Use black magic to get a function pointer. ++ // Use black magic to create a function pointer. + let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; + + // Jump to loaded kernel! diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 47a0ae1037b18fa72ffdd9262c664c64bfc89289..0719de414d1e2a8495d6c68862e25eec89998ba2 100755 GIT binary patch delta 14 VcmX?Mdct(W3~|QJ%`?UInE@~i1$h7f delta 14 VcmX?Mdct(W3~|P`%`?UInE@~Q1$F=c diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index 14e578c1784483707bb257c94ea90a0cb9d7cb97..c8d8f9685adcdfcaeeb113672567c4f264e52f06 100755 GIT binary patch delta 14 VcmX?Ma>8VTgg9g8W=U~8VTgg9f{W=U~ //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::relocate; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - relocate::relocate_self() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..b343792f3 --- /dev/null +++ b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::relocate; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + relocate::relocate_self() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs b/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/07_uart_chainloader/src/bsp.rs b/07_uart_chainloader/src/bsp.rs index 257502491..c558922f7 100644 --- a/07_uart_chainloader/src/bsp.rs +++ b/07_uart_chainloader/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/07_uart_chainloader/src/console.rs b/07_uart_chainloader/src/console.rs index 3552823cc..c3154ba23 100644 --- a/07_uart_chainloader/src/console.rs +++ b/07_uart_chainloader/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/07_uart_chainloader/src/cpu.rs b/07_uart_chainloader/src/cpu.rs index c9e5af72c..c543bd988 100644 --- a/07_uart_chainloader/src/cpu.rs +++ b/07_uart_chainloader/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; diff --git a/07_uart_chainloader/src/cpu/boot.rs b/07_uart_chainloader/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/07_uart_chainloader/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/07_uart_chainloader/src/cpu/smp.rs b/07_uart_chainloader/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/07_uart_chainloader/src/cpu/smp.rs +++ b/07_uart_chainloader/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index c8f2a91f6..fdb787b56 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -7,14 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -28,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -45,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -99,6 +97,15 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. +//! 3. Finally, [`runtime_init::runtime_init()`] is called. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -109,10 +116,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `relocate::relocate_self()`. `relocate::relocate_self()` calls `runtime_init()`, which jumps to -// `kernel_init()`. - mod bsp; mod console; mod cpu; @@ -189,7 +192,7 @@ fn kernel_main() -> ! { println!("[ML] Loaded! Executing the payload now\n"); console().flush(); - // Use black magic to get a function pointer. + // Use black magic to create a function pointer. let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; // Jump to loaded kernel! diff --git a/07_uart_chainloader/src/print.rs b/07_uart_chainloader/src/print.rs index 59ec9e8ee..fa6451c75 100644 --- a/07_uart_chainloader/src/print.rs +++ b/07_uart_chainloader/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/08_timestamps/Makefile b/08_timestamps/Makefile index 071167b9e..27944c93d 100644 --- a/08_timestamps/Makefile +++ b/08_timestamps/Makefile @@ -40,8 +40,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/08_timestamps/README.md b/08_timestamps/README.md index b79121c0f..432ac091a 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -2,7 +2,8 @@ ## tl;dr -- We add abstractions for the architectural timer and implement them for `_arch/aarch64`. +- We add abstractions for timer hardware, and implement them for the ARM architectural timer in + `_arch/aarch64`. - The new timer functions are used to annotate UART prints with timestamps, and to get rid of the cycle-based delays in the `GPIO` and `UART` device drivers, which boosts accuracy. - A `warn!()` macro is added. @@ -68,7 +69,7 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile endif # Export for build.rs -@@ -75,8 +73,7 @@ +@@ -76,8 +74,7 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) EXEC_MINIPUSH = ruby ../utils/minipush.rb @@ -78,7 +79,7 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile all: $(KERNEL_BIN) -@@ -90,18 +87,15 @@ +@@ -91,18 +88,15 @@ $(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) @@ -100,7 +101,7 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile clippy: RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -113,10 +107,7 @@ +@@ -114,10 +108,7 @@ readelf --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) @@ -113,10 +114,10 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile nm: $(KERNEL_ELF) @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs ---- 07_uart_chainloader/src/_arch/aarch64/cpu.rs -+++ 08_timestamps/src/_arch/aarch64/cpu.rs -@@ -22,11 +22,11 @@ +diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs 08_timestamps/src/_arch/aarch64/cpu/boot.rs +--- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs ++++ 08_timestamps/src/_arch/aarch64/cpu/boot.rs +@@ -29,11 +29,11 @@ /// actually set (`SP.set()`). #[no_mangle] pub unsafe fn _start() -> ! { @@ -129,8 +130,12 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a + runtime_init::runtime_init() } else { // If not core0, infinitely wait for events. - wait_forever() -@@ -39,14 +39,6 @@ + cpu::wait_forever() + +diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs +--- 07_uart_chainloader/src/_arch/aarch64/cpu.rs ++++ 08_timestamps/src/_arch/aarch64/cpu.rs +@@ -19,14 +19,6 @@ pub use asm::nop; @@ -145,7 +150,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -54,19 +46,3 @@ +@@ -34,19 +26,3 @@ asm::wfe() } } @@ -169,12 +174,19 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/aarch64/time.rs --- 07_uart_chainloader/src/_arch/aarch64/time.rs +++ 08_timestamps/src/_arch/aarch64/time.rs -@@ -0,0 +1,98 @@ +@@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural timer primitives. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::time::arch_time + +use crate::{time, warn}; +use core::time::Duration; @@ -227,7 +239,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ + } + + // Calculate the register compare value. -+ let frq = CNTFRQ_EL0.get() as u64; ++ let frq = CNTFRQ_EL0.get(); + let x = match frq.checked_mul(duration.as_nanos() as u64) { + None => { + warn!("Spin duration too long, skipping"); @@ -483,24 +495,28 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs unsafe { range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); +diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs +--- 07_uart_chainloader/src/cpu.rs ++++ 08_timestamps/src/cpu.rs +@@ -15,4 +15,4 @@ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; ++pub use arch_cpu::{nop, wait_forever}; + diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs --- 07_uart_chainloader/src/main.rs +++ 08_timestamps/src/main.rs -@@ -11,9 +11,11 @@ - //! - //! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. - //! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -+//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. +@@ -102,14 +102,11 @@ //! - //! [console interface]: ../libkernel/console/interface/index.html - //! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -+//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html + //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +-//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. +-//! 3. Finally, [`runtime_init::runtime_init()`] is called. ++//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! - //! # Code organization and architecture - //! -@@ -100,9 +102,7 @@ - //! - `crate::memory::*` - //! - `crate::bsp::memory::*` + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -508,17 +524,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -110,8 +110,7 @@ - #![no_std] - - // `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls --// `relocate::relocate_self()`. `relocate::relocate_self()` calls `runtime_init()`, which jumps to --// `kernel_init()`. -+// `runtime_init()`, which jumps to `kernel_init()`. - - mod bsp; - mod console; -@@ -120,9 +119,9 @@ +@@ -123,9 +120,9 @@ mod memory; mod panic_wait; mod print; @@ -529,7 +535,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs /// Early init code. /// -@@ -147,51 +146,31 @@ +@@ -150,51 +147,31 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -593,7 +599,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs + // Test a failing timer case. + time::time_manager().spin_for(Duration::from_nanos(1)); -- // Use black magic to get a function pointer. +- // Use black magic to create a function pointer. - let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; - - // Jump to loaded kernel! @@ -760,7 +766,7 @@ diff -uNr 07_uart_chainloader/src/runtime_init.rs 08_timestamps/src/runtime_init diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs --- 07_uart_chainloader/src/time.rs +++ 08_timestamps/src/time.rs -@@ -0,0 +1,35 @@ +@@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -770,7 +776,11 @@ diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/time.rs"] +mod arch_time; -+pub use arch_time::*; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_time::time_manager; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -781,8 +791,6 @@ diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs + use core::time::Duration; + + /// Time management functions. -+ /// -+ /// The `BSP` is supposed to supply one global instance. + pub trait TimeManager { + /// The timer's resolution. + fn resolution(&self) -> Duration; diff --git a/08_timestamps/src/_arch/aarch64/cpu.rs b/08_timestamps/src/_arch/aarch64/cpu.rs index a65a06827..d3b07461a 100644 --- a/08_timestamps/src/_arch/aarch64/cpu.rs +++ b/08_timestamps/src/_arch/aarch64/cpu.rs @@ -3,35 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/08_timestamps/src/_arch/aarch64/cpu/boot.rs b/08_timestamps/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/08_timestamps/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/08_timestamps/src/_arch/aarch64/cpu/smp.rs b/08_timestamps/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/08_timestamps/src/_arch/aarch64/cpu/smp.rs +++ b/08_timestamps/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/08_timestamps/src/_arch/aarch64/time.rs b/08_timestamps/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/08_timestamps/src/_arch/aarch64/time.rs +++ b/08_timestamps/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/08_timestamps/src/bsp.rs b/08_timestamps/src/bsp.rs index 257502491..c558922f7 100644 --- a/08_timestamps/src/bsp.rs +++ b/08_timestamps/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/08_timestamps/src/console.rs b/08_timestamps/src/console.rs index 3552823cc..c3154ba23 100644 --- a/08_timestamps/src/console.rs +++ b/08_timestamps/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/08_timestamps/src/cpu.rs b/08_timestamps/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/08_timestamps/src/cpu.rs +++ b/08_timestamps/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/08_timestamps/src/cpu/boot.rs b/08_timestamps/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/08_timestamps/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/08_timestamps/src/cpu/smp.rs b/08_timestamps/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/08_timestamps/src/cpu/smp.rs +++ b/08_timestamps/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index b9db5fef4..5a7b9f550 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -7,16 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -30,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -47,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -101,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] @@ -109,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/08_timestamps/src/print.rs b/08_timestamps/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/08_timestamps/src/print.rs +++ b/08_timestamps/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/08_timestamps/src/time.rs b/08_timestamps/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/08_timestamps/src/time.rs +++ b/08_timestamps/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/09_hw_debug_JTAG/Makefile b/09_hw_debug_JTAG/Makefile index 9254018cf..9105d297f 100644 --- a/09_hw_debug_JTAG/Makefile +++ b/09_hw_debug_JTAG/Makefile @@ -44,8 +44,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/09_hw_debug_JTAG/README.md b/09_hw_debug_JTAG/README.md index 12cf0cc1e..918f72582 100644 --- a/09_hw_debug_JTAG/README.md +++ b/09_hw_debug_JTAG/README.md @@ -287,8 +287,8 @@ Hence, please ensure the following order of connecting the devices to your box: 1. Connect the USB serial. 2. Afterwards, the Olimex debugger. -This way, Linux enumerates the devices accordingly. This has to be done only once. It is fine to -disconnect and connect the serial multiple times, e.g. for kicking off different `make jtagboot` +This way, the host OS enumerates the devices accordingly. This has to be done only once. It is fine +to disconnect and connect the serial multiple times, e.g. for kicking off different `make jtagboot` runs, while keeping the debugger connected. ## Additional resources @@ -324,7 +324,7 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -@@ -58,9 +62,12 @@ +@@ -59,9 +63,12 @@ DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils @@ -337,7 +337,7 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux -@@ -68,12 +75,17 @@ +@@ -69,12 +76,17 @@ DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) @@ -356,7 +356,7 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile all: $(KERNEL_BIN) -@@ -97,6 +109,23 @@ +@@ -98,6 +110,23 @@ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs index a65a06827..d3b07461a 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs @@ -3,35 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/09_hw_debug_JTAG/src/bsp.rs b/09_hw_debug_JTAG/src/bsp.rs index 257502491..c558922f7 100644 --- a/09_hw_debug_JTAG/src/bsp.rs +++ b/09_hw_debug_JTAG/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/09_hw_debug_JTAG/src/console.rs b/09_hw_debug_JTAG/src/console.rs index 3552823cc..c3154ba23 100644 --- a/09_hw_debug_JTAG/src/console.rs +++ b/09_hw_debug_JTAG/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/09_hw_debug_JTAG/src/cpu.rs b/09_hw_debug_JTAG/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/09_hw_debug_JTAG/src/cpu.rs +++ b/09_hw_debug_JTAG/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/09_hw_debug_JTAG/src/cpu/boot.rs b/09_hw_debug_JTAG/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/09_hw_debug_JTAG/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/09_hw_debug_JTAG/src/cpu/smp.rs b/09_hw_debug_JTAG/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/09_hw_debug_JTAG/src/cpu/smp.rs +++ b/09_hw_debug_JTAG/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index b9db5fef4..5a7b9f550 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -7,16 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -30,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -47,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -101,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] @@ -109,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/09_hw_debug_JTAG/src/print.rs b/09_hw_debug_JTAG/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/09_hw_debug_JTAG/src/print.rs +++ b/09_hw_debug_JTAG/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/09_hw_debug_JTAG/src/time.rs b/09_hw_debug_JTAG/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/09_hw_debug_JTAG/src/time.rs +++ b/09_hw_debug_JTAG/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/10_privilege_level/Makefile b/10_privilege_level/Makefile index 9254018cf..9105d297f 100644 --- a/10_privilege_level/Makefile +++ b/10_privilege_level/Makefile @@ -44,8 +44,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 8cd313810..2556ec993 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -33,16 +33,14 @@ architectures, please have a look at the following links: - [RISC-V privilege modes](https://content.riscv.org/wp-content/uploads/2017/12/Tue0942-riscv-hypervisor-waterman.pdf). At this point, I strongly recommend that you glimpse over `Chapter 3` of the [Programmer’s Guide for -ARMv8-A](http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/DEN0024A_v8_architecture_PG.pdf) -before you continue. It gives a concise overview about the topic. +ARMv8-A] before you continue. It gives a concise overview about the topic. -## Scope of this tutorial +[Programmer’s Guide forARMv8-A]: http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/DEN0024A_v8_architecture_PG.pdf -If you set up your SD Card exactly like mentioned in [tutorial 06], the Rpi will always start -executing in `EL2`. Since we are writing a traditional `Kernel`, we have to transition into the more -appropriate `EL1`. +## Scope of this tutorial -[tutorial 06]: https://github.com/rust-embedded/rust-raspberrypi-OS-tutorials/tree/master/06_drivers_gpio_uart#boot-it-from-sd-card +By default, the Rpi will always start executing in `EL2`. Since we are writing a traditional +`Kernel`, we have to transition into the more appropriate `EL1`. ## Checking for EL2 in the entrypoint @@ -50,7 +48,8 @@ First of all, we need to ensure that we actually execute in `EL2` before we can to transition to `EL1`: ```rust -pub unsafe extern "C" fn _start() -> ! { +#[no_mangle] +pub unsafe fn _start() -> ! { // Expect the boot core to start in EL2. if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) && (CurrentEL.get() == CurrentEL::EL::EL2.value) @@ -58,7 +57,7 @@ pub unsafe extern "C" fn _start() -> ! { el2_to_el1_transition() } else { // If not core0, infinitely wait for events. - wait_forever() + cpu::wait_forever() } } ``` @@ -74,7 +73,7 @@ We are already using them since [tutorial 08](../08_timestamps/), so of course w Therefore we set the respective flags in the [Counter-timer Hypervisor Control register] and additionally set the virtual offset to zero so that we get the real physical value everytime: -[Counter-timer Hypervisor Control register]: https://docs.rs/cortex-a/2.4.0/src/cortex_a/regs/cnthctl_el2.rs.html +[Counter-timer Hypervisor Control register]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/cnthctl_el2.rs.html ```rust // Enable timer counter registers for EL1. @@ -84,10 +83,10 @@ CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); CNTVOFF_EL2.set(0); ``` -Next, we configure the [Hypervisor Configuration Register] such that `EL1` should actually run in -`AArch64` mode, and not in `AArch32`, which would also be possible. +Next, we configure the [Hypervisor Configuration Register] such that `EL1` runs in `AArch64` mode, +and not in `AArch32`, which would also be possible. -[Hypervisor Configuration Register]: https://docs.rs/cortex-a/2.4.0/src/cortex_a/regs/hcr_el2.rs.html +[Hypervisor Configuration Register]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/hcr_el2.rs.html ```rust // Set EL1 execution state to AArch64. @@ -99,7 +98,7 @@ HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); There is actually only one way to transition from a higher EL to a lower EL, which is by way of executing the [ERET] instruction. -[ERET]: https://docs.rs/cortex-a/2.4.0/src/cortex_a/asm.rs.html#49-62 +[ERET]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/asm.rs.html#87-96 This instruction will copy the contents of the [Saved Program Status Register - EL2] to `Current Program Status Register - EL1` and jump to the instruction address that is stored in the [Exception @@ -108,8 +107,8 @@ Link Register - EL2]. This is basically the reverse of what is happening when an exception is taken. You'll learn about it in an upcoming tutorial. -[Saved Program Status Register - EL2]: https://docs.rs/cortex-a/2.4.0/src/cortex_a/regs/spsr_el2.rs.html -[Exception Link Register - EL2]: https://docs.rs/cortex-a/2.4.0/src/cortex_a/regs/elr_el2.rs.html +[Saved Program Status Register - EL2]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/spsr_el2.rs.html +[Exception Link Register - EL2]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/elr_el2.rs.html ```rust // Set up a simulated exception return. @@ -156,23 +155,23 @@ Disassembly of section .text: 0000000000080000 <_start>: 80000: d53800a8 mrs x8, mpidr_el1 80004: f240051f tst x8, #0x3 - 80008: 54000081 b.ne 80018 <_start+0x18> + 80008: 54000081 b.ne 80018 <_start+0x18> // b.any 8000c: d5384248 mrs x8, currentel 80010: f100211f cmp x8, #0x8 - 80014: 54000060 b.eq 80020 <_start+0x20> + 80014: 54000060 b.eq 80020 <_start+0x20> // b.none 80018: d503205f wfe 8001c: 17ffffff b 80018 <_start+0x18> 80020: aa1f03e8 mov x8, xzr - 80024: 52800069 mov w9, #0x3 + 80024: 52800069 mov w9, #0x3 // #3 80028: d51ce109 msr cnthctl_el2, x9 8002c: d51ce068 msr cntvoff_el2, x8 - 80030: 90000008 adrp x8, 80000 <_start> - 80034: 52b0000a mov w10, #0x80000000 - 80038: 528078ab mov w11, #0x3c5 - 8003c: 52a0010c mov w12, #0x80000 + 80030: d0000008 adrp x8, 82000 + 80034: 52b0000a mov w10, #0x80000000 // #-2147483648 + 80038: 528078ab mov w11, #0x3c5 // #965 + 8003c: 52a0010c mov w12, #0x80000 // #524288 80040: d51c110a msr hcr_el2, x10 80044: d51c400b msr spsr_el2, x11 - 80048: 91374108 add x8, x8, #0xdd0 + 80048: 9114c108 add x8, x8, #0x530 8004c: d51c4028 msr elr_el2, x8 80050: d51c410c msr sp_el1, x12 80054: d69f03e0 eret @@ -223,35 +222,20 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch/aarch64/cpu.rs ---- 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs -+++ 10_privilege_level/src/_arch/aarch64/cpu.rs -@@ -18,21 +18,65 @@ - /// # Safety - /// - /// - Linker script must ensure to place this function where it is expected by the target machine. --/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is --/// actually set (`SP.set()`). -+/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -+/// a stack for EL2. - #[no_mangle] - pub unsafe fn _start() -> ! { -- use crate::runtime_init; -- -- if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { -- SP.set(bsp::memory::boot_core_stack_end() as u64); -- runtime_init::runtime_init() -+ // Expect the boot core to start in EL2. -+ if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) -+ && (CurrentEL.get() == CurrentEL::EL::EL2.value) -+ { -+ el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } - } +diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 10_privilege_level/src/_arch/aarch64/cpu/boot.rs +--- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs ++++ 10_privilege_level/src/_arch/aarch64/cpu/boot.rs +@@ -12,7 +12,55 @@ + //! crate::cpu::boot::arch_boot + use crate::{bsp, cpu}; +-use cortex_a::regs::*; ++use cortex_a::{asm, regs::*}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Code ++//-------------------------------------------------------------------------------------------------- ++ +/// Transition from EL2 to EL1. +/// +/// # Safety @@ -295,20 +279,49 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs 10_privilege_level/src/_arch + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} -+ + //-------------------------------------------------------------------------------------------------- // Public Code - //-------------------------------------------------------------------------------------------------- +@@ -25,15 +73,15 @@ + /// # Safety + /// + /// - Linker script must ensure to place this function where it is expected by the target machine. +-/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +-/// actually set (`SP.set()`). ++/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up ++/// a stack for EL2. + #[no_mangle] + pub unsafe fn _start() -> ! { +- use crate::runtime_init; +- +- if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { +- SP.set(bsp::memory::boot_core_stack_end() as u64); +- runtime_init::runtime_init() ++ // Expect the boot core to start in EL2. ++ if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) ++ && (CurrentEL.get() == CurrentEL::EL::EL2.value) ++ { ++ el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs +++ 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs -@@ -0,0 +1,74 @@ +@@ -0,0 +1,81 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural asynchronous exception handling. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::exception::asynchronous::arch_asynchronous + +use cortex_a::regs::*; + @@ -382,12 +395,19 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privil diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs 10_privilege_level/src/_arch/aarch64/exception.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs +++ 10_privilege_level/src/_arch/aarch64/exception.rs -@@ -0,0 +1,23 @@ +@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural synchronous and asynchronous exception handling. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::exception::arch_exception + +use cortex_a::regs::*; + @@ -410,7 +430,7 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs 10_privilege_level/src diff -uNr 09_hw_debug_JTAG/src/exception/asynchronous.rs 10_privilege_level/src/exception/asynchronous.rs --- 09_hw_debug_JTAG/src/exception/asynchronous.rs +++ 10_privilege_level/src/exception/asynchronous.rs -@@ -0,0 +1,10 @@ +@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -419,13 +439,17 @@ diff -uNr 09_hw_debug_JTAG/src/exception/asynchronous.rs 10_privilege_level/src/ + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/exception/asynchronous.rs"] -+mod arch_exception_async; -+pub use arch_exception_async::*; ++mod arch_asynchronous; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_asynchronous::print_state; diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs --- 09_hw_debug_JTAG/src/exception.rs +++ 10_privilege_level/src/exception.rs -@@ -0,0 +1,26 @@ +@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -435,11 +459,15 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/exception.rs"] +mod arch_exception; -+pub use arch_exception::*; + +pub mod asynchronous; + +//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_exception::current_privilege_level; ++ ++//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + @@ -456,7 +484,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs --- 09_hw_debug_JTAG/src/main.rs +++ 10_privilege_level/src/main.rs -@@ -116,6 +116,7 @@ +@@ -117,6 +117,7 @@ mod console; mod cpu; mod driver; @@ -464,7 +492,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -146,12 +147,20 @@ +@@ -147,12 +148,20 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -485,7 +513,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -166,11 +175,15 @@ +@@ -167,11 +176,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/10_privilege_level/src/_arch/aarch64/cpu.rs b/10_privilege_level/src/_arch/aarch64/cpu.rs index 3846968b3..d3b07461a 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/10_privilege_level/src/_arch/aarch64/cpu/boot.rs b/10_privilege_level/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c2f5fe0b7 --- /dev/null +++ b/10_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs b/10_privilege_level/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/10_privilege_level/src/_arch/aarch64/exception.rs b/10_privilege_level/src/_arch/aarch64/exception.rs index b6ee28b8c..d8c617b2a 100644 --- a/10_privilege_level/src/_arch/aarch64/exception.rs +++ b/10_privilege_level/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use cortex_a::regs::*; diff --git a/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs b/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs index 445f490eb..b63b00fe7 100644 --- a/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs +++ b/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/10_privilege_level/src/_arch/aarch64/time.rs b/10_privilege_level/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/10_privilege_level/src/_arch/aarch64/time.rs +++ b/10_privilege_level/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/10_privilege_level/src/bsp.rs b/10_privilege_level/src/bsp.rs index 257502491..c558922f7 100644 --- a/10_privilege_level/src/bsp.rs +++ b/10_privilege_level/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/10_privilege_level/src/console.rs b/10_privilege_level/src/console.rs index 3552823cc..c3154ba23 100644 --- a/10_privilege_level/src/console.rs +++ b/10_privilege_level/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/10_privilege_level/src/cpu.rs b/10_privilege_level/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/10_privilege_level/src/cpu.rs +++ b/10_privilege_level/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/10_privilege_level/src/cpu/boot.rs b/10_privilege_level/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/10_privilege_level/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/10_privilege_level/src/cpu/smp.rs b/10_privilege_level/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/10_privilege_level/src/cpu/smp.rs +++ b/10_privilege_level/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/10_privilege_level/src/exception.rs b/10_privilege_level/src/exception.rs index 432c606b9..87d4db63a 100644 --- a/10_privilege_level/src/exception.rs +++ b/10_privilege_level/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::current_privilege_level; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/10_privilege_level/src/exception/asynchronous.rs b/10_privilege_level/src/exception/asynchronous.rs index fbdba9579..566efe329 100644 --- a/10_privilege_level/src/exception/asynchronous.rs +++ b/10_privilege_level/src/exception/asynchronous.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::print_state; diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index fc2bd6912..741883437 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -7,16 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -30,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -47,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -101,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] @@ -109,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/10_privilege_level/src/print.rs b/10_privilege_level/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/10_privilege_level/src/print.rs +++ b/10_privilege_level/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/10_privilege_level/src/time.rs b/10_privilege_level/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/10_privilege_level/src/time.rs +++ b/10_privilege_level/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/11_virtual_mem_part1_identity_mapping/Makefile b/11_virtual_mem_part1_identity_mapping/Makefile index 9254018cf..9105d297f 100644 --- a/11_virtual_mem_part1_identity_mapping/Makefile +++ b/11_virtual_mem_part1_identity_mapping/Makefile @@ -44,8 +44,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index c64442967..0d24f17fb 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -3,7 +3,7 @@ ## tl;dr - The `MMU` is turned on. -- A simple scheme is used: static `64 KiB` translation tables. +- A simple scheme is used: Static `64 KiB` translation tables. - For educational purposes, we write to a remapped `UART`, and `identity map` everything else. ## Table of Contents @@ -13,7 +13,7 @@ - [Approach](#approach) * [Generic Kernel code: `memory/mmu.rs`](#generic-kernel-code-memorymmurs) * [BSP: `bsp/raspberrypi/memory/mmu.rs`](#bsp-bspraspberrypimemorymmurs) - * [AArch64: `_arch/aarch64/memory/mmu.rs`](#aarch64-_archaarch64memorymmurs) + * [AArch64: `_arch/aarch64/memory/*`](#aarch64-_archaarch64memory) * [`link.ld`](#linkld) - [Address translation examples](#address-translation-examples) * [Address translation using a 64 KiB page descriptor](#address-translation-using-a-64-kib-page-descriptor) @@ -25,8 +25,8 @@ Virtual memory is an immensely complex, but important and powerful topic. In this tutorial, we start slow and easy by switching on the `MMU`, using static translation tables and `identity-map` -everything at once (except for the `UART`, which we remap for educational purposes; This will be -gone again in the next tutorial). +everything at once (except for the `UART`, which we also remap a second time for educational +purposes; This will be gone again in the next tutorial). ## MMU and paging theory @@ -44,14 +44,15 @@ Back from reading `Chapter 12` already? Good job :+1:! ## Approach -1. The generic `kernel` part: `src/memory/mmu.rs` provides architecture-agnostic descriptor types - for composing a high-level data structure that describes the kernel's virtual memory layout: - `memory::mmu::KernelVirtualLayout`. +1. The generic `kernel` part: `src/memory/mmu.rs` and its submodules provide architecture-agnostic + descriptor types for composing a high-level data structure that describes the kernel's virtual + memory layout: `memory::mmu::KernelVirtualLayout`. 2. The `BSP` part: `src/bsp/raspberrypi/memory/mmu.rs` contains a static instance of `KernelVirtualLayout` and makes it accessible through the function `bsp::memory::mmu::virt_mem_layout()`. -3. The `aarch64` part: `src/_arch/aarch64/memory/mmu.rs` contains the actual `MMU` driver. It picks - up the `BSP`'s high-level `KernelVirtualLayout` and maps it using a `64 KiB` granule. +3. The `aarch64` part: `src/_arch/aarch64/memory/mmu.rs` and its submodules contain the actual `MMU` + driver. It picks up the `BSP`'s high-level `KernelVirtualLayout` and maps it using a `64 KiB` + granule. ### Generic Kernel code: `memory/mmu.rs` @@ -97,42 +98,63 @@ pub fn virt_addr_properties( ) -> Result<(usize, AttributeFields), &'static str> ``` -It will be used by the `_arch/aarch64`'s `MMU` code to request attributes for a virtual address and -the translation, which delivers the physical output address (the `usize` in the return-tuple). The +It will be used by `_arch/aarch64`'s `MMU` code to request attributes for a virtual address and the +translation, which delivers the physical output address (the `usize` in the return-tuple). The function scans for a descriptor that contains the queried address, and returns the respective findings for the first entry that is a hit. If no entry is found, it returns default attributes for normal chacheable DRAM and the input address, hence telling the `MMU` code that the requested address should be `identity mapped`. -Due to this default return, it is technicall not needed to define normal cacheable DRAM regions. +Due to this default behavior, it is not needed to define normal cacheable DRAM regions. -### AArch64: `_arch/aarch64/memory/mmu.rs` +### AArch64: `_arch/aarch64/memory/*` -This file contains the `AArch64` `MMU` driver. The granule is hardcoded here (`64 KiB` page +These modules contain the `AArch64` `MMU` driver. The granule is hardcoded here (`64 KiB` page descriptors). -The actual translation tables are stored in a global instance of the `ArchTranslationTable` struct: +In `translation_table.rs`, there is a definition of the actual translation table struct which is +generic over the number of `LVL2` tables. The latter depends on the size of the target board's +memory. Naturally, the `BSP` knows these details about the target board, and provides the size +through the constant `bsp::memory::mmu::KernelAddrSpaceSize::SIZE`. + +This information is used by `translation_table.rs` to calculate the number of needed `LVL2` tables. +Since one `LVL2` table in a `64 KiB` configuration covers `512 MiB`, all that needs to be done is to +divide `KernelAddrSpaceSize::SIZE` by `512 MiB` (there are several compile-time checks in place that +ensure that `KernelAddrSpaceSize` is a multiple of `512 MiB`). + +The final table type is exported as `KernelTranslationTable`. Below is the respective excerpt from +`translation_table.rs`: ```rust /// A table descriptor for 64 KiB aperture. /// /// The output points to the next table. #[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(InMemoryRegister); +#[repr(C)] +struct TableDescriptor { + value: u64, +} /// A page descriptor with 64 KiB aperture. /// /// The output points to physical memory. #[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(InMemoryRegister); +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB /// aligned, hence the "reverse" order of appearance. #[repr(C)] #[repr(align(65536))] -struct FixedSizeTranslationTable { +pub struct FixedSizeTranslationTable { /// Page descriptors, covering 64 KiB windows per entry. lvl3: [[PageDescriptor; 8192]; NUM_TABLES], @@ -140,55 +162,69 @@ struct FixedSizeTranslationTable { lvl2: [TableDescriptor; NUM_TABLES], } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; +``` + +In `mmu.rs`, `KernelTranslationTable` is then used to create the final instance of the kernel's +tables: +```rust //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); +static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); ``` -They are populated using `bsp::memory::mmu::virt_mem_layout().virt_addr_properties()` and a bunch of -utility functions that convert our own descriptors to the actual `64 bit` integer entries needed by -the `MMU` hardware for the translation table arrays. +They are populated during `MMU::init()` by calling `KERNEL_TABLES.populate_tt_entries()`, which +utilizes `bsp::memory::mmu::virt_mem_layout().virt_addr_properties()` and a bunch of utility +functions that convert the kernel generic descriptors to the actual `64 bit` integer entries needed +by the `AArch64 MMU` hardware for the translation table arrays. -Each page descriptor has an entry (`AttrIndex`) that indexes into the [MAIR_EL1] register, which -holds information about the cacheability of the respective page. We currently define normal -cacheable memory and device memory (which is not cached). +One notable thing is that each page descriptor has an entry (`AttrIndex`) that indexes into the +[MAIR_EL1] register, which holds information about the cacheability of the respective page. We +currently define normal cacheable memory and device memory (which is not cached). [MAIR_EL1]: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0500d/CIHDHJBB.html ```rust -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + // Attribute 0 - Device. MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} + ); + } ``` Afterwards, the [Translation Table Base Register 0 - EL1] is set up with the base address of the -`lvl2` tables and the [Translation Control Register - EL1] is configured. +`lvl2` tables and the [Translation Control Register - EL1] is configured: + +```rust + // Set the "Translation Table Base Register". + TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + + self.configure_translation_control(); +``` Finally, the `MMU` is turned on through the [System Control Register - EL1]. The last step also enables caching for data and instructions. -[Translation Table Base Register 0 - EL1]: https://docs.rs/crate/cortex-a/2.4.0/source/src/regs/ttbr0_el1.rs -[Translation Control Register - EL1]: https://docs.rs/crate/cortex-a/2.4.0/source/src/regs/tcr_el1.rs -[System Control Register - EL1]: https://docs.rs/crate/cortex-a/2.4.0/source/src/regs/sctlr_el1.rs +[Translation Table Base Register 0 - EL1]: https://docs.rs/crate/cortex-a/5.1.2/source/src/regs/ttbr0_el1.rs +[Translation Control Register - EL1]: https://docs.rs/crate/cortex-a/5.1.2/source/src/regs/tcr_el1.rs +[System Control Register - EL1]: https://docs.rs/crate/cortex-a/5.1.2/source/src/regs/sctlr_el1.rs ### `link.ld` @@ -227,15 +263,15 @@ Let's take a look again at the piece of code for setting up the `MAIR_EL1` regis ```rust /// Setup function for the MAIR_EL1 register. -fn set_up_mair() { +fn set_up_mair(&self) { // Define the memory types being mapped. MAIR_EL1.write( // Attribute 1 - Cacheable normal DRAM. MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + - MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + - // Attribute 0 - Device. - MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, + // Attribute 0 - Device. + MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, ); } ``` @@ -248,15 +284,26 @@ data sheet. Looking at the generated code, we can see that despite all the type- abstractions, it boils down to two assembly instructions: ```text -0000000000081660 <::init>: - ... - 816bc: 529fe088 mov w8, #0xff04 - ... - 816c4: d518a208 msr mair_el1, x8 + 800a8: 529fe089 mov w9, #0xff04 // #65284 + 800ac: d518a209 msr mair_el1, x9 ``` ## Test it +Turning on virtual memory is now the first thing we do during kernel init: + +```rust +unsafe fn kernel_init() -> ! { + use driver::interface::DriverManager; + use memory::mmu::interface::MMU; + + if let Err(string) = memory::mmu::mmu().init() { + panic!("MMU: {}", string); + } +``` + +Later in the boot process, prints about the mappings can be observed: + ```console $ make chainboot [...] @@ -299,22 +346,33 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs ---- 10_privilege_level/src/_arch/aarch64/memory/mmu.rs -+++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs -@@ -0,0 +1,343 @@ +diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +@@ -0,0 +1,288 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// -+// Copyright (c) 2018-2021 Andre Richter ++// Copyright (c) 2021 Andre Richter + -+//! Memory Management Unit Driver. ++//! Architectural translation table. +//! -+//! Static translation tables, compiled on boot; Everything 64 KiB granule. -+ -+use super::{AccessPermissions, AttributeFields, MemAttributes}; -+use crate::{bsp, memory}; ++//! Only 64 KiB granule is supported. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::memory::mmu::translation_table::arch_translation_table ++ ++use crate::{ ++ bsp, memory, ++ memory::mmu::{ ++ arch_mmu::{Granule512MiB, Granule64KiB}, ++ AccessPermissions, AttributeFields, MemAttributes, ++ }, ++}; +use core::convert; -+use cortex_a::{barrier, regs::*}; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- @@ -392,65 +450,49 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + ] +} + -+const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -+const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) -+ +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] -+#[repr(transparent)] -+struct TableDescriptor(u64); ++#[repr(C)] ++struct TableDescriptor { ++ value: u64, ++} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] -+#[repr(transparent)] -+struct PageDescriptor(u64); -+ -+/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -+/// aligned, hence the "reverse" order of appearance. +#[repr(C)] -+#[repr(align(65536))] -+struct FixedSizeTranslationTable { -+ /// Page descriptors, covering 64 KiB windows per entry. -+ lvl3: [[PageDescriptor; 8192]; NUM_TABLES], -+ -+ /// Table descriptors, covering 512 MiB windows. -+ lvl2: [TableDescriptor; NUM_TABLES], ++struct PageDescriptor { ++ value: u64, +} + -+const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -+type ArchTranslationTable = FixedSizeTranslationTable; -+ +trait BaseAddr { + fn base_addr_u64(&self) -> u64; + fn base_addr_usize(&self) -> usize; +} + -+/// Constants for indexing the MAIR_EL1. -+#[allow(dead_code)] -+mod mair { -+ pub const DEVICE: u64 = 0; -+ pub const NORMAL: u64 = 1; -+} -+ -+/// Memory Management Unit type. -+struct MemoryManagementUnit; ++const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- -+// Global instances ++// Public Definitions +//-------------------------------------------------------------------------------------------------- + -+/// The translation tables. -+/// -+/// # Safety -+/// -+/// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -+static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); ++/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB ++/// aligned, hence the "reverse" order of appearance. ++#[repr(C)] ++#[repr(align(65536))] ++pub struct FixedSizeTranslationTable { ++ /// Page descriptors, covering 64 KiB windows per entry. ++ lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + -+static MMU: MemoryManagementUnit = MemoryManagementUnit; ++ /// Table descriptors, covering 512 MiB windows. ++ lvl2: [TableDescriptor; NUM_TABLES], ++} ++ ++/// A translation table type for the kernel space. ++pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code @@ -466,18 +508,26 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + } +} + -+impl convert::From for TableDescriptor { -+ fn from(next_lvl_table_addr: usize) -> Self { ++impl TableDescriptor { ++ /// Create an instance. ++ /// ++ /// Descriptor is invalid by default. ++ pub const fn new_zeroed() -> Self { ++ Self { value: 0 } ++ } ++ ++ /// Create an instance pointing to the supplied address. ++ pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + -+ let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; ++ let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + -+ TableDescriptor(val.get()) ++ TableDescriptor { value: val.get() } + } +} + @@ -490,11 +540,11 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable -+ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) ++ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable -+ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) ++ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + @@ -520,10 +570,17 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + +impl PageDescriptor { + /// Create an instance. -+ fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { ++ /// ++ /// Descriptor is invalid by default. ++ pub const fn new_zeroed() -> Self { ++ Self { value: 0 } ++ } ++ ++ /// Create an instance. ++ pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + -+ let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; ++ let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True @@ -532,73 +589,159 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + -+ Self(val.get()) ++ Self { value: val.get() } + } +} + -+impl FixedSizeTranslationTable<{ NUM_TABLES }> { ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl FixedSizeTranslationTable { + /// Create an instance. ++ #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(NUM_TABLES > 0); ++ assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); + + Self { -+ lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], -+ lvl2: [TableDescriptor(0); NUM_TABLES], ++ lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], ++ lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + } + } ++ ++ /// Iterates over all static translation table entries and fills them at once. ++ /// ++ /// # Safety ++ /// ++ /// - Modifies a `static mut`. Ensure it only happens from here. ++ pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { ++ for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { ++ *l2_entry = ++ TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); ++ ++ for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { ++ let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); ++ ++ let (output_addr, attribute_fields) = ++ bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; ++ ++ *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); ++ } ++ } ++ ++ Ok(()) ++ } ++ ++ /// The translation table's base address to be used for programming the MMU. ++ pub fn base_address(&self) -> u64 { ++ self.lvl2.base_addr_u64() ++ } +} + +diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +--- 10_privilege_level/src/_arch/aarch64/memory/mmu.rs ++++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +@@ -0,0 +1,146 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter + -+/// Setup function for the MAIR_EL1 register. -+fn set_up_mair() { -+ // Define the memory types being mapped. -+ MAIR_EL1.write( -+ // Attribute 1 - Cacheable normal DRAM. -+ MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + -+ MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + ++//! Memory Management Unit Driver. ++//! ++//! Only 64 KiB granule is supported. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::memory::mmu::arch_mmu + -+ // Attribute 0 - Device. -+ MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, -+ ); ++use crate::{ ++ bsp, memory, ++ memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, ++}; ++use cortex_a::{barrier, regs::*}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++/// Memory Management Unit type. ++struct MemoryManagementUnit; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; ++pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; ++ ++/// The min supported address space size. ++pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB ++ ++/// The max supported address space size. ++pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB ++ ++/// The supported address space size granule. ++pub type AddrSpaceSizeGranule = Granule512MiB; ++ ++/// Constants for indexing the MAIR_EL1. ++#[allow(dead_code)] ++pub mod mair { ++ pub const DEVICE: u64 = 0; ++ pub const NORMAL: u64 = 1; +} + -+/// Iterates over all static translation table entries and fills them at once. ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++/// The kernel translation tables. +/// +/// # Safety +/// -+/// - Modifies a `static mut`. Ensure it only happens from here. -+unsafe fn populate_tt_entries() -> Result<(), &'static str> { -+ for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { -+ *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); ++/// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". ++static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); ++ ++static MMU: MemoryManagementUnit = MemoryManagementUnit; + -+ for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { -+ let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); ++//-------------------------------------------------------------------------------------------------- ++// Private Code ++//-------------------------------------------------------------------------------------------------- + -+ let (output_addr, attribute_fields) = -+ bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; ++impl MemoryManagementUnit { ++ /// Setup function for the MAIR_EL1 register. ++ fn set_up_mair(&self) { ++ // Define the memory types being mapped. ++ MAIR_EL1.write( ++ // Attribute 1 - Cacheable normal DRAM. ++ MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + ++ MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + + -+ *l3_entry = PageDescriptor::new(output_addr, attribute_fields); -+ } ++ // Attribute 0 - Device. ++ MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, ++ ); + } + -+ Ok(()) -+} -+ -+/// Configure various settings of stage 1 of the EL1 translation regime. -+fn configure_translation_control() { -+ let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -+ let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); -+ -+ TCR_EL1.write( -+ TCR_EL1::TBI0::Ignored -+ + TCR_EL1::IPS.val(ips) -+ + TCR_EL1::EPD1::DisableTTBR1Walks -+ + TCR_EL1::TG0::KiB_64 -+ + TCR_EL1::SH0::Inner -+ + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable -+ + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable -+ + TCR_EL1::EPD0::EnableTTBR0Walks -+ + TCR_EL1::T0SZ.val(t0sz), -+ ); ++ /// Configure various settings of stage 1 of the EL1 translation regime. ++ fn configure_translation_control(&self) { ++ let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); ++ let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; ++ ++ TCR_EL1.write( ++ TCR_EL1::TBI0::Ignored ++ + TCR_EL1::IPS.val(ips) ++ + TCR_EL1::EPD1::DisableTTBR1Walks ++ + TCR_EL1::TG0::KiB_64 ++ + TCR_EL1::SH0::Inner ++ + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable ++ + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable ++ + TCR_EL1::EPD0::EnableTTBR0Walks ++ + TCR_EL1::T0SZ.val(t0sz), ++ ); ++ } +} + +//-------------------------------------------------------------------------------------------------- @@ -622,15 +765,15 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + } + + // Prepare the memory attribute indirection register. -+ set_up_mair(); ++ self.set_up_mair(); + + // Populate translation tables. -+ populate_tt_entries()?; ++ KERNEL_TABLES.populate_tt_entries()?; + + // Set the "Translation Table Base Register". -+ TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); ++ TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + -+ configure_translation_control(); ++ self.configure_translation_control(); + + // Switch the MMU on. + // @@ -671,7 +814,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_id diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs --- 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs -@@ -0,0 +1,93 @@ +@@ -0,0 +1,86 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -686,13 +829,16 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa +// Public Definitions +//-------------------------------------------------------------------------------------------------- + ++/// The address space size chosen by this BSP. ++pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; ++ +const NUM_MEM_RANGES: usize = 3; + +/// The virtual memory layout. +/// +/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. +/// It is agnostic of the paging granularity that the architecture's MMU will use. -+pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( ++pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( + memory_map::END_INCLUSIVE, + [ + TranslationDescriptor { @@ -751,18 +897,8 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa +// Public Code +//-------------------------------------------------------------------------------------------------- + -+/// Return the address space size in bytes. -+/// -+/// Guarantees size to be a power of two. -+pub const fn addr_space_size() -> usize { -+ let size = memory_map::END_INCLUSIVE + 1; -+ assert!(size.is_power_of_two()); -+ -+ size -+} -+ +/// Return a reference to the virtual memory layout. -+pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { ++pub fn virt_mem_layout() -> &'static KernelVirtualLayout { + &LAYOUT +} @@ -859,7 +995,7 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr +++ 11_virtual_mem_part1_identity_mapping/src/bsp.rs @@ -4,7 +4,7 @@ - //! Conditional re-exporting of Board Support Packages. + //! Conditional reexporting of Board Support Packages. -mod device_driver; +pub mod device_driver; @@ -870,22 +1006,9 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs --- 10_privilege_level/src/main.rs +++ 11_virtual_mem_part1_identity_mapping/src/main.rs -@@ -11,10 +11,12 @@ - //! - //! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. - //! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -+//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. - //! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. +@@ -106,7 +106,10 @@ //! - //! [console interface]: ../libkernel/console/interface/index.html - //! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -+//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html - //! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html - //! - //! # Code organization and architecture -@@ -102,7 +104,10 @@ - //! - `crate::memory::*` - //! - `crate::bsp::memory::*` + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +#![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] @@ -894,7 +1017,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -129,9 +134,18 @@ +@@ -130,9 +133,18 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -914,7 +1037,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -155,6 +169,9 @@ +@@ -156,6 +168,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -924,7 +1047,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -178,6 +195,13 @@ +@@ -179,6 +194,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -939,10 +1062,29 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s // Discard any spurious received characters before going into echo mode. +diff -uNr 10_privilege_level/src/memory/mmu/translation_table.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs +--- 10_privilege_level/src/memory/mmu/translation_table.rs ++++ 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs +@@ -0,0 +1,14 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2021 Andre Richter ++ ++//! Translation table. ++ ++#[cfg(target_arch = "aarch64")] ++#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] ++mod arch_translation_table; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_translation_table::KernelTranslationTable; + diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs --- 10_privilege_level/src/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs -@@ -0,0 +1,199 @@ +@@ -0,0 +1,247 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -950,8 +1092,8 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map +//! Memory Management Unit. +//! +//! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file -+//! provides types for composing an architecture-agnostic description of the kernel 's virtual -+//! memory layout. ++//! provides types for composing an architecture-agnostic description of the kernel's virtual memory ++//! layout. +//! +//! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` +//! function. @@ -962,11 +1104,17 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/memory/mmu.rs"] +mod arch_mmu; -+pub use arch_mmu::*; ++ ++mod translation_table; + +use core::{fmt, ops::RangeInclusive}; + +//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_mmu::mmu; ++ ++//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + @@ -985,8 +1133,15 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map + } +} + ++/// Describes the characteristics of a translation granule. ++pub struct TranslationGranule; ++ ++/// Describes the size of an address space. ++pub struct AddressSpaceSize; ++ +/// Architecture agnostic translation types. +#[allow(missing_docs)] ++#[allow(dead_code)] +#[derive(Copy, Clone)] +pub enum Translation { + Identity, @@ -1040,6 +1195,41 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map +// Public Code +//-------------------------------------------------------------------------------------------------- + ++impl TranslationGranule { ++ /// The granule's size. ++ pub const SIZE: usize = Self::size_checked(); ++ ++ /// The granule's shift, aka log2(size). ++ pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; ++ ++ const fn size_checked() -> usize { ++ assert!(GRANULE_SIZE.is_power_of_two()); ++ ++ GRANULE_SIZE ++ } ++} ++ ++impl AddressSpaceSize { ++ /// The address space size. ++ pub const SIZE: usize = Self::size_checked(); ++ ++ /// The address space shift, aka log2(size). ++ pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; ++ ++ const fn size_checked() -> usize { ++ assert!(AS_SIZE.is_power_of_two()); ++ assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); ++ assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); ++ ++ // Must adhere to architectural restrictions. ++ assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); ++ assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); ++ assert!((AS_SIZE modulo arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); ++ ++ AS_SIZE ++ } ++} ++ +impl Default for AttributeFields { + fn default() -> AttributeFields { + AttributeFields { diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs index 3846968b3..d3b07461a 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c2f5fe0b7 --- /dev/null +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs index b6ee28b8c..d8c617b2a 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use cortex_a::regs::*; diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs index 445f490eb..b63b00fe7 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs index 69023cef1..3504d2571 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -4,146 +4,61 @@ //! Memory Management Unit Driver. //! -//! Static translation tables, compiled on boot; Everything 64 KiB granule. +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu -use super::{AccessPermissions, AttributeFields, MemAttributes}; -use crate::{bsp, memory}; -use core::convert; +use crate::{ + bsp, memory, + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; use cortex_a::{barrier, regs::*}; -use register::{register_bitfields, InMemoryRegister}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. -register_bitfields! {u64, - STAGE1_TABLE_DESCRIPTOR [ - /// Physical address of the next descriptor. - NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. -register_bitfields! {u64, - STAGE1_PAGE_DESCRIPTOR [ - /// Unprivileged execute-never. - UXN OFFSET(54) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Privileged execute-never. - PXN OFFSET(53) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). - OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - /// Access flag. - AF OFFSET(10) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Shareability field. - SH OFFSET(8) NUMBITS(2) [ - OuterShareable = 0b10, - InnerShareable = 0b11 - ], - - /// Access Permissions. - AP OFFSET(6) NUMBITS(2) [ - RW_EL1 = 0b00, - RW_EL1_EL0 = 0b01, - RO_EL1 = 0b10, - RO_EL1_EL0 = 0b11 - ], - - /// Memory attributes index into the MAIR_EL1 register. - AttrIndx OFFSET(2) NUMBITS(3) [], - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) - -/// A table descriptor for 64 KiB aperture. -/// -/// The output points to the next table. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(u64); +/// Memory Management Unit type. +struct MemoryManagementUnit; -/// A page descriptor with 64 KiB aperture. -/// -/// The output points to physical memory. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(u64); +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- -/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. -#[repr(C)] -#[repr(align(65536))] -struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; - /// Table descriptors, covering 512 MiB windows. - lvl2: [TableDescriptor; NUM_TABLES], -} +/// The min supported address space size. +pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; +/// The max supported address space size. +pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; -} +/// The supported address space size granule. +pub type AddrSpaceSizeGranule = Granule512MiB; /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] -mod mair { +pub mod mair { pub const DEVICE: u64 = 0; pub const NORMAL: u64 = 1; } -/// Memory Management Unit type. -struct MemoryManagementUnit; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); +static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); static MMU: MemoryManagementUnit = MemoryManagementUnit; @@ -151,149 +66,37 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { - self as *const T as u64 - } - - fn base_addr_usize(&self) -> usize { - self as *const _ as usize - } -} - -impl convert::From for TableDescriptor { - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), - ); - - TableDescriptor(val.get()) - } -} - -/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. -impl convert::From - for register::FieldValue -{ - fn from(attribute_fields: AttributeFields) -> Self { - // Memory attributes. - let mut desc = match attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => { - STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) - } - MemAttributes::Device => { - STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) - } - }; - - // Access Permissions. - desc += match attribute_fields.acc_perms { - AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, - AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, - }; - - // The execute-never attribute is mapped to PXN in AArch64. - desc += if attribute_fields.execute_never { - STAGE1_PAGE_DESCRIPTOR::PXN::True - } else { - STAGE1_PAGE_DESCRIPTOR::PXN::False - }; - - // Always set unprivileged exectue-never as long as userspace is not implemented yet. - desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; - - desc - } -} - -impl PageDescriptor { - /// Create an instance. - fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), - ); - - Self(val.get()) - } -} - -impl FixedSizeTranslationTable<{ NUM_TABLES }> { - /// Create an instance. - pub const fn new() -> Self { - assert!(NUM_TABLES > 0); - - Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], - } - } -} - -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + // Attribute 0 - Device. MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} - -/// Iterates over all static translation table entries and fills them at once. -/// -/// # Safety -/// -/// - Modifies a `static mut`. Ensure it only happens from here. -unsafe fn populate_tt_entries() -> Result<(), &'static str> { - for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { - *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); - - for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { - let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); - - let (output_addr, attribute_fields) = - bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - - *l3_entry = PageDescriptor::new(output_addr, attribute_fields); - } + ); } - Ok(()) -} - -/// Configure various settings of stage 1 of the EL1 translation regime. -fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks - + TCR_EL1::TG0::KiB_64 - + TCR_EL1::SH0::Inner - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), - ); + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::T0SZ.val(t0sz), + ); + } } //-------------------------------------------------------------------------------------------------- @@ -317,15 +120,15 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { } // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); // Populate translation tables. - populate_tt_entries()?; + KERNEL_TABLES.populate_tt_entries()?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); - configure_translation_control(); + self.configure_translation_control(); // Switch the MMU on. // diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..cbe1d7832 --- /dev/null +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,288 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, + }, +}; +use core::convert; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait BaseAddr { + fn base_addr_u64(&self) -> u64; + fn base_addr_usize(&self) -> usize; +} + +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, hence the "reverse" order of appearance. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], +} + +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl BaseAddr for [T; N] { + fn base_addr_u64(&self) -> u64 { + self as *const T as u64 + } + + fn base_addr_usize(&self) -> usize { + self as *const _ as usize + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True + + attribute_fields.into() + + STAGE1_PAGE_DESCRIPTOR::TYPE::Table + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + + Self { value: val.get() } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl FixedSizeTranslationTable { + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(NUM_TABLES > 0); + assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + } + } + + /// Iterates over all static translation table entries and fills them at once. + /// + /// # Safety + /// + /// - Modifies a `static mut`. Ensure it only happens from here. + pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { + for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { + *l2_entry = + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + + for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { + let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + + let (output_addr, attribute_fields) = + bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + + *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + } + } + + Ok(()) + } + + /// The translation table's base address to be used for programming the MMU. + pub fn base_address(&self) -> u64 { + self.lvl2.base_addr_u64() + } +} diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp.rs b/11_virtual_mem_part1_identity_mapping/src/bsp.rs index 3d7587673..91ce3c495 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. pub mod device_driver; diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs index 982fc0655..911d10544 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -12,13 +12,16 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// The address space size chosen by this BSP. +pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; + const NUM_MEM_RANGES: usize = 3; /// The virtual memory layout. /// /// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. /// It is agnostic of the paging granularity that the architecture's MMU will use. -pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( +pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( memory_map::END_INCLUSIVE, [ TranslationDescriptor { @@ -77,17 +80,7 @@ fn mmio_range_inclusive() -> RangeInclusive { // Public Code //-------------------------------------------------------------------------------------------------- -/// Return the address space size in bytes. -/// -/// Guarantees size to be a power of two. -pub const fn addr_space_size() -> usize { - let size = memory_map::END_INCLUSIVE + 1; - assert!(size.is_power_of_two()); - - size -} - /// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT } diff --git a/11_virtual_mem_part1_identity_mapping/src/console.rs b/11_virtual_mem_part1_identity_mapping/src/console.rs index 3552823cc..c3154ba23 100644 --- a/11_virtual_mem_part1_identity_mapping/src/console.rs +++ b/11_virtual_mem_part1_identity_mapping/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs b/11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs +++ b/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/11_virtual_mem_part1_identity_mapping/src/exception.rs b/11_virtual_mem_part1_identity_mapping/src/exception.rs index 432c606b9..87d4db63a 100644 --- a/11_virtual_mem_part1_identity_mapping/src/exception.rs +++ b/11_virtual_mem_part1_identity_mapping/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::current_privilege_level; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs b/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs index fbdba9579..566efe329 100644 --- a/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs +++ b/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::print_state; diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 259715354..83ee1de57 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -7,18 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -32,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -49,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -103,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] @@ -114,9 +116,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs index 4880ce913..efc9c447b 100644 --- a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs @@ -5,8 +5,8 @@ //! Memory Management Unit. //! //! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file -//! provides types for composing an architecture-agnostic description of the kernel 's virtual -//! memory layout. +//! provides types for composing an architecture-agnostic description of the kernel's virtual memory +//! layout. //! //! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` //! function. @@ -17,10 +17,16 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; -pub use arch_mmu::*; + +mod translation_table; use core::{fmt, ops::RangeInclusive}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_mmu::mmu; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -40,8 +46,15 @@ pub mod interface { } } +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes the size of an address space. +pub struct AddressSpaceSize; + /// Architecture agnostic translation types. #[allow(missing_docs)] +#[allow(dead_code)] #[derive(Copy, Clone)] pub enum Translation { Identity, @@ -95,6 +108,41 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpaceSize { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); + assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + + // Must adhere to architectural restrictions. + assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); + assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); + assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + + AS_SIZE + } +} + impl Default for AttributeFields { fn default() -> AttributeFields { AttributeFields { diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs b/11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bbb6f9394 --- /dev/null +++ b/11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; diff --git a/11_virtual_mem_part1_identity_mapping/src/print.rs b/11_virtual_mem_part1_identity_mapping/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/11_virtual_mem_part1_identity_mapping/src/print.rs +++ b/11_virtual_mem_part1_identity_mapping/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/11_virtual_mem_part1_identity_mapping/src/time.rs b/11_virtual_mem_part1_identity_mapping/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/11_virtual_mem_part1_identity_mapping/src/time.rs +++ b/11_virtual_mem_part1_identity_mapping/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/12_exceptions_part1_groundwork/Makefile b/12_exceptions_part1_groundwork/Makefile index 9254018cf..9105d297f 100644 --- a/12_exceptions_part1_groundwork/Makefile +++ b/12_exceptions_part1_groundwork/Makefile @@ -44,8 +44,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index be16673f5..829ed0920 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -2,8 +2,8 @@ ## tl;dr -- We lay the groundwork for all the architectural `CPU exceptions`. For now, only print an elaborate - system state through a `panic!` call, and halt execution +- We lay the groundwork for all the architectural `CPU exceptions`. +- For now, only print an elaborate system state through a `panic!` call, and halt execution - This will help finding bugs during development and runtime. - For demo purposes, MMU `page faults` are used to demonstrate (i) returning from an exception and (ii) the default `panic!` behavior. @@ -27,8 +27,8 @@ Now that we are executing in `EL1`, and have activated the `MMU`, time is due for implementing `CPU exceptions`. For now, we only set up a scaffold with very basic functionality that will help us to -find bugs along the way. A follow-up `Interrupt` tutorial in the future will continue the work we -start here. +find bugs along the way. A follow-up `Interrupt` tutorial later will continue the work we start +here. Please note that this tutorial is specific to the `AArch64` architecture. It does not contain any generic exception handling code yet. @@ -38,7 +38,7 @@ generic exception handling code yet. In `AArch64`, it is differentiated between four types of exceptions. These are: - Synchronous - For example, a `data abort` (e.g. `page fault`) or a `system call`. They happen in direct - consequence of executing a certain instruction, hence _synchronously_. + consequence of executing a certain CPU instruction, hence _synchronously_. - Interrupt Request (`IRQ`) - For example, an external device, like a timer, is asserting a physical interrupt line. IRQs happen _asynchronously_. @@ -176,8 +176,8 @@ resources in its own code without bothering, and as a last action before returni handling code, restore the context, so that the processor can continue where it left off before taking the exception. -Context save and restore is one of the few places in system software where it is strongly advised to -to use some hand-crafted assembly. Introducing `exception.S`: +Context save and restore is one of the few places in system software where there is no way around +some hand-crafted assembly. Introducing `exception.S`: ```asm /// Call the function provided by parameter `\handler` after saving the exception context. Provide @@ -356,7 +356,7 @@ unsafe { core::ptr::read_volatile(big_addr as *mut u64) }; ``` This triggers our exception code, because we try to read from a virtual address for which no mapping -has been installed. Remember, we only installed up to `4 GiB` of address space in the previous +has been installed. Remember, we only mapped up to `4 GiB` of address space in the previous tutorial. To survive this exception, the respective handler has a special demo case: @@ -391,8 +391,6 @@ catch, eventually triggering the `panic!` call from the default handler. ## Test it -Emphasis on the events at timestamps > `4.xxxxxx`. - ```console $ make chainboot [...] @@ -482,9 +480,9 @@ General purpose register: diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs --- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs -@@ -4,7 +4,230 @@ - - //! Architectural synchronous and asynchronous exception handling. +@@ -11,7 +11,230 @@ + //! + //! crate::exception::arch_exception -use cortex_a::regs::*; +use core::{cell::UnsafeCell, fmt}; @@ -714,7 +712,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 //-------------------------------------------------------------------------------------------------- // Public Code -@@ -21,3 +244,23 @@ +@@ -28,3 +251,23 @@ _ => (PrivilegeLevel::Unknown, "Unknown"), } } @@ -901,16 +899,16 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld 12_e diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs --- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs -@@ -12,7 +12,7 @@ - // Public Definitions - //-------------------------------------------------------------------------------------------------- +@@ -15,7 +15,7 @@ + /// The address space size chosen by this BSP. + pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; -const NUM_MEM_RANGES: usize = 3; +const NUM_MEM_RANGES: usize = 2; /// The virtual memory layout. /// -@@ -32,16 +32,6 @@ +@@ -35,16 +35,6 @@ }, }, TranslationDescriptor { @@ -927,7 +925,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.r name: "Device MMIO", virtual_range: mmio_range_inclusive, physical_range_translation: Translation::Identity, -@@ -64,11 +54,6 @@ +@@ -67,11 +57,6 @@ RangeInclusive::new(super::ro_start(), super::ro_end() - 1) } @@ -945,7 +943,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp.rs 12_exceptions_part1_g +++ 12_exceptions_part1_groundwork/src/bsp.rs @@ -4,7 +4,7 @@ - //! Conditional re-exporting of Board Support Packages. + //! Conditional reexporting of Board Support Packages. -pub mod device_driver; +mod device_driver; @@ -953,10 +951,23 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp.rs 12_exceptions_part1_g #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; +diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_part1_groundwork/src/exception.rs +--- 11_virtual_mem_part1_identity_mapping/src/exception.rs ++++ 12_exceptions_part1_groundwork/src/exception.rs +@@ -13,7 +13,7 @@ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_exception::current_privilege_level; ++pub use arch_exception::{current_privilege_level, handling_init}; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions + diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs --- 11_virtual_mem_part1_identity_mapping/src/main.rs +++ 12_exceptions_part1_groundwork/src/main.rs -@@ -109,6 +109,7 @@ +@@ -111,6 +111,7 @@ #![feature(const_generics)] #![feature(const_panic)] #![feature(format_args_nl)] @@ -964,7 +975,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] -@@ -143,6 +144,8 @@ +@@ -142,6 +143,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -973,7 +984,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } -@@ -195,13 +198,28 @@ +@@ -194,13 +197,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -1009,16 +1020,4 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ // Discard any spurious received characters before going into echo mode. -diff -uNr 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs 12_exceptions_part1_groundwork/src/memory/mmu.rs ---- 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs -+++ 12_exceptions_part1_groundwork/src/memory/mmu.rs -@@ -42,6 +42,7 @@ - - /// Architecture agnostic translation types. - #[allow(missing_docs)] -+#[allow(dead_code)] - #[derive(Copy, Clone)] - pub enum Translation { - Identity, - ``` diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs index 3846968b3..d3b07461a 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c2f5fe0b7 --- /dev/null +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs index 10d003542..64f1fd23e 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; use cortex_a::{asm, barrier, regs::*}; diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs index 445f490eb..b63b00fe7 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs index 69023cef1..3504d2571 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs @@ -4,146 +4,61 @@ //! Memory Management Unit Driver. //! -//! Static translation tables, compiled on boot; Everything 64 KiB granule. +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu -use super::{AccessPermissions, AttributeFields, MemAttributes}; -use crate::{bsp, memory}; -use core::convert; +use crate::{ + bsp, memory, + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; use cortex_a::{barrier, regs::*}; -use register::{register_bitfields, InMemoryRegister}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. -register_bitfields! {u64, - STAGE1_TABLE_DESCRIPTOR [ - /// Physical address of the next descriptor. - NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. -register_bitfields! {u64, - STAGE1_PAGE_DESCRIPTOR [ - /// Unprivileged execute-never. - UXN OFFSET(54) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Privileged execute-never. - PXN OFFSET(53) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). - OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - /// Access flag. - AF OFFSET(10) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Shareability field. - SH OFFSET(8) NUMBITS(2) [ - OuterShareable = 0b10, - InnerShareable = 0b11 - ], - - /// Access Permissions. - AP OFFSET(6) NUMBITS(2) [ - RW_EL1 = 0b00, - RW_EL1_EL0 = 0b01, - RO_EL1 = 0b10, - RO_EL1_EL0 = 0b11 - ], - - /// Memory attributes index into the MAIR_EL1 register. - AttrIndx OFFSET(2) NUMBITS(3) [], - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) - -/// A table descriptor for 64 KiB aperture. -/// -/// The output points to the next table. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(u64); +/// Memory Management Unit type. +struct MemoryManagementUnit; -/// A page descriptor with 64 KiB aperture. -/// -/// The output points to physical memory. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(u64); +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- -/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. -#[repr(C)] -#[repr(align(65536))] -struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; - /// Table descriptors, covering 512 MiB windows. - lvl2: [TableDescriptor; NUM_TABLES], -} +/// The min supported address space size. +pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; +/// The max supported address space size. +pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; -} +/// The supported address space size granule. +pub type AddrSpaceSizeGranule = Granule512MiB; /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] -mod mair { +pub mod mair { pub const DEVICE: u64 = 0; pub const NORMAL: u64 = 1; } -/// Memory Management Unit type. -struct MemoryManagementUnit; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); +static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); static MMU: MemoryManagementUnit = MemoryManagementUnit; @@ -151,149 +66,37 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { - self as *const T as u64 - } - - fn base_addr_usize(&self) -> usize { - self as *const _ as usize - } -} - -impl convert::From for TableDescriptor { - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), - ); - - TableDescriptor(val.get()) - } -} - -/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. -impl convert::From - for register::FieldValue -{ - fn from(attribute_fields: AttributeFields) -> Self { - // Memory attributes. - let mut desc = match attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => { - STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) - } - MemAttributes::Device => { - STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) - } - }; - - // Access Permissions. - desc += match attribute_fields.acc_perms { - AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, - AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, - }; - - // The execute-never attribute is mapped to PXN in AArch64. - desc += if attribute_fields.execute_never { - STAGE1_PAGE_DESCRIPTOR::PXN::True - } else { - STAGE1_PAGE_DESCRIPTOR::PXN::False - }; - - // Always set unprivileged exectue-never as long as userspace is not implemented yet. - desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; - - desc - } -} - -impl PageDescriptor { - /// Create an instance. - fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), - ); - - Self(val.get()) - } -} - -impl FixedSizeTranslationTable<{ NUM_TABLES }> { - /// Create an instance. - pub const fn new() -> Self { - assert!(NUM_TABLES > 0); - - Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], - } - } -} - -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + // Attribute 0 - Device. MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} - -/// Iterates over all static translation table entries and fills them at once. -/// -/// # Safety -/// -/// - Modifies a `static mut`. Ensure it only happens from here. -unsafe fn populate_tt_entries() -> Result<(), &'static str> { - for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { - *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); - - for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { - let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); - - let (output_addr, attribute_fields) = - bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - - *l3_entry = PageDescriptor::new(output_addr, attribute_fields); - } + ); } - Ok(()) -} - -/// Configure various settings of stage 1 of the EL1 translation regime. -fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks - + TCR_EL1::TG0::KiB_64 - + TCR_EL1::SH0::Inner - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), - ); + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::T0SZ.val(t0sz), + ); + } } //-------------------------------------------------------------------------------------------------- @@ -317,15 +120,15 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { } // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); // Populate translation tables. - populate_tt_entries()?; + KERNEL_TABLES.populate_tt_entries()?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); - configure_translation_control(); + self.configure_translation_control(); // Switch the MMU on. // diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..cbe1d7832 --- /dev/null +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,288 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, + }, +}; +use core::convert; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait BaseAddr { + fn base_addr_u64(&self) -> u64; + fn base_addr_usize(&self) -> usize; +} + +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, hence the "reverse" order of appearance. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], +} + +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl BaseAddr for [T; N] { + fn base_addr_u64(&self) -> u64 { + self as *const T as u64 + } + + fn base_addr_usize(&self) -> usize { + self as *const _ as usize + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True + + attribute_fields.into() + + STAGE1_PAGE_DESCRIPTOR::TYPE::Table + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + + Self { value: val.get() } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl FixedSizeTranslationTable { + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(NUM_TABLES > 0); + assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + } + } + + /// Iterates over all static translation table entries and fills them at once. + /// + /// # Safety + /// + /// - Modifies a `static mut`. Ensure it only happens from here. + pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { + for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { + *l2_entry = + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + + for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { + let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + + let (output_addr, attribute_fields) = + bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + + *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + } + } + + Ok(()) + } + + /// The translation table's base address to be used for programming the MMU. + pub fn base_address(&self) -> u64 { + self.lvl2.base_addr_u64() + } +} diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/12_exceptions_part1_groundwork/src/bsp.rs b/12_exceptions_part1_groundwork/src/bsp.rs index 257502491..c558922f7 100644 --- a/12_exceptions_part1_groundwork/src/bsp.rs +++ b/12_exceptions_part1_groundwork/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs index 59d736a7c..fe98604d9 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -12,13 +12,16 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// The address space size chosen by this BSP. +pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; + const NUM_MEM_RANGES: usize = 2; /// The virtual memory layout. /// /// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. /// It is agnostic of the paging granularity that the architecture's MMU will use. -pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( +pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( memory_map::END_INCLUSIVE, [ TranslationDescriptor { @@ -62,17 +65,7 @@ fn mmio_range_inclusive() -> RangeInclusive { // Public Code //-------------------------------------------------------------------------------------------------- -/// Return the address space size in bytes. -/// -/// Guarantees size to be a power of two. -pub const fn addr_space_size() -> usize { - let size = memory_map::END_INCLUSIVE + 1; - assert!(size.is_power_of_two()); - - size -} - /// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT } diff --git a/12_exceptions_part1_groundwork/src/console.rs b/12_exceptions_part1_groundwork/src/console.rs index 3552823cc..c3154ba23 100644 --- a/12_exceptions_part1_groundwork/src/console.rs +++ b/12_exceptions_part1_groundwork/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/12_exceptions_part1_groundwork/src/cpu.rs b/12_exceptions_part1_groundwork/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/12_exceptions_part1_groundwork/src/cpu.rs +++ b/12_exceptions_part1_groundwork/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/12_exceptions_part1_groundwork/src/cpu/boot.rs b/12_exceptions_part1_groundwork/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/12_exceptions_part1_groundwork/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/12_exceptions_part1_groundwork/src/cpu/smp.rs b/12_exceptions_part1_groundwork/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/12_exceptions_part1_groundwork/src/cpu/smp.rs +++ b/12_exceptions_part1_groundwork/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/12_exceptions_part1_groundwork/src/exception.rs b/12_exceptions_part1_groundwork/src/exception.rs index 432c606b9..8e91d8bb9 100644 --- a/12_exceptions_part1_groundwork/src/exception.rs +++ b/12_exceptions_part1_groundwork/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/12_exceptions_part1_groundwork/src/exception/asynchronous.rs b/12_exceptions_part1_groundwork/src/exception/asynchronous.rs index fbdba9579..566efe329 100644 --- a/12_exceptions_part1_groundwork/src/exception/asynchronous.rs +++ b/12_exceptions_part1_groundwork/src/exception/asynchronous.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::print_state; diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index a2cf77527..80706a248 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -7,18 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -32,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -49,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -103,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] @@ -115,9 +117,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/12_exceptions_part1_groundwork/src/memory/mmu.rs b/12_exceptions_part1_groundwork/src/memory/mmu.rs index 826cda6c9..efc9c447b 100644 --- a/12_exceptions_part1_groundwork/src/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/memory/mmu.rs @@ -5,8 +5,8 @@ //! Memory Management Unit. //! //! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file -//! provides types for composing an architecture-agnostic description of the kernel 's virtual -//! memory layout. +//! provides types for composing an architecture-agnostic description of the kernel's virtual memory +//! layout. //! //! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` //! function. @@ -17,10 +17,16 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; -pub use arch_mmu::*; + +mod translation_table; use core::{fmt, ops::RangeInclusive}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_mmu::mmu; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -40,6 +46,12 @@ pub mod interface { } } +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes the size of an address space. +pub struct AddressSpaceSize; + /// Architecture agnostic translation types. #[allow(missing_docs)] #[allow(dead_code)] @@ -96,6 +108,41 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpaceSize { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); + assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + + // Must adhere to architectural restrictions. + assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); + assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); + assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + + AS_SIZE + } +} + impl Default for AttributeFields { fn default() -> AttributeFields { AttributeFields { diff --git a/12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs b/12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bbb6f9394 --- /dev/null +++ b/12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; diff --git a/12_exceptions_part1_groundwork/src/print.rs b/12_exceptions_part1_groundwork/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/12_exceptions_part1_groundwork/src/print.rs +++ b/12_exceptions_part1_groundwork/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/12_exceptions_part1_groundwork/src/time.rs b/12_exceptions_part1_groundwork/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/12_exceptions_part1_groundwork/src/time.rs +++ b/12_exceptions_part1_groundwork/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/13_integrated_testing/Makefile b/13_integrated_testing/Makefile index 9f7ea9a39..4f7561dbd 100644 --- a/13_integrated_testing/Makefile +++ b/13_integrated_testing/Makefile @@ -57,8 +57,9 @@ QEMU_MISSING_STRING = "This board is not yet supported for QEMU." RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 849cbbfb2..9a99cbdf5 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -138,7 +138,6 @@ In `lib.rs`, we add the following headers to get started with `custom_test_frame ```rust // Testing #![cfg_attr(test, no_main)] -#![cfg_attr(test, feature(slice_ptr_range))] #![feature(custom_test_frameworks)] #![reexport_test_harness_main = "test_main"] #![test_runner(crate::test_runner)] @@ -255,9 +254,9 @@ this is an opportunity to cut down on setup code. [tutorial 03]: ../03_hacky_hello_world -As a matter of fact, for the `Raspberrys`, nothing needs to be done and the function is empy. But +As a matter of fact, for the `Raspberrys`, nothing needs to be done, so the function is empy. But this might be different for other hardware emulated by QEMU, so it makes sense to introduce the -function now to make it easier in case new `BSPs` are added to the kernel in the future. +function now to make it easier in case new `BSPs` are added to the kernel in the future. Next, the reexported `test_main()` is called, which will call our `test_runner()` which finally prints the unit test names and executes them. @@ -875,7 +874,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -@@ -52,6 +65,7 @@ +@@ -53,6 +66,7 @@ DOC_CMD = cargo doc $(COMPILER_ARGS) CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) CHECK_CMD = cargo check $(COMPILER_ARGS) @@ -883,7 +882,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -@@ -68,6 +82,7 @@ +@@ -69,6 +83,7 @@ DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) @@ -891,7 +890,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux -@@ -84,8 +99,8 @@ +@@ -85,8 +100,8 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) EXEC_MINIPUSH = ruby ../utils/minipush.rb @@ -902,7 +901,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile all: $(KERNEL_BIN) -@@ -99,11 +114,26 @@ +@@ -100,11 +115,26 @@ $(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) @@ -935,7 +934,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_testing/src/_arch/aarch64/cpu.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ 13_integrated_testing/src/_arch/aarch64/cpu.rs -@@ -90,3 +90,20 @@ +@@ -26,3 +26,20 @@ asm::wfe() } } @@ -960,8 +959,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integrated_testing/src/_arch/aarch64/exception.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ 13_integrated_testing/src/_arch/aarch64/exception.rs -@@ -5,7 +5,7 @@ - //! Architectural synchronous and asynchronous exception handling. +@@ -12,7 +12,7 @@ + //! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; -use cortex_a::{asm, barrier, regs::*}; @@ -969,7 +968,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integ use register::InMemoryRegister; // Assembly counterpart to this file. -@@ -80,16 +80,6 @@ +@@ -87,16 +87,6 @@ #[no_mangle] unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) { @@ -987,11 +986,11 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integ } -diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs ---- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs -+++ 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs -@@ -341,3 +341,40 @@ - Ok(()) +diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +@@ -286,3 +286,31 @@ + self.lvl2.base_addr_u64() } } + @@ -1021,6 +1020,24 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_inte + core::mem::size_of::() + ); + } ++} + +diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +--- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs ++++ 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +@@ -144,3 +144,22 @@ + Ok(()) + } + } ++ ++//-------------------------------------------------------------------------------------------------- ++// Testing ++//-------------------------------------------------------------------------------------------------- ++ ++#[cfg(test)] ++mod tests { ++ use super::*; ++ use test_macros::kernel_test; + + /// Check if KERNEL_TABLES is in .bss. + #[kernel_test] @@ -1053,8 +1070,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 13_integ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs -@@ -76,3 +76,46 @@ - pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +@@ -69,3 +69,46 @@ + pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT } + @@ -1101,10 +1118,20 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_in + } +} +diff -uNr 12_exceptions_part1_groundwork/src/cpu.rs 13_integrated_testing/src/cpu.rs +--- 12_exceptions_part1_groundwork/src/cpu.rs ++++ 13_integrated_testing/src/cpu.rs +@@ -15,4 +15,4 @@ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_cpu::{nop, wait_forever}; ++pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; + diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/src/exception.rs --- 12_exceptions_part1_groundwork/src/exception.rs +++ 13_integrated_testing/src/exception.rs -@@ -24,3 +24,21 @@ +@@ -28,3 +28,21 @@ Hypervisor, Unknown, } @@ -1130,7 +1157,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,169 @@ +@@ -0,0 +1,168 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1140,19 +1167,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li + +//! The `kernel` library. +//! -+//! Used by `main.rs` to compose the final kernel binary. -+//! -+//! # TL;DR - Overview of important Kernel entities -+//! -+//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -+//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -+//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -+//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -+//! -+//! [console interface]: ../libkernel/console/interface/index.html -+//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -+//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -+//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html ++//! Used to compose the final kernel binary. +//! +//! # Code organization and architecture +//! @@ -1167,15 +1182,22 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! `src/_arch`, for example, `src/_arch/aarch64`. +//! +//! The architecture folders mirror the subsystem modules laid out in `src`. For example, -+//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -+//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -+//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -+//! module organization. That means a public function `foo()` defined in -+//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. ++//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go ++//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in ++//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic ++//! module's name prefixed with `arch_`. +//! -+//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -+//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -+//! "_arch/xxx/yyy.rs"]` attribute. ++//! For example, this is the top of `src/memory/mmu.rs`: ++//! ++//! ``` ++//! #[cfg(target_arch = "aarch64")] ++//! #[path = "../_arch/aarch64/memory/mmu.rs"] ++//! mod arch_mmu; ++//! ``` ++//! ++//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. ++//! This way, each architecture specific module can provide its implementation of an item, while the ++//! caller must not be concerned which architecture has been conditionally compiled. +//! +//! ## BSP code +//! @@ -1184,9 +1206,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! or instances of drivers for devices that are featured on the respective board. +//! +//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -+//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -+//! whatever is provided must be called starting from the `bsp` namespace, e.g. -+//! `bsp::driver::driver_manager()`. ++//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is ++//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. +//! +//! ## Kernel interfaces +//! @@ -1238,6 +1259,14 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! +//! - `crate::memory::*` +//! - `crate::bsp::memory::*` ++//! ++//! # Boot flow ++//! ++//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. ++//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. ++//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. ++//! ++//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html + +#![allow(incomplete_features)] +#![feature(const_fn_fn_ptr_basics)] @@ -1255,9 +1284,6 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +#![reexport_test_harness_main = "test_main"] +#![test_runner(crate::test_runner)] + -+// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -+// `runtime_init()`, which jumps to `kernel_init()` (defined in `main.rs`). -+ +mod panic_wait; +mod runtime_init; +mod synchronization; @@ -1304,23 +1330,11 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,129 +6,12 @@ +@@ -6,128 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. -//! --//! # TL;DR - Overview of important Kernel entities --//! --//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. --//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. --//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. --//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. --//! --//! [console interface]: ../libkernel/console/interface/index.html --//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html --//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html --//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html --//! -//! # Code organization and architecture -//! -//! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -1334,15 +1348,22 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! `src/_arch`, for example, `src/_arch/aarch64`. -//! -//! The architecture folders mirror the subsystem modules laid out in `src`. For example, --//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go --//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in --//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's --//! module organization. That means a public function `foo()` defined in --//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +-//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +-//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +-//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +-//! module's name prefixed with `arch_`. -//! --//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. --//! Rather, it's contents are conditionally pulled into respective files using the `#[path = --//! "_arch/xxx/yyy.rs"]` attribute. +-//! For example, this is the top of `src/memory/mmu.rs`: +-//! +-//! ``` +-//! #[cfg(target_arch = "aarch64")] +-//! #[path = "../_arch/aarch64/memory/mmu.rs"] +-//! mod arch_mmu; +-//! ``` +-//! +-//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +-//! This way, each architecture specific module can provide its implementation of an item, while the +-//! caller must not be concerned which architecture has been conditionally compiled. -//! -//! ## BSP code -//! @@ -1351,9 +1372,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! or instances of drivers for devices that are featured on the respective board. -//! -//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the --//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means --//! whatever is provided must be called starting from the `bsp` namespace, e.g. --//! `bsp::driver::driver_manager()`. +-//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +-//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. -//! -//! ## Kernel interfaces -//! @@ -1405,6 +1425,14 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! -//! - `crate::memory::*` -//! - `crate::bsp::memory::*` +-//! +-//! # Boot flow +-//! +-//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +-//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +-//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +-//! +-//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html - -#![allow(incomplete_features)] -#![feature(const_fn_fn_ptr_basics)] @@ -1418,9 +1446,6 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m #![no_main] #![no_std] --// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls --// `runtime_init()`, which jumps to `kernel_init()`. -- -mod bsp; -mod console; -mod cpu; @@ -1436,7 +1461,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -140,6 +23,7 @@ +@@ -139,6 +23,7 @@ /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -1444,7 +1469,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -166,9 +50,7 @@ +@@ -165,9 +50,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1454,7 +1479,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -195,31 +77,6 @@ +@@ -194,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1490,7 +1515,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing/src/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/memory/mmu.rs +++ 13_integrated_testing/src/memory/mmu.rs -@@ -42,7 +42,6 @@ +@@ -54,7 +54,6 @@ /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -1498,7 +1523,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing #[derive(Copy, Clone)] pub enum Translation { Identity, -@@ -197,4 +196,9 @@ +@@ -244,4 +243,9 @@ info!("{}", i); } } diff --git a/13_integrated_testing/src/_arch/aarch64/cpu.rs b/13_integrated_testing/src/_arch/aarch64/cpu.rs index 491f17ca8..948bad747 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs b/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c2f5fe0b7 --- /dev/null +++ b/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs b/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/13_integrated_testing/src/_arch/aarch64/exception.rs b/13_integrated_testing/src/_arch/aarch64/exception.rs index 7070d8140..0e640ec69 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; use cortex_a::{barrier, regs::*}; diff --git a/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs b/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs index 445f490eb..b63b00fe7 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs index 9b658e864..42ba0519b 100644 --- a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -4,146 +4,61 @@ //! Memory Management Unit Driver. //! -//! Static translation tables, compiled on boot; Everything 64 KiB granule. +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu -use super::{AccessPermissions, AttributeFields, MemAttributes}; -use crate::{bsp, memory}; -use core::convert; +use crate::{ + bsp, memory, + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; use cortex_a::{barrier, regs::*}; -use register::{register_bitfields, InMemoryRegister}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. -register_bitfields! {u64, - STAGE1_TABLE_DESCRIPTOR [ - /// Physical address of the next descriptor. - NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. -register_bitfields! {u64, - STAGE1_PAGE_DESCRIPTOR [ - /// Unprivileged execute-never. - UXN OFFSET(54) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Privileged execute-never. - PXN OFFSET(53) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). - OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - /// Access flag. - AF OFFSET(10) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Shareability field. - SH OFFSET(8) NUMBITS(2) [ - OuterShareable = 0b10, - InnerShareable = 0b11 - ], - - /// Access Permissions. - AP OFFSET(6) NUMBITS(2) [ - RW_EL1 = 0b00, - RW_EL1_EL0 = 0b01, - RO_EL1 = 0b10, - RO_EL1_EL0 = 0b11 - ], - - /// Memory attributes index into the MAIR_EL1 register. - AttrIndx OFFSET(2) NUMBITS(3) [], - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} +/// Memory Management Unit type. +struct MemoryManagementUnit; -const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- -/// A table descriptor for 64 KiB aperture. -/// -/// The output points to the next table. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(u64); +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// A page descriptor with 64 KiB aperture. -/// -/// The output points to physical memory. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(u64); - -/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. -#[repr(C)] -#[repr(align(65536))] -struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], - - /// Table descriptors, covering 512 MiB windows. - lvl2: [TableDescriptor; NUM_TABLES], -} +/// The min supported address space size. +pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; +/// The max supported address space size. +pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; -} +/// The supported address space size granule. +pub type AddrSpaceSizeGranule = Granule512MiB; /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] -mod mair { +pub mod mair { pub const DEVICE: u64 = 0; pub const NORMAL: u64 = 1; } -/// Memory Management Unit type. -struct MemoryManagementUnit; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); +static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); static MMU: MemoryManagementUnit = MemoryManagementUnit; @@ -151,149 +66,37 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { - self as *const T as u64 - } - - fn base_addr_usize(&self) -> usize { - self as *const _ as usize - } -} - -impl convert::From for TableDescriptor { - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), - ); - - TableDescriptor(val.get()) - } -} - -/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. -impl convert::From - for register::FieldValue -{ - fn from(attribute_fields: AttributeFields) -> Self { - // Memory attributes. - let mut desc = match attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => { - STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) - } - MemAttributes::Device => { - STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) - } - }; - - // Access Permissions. - desc += match attribute_fields.acc_perms { - AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, - AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, - }; - - // The execute-never attribute is mapped to PXN in AArch64. - desc += if attribute_fields.execute_never { - STAGE1_PAGE_DESCRIPTOR::PXN::True - } else { - STAGE1_PAGE_DESCRIPTOR::PXN::False - }; - - // Always set unprivileged exectue-never as long as userspace is not implemented yet. - desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; - - desc - } -} - -impl PageDescriptor { - /// Create an instance. - fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), - ); - - Self(val.get()) - } -} - -impl FixedSizeTranslationTable<{ NUM_TABLES }> { - /// Create an instance. - pub const fn new() -> Self { - assert!(NUM_TABLES > 0); - - Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], - } - } -} - -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + // Attribute 0 - Device. MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} - -/// Iterates over all static translation table entries and fills them at once. -/// -/// # Safety -/// -/// - Modifies a `static mut`. Ensure it only happens from here. -unsafe fn populate_tt_entries() -> Result<(), &'static str> { - for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { - *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); - - for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { - let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); - - let (output_addr, attribute_fields) = - bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - - *l3_entry = PageDescriptor::new(output_addr, attribute_fields); - } + ); } - Ok(()) -} - -/// Configure various settings of stage 1 of the EL1 translation regime. -fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks - + TCR_EL1::TG0::KiB_64 - + TCR_EL1::SH0::Inner - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), - ); + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::T0SZ.val(t0sz), + ); + } } //-------------------------------------------------------------------------------------------------- @@ -317,15 +120,15 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { } // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); // Populate translation tables. - populate_tt_entries()?; + KERNEL_TABLES.populate_tt_entries()?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); - configure_translation_control(); + self.configure_translation_control(); // Switch the MMU on. // @@ -351,24 +154,6 @@ mod tests { use super::*; use test_macros::kernel_test; - /// Check if the size of `struct TableDescriptor` is as expected. - #[kernel_test] - fn size_of_tabledescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - - /// Check if the size of `struct PageDescriptor` is as expected. - #[kernel_test] - fn size_of_pagedescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..337f9aedf --- /dev/null +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, + }, +}; +use core::convert; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait BaseAddr { + fn base_addr_u64(&self) -> u64; + fn base_addr_usize(&self) -> usize; +} + +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, hence the "reverse" order of appearance. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], +} + +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl BaseAddr for [T; N] { + fn base_addr_u64(&self) -> u64 { + self as *const T as u64 + } + + fn base_addr_usize(&self) -> usize { + self as *const _ as usize + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True + + attribute_fields.into() + + STAGE1_PAGE_DESCRIPTOR::TYPE::Table + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + + Self { value: val.get() } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl FixedSizeTranslationTable { + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(NUM_TABLES > 0); + assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + } + } + + /// Iterates over all static translation table entries and fills them at once. + /// + /// # Safety + /// + /// - Modifies a `static mut`. Ensure it only happens from here. + pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { + for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { + *l2_entry = + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + + for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { + let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + + let (output_addr, attribute_fields) = + bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + + *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + } + } + + Ok(()) + } + + /// The translation table's base address to be used for programming the MMU. + pub fn base_address(&self) -> u64 { + self.lvl2.base_addr_u64() + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct TableDescriptor` is as expected. + #[kernel_test] + fn size_of_tabledescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } + + /// Check if the size of `struct PageDescriptor` is as expected. + #[kernel_test] + fn size_of_pagedescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } +} diff --git a/13_integrated_testing/src/_arch/aarch64/time.rs b/13_integrated_testing/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/13_integrated_testing/src/_arch/aarch64/time.rs +++ b/13_integrated_testing/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/13_integrated_testing/src/bsp.rs b/13_integrated_testing/src/bsp.rs index 257502491..c558922f7 100644 --- a/13_integrated_testing/src/bsp.rs +++ b/13_integrated_testing/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs index 485b8a46a..451d927b6 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -12,13 +12,16 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// The address space size chosen by this BSP. +pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; + const NUM_MEM_RANGES: usize = 2; /// The virtual memory layout. /// /// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. /// It is agnostic of the paging granularity that the architecture's MMU will use. -pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( +pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( memory_map::END_INCLUSIVE, [ TranslationDescriptor { @@ -62,18 +65,8 @@ fn mmio_range_inclusive() -> RangeInclusive { // Public Code //-------------------------------------------------------------------------------------------------- -/// Return the address space size in bytes. -/// -/// Guarantees size to be a power of two. -pub const fn addr_space_size() -> usize { - let size = memory_map::END_INCLUSIVE + 1; - assert!(size.is_power_of_two()); - - size -} - /// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT } diff --git a/13_integrated_testing/src/console.rs b/13_integrated_testing/src/console.rs index 3552823cc..c3154ba23 100644 --- a/13_integrated_testing/src/console.rs +++ b/13_integrated_testing/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/13_integrated_testing/src/cpu.rs b/13_integrated_testing/src/cpu.rs index c9e5af72c..9c8eb6f6e 100644 --- a/13_integrated_testing/src/cpu.rs +++ b/13_integrated_testing/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; diff --git a/13_integrated_testing/src/cpu/boot.rs b/13_integrated_testing/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/13_integrated_testing/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/13_integrated_testing/src/cpu/smp.rs b/13_integrated_testing/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/13_integrated_testing/src/cpu/smp.rs +++ b/13_integrated_testing/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/13_integrated_testing/src/exception.rs b/13_integrated_testing/src/exception.rs index dfa852a87..3c5e7bc83 100644 --- a/13_integrated_testing/src/exception.rs +++ b/13_integrated_testing/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/13_integrated_testing/src/exception/asynchronous.rs b/13_integrated_testing/src/exception/asynchronous.rs index fbdba9579..566efe329 100644 --- a/13_integrated_testing/src/exception/asynchronous.rs +++ b/13_integrated_testing/src/exception/asynchronous.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::print_state; diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index f958ae5d5..0bc8d3292 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -7,19 +7,7 @@ //! The `kernel` library. //! -//! Used by `main.rs` to compose the final kernel binary. -//! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html +//! Used to compose the final kernel binary. //! //! # Code organization and architecture //! @@ -34,15 +22,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -51,9 +46,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -105,6 +99,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] @@ -122,9 +124,6 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(crate::test_runner)] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()` (defined in `main.rs`). - mod panic_wait; mod runtime_init; mod synchronization; diff --git a/13_integrated_testing/src/memory/mmu.rs b/13_integrated_testing/src/memory/mmu.rs index 163419ec0..cca2951aa 100644 --- a/13_integrated_testing/src/memory/mmu.rs +++ b/13_integrated_testing/src/memory/mmu.rs @@ -5,8 +5,8 @@ //! Memory Management Unit. //! //! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file -//! provides types for composing an architecture-agnostic description of the kernel 's virtual -//! memory layout. +//! provides types for composing an architecture-agnostic description of the kernel's virtual memory +//! layout. //! //! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` //! function. @@ -17,10 +17,16 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; -pub use arch_mmu::*; + +mod translation_table; use core::{fmt, ops::RangeInclusive}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_mmu::mmu; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -40,6 +46,12 @@ pub mod interface { } } +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes the size of an address space. +pub struct AddressSpaceSize; + /// Architecture agnostic translation types. #[allow(missing_docs)] #[derive(Copy, Clone)] @@ -95,6 +107,41 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpaceSize { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); + assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + + // Must adhere to architectural restrictions. + assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); + assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); + assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + + AS_SIZE + } +} + impl Default for AttributeFields { fn default() -> AttributeFields { AttributeFields { diff --git a/13_integrated_testing/src/memory/mmu/translation_table.rs b/13_integrated_testing/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bbb6f9394 --- /dev/null +++ b/13_integrated_testing/src/memory/mmu/translation_table.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; diff --git a/13_integrated_testing/src/print.rs b/13_integrated_testing/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/13_integrated_testing/src/print.rs +++ b/13_integrated_testing/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/13_integrated_testing/src/time.rs b/13_integrated_testing/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/13_integrated_testing/src/time.rs +++ b/13_integrated_testing/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 8e68edcb8..2c74961d9 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -744,25 +744,10 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 13_integrated_testing/Makefile 14_exceptions_part2_peripheral_IRQs/Makefile ---- 13_integrated_testing/Makefile -+++ 14_exceptions_part2_peripheral_IRQs/Makefile -@@ -57,8 +57,9 @@ - RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) - RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs - -+FEATURES = bsp_$(BSP) - COMPILER_ARGS = --target=$(TARGET) \ -- --features bsp_$(BSP) \ -+ --features $(FEATURES) \ - --release - - RUSTC_CMD = cargo rustc $(COMPILER_ARGS) - diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs --- 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs -@@ -10,6 +10,10 @@ +@@ -17,6 +17,10 @@ // Private Definitions //-------------------------------------------------------------------------------------------------- @@ -773,7 +758,7 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_e trait DaifField { fn daif_field() -> register::Field; } -@@ -58,6 +62,71 @@ +@@ -65,6 +69,71 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -849,15 +834,15 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_e diff -uNr 13_integrated_testing/src/_arch/aarch64/exception.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs --- 13_integrated_testing/src/_arch/aarch64/exception.rs +++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs -@@ -4,6 +4,7 @@ - - //! Architectural synchronous and asynchronous exception handling. +@@ -11,6 +11,7 @@ + //! + //! crate::exception::arch_exception +use crate::{bsp, exception}; use core::{cell::UnsafeCell, fmt}; use cortex_a::{barrier, regs::*}; use register::InMemoryRegister; -@@ -84,8 +85,11 @@ +@@ -91,8 +92,11 @@ } #[no_mangle] @@ -2156,12 +2141,20 @@ diff -uNr 13_integrated_testing/src/driver.rs 14_exceptions_part2_peripheral_IRQ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs --- 13_integrated_testing/src/exception/asynchronous.rs +++ 14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs -@@ -8,3 +8,138 @@ +@@ -8,7 +8,145 @@ #[path = "../_arch/aarch64/exception/asynchronous.rs"] - mod arch_exception_async; - pub use arch_exception_async::*; -+ + mod arch_asynchronous; + +use core::{fmt, marker::PhantomData}; ++ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- +-pub use arch_asynchronous::print_state; ++pub use arch_asynchronous::{ ++ is_local_irq_masked, local_irq_mask, local_irq_mask_save, local_irq_restore, local_irq_unmask, ++ print_state, ++}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -2268,7 +2261,7 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part + } + + /// Return the wrapped number. -+ pub fn get(self) -> usize { ++ pub const fn get(self) -> usize { + self.0 + } +} @@ -2299,26 +2292,8 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs --- 13_integrated_testing/src/lib.rs +++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -13,12 +13,17 @@ - //! - //! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. - //! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -+//! - [`bsp::exception::asynchronous::irq_manager()`] - Returns a reference to the kernel's [IRQ -+//! Handling interface]. - //! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -+//! - [`state::state_manager()`] - Returns a reference to the kernel's [state management] instance. - //! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. - //! - //! [console interface]: ../libkernel/console/interface/index.html - //! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -+//! [IRQ Handling interface]: ../libkernel/exception/asynchronous/interface/trait.IRQManager.html - //! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -+//! [state management]: ../libkernel/state/struct.StateManager.html - //! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html - //! - //! # Code organization and architecture -@@ -107,9 +112,11 @@ - //! - `crate::bsp::memory::*` +@@ -109,9 +109,11 @@ + //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] +#![feature(asm)] @@ -2329,7 +2304,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -@@ -136,6 +143,7 @@ +@@ -135,6 +137,7 @@ pub mod exception; pub mod memory; pub mod print; @@ -2506,7 +2481,7 @@ diff -uNr 13_integrated_testing/src/state.rs 14_exceptions_part2_peripheral_IRQs + } + } + -+ /// Return if the kernel is still in an init state. ++ /// Return if the kernel is init state. + pub fn is_init(&self) -> bool { + self.state() == State::Init + } diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs index 491f17ca8..948bad747 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..c2f5fe0b7 --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs index 99fbd85bd..5cf9eb5ca 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use crate::{bsp, exception}; use core::{cell::UnsafeCell, fmt}; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs index 968eedf34..a4b1a548e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 9b658e864..42ba0519b 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -4,146 +4,61 @@ //! Memory Management Unit Driver. //! -//! Static translation tables, compiled on boot; Everything 64 KiB granule. +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu -use super::{AccessPermissions, AttributeFields, MemAttributes}; -use crate::{bsp, memory}; -use core::convert; +use crate::{ + bsp, memory, + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; use cortex_a::{barrier, regs::*}; -use register::{register_bitfields, InMemoryRegister}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. -register_bitfields! {u64, - STAGE1_TABLE_DESCRIPTOR [ - /// Physical address of the next descriptor. - NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. -register_bitfields! {u64, - STAGE1_PAGE_DESCRIPTOR [ - /// Unprivileged execute-never. - UXN OFFSET(54) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Privileged execute-never. - PXN OFFSET(53) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). - OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - /// Access flag. - AF OFFSET(10) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Shareability field. - SH OFFSET(8) NUMBITS(2) [ - OuterShareable = 0b10, - InnerShareable = 0b11 - ], - - /// Access Permissions. - AP OFFSET(6) NUMBITS(2) [ - RW_EL1 = 0b00, - RW_EL1_EL0 = 0b01, - RO_EL1 = 0b10, - RO_EL1_EL0 = 0b11 - ], - - /// Memory attributes index into the MAIR_EL1 register. - AttrIndx OFFSET(2) NUMBITS(3) [], - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} +/// Memory Management Unit type. +struct MemoryManagementUnit; -const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- -/// A table descriptor for 64 KiB aperture. -/// -/// The output points to the next table. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(u64); +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// A page descriptor with 64 KiB aperture. -/// -/// The output points to physical memory. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(u64); - -/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. -#[repr(C)] -#[repr(align(65536))] -struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], - - /// Table descriptors, covering 512 MiB windows. - lvl2: [TableDescriptor; NUM_TABLES], -} +/// The min supported address space size. +pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -type ArchTranslationTable = FixedSizeTranslationTable; +/// The max supported address space size. +pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; -} +/// The supported address space size granule. +pub type AddrSpaceSizeGranule = Granule512MiB; /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] -mod mair { +pub mod mair { pub const DEVICE: u64 = 0; pub const NORMAL: u64 = 1; } -/// Memory Management Unit type. -struct MemoryManagementUnit; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); +static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); static MMU: MemoryManagementUnit = MemoryManagementUnit; @@ -151,149 +66,37 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { - self as *const T as u64 - } - - fn base_addr_usize(&self) -> usize { - self as *const _ as usize - } -} - -impl convert::From for TableDescriptor { - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), - ); - - TableDescriptor(val.get()) - } -} - -/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. -impl convert::From - for register::FieldValue -{ - fn from(attribute_fields: AttributeFields) -> Self { - // Memory attributes. - let mut desc = match attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => { - STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) - } - MemAttributes::Device => { - STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) - } - }; - - // Access Permissions. - desc += match attribute_fields.acc_perms { - AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, - AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, - }; - - // The execute-never attribute is mapped to PXN in AArch64. - desc += if attribute_fields.execute_never { - STAGE1_PAGE_DESCRIPTOR::PXN::True - } else { - STAGE1_PAGE_DESCRIPTOR::PXN::False - }; - - // Always set unprivileged exectue-never as long as userspace is not implemented yet. - desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; - - desc - } -} - -impl PageDescriptor { - /// Create an instance. - fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), - ); - - Self(val.get()) - } -} - -impl FixedSizeTranslationTable<{ NUM_TABLES }> { - /// Create an instance. - pub const fn new() -> Self { - assert!(NUM_TABLES > 0); - - Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], - } - } -} - -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + // Attribute 0 - Device. MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} - -/// Iterates over all static translation table entries and fills them at once. -/// -/// # Safety -/// -/// - Modifies a `static mut`. Ensure it only happens from here. -unsafe fn populate_tt_entries() -> Result<(), &'static str> { - for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { - *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); - - for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { - let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); - - let (output_addr, attribute_fields) = - bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - - *l3_entry = PageDescriptor::new(output_addr, attribute_fields); - } + ); } - Ok(()) -} - -/// Configure various settings of stage 1 of the EL1 translation regime. -fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks - + TCR_EL1::TG0::KiB_64 - + TCR_EL1::SH0::Inner - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), - ); + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::T0SZ.val(t0sz), + ); + } } //-------------------------------------------------------------------------------------------------- @@ -317,15 +120,15 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { } // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); // Populate translation tables. - populate_tt_entries()?; + KERNEL_TABLES.populate_tt_entries()?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); - configure_translation_control(); + self.configure_translation_control(); // Switch the MMU on. // @@ -351,24 +154,6 @@ mod tests { use super::*; use test_macros::kernel_test; - /// Check if the size of `struct TableDescriptor` is as expected. - #[kernel_test] - fn size_of_tabledescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - - /// Check if the size of `struct PageDescriptor` is as expected. - #[kernel_test] - fn size_of_pagedescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..337f9aedf --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, + }, +}; +use core::convert; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait BaseAddr { + fn base_addr_u64(&self) -> u64; + fn base_addr_usize(&self) -> usize; +} + +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, hence the "reverse" order of appearance. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], +} + +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl BaseAddr for [T; N] { + fn base_addr_u64(&self) -> u64 { + self as *const T as u64 + } + + fn base_addr_usize(&self) -> usize { + self as *const _ as usize + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True + + attribute_fields.into() + + STAGE1_PAGE_DESCRIPTOR::TYPE::Table + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + + Self { value: val.get() } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl FixedSizeTranslationTable { + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(NUM_TABLES > 0); + assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + } + } + + /// Iterates over all static translation table entries and fills them at once. + /// + /// # Safety + /// + /// - Modifies a `static mut`. Ensure it only happens from here. + pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { + for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { + *l2_entry = + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + + for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { + let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + + let (output_addr, attribute_fields) = + bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + + *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + } + } + + Ok(()) + } + + /// The translation table's base address to be used for programming the MMU. + pub fn base_address(&self) -> u64 { + self.lvl2.base_addr_u64() + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct TableDescriptor` is as expected. + #[kernel_test] + fn size_of_tabledescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } + + /// Check if the size of `struct PageDescriptor` is as expected. + #[kernel_test] + fn size_of_pagedescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } +} diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp.rs index 257502491..c558922f7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs index 485b8a46a..451d927b6 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs @@ -12,13 +12,16 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// The address space size chosen by this BSP. +pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; + const NUM_MEM_RANGES: usize = 2; /// The virtual memory layout. /// /// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. /// It is agnostic of the paging granularity that the architecture's MMU will use. -pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( +pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( memory_map::END_INCLUSIVE, [ TranslationDescriptor { @@ -62,18 +65,8 @@ fn mmio_range_inclusive() -> RangeInclusive { // Public Code //-------------------------------------------------------------------------------------------------- -/// Return the address space size in bytes. -/// -/// Guarantees size to be a power of two. -pub const fn addr_space_size() -> usize { - let size = memory_map::END_INCLUSIVE + 1; - assert!(size.is_power_of_two()); - - size -} - /// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { +pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT } diff --git a/14_exceptions_part2_peripheral_IRQs/src/console.rs b/14_exceptions_part2_peripheral_IRQs/src/console.rs index 3552823cc..c3154ba23 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/console.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs index c9e5af72c..9c8eb6f6e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/exception.rs index dfa852a87..3c5e7bc83 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs index c16ce0077..2b9ef05f3 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs @@ -6,11 +6,18 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; use core::{fmt, marker::PhantomData}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::{ + is_local_irq_masked, local_irq_mask, local_irq_mask_save, local_irq_restore, local_irq_unmask, + print_state, +}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -116,7 +123,7 @@ impl IRQNumber<{ MAX_INCLUSIVE }> { } /// Return the wrapped number. - pub fn get(self) -> usize { + pub const fn get(self) -> usize { self.0 } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index 635732bd8..bc50cabd8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -7,24 +7,7 @@ //! The `kernel` library. //! -//! Used by `main.rs` to compose the final kernel binary. -//! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`bsp::exception::asynchronous::irq_manager()`] - Returns a reference to the kernel's [IRQ -//! Handling interface]. -//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -//! - [`state::state_manager()`] - Returns a reference to the kernel's [state management] instance. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [IRQ Handling interface]: ../libkernel/exception/asynchronous/interface/trait.IRQManager.html -//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -//! [state management]: ../libkernel/state/struct.StateManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html +//! Used to compose the final kernel binary. //! //! # Code organization and architecture //! @@ -39,15 +22,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -56,9 +46,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -110,6 +99,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] #![feature(asm)] @@ -129,9 +126,6 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(crate::test_runner)] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()` (defined in `main.rs`). - mod panic_wait; mod runtime_init; mod synchronization; diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs index 163419ec0..cca2951aa 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs @@ -5,8 +5,8 @@ //! Memory Management Unit. //! //! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file -//! provides types for composing an architecture-agnostic description of the kernel 's virtual -//! memory layout. +//! provides types for composing an architecture-agnostic description of the kernel's virtual memory +//! layout. //! //! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` //! function. @@ -17,10 +17,16 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; -pub use arch_mmu::*; + +mod translation_table; use core::{fmt, ops::RangeInclusive}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_mmu::mmu; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -40,6 +46,12 @@ pub mod interface { } } +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes the size of an address space. +pub struct AddressSpaceSize; + /// Architecture agnostic translation types. #[allow(missing_docs)] #[derive(Copy, Clone)] @@ -95,6 +107,41 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpaceSize { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); + assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + + // Must adhere to architectural restrictions. + assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); + assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); + assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + + AS_SIZE + } +} + impl Default for AttributeFields { fn default() -> AttributeFields { AttributeFields { diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bbb6f9394 --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; diff --git a/14_exceptions_part2_peripheral_IRQs/src/print.rs b/14_exceptions_part2_peripheral_IRQs/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/print.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/14_exceptions_part2_peripheral_IRQs/src/state.rs b/14_exceptions_part2_peripheral_IRQs/src/state.rs index d08e67d6e..c94d04c84 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/state.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/state.rs @@ -69,7 +69,7 @@ impl StateManager { } } - /// Return if the kernel is still in an init state. + /// Return if the kernel is init state. pub fn is_init(&self) -> bool { self.state() == State::Init } diff --git a/14_exceptions_part2_peripheral_IRQs/src/time.rs b/14_exceptions_part2_peripheral_IRQs/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/time.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index acb48ded6..beb2cffc6 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -15,7 +15,7 @@ - [Introduction](#introduction) - [Implementation](#implementation) - - [A New Mapping API in `src/memory/mmu.rs`](#a-new-mapping-api-in-srcmemorymmurs) + - [A New Mapping API in `src/memory/mmu.rs`](#a-new-mapping-api-in-srcmemorymmutranslationtablers) - [Using the new API in `bsp` code and drivers](#using-the-new-api-in-bsp-code-and-drivers) - [Additional Changes](#additional-changes) - [Test it](#test-it) @@ -77,7 +77,7 @@ Until now, the whole address space of the board was identity mapped at once. The together directly while setting up the translation tables, without any indirection through **generic kernel code** (`src/memory/**`). -The way it worked was that the `architectural MMU driver` would query the `bsp code` about the start +The way it worked was that the `architectural MMU code` would query the `bsp code` about the start and end of the physical address space, and any special regions in this space that need a mapping that _is not_ normal chacheable DRAM. It would then go ahead and map the whole address space at once and never touch the page tables again during runtime. @@ -85,7 +85,7 @@ and never touch the page tables again during runtime. Changing in this tutorial, **architecture** and **bsp** code will no longer talk to each other directly. Instead, this is decoupled now through the kernel's **generic MMU subsystem code**. -### A New Mapping API in `src/memory/mmu.rs` +### A New Mapping API in `src/memory/mmu/translation_table.rs` First, we define an interface for operating on `translation tables`: @@ -98,11 +98,11 @@ pub trait TranslationTable { /// The translation table's base address to be used for programming the MMU. fn phys_base_address(&self) -> Address; - /// Map the given physical pages to the given virtual pages. + /// Map the given virtual pages to the given physical pages. unsafe fn map_pages_at( &mut self, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str>; @@ -126,7 +126,7 @@ pub trait TranslationTable { The MMU driver (`src/_arch/_/memory/mmu.rs`) has one global instance for the kernel tables which implements this interface, and which can be accessed by calling `arch_mmu::kernel_translation_tables()` in the generic kernel code (`src/memory/mmu.rs`). From -there, we provice a couple of memory mapping functions that wrap around this interface , and which +there, we provide a couple of memory mapping functions that wrap around this interface , and which are exported for the rest of the kernel to use: ```rust @@ -163,8 +163,8 @@ provides a dedicated call to **map the kernel binary** (because it is the `BSP` pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { kernel_mmu::kernel_map_pages_at( "Kernel boot-core stack", - &phys_stack_page_desc(), &virt_stack_page_desc(), + &phys_stack_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, acc_perms: AccessPermissions::ReadWrite, @@ -225,7 +225,7 @@ There's a couple of changes not covered in this tutorial text, but the reader sh through them: - [`src/memory/mmu/types.rs`](src/memory/mmu/types.rs) introduces a couple of supporting types, like - `Address`, which is used to differentiate between `Physical` and `Virtual` addresses. + `Page`. - [`src/memory/mmu/mapping_record.rs`](src/memory/mmu/mapping_record.rs) provides the generic kernel code's way of tracking previous memory mappings for use cases such as reusing existing mappings (in case of drivers that have their MMIO ranges in the same `64 KiB` page) or printing mappings @@ -310,10 +310,10 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs -@@ -71,7 +71,7 @@ +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +--- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs ++++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +@@ -56,7 +56,7 @@ ELR_EL2.set(runtime_init::runtime_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. @@ -323,182 +323,81 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs 15_virtua // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. asm::eret() -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs -@@ -4,10 +4,19 @@ - - //! Memory Management Unit Driver. - //! --//! Static translation tables, compiled on boot; Everything 64 KiB granule. -+//! Only 64 KiB granule is supported. +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +@@ -15,9 +15,12 @@ --use super::{AccessPermissions, AttributeFields, MemAttributes}; --use crate::{bsp, memory}; -+use crate::{ -+ bsp, + use crate::{ + bsp, memory, +- memory::mmu::{ +- arch_mmu::{Granule512MiB, Granule64KiB}, +- AccessPermissions, AttributeFields, MemAttributes, + memory::{ -+ mmu, + mmu::{ -+ AccessPermissions, Address, AddressType, AttributeFields, MemAttributes, Page, -+ PageSliceDescriptor, Physical, Virtual, ++ arch_mmu::{Granule512MiB, Granule64KiB}, ++ AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, + }, -+ }, -+ synchronization::InitStateLock, -+}; ++ Address, AddressType, Physical, Virtual, + }, + }; use core::convert; - use cortex_a::{barrier, regs::*}; - use register::{register_bitfields, InMemoryRegister}; -@@ -15,6 +24,7 @@ - //-------------------------------------------------------------------------------------------------- - // Private Definitions - //-------------------------------------------------------------------------------------------------- -+use mmu::interface::TranslationGranule; +@@ -117,11 +120,11 @@ + } - // A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. - register_bitfields! {u64, -@@ -87,9 +97,6 @@ - ] + trait BaseAddr { +- fn base_addr_u64(&self) -> u64; +- fn base_addr_usize(&self) -> usize; ++ fn phys_base_addr(&self) -> Address; } --const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) --const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) -- - /// A table descriptor for 64 KiB aperture. - /// - /// The output points to the next table. -@@ -104,35 +111,65 @@ - #[repr(transparent)] - struct PageDescriptor(u64); +-const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; ++const NUM_LVL2_TABLES: usize = ++ bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; -+#[derive(Copy, Clone)] -+enum Granule512MiB {} -+ -+trait BaseAddr { -+ fn phys_base_addr(&self) -> Address; -+} -+ -+/// Constants for indexing the MAIR_EL1. -+#[allow(dead_code)] -+mod mair { -+ pub const DEVICE: u64 = 0; -+ pub const NORMAL: u64 = 1; -+} -+ -+/// Memory Management Unit type. -+struct MemoryManagementUnit; -+ -+/// This constant is the power-of-two exponent that defines the virtual address space size. -+/// -+/// Values tested and known to be working: -+/// - 30 (1 GiB) -+/// - 31 (2 GiB) -+/// - 32 (4 GiB) -+/// - 33 (8 GiB) -+const ADDR_SPACE_SIZE_EXPONENT: usize = 33; -+ -+const NUM_LVL2_TABLES: usize = (1 << ADDR_SPACE_SIZE_EXPONENT) >> Granule512MiB::SHIFT; -+const T0SZ: u64 = (64 - ADDR_SPACE_SIZE_EXPONENT) as u64; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Definitions -+//-------------------------------------------------------------------------------------------------- -+ - /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB - /// aligned, hence the "reverse" order of appearance. - #[repr(C)] - #[repr(align(65536))] --struct FixedSizeTranslationTable { -+pub(in crate::memory::mmu) struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + //-------------------------------------------------------------------------------------------------- + // Public Definitions +@@ -137,6 +140,12 @@ /// Table descriptors, covering 512 MiB windows. lvl2: [TableDescriptor; NUM_TABLES], --} - --const NUM_LVL2_TABLES: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; --type ArchTranslationTable = FixedSizeTranslationTable; ++ + /// Index of the next free MMIO page. + cur_l3_mmio_index: usize, - --trait BaseAddr { -- fn base_addr_u64(&self) -> u64; -- fn base_addr_usize(&self) -> usize; ++ + /// Have the tables been initialized? + initialized: bool, } --/// Constants for indexing the MAIR_EL1. --#[allow(dead_code)] --mod mair { -- pub const DEVICE: u64 = 0; -- pub const NORMAL: u64 = 1; --} -+pub(in crate::memory::mmu) type ArchTranslationTable = FixedSizeTranslationTable; - --/// Memory Management Unit type. --struct MemoryManagementUnit; -+// Supported translation granules are exported below, so that BSP code can pick between the options. -+// This driver only supports 64 KiB at the moment. -+ -+#[derive(Copy, Clone)] -+/// 64 KiB translation granule. -+pub enum Granule64KiB {} - - //-------------------------------------------------------------------------------------------------- - // Global instances -@@ -143,7 +180,8 @@ - /// # Safety - /// - /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". --static mut KERNEL_TABLES: ArchTranslationTable = ArchTranslationTable::new(); -+static KERNEL_TABLES: InitStateLock = -+ InitStateLock::new(ArchTranslationTable::new()); - - static MMU: MemoryManagementUnit = MemoryManagementUnit; - -@@ -151,13 +189,15 @@ - // Private Code + /// A translation table type for the kernel space. +@@ -147,12 +156,9 @@ //-------------------------------------------------------------------------------------------------- --impl BaseAddr for [T; N] { + impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { - self as *const T as u64 - } -+impl mmu::interface::TranslationGranule for Granule512MiB { -+ const SIZE: usize = 512 * 1024 * 1024; -+ const SHIFT: usize = 29; // log2(SIZE) -+} - +- - fn base_addr_usize(&self) -> usize { - self as *const _ as usize -+impl BaseAddr for [T; N] { + fn phys_base_addr(&self) -> Address { + // The binary is still identity mapped, so we don't need to convert here. + Address::new(self as *const _ as usize) } } -@@ -165,7 +205,7 @@ - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); - -- let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; -+ let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table -@@ -215,23 +255,33 @@ +@@ -225,20 +231,29 @@ + } - impl PageDescriptor { /// Create an instance. -- fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { -+ fn new(output_addr: *const Page, attribute_fields: &AttributeFields) -> Self { +- pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { ++ pub fn from_output_addr( ++ output_addr: *const Page, ++ attribute_fields: &AttributeFields, ++ ) -> Self { let val = InMemoryRegister::::new(0); -- let shifted = output_addr as u64 >> SIXTYFOUR_KIB_SHIFT; -+ let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; val.write( STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::AF::True @@ -508,36 +407,53 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), ); - Self(val.get()) + Self { value: val.get() } } + + /// Returns the valid bit. + fn is_valid(&self) -> bool { -+ InMemoryRegister::::new(self.0) ++ InMemoryRegister::::new(self.value) + .is_set(STAGE1_PAGE_DESCRIPTOR::VALID) + } } - impl FixedSizeTranslationTable<{ NUM_TABLES }> { + //-------------------------------------------------------------------------------------------------- +@@ -246,44 +261,172 @@ + //-------------------------------------------------------------------------------------------------- + + impl FixedSizeTranslationTable { + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + /// Create an instance. + #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { ++ assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); assert!(NUM_TABLES > 0); -@@ -239,7 +289,55 @@ +- assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); ++ assert!((bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); + Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + cur_l3_mmio_index: 0, + initialized: false, -+ } -+ } -+ + } + } + +- /// Iterates over all static translation table entries and fills them at once. +- /// +- /// # Safety +- /// +- /// - Modifies a `static mut`. Ensure it only happens from here. +- pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { +- for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { +- *l2_entry = +- TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + /// The start address of the table's MMIO range. + #[inline(always)] -+ fn mmio_start_addr(&self) -> Address { ++ const fn mmio_start_addr(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), @@ -546,7 +462,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] -+ fn mmio_end_addr_inclusive(&self) -> Address { ++ const fn mmio_end_addr_inclusive(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) @@ -565,7 +481,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + + if lvl2_index > (NUM_TABLES - 1) { + return Err("Virtual page is out of bounds of translation table"); - } ++ } + + Ok((lvl2_index, lvl3_index)) + } @@ -579,100 +495,46 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + let (lvl2_index, lvl3_index) = self.lvl2_lvl3_index_from(addr)?; + + Ok(&mut self.lvl3[lvl2_index][lvl3_index]) - } - } - -@@ -256,32 +354,9 @@ - ); - } - --/// Iterates over all static translation table entries and fills them at once. --/// --/// # Safety --/// --/// - Modifies a `static mut`. Ensure it only happens from here. --unsafe fn populate_tt_entries() -> Result<(), &'static str> { -- for (l2_nr, l2_entry) in KERNEL_TABLES.lvl2.iter_mut().enumerate() { -- *l2_entry = KERNEL_TABLES.lvl3[l2_nr].base_addr_usize().into(); -- -- for (l3_nr, l3_entry) in KERNEL_TABLES.lvl3[l2_nr].iter_mut().enumerate() { -- let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); -- -- let (output_addr, attribute_fields) = -- bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; -- -- *l3_entry = PageDescriptor::new(output_addr, attribute_fields); -- } -- } -- -- Ok(()) --} -- - /// Configure various settings of stage 1 of the EL1 translation regime. - fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -- let t0sz: u64 = bsp::memory::mmu::addr_space_size().trailing_zeros().into(); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored -@@ -292,7 +367,7 @@ - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks -- + TCR_EL1::T0SZ.val(t0sz), -+ + TCR_EL1::T0SZ.val(T0SZ), - ); - } - -@@ -300,17 +375,126 @@ - // Public Code - //-------------------------------------------------------------------------------------------------- - -+/// Return a guarded reference to the kernel's translation tables. -+pub(in crate::memory::mmu) fn kernel_translation_tables( -+) -> &'static InitStateLock { -+ &KERNEL_TABLES ++ } +} + - /// Return a reference to the MMU instance. --pub fn mmu() -> &'static impl memory::mmu::interface::MMU { -+pub(in crate::memory::mmu) fn mmu() -> &'static impl mmu::interface::MMU { - &MMU - } - - //------------------------------------------------------------------------------ - // OS Interface Code - //------------------------------------------------------------------------------ -+impl mmu::interface::TranslationGranule for Granule64KiB { -+ const SIZE: usize = 64 * 1024; -+ const SHIFT: usize = 16; // log2(SIZE) -+} ++//------------------------------------------------------------------------------ ++// OS Interface Code ++//------------------------------------------------------------------------------ + -+impl mmu::interface::TranslationTable -+ for FixedSizeTranslationTable<{ NUM_TABLES }> ++impl memory::mmu::translation_table::interface::TranslationTable ++ for FixedSizeTranslationTable +{ + unsafe fn init(&mut self) { + if self.initialized { + return; + } -+ + +- for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { +- let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { -+ *lvl2_entry = self.lvl3[lvl2_nr].phys_base_addr().into_usize().into(); ++ let desc = TableDescriptor::from_next_lvl_table_addr( ++ self.lvl3[lvl2_nr].phys_base_addr().into_usize(), ++ ); ++ *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; + } -+ + +- let (output_addr, attribute_fields) = +- bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + fn phys_base_address(&self) -> Address { + self.lvl2.phys_base_addr() + } -+ + +- *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + unsafe fn map_pages_at( + &mut self, -+ phys_pages: &PageSliceDescriptor, + virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); @@ -683,9 +545,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + if p.len() != v.len() { + return Err("Tried to map page slices with unequal sizes"); + } - --impl memory::mmu::interface::MMU for MemoryManagementUnit { -- unsafe fn init(&self) -> Result<(), &'static str> { ++ + // No work to do for empty slices. + if p.is_empty() { + return Ok(()); @@ -700,14 +560,17 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + let page_descriptor = self.page_descriptor_from(virt_page.as_ptr())?; + if page_descriptor.is_valid() { + return Err("Virtual page is already mapped"); -+ } -+ -+ *page_descriptor = PageDescriptor::new(phys_page.as_ptr(), &attr); -+ } -+ -+ Ok(()) -+ } + } + ++ *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + } + + Ok(()) + } + +- /// The translation table's base address to be used for programming the MMU. +- pub fn base_address(&self) -> u64 { +- self.lvl2.base_addr_u64() + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, @@ -743,41 +606,102 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 + } + + false -+ } + } + } + +@@ -292,6 +435,9 @@ + //-------------------------------------------------------------------------------------------------- + + #[cfg(test)] ++pub type MinSizeKernelTranslationTable = FixedSizeTranslationTable<1>; ++ ++#[cfg(test)] + mod tests { + use super::*; + use test_macros::kernel_test; + +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +--- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs ++++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +@@ -15,7 +15,11 @@ + + use crate::{ + bsp, memory, +- memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, ++ memory::{ ++ mmu::{translation_table::KernelTranslationTable, TranslationGranule}, ++ Address, Physical, ++ }, ++ synchronization::InitStateLock, + }; + use cortex_a::{barrier, regs::*}; + +@@ -37,7 +41,7 @@ + pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB + + /// The max supported address space size. +-pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB ++pub const MAX_ADDR_SPACE_SIZE: usize = 8 * 1024 * 1024 * 1024; // 8 GiB + + /// The supported address space size granule. + pub type AddrSpaceSizeGranule = Granule512MiB; +@@ -58,7 +62,8 @@ + /// # Safety + /// + /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". +-static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); ++static KERNEL_TABLES: InitStateLock = ++ InitStateLock::new(KernelTranslationTable::new()); + + static MMU: MemoryManagementUnit = MemoryManagementUnit; + +@@ -83,7 +88,7 @@ + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); +- let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; ++ let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored +@@ -103,6 +108,11 @@ + // Public Code + //-------------------------------------------------------------------------------------------------- + ++/// Return a guarded reference to the kernel's translation tables. ++pub fn kernel_translation_tables() -> &'static InitStateLock { ++ &KERNEL_TABLES +} + -+impl mmu::interface::MMU for MemoryManagementUnit { + /// Return a reference to the MMU instance. + pub fn mmu() -> &'static impl memory::mmu::interface::MMU { + &MMU +@@ -113,7 +123,10 @@ + //------------------------------------------------------------------------------ + + impl memory::mmu::interface::MMU for MemoryManagementUnit { +- unsafe fn init(&self) -> Result<(), &'static str> { + unsafe fn enable( + &self, -+ phys_kernel_table_base_addr: Address, ++ kernel_table_phys_base_addr: Address, + ) -> Result<(), &'static str> { // Fail early if translation granule is not supported. Both RPis support it, though. if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { return Err("Translation granule not supported in HW"); -@@ -319,11 +503,8 @@ +@@ -122,11 +135,8 @@ // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); - // Populate translation tables. -- populate_tt_entries()?; +- KERNEL_TABLES.populate_tt_entries()?; - // Set the "Translation Table Base Register". -- TTBR0_EL1.set_baddr(KERNEL_TABLES.lvl2.base_addr_u64()); -+ TTBR0_EL1.set_baddr(phys_kernel_table_base_addr.into_usize() as u64); - - configure_translation_control(); +- TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); ++ TTBR0_EL1.set_baddr(kernel_table_phys_base_addr.into_usize() as u64); -@@ -347,6 +528,9 @@ - //-------------------------------------------------------------------------------------------------- + self.configure_translation_control(); - #[cfg(test)] -+pub(in crate::memory::mmu) type MinSizeArchTranslationTable = FixedSizeTranslationTable<1>; -+ -+#[cfg(test)] - mod tests { - use super::*; - use test_macros::kernel_test; -@@ -373,7 +557,7 @@ +@@ -158,7 +168,7 @@ #[kernel_test] fn kernel_tables_in_bss() { let bss_range = bsp::memory::bss_range_inclusive(); @@ -787,19 +711,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 assert!(bss_range.contains(&kernel_tables_addr)); } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs -@@ -55,7 +55,7 @@ - } - - // Calculate the register compare value. -- let frq = CNTFRQ_EL0.get() as u64; -+ let frq = CNTFRQ_EL0.get(); - let x = match frq.checked_mul(duration.as_nanos() as u64) { - None => { - warn!("Spin duration too long, skipping"); - diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -974,7 +885,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs -use crate::{bsp, cpu, driver, exception, synchronization, synchronization::InitStateLock}; +use crate::{ -+ bsp, cpu, driver, exception, memory, memory::mmu::Physical, synchronization, ++ bsp, cpu, driver, exception, memory, memory::Physical, synchronization, + synchronization::InitStateLock, +}; +use core::sync::atomic::{AtomicBool, Ordering}; @@ -1055,7 +966,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, - synchronization::IRQSafeNullLock, -+ bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::mmu::Physical, ++ bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::Physical, + synchronization, synchronization::IRQSafeNullLock, }; +use core::sync::atomic::{AtomicUsize, Ordering}; @@ -1153,7 +1064,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ bsp::device_driver::common::MMIODerefWrapper, - exception, synchronization, + driver, exception, memory, -+ memory::mmu::Physical, ++ memory::Physical, + synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; @@ -1242,7 +1153,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ mod peripheral_ic; -use crate::{driver, exception}; -+use crate::{driver, exception, memory, memory::mmu::Physical}; ++use crate::{driver, exception, memory, memory::Physical}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -1285,7 +1196,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ - bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, - synchronization, synchronization::IRQSafeNullLock, + bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, -+ memory::mmu::Physical, synchronization, synchronization::IRQSafeNullLock, ++ memory::Physical, synchronization, synchronization::IRQSafeNullLock, +}; +use core::{ + fmt, @@ -1468,7 +1379,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs -@@ -4,77 +4,128 @@ +@@ -4,70 +4,131 @@ //! BSP Memory Management Unit. @@ -1480,9 +1391,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + memory::{ + mmu as kernel_mmu, + mmu::{ -+ interface, AccessPermissions, AttributeFields, Granule64KiB, MemAttributes, Page, -+ PageSliceDescriptor, Physical, Virtual, ++ AccessPermissions, AddressSpaceSize, AttributeFields, MemAttributes, Page, ++ PageSliceDescriptor, TranslationGranule, + }, ++ Physical, Virtual, + }, +}; @@ -1490,13 +1402,19 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs // Public Definitions //-------------------------------------------------------------------------------------------------- --const NUM_MEM_RANGES: usize = 2; +-/// The address space size chosen by this BSP. +-pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; - +-const NUM_MEM_RANGES: usize = 2; ++/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to ++/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. ++pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; + -/// The virtual memory layout. -/// -/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. -/// It is agnostic of the paging granularity that the architecture's MMU will use. --pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( +-pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( - memory_map::END_INCLUSIVE, - [ - TranslationDescriptor { @@ -1521,15 +1439,17 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - }, - ], -); -+/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to -+/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. -+pub type KernelGranule = Granule64KiB; ++/// The address space size chosen by this BSP. ++pub type KernelVirtAddrSpaceSize = AddressSpaceSize<{ 8 * 1024 * 1024 * 1024 }>; //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -+use interface::TranslationGranule; -+ + +-fn ro_range_inclusive() -> RangeInclusive { +- // Notice the subtraction to turn the exclusive end into an inclusive end. +- #[allow(clippy::range_minus_one)] +- RangeInclusive::new(super::ro_start(), super::ro_end() - 1) +/// Helper function for calculating the number of pages the given parameter spans. +const fn size_to_num_pages(size: usize) -> usize { + assert!(size > 0); @@ -1555,11 +1475,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// The data pages of the kernel binary. +fn virt_data_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::data_size()); - --fn ro_range_inclusive() -> RangeInclusive { -- // Notice the subtraction to turn the exclusive end into an inclusive end. -- #[allow(clippy::range_minus_one)] -- RangeInclusive::new(super::ro_start(), super::ro_end() - 1) ++ + PageSliceDescriptor::from_addr(super::virt_data_start(), num_pages) } @@ -1586,7 +1502,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs // Public Code //-------------------------------------------------------------------------------------------------- --/// Return the address space size in bytes. +-/// Return a reference to the virtual memory layout. +-pub fn virt_mem_layout() -> &'static KernelVirtualLayout { +- &LAYOUT +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( @@ -1598,44 +1516,35 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// Map the kernel binary. +/// +/// # Safety - /// --/// Guarantees size to be a power of two. --pub const fn addr_space_size() -> usize { -- let size = memory_map::END_INCLUSIVE + 1; -- assert!(size.is_power_of_two()); ++/// +/// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. +pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { + kernel_mmu::kernel_map_pages_at( + "Kernel boot-core stack", -+ &phys_stack_page_desc(), + &virt_stack_page_desc(), ++ &phys_stack_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + )?; - -- size --} ++ + kernel_mmu::kernel_map_pages_at( + "Kernel code and RO data", -+ &phys_ro_page_desc(), + &virt_ro_page_desc(), ++ &phys_ro_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadOnly, + execute_never: false, + }, + )?; - --/// Return a reference to the virtual memory layout. --pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { -- &LAYOUT ++ + kernel_mmu::kernel_map_pages_at( + "Kernel data and bss", -+ &phys_data_page_desc(), + &virt_data_page_desc(), ++ &phys_data_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, @@ -1647,19 +1556,19 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs } //-------------------------------------------------------------------------------------------------- -@@ -89,14 +140,12 @@ +@@ -82,14 +143,12 @@ /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { - const SIXTYFOUR_KIB: usize = 65536; -- -- for i in LAYOUT.inner().iter() { -- let start: usize = *(i.virtual_range)().start(); -- let end: usize = *(i.virtual_range)().end() + 1; + for i in [virt_stack_page_desc, virt_ro_page_desc, virt_data_page_desc].iter() { + let start: usize = i().start_addr().into_usize(); + let end: usize = i().end_addr().into_usize(); +- for i in LAYOUT.inner().iter() { +- let start: usize = *(i.virtual_range)().start(); +- let end: usize = *(i.virtual_range)().end() + 1; +- - assert_eq!(start modulo SIXTYFOUR_KIB, 0); - assert_eq!(end modulo SIXTYFOUR_KIB, 0); + assert_eq!(start modulo KernelGranule::SIZE, 0); @@ -1667,7 +1576,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs assert!(end >= start); } } -@@ -104,17 +153,18 @@ +@@ -97,17 +156,18 @@ /// Ensure the kernel's virtual memory layout is free of overlaps. #[kernel_test] fn virt_mem_layout_has_no_overlaps() { @@ -1739,7 +1648,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v pub mod mmu; -+use crate::memory::mmu::{Address, Physical, Virtual}; ++use crate::memory::{Address, Physical, Virtual}; use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- @@ -2030,7 +1939,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs --- 14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 15_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -113,6 +113,7 @@ +@@ -110,6 +110,7 @@ #![allow(incomplete_features)] #![feature(asm)] @@ -2038,7 +1947,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -137,6 +138,7 @@ +@@ -131,6 +132,7 @@ mod synchronization; pub mod bsp; @@ -2151,8 +2060,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 +impl MappingRecordEntry { + pub fn new( + name: &'static str, -+ phys_pages: &PageSliceDescriptor, + virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Self { + Self { @@ -2206,13 +2115,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + pub fn add( + &mut self, + name: &'static str, -+ phys_pages: &PageSliceDescriptor, + virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + let x = self.find_next_free()?; + -+ *x = Some(MappingRecordEntry::new(name, phys_pages, virt_pages, attr)); ++ *x = Some(MappingRecordEntry::new(name, virt_pages, phys_pages, attr)); + Ok(()) + } + @@ -2300,11 +2209,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 +/// Add an entry to the mapping info record. +pub fn kernel_add( + name: &'static str, -+ phys_pages: &PageSliceDescriptor, + virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { -+ KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, phys_pages, virt_pages, attr)) ++ KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, virt_pages, phys_pages, attr)) +} + +pub fn kernel_find_and_insert_mmio_duplicate( @@ -2329,41 +2238,134 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + KERNEL_MAPPING_RECORD.read(|mr| mr.print()); +} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs -@@ -0,0 +1,283 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2020-2021 Andre Richter -+ -+//! Memory Management Unit Types. +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs +--- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs ++++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs +@@ -8,7 +8,104 @@ + #[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] + mod arch_translation_table; + ++use crate::memory::{ ++ mmu::{AttributeFields, PageSliceDescriptor}, ++ Address, Physical, Virtual, ++}; + -+use crate::{bsp, common}; -+use core::{convert::From, marker::PhantomData, ops::RangeInclusive}; + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- + pub use arch_translation_table::KernelTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- -+use super::interface::TranslationGranule; + -+/// Metadata trait for marking the type of an address. -+pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} ++/// Translation table interfaces. ++pub mod interface { ++ use super::*; + -+/// Zero-sized type to mark a physical address. -+#[derive(Copy, Clone, PartialOrd, PartialEq)] -+pub enum Physical {} ++ /// Translation table operations. ++ pub trait TranslationTable { ++ /// Anything that needs to run before any of the other provided functions can be used. ++ /// ++ /// # Safety ++ /// ++ /// - Implementor must ensure that this function can run only once or is harmless if invoked ++ /// multiple times. ++ unsafe fn init(&mut self); + -+/// Zero-sized type to mark a virtual address. -+#[derive(Copy, Clone, PartialOrd, PartialEq)] -+pub enum Virtual {} ++ /// The translation table's base address to be used for programming the MMU. ++ fn phys_base_address(&self) -> Address; + -+/// Generic address type. -+#[derive(Copy, Clone, PartialOrd, PartialEq)] -+pub struct Address { -+ value: usize, -+ _address_type: PhantomData, ++ /// Map the given virtual pages to the given physical pages. ++ /// ++ /// # Safety ++ /// ++ /// - Using wrong attributes can cause multiple issues of different nature in the system. ++ /// - It is not required that the architectural implementation prevents aliasing. That is, ++ /// mapping to the same physical memory using multiple virtual addresses, which would ++ /// break Rust's ownership assumptions. This should be protected against in the kernel's ++ /// generic MMU code. ++ unsafe fn map_pages_at( ++ &mut self, ++ virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, ++ attr: &AttributeFields, ++ ) -> Result<(), &'static str>; ++ ++ /// Obtain a free virtual page slice in the MMIO region. ++ /// ++ /// The "MMIO region" is a distinct region of the implementor's choice, which allows ++ /// differentiating MMIO addresses from others. This can speed up debugging efforts. ++ /// Ideally, those MMIO addresses are also standing out visually so that a human eye can ++ /// identify them. For example, by allocating them from near the end of the virtual address ++ /// space. ++ fn next_mmio_virt_page_slice( ++ &mut self, ++ num_pages: usize, ++ ) -> Result, &'static str>; ++ ++ /// Check if a virtual page splice is in the "MMIO region". ++ fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; ++ } ++} ++ ++//-------------------------------------------------------------------------------------------------- ++// Testing ++//-------------------------------------------------------------------------------------------------- ++ ++#[cfg(test)] ++mod tests { ++ use super::*; ++ use crate::bsp; ++ use arch_translation_table::MinSizeKernelTranslationTable; ++ use interface::TranslationTable; ++ use test_macros::kernel_test; ++ ++ /// Sanity checks for the kernel TranslationTable implementation. ++ #[kernel_test] ++ fn translationtable_implementation_sanity() { ++ // Need to take care that `tables` fits into the stack. ++ let mut tables = MinSizeKernelTranslationTable::new(); ++ ++ unsafe { tables.init() }; ++ ++ let x = tables.next_mmio_virt_page_slice(0); ++ assert!(x.is_err()); ++ ++ let x = tables.next_mmio_virt_page_slice(1_0000_0000); ++ assert!(x.is_err()); ++ ++ let x = tables.next_mmio_virt_page_slice(2).unwrap(); ++ assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); ++ ++ assert_eq!(tables.is_virt_page_slice_mmio(&x), true); ++ ++ assert_eq!( ++ tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), ++ false ++ ); ++ } +} + +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +--- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs ++++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +@@ -0,0 +1,213 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2020-2021 Andre Richter ++ ++//! Memory Management Unit types. ++ ++use crate::{ ++ bsp, common, ++ memory::{Address, AddressType, Physical, Virtual}, ++}; ++use core::{convert::From, marker::PhantomData, ops::RangeInclusive}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- + +/// Generic page type. +#[repr(C)] @@ -2415,60 +2417,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual +// Public Code +//-------------------------------------------------------------------------------------------------- + -+impl AddressType for Physical {} -+impl AddressType for Virtual {} -+ -+//------------------------------------------------------------------------------ -+// Address -+//------------------------------------------------------------------------------ -+ -+impl Address { -+ /// Create an instance. -+ pub const fn new(value: usize) -> Self { -+ Self { -+ value, -+ _address_type: PhantomData, -+ } -+ } -+ -+ /// Align down. -+ pub const fn align_down(self, alignment: usize) -> Self { -+ let aligned = common::align_down(self.value, alignment); -+ -+ Self { -+ value: aligned, -+ _address_type: PhantomData, -+ } -+ } -+ -+ /// Converts `Address` into an usize. -+ pub const fn into_usize(self) -> usize { -+ self.value -+ } -+} -+ -+impl core::ops::Add for Address { -+ type Output = Self; -+ -+ fn add(self, other: usize) -> Self { -+ Self { -+ value: self.value + other, -+ _address_type: PhantomData, -+ } -+ } -+} -+ -+impl core::ops::Sub for Address { -+ type Output = Self; -+ -+ fn sub(self, other: usize) -> Self { -+ Self { -+ value: self.value - other, -+ _address_type: PhantomData, -+ } -+ } -+} -+ +//------------------------------------------------------------------------------ +// Page +//------------------------------------------------------------------------------ @@ -2620,14 +2568,14 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs -@@ -3,23 +3,18 @@ +@@ -3,29 +3,22 @@ // Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. -//! -//! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file --//! provides types for composing an architecture-agnostic description of the kernel 's virtual --//! memory layout. +-//! provides types for composing an architecture-agnostic description of the kernel's virtual memory +-//! layout. -//! -//! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` -//! function. @@ -2638,76 +2586,31 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; - pub use arch_mmu::*; --use core::{fmt, ops::RangeInclusive}; +mod mapping_record; + mod translation_table; +mod types; -+ -+use crate::{bsp, synchronization, warn}; -+ + +-use core::{fmt, ops::RangeInclusive}; ++use crate::{ ++ bsp, ++ memory::{Address, Physical, Virtual}, ++ synchronization, warn, ++}; + +-//-------------------------------------------------------------------------------------------------- +-// Architectural Public Reexports +-//-------------------------------------------------------------------------------------------------- +-pub use arch_mmu::mmu; +pub use types::*; //-------------------------------------------------------------------------------------------------- // Public Definitions -@@ -27,178 +22,229 @@ +@@ -33,16 +26,20 @@ /// Memory Management interfaces. pub mod interface { + use super::*; -+ -+ /// Describes the characteristics of a translation granule. -+ #[allow(missing_docs)] -+ pub trait TranslationGranule { -+ const SIZE: usize; -+ const MASK: usize = Self::SIZE - 1; -+ const SHIFT: usize; -+ } -+ -+ /// Translation table operations. -+ pub trait TranslationTable { -+ /// Anything that needs to run before any of the other provided functions can be used. -+ /// -+ /// # Safety -+ /// -+ /// - Implementor must ensure that this function can run only once or is harmless if invoked -+ /// multiple times. -+ unsafe fn init(&mut self); -+ -+ /// The translation table's base address to be used for programming the MMU. -+ fn phys_base_address(&self) -> Address; -+ -+ /// Map the given physical pages to the given virtual pages. -+ /// -+ /// # Safety -+ /// -+ /// - Using wrong attributes can cause multiple issues of different nature in the system. -+ /// - It is not required that the architectural implementation prevents aliasing. That is, -+ /// mapping to the same physical memory using multiple virtual addresses, which would -+ /// break Rust's ownership assumptions. This should be protected against in this module -+ /// (the kernel's generic MMU code). -+ unsafe fn map_pages_at( -+ &mut self, -+ phys_pages: &PageSliceDescriptor, -+ virt_pages: &PageSliceDescriptor, -+ attr: &AttributeFields, -+ ) -> Result<(), &'static str>; -+ -+ /// Obtain a free virtual page slice in the MMIO region. -+ /// -+ /// The "MMIO region" is a distinct region of the implementor's choice, which allows -+ /// differentiating MMIO addresses from others. This can speed up debugging efforts. -+ /// Ideally, those MMIO addresses are also standing out visually so that a human eye can -+ /// identify them. For example, by allocating them from near the end of the virtual address -+ /// space. -+ fn next_mmio_virt_page_slice( -+ &mut self, -+ num_pages: usize, -+ ) -> Result, &'static str>; -+ -+ /// Check if a virtual page splice is in the "MMIO region". -+ fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; -+ } /// MMU functions. pub trait MMU { @@ -2722,11 +2625,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - unsafe fn init(&self) -> Result<(), &'static str>; + unsafe fn enable( + &self, -+ phys_kernel_table_base_addr: Address, ++ kernel_table_phys_base_addr: Address, + ) -> Result<(), &'static str>; } } +@@ -52,55 +49,35 @@ + /// Describes the size of an address space. + pub struct AddressSpaceSize; + -/// Architecture agnostic translation types. -#[allow(missing_docs)] -#[derive(Copy, Clone)] @@ -2742,12 +2649,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - CacheableDRAM, - Device, -} -+//-------------------------------------------------------------------------------------------------- -+// Private Code -+//-------------------------------------------------------------------------------------------------- -+use interface::{TranslationTable, MMU}; -+use synchronization::interface::ReadWriteEx; - +- -/// Architecture agnostic access permissions. -#[allow(missing_docs)] -#[derive(Copy, Clone)] @@ -2755,23 +2657,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - ReadOnly, - ReadWrite, -} -+/// Map pages in the kernel's translation tables. -+/// -+/// No input checks done, input is passed through to the architectural implementation. -+/// -+/// # Safety -+/// -+/// - See `map_pages_at()`. -+/// - Does not prevent aliasing. -+unsafe fn kernel_map_pages_at_unchecked( -+ name: &'static str, -+ phys_pages: &PageSliceDescriptor, -+ virt_pages: &PageSliceDescriptor, -+ attr: &AttributeFields, -+) -> Result<(), &'static str> { -+ arch_mmu::kernel_translation_tables() -+ .write(|tables| tables.map_pages_at(phys_pages, virt_pages, attr))?; - +- -/// Collection of memory attributes. -#[allow(missing_docs)] -#[derive(Copy, Clone)] @@ -2780,10 +2666,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - pub acc_perms: AccessPermissions, - pub execute_never: bool, -} -+ if let Err(x) = mapping_record::kernel_add(name, phys_pages, virt_pages, attr) { -+ warn!("{}", x); -+ } - +- -/// Architecture agnostic descriptor for a memory range. -#[allow(missing_docs)] -pub struct TranslationDescriptor { @@ -2791,111 +2674,59 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - pub virtual_range: fn() -> RangeInclusive, - pub physical_range_translation: Translation, - pub attribute_fields: AttributeFields, -+ Ok(()) - } - +-} +- -/// Type for expressing the kernel's virtual memory layout. -pub struct KernelVirtualLayout { - /// The last (inclusive) address of the address space. - max_virt_addr_inclusive: usize, +//-------------------------------------------------------------------------------------------------- -+// Public Code ++// Private Code +//-------------------------------------------------------------------------------------------------- -+use interface::TranslationGranule; - -- /// Array of descriptors for non-standard (normal cacheable DRAM) memory regions. -- inner: [TranslationDescriptor; NUM_SPECIAL_RANGES], -+/// Raw mapping of virtual to physical pages in the kernel translation tables. ++use interface::MMU; ++use synchronization::interface::ReadWriteEx; ++use translation_table::interface::TranslationTable; ++ ++/// Map pages in the kernel's translation tables. +/// -+/// Prevents mapping into the MMIO range of the tables. ++/// No input checks done, input is passed through to the architectural implementation. +/// +/// # Safety +/// -+/// - See `kernel_map_pages_at_unchecked()`. -+/// - Does not prevent aliasing. Currently, we have to trust the callers. -+pub unsafe fn kernel_map_pages_at( -+ name: &'static str, -+ phys_pages: &PageSliceDescriptor, ++/// - See `map_pages_at()`. ++/// - Does not prevent aliasing. ++unsafe fn kernel_map_pages_at_unchecked( ++ name: &'static str, + virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { -+ let is_mmio = arch_mmu::kernel_translation_tables() -+ .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); -+ if is_mmio { -+ return Err("Attempt to manually map into MMIO region"); -+ } -+ -+ kernel_map_pages_at_unchecked(name, phys_pages, virt_pages, attr)?; ++ arch_mmu::kernel_translation_tables() ++ .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; + ++ if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { ++ warn!("{}", x); ++ } + +- /// Array of descriptors for non-standard (normal cacheable DRAM) memory regions. +- inner: [TranslationDescriptor; NUM_SPECIAL_RANGES], + Ok(()) -+} -+ -+/// MMIO remapping in the kernel translation tables. -+/// -+/// Typically used by device drivers. -+/// -+/// # Safety -+/// -+/// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. -+pub unsafe fn kernel_map_mmio( -+ name: &'static str, -+ phys_mmio_descriptor: &MMIODescriptor, -+) -> Result, &'static str> { -+ let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); -+ let offset_into_start_page = -+ phys_mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; -+ -+ // Check if an identical page slice has been mapped for another driver. If so, reuse it. -+ let virt_addr = if let Some(addr) = -+ mapping_record::kernel_find_and_insert_mmio_duplicate(phys_mmio_descriptor, name) -+ { -+ addr -+ // Otherwise, allocate a new virtual page slice and map it. -+ } else { -+ let virt_pages: PageSliceDescriptor = arch_mmu::kernel_translation_tables() -+ .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; -+ -+ kernel_map_pages_at_unchecked( -+ name, -+ &phys_pages, -+ &virt_pages, -+ &AttributeFields { -+ mem_attributes: MemAttributes::Device, -+ acc_perms: AccessPermissions::ReadWrite, -+ execute_never: true, -+ }, -+ )?; -+ -+ virt_pages.start_addr() -+ }; -+ -+ Ok(virt_addr + offset_into_start_page) -+} -+ -+/// Map the kernel's binary and enable the MMU. -+/// -+/// # Safety -+/// -+/// - Crucial function during kernel init. Changes the the complete memory view of the processor. -+pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> { -+ let phys_base_addr = arch_mmu::kernel_translation_tables().write(|tables| { -+ tables.init(); -+ tables.phys_base_address() -+ }); -+ -+ bsp::memory::mmu::kernel_map_binary()?; -+ arch_mmu::mmu().enable(phys_base_addr) -+} -+ -+/// Human-readable print of all recorded kernel mappings. -+pub fn kernel_print_mappings() { -+ mapping_record::kernel_print() } //-------------------------------------------------------------------------------------------------- --// Public Code -+// Testing - //-------------------------------------------------------------------------------------------------- +@@ -111,6 +88,9 @@ + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + ++ /// The granule's mask. ++ pub const MASK: usize = Self::SIZE - 1; ++ + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + +@@ -142,110 +122,89 @@ + } + } -impl Default for AttributeFields { - fn default() -> AttributeFields { @@ -3002,37 +2833,185 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - #[cfg(test)] - pub fn inner(&self) -> &[TranslationDescriptor; NUM_SPECIAL_RANGES] { - &self.inner -+#[cfg(test)] -+mod tests { -+ use super::*; -+ use test_macros::kernel_test; +- } ++/// Raw mapping of virtual to physical pages in the kernel translation tables. ++/// ++/// Prevents mapping into the MMIO range of the tables. ++/// ++/// # Safety ++/// ++/// - See `kernel_map_pages_at_unchecked()`. ++/// - Does not prevent aliasing. Currently, the callers must be trusted. ++pub unsafe fn kernel_map_pages_at( ++ name: &'static str, ++ virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, ++ attr: &AttributeFields, ++) -> Result<(), &'static str> { ++ let is_mmio = arch_mmu::kernel_translation_tables() ++ .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); ++ if is_mmio { ++ return Err("Attempt to manually map into MMIO region"); ++ } + -+ /// Sanity checks for the kernel TranslationTable implementation. -+ #[kernel_test] -+ fn translationtable_implementation_sanity() { -+ // Need to take care that `tables` fits into the stack. -+ let mut tables = MinSizeArchTranslationTable::new(); ++ kernel_map_pages_at_unchecked(name, virt_pages, phys_pages, attr)?; + -+ unsafe { tables.init() }; ++ Ok(()) ++} + -+ let x = tables.next_mmio_virt_page_slice(0); -+ assert!(x.is_err()); ++/// MMIO remapping in the kernel translation tables. ++/// ++/// Typically used by device drivers. ++/// ++/// # Safety ++/// ++/// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. ++pub unsafe fn kernel_map_mmio( ++ name: &'static str, ++ phys_mmio_descriptor: &MMIODescriptor, ++) -> Result, &'static str> { ++ let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); ++ let offset_into_start_page = ++ phys_mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + -+ let x = tables.next_mmio_virt_page_slice(1_0000_0000); -+ assert!(x.is_err()); ++ // Check if an identical page slice has been mapped for another driver. If so, reuse it. ++ let virt_addr = if let Some(addr) = ++ mapping_record::kernel_find_and_insert_mmio_duplicate(phys_mmio_descriptor, name) ++ { ++ addr ++ // Otherwise, allocate a new virtual page slice and map it. ++ } else { ++ let virt_pages: PageSliceDescriptor = arch_mmu::kernel_translation_tables() ++ .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; + -+ let x = tables.next_mmio_virt_page_slice(2).unwrap(); -+ assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); ++ kernel_map_pages_at_unchecked( ++ name, ++ &virt_pages, ++ &phys_pages, ++ &AttributeFields { ++ mem_attributes: MemAttributes::Device, ++ acc_perms: AccessPermissions::ReadWrite, ++ execute_never: true, ++ }, ++ )?; + -+ assert_eq!(tables.is_virt_page_slice_mmio(&x), true); ++ virt_pages.start_addr() ++ }; + -+ assert_eq!( -+ tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), -+ false -+ ); - } ++ Ok(virt_addr + offset_into_start_page) ++} ++ ++/// Map the kernel's binary and enable the MMU. ++/// ++/// # Safety ++/// ++/// - Crucial function during kernel init. Changes the the complete memory view of the processor. ++pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> { ++ let phys_base_addr = arch_mmu::kernel_translation_tables().write(|tables| { ++ tables.init(); ++ tables.phys_base_address() ++ }); ++ ++ bsp::memory::mmu::kernel_map_binary()?; ++ arch_mmu::mmu().enable(phys_base_addr) ++} ++ ++/// Human-readable print of all recorded kernel mappings. ++pub fn kernel_print_mappings() { ++ mapping_record::kernel_print() } +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2_mmio_remap/src/memory.rs +--- 14_exceptions_part2_peripheral_IRQs/src/memory.rs ++++ 15_virtual_mem_part2_mmio_remap/src/memory.rs +@@ -6,12 +6,85 @@ + + pub mod mmu; + +-use core::ops::RangeInclusive; ++use crate::common; ++use core::{marker::PhantomData, ops::RangeInclusive}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++/// Metadata trait for marking the type of an address. ++pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} ++ ++/// Zero-sized type to mark a physical address. ++#[derive(Copy, Clone, PartialOrd, PartialEq)] ++pub enum Physical {} ++ ++/// Zero-sized type to mark a virtual address. ++#[derive(Copy, Clone, PartialOrd, PartialEq)] ++pub enum Virtual {} ++ ++/// Generic address type. ++#[derive(Copy, Clone, PartialOrd, PartialEq)] ++pub struct Address { ++ value: usize, ++ _address_type: PhantomData ATYPE>, ++} + + //-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- + ++impl AddressType for Physical {} ++impl AddressType for Virtual {} ++ ++impl Address { ++ /// Create an instance. ++ pub const fn new(value: usize) -> Self { ++ Self { ++ value, ++ _address_type: PhantomData, ++ } ++ } ++ ++ /// Align down. ++ pub const fn align_down(self, alignment: usize) -> Self { ++ let aligned = common::align_down(self.value, alignment); ++ ++ Self { ++ value: aligned, ++ _address_type: PhantomData, ++ } ++ } ++ ++ /// Converts `Address` into an usize. ++ pub const fn into_usize(self) -> usize { ++ self.value ++ } ++} ++ ++impl core::ops::Add for Address { ++ type Output = Self; ++ ++ fn add(self, other: usize) -> Self { ++ Self { ++ value: self.value + other, ++ _address_type: PhantomData, ++ } ++ } ++} ++ ++impl core::ops::Sub for Address { ++ type Output = Self; ++ ++ fn sub(self, other: usize) -> Self { ++ Self { ++ value: self.value - other, ++ _address_type: PhantomData, ++ } ++ } ++} ++ + /// Zero out an inclusive memory range. + /// + /// # Safety + diff -uNr 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs 15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs --- 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs +++ 15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs index 6ecea1825..948bad747 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs @@ -3,79 +3,15 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::phys_boot_core_stack_end().into_usize() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..276965146 --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::memory::phys_boot_core_stack_end().into_usize() as u64); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up +/// a stack for EL2. +#[no_mangle] +pub unsafe fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs index 99fbd85bd..5cf9eb5ca 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception use crate::{bsp, exception}; use core::{cell::UnsafeCell, fmt}; diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs index 968eedf34..a4b1a548e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous use cortex_a::regs::*; diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs index 8abb997e2..ebf454ce9 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -5,183 +5,65 @@ //! Memory Management Unit Driver. //! //! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu use crate::{ - bsp, + bsp, memory, memory::{ - mmu, - mmu::{ - AccessPermissions, Address, AddressType, AttributeFields, MemAttributes, Page, - PageSliceDescriptor, Physical, Virtual, - }, + mmu::{translation_table::KernelTranslationTable, TranslationGranule}, + Address, Physical, }, synchronization::InitStateLock, }; -use core::convert; use cortex_a::{barrier, regs::*}; -use register::{register_bitfields, InMemoryRegister}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -use mmu::interface::TranslationGranule; - -// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. -register_bitfields! {u64, - STAGE1_TABLE_DESCRIPTOR [ - /// Physical address of the next descriptor. - NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. -register_bitfields! {u64, - STAGE1_PAGE_DESCRIPTOR [ - /// Unprivileged execute-never. - UXN OFFSET(54) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Privileged execute-never. - PXN OFFSET(53) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). - OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] - - /// Access flag. - AF OFFSET(10) NUMBITS(1) [ - False = 0, - True = 1 - ], - - /// Shareability field. - SH OFFSET(8) NUMBITS(2) [ - OuterShareable = 0b10, - InnerShareable = 0b11 - ], - - /// Access Permissions. - AP OFFSET(6) NUMBITS(2) [ - RW_EL1 = 0b00, - RW_EL1_EL0 = 0b01, - RO_EL1 = 0b10, - RO_EL1_EL0 = 0b11 - ], - - /// Memory attributes index into the MAIR_EL1 register. - AttrIndx OFFSET(2) NUMBITS(3) [], - - TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 - ], - - VALID OFFSET(0) NUMBITS(1) [ - False = 0, - True = 1 - ] - ] -} - -/// A table descriptor for 64 KiB aperture. -/// -/// The output points to the next table. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct TableDescriptor(u64); - -/// A page descriptor with 64 KiB aperture. -/// -/// The output points to physical memory. -#[derive(Copy, Clone)] -#[repr(transparent)] -struct PageDescriptor(u64); - -#[derive(Copy, Clone)] -enum Granule512MiB {} - -trait BaseAddr { - fn phys_base_addr(&self) -> Address; -} - -/// Constants for indexing the MAIR_EL1. -#[allow(dead_code)] -mod mair { - pub const DEVICE: u64 = 0; - pub const NORMAL: u64 = 1; -} /// Memory Management Unit type. struct MemoryManagementUnit; -/// This constant is the power-of-two exponent that defines the virtual address space size. -/// -/// Values tested and known to be working: -/// - 30 (1 GiB) -/// - 31 (2 GiB) -/// - 32 (4 GiB) -/// - 33 (8 GiB) -const ADDR_SPACE_SIZE_EXPONENT: usize = 33; - -const NUM_LVL2_TABLES: usize = (1 << ADDR_SPACE_SIZE_EXPONENT) >> Granule512MiB::SHIFT; -const T0SZ: u64 = (64 - ADDR_SPACE_SIZE_EXPONENT) as u64; - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. -#[repr(C)] -#[repr(align(65536))] -pub(in crate::memory::mmu) struct FixedSizeTranslationTable { - /// Page descriptors, covering 64 KiB windows per entry. - lvl3: [[PageDescriptor; 8192]; NUM_TABLES], +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; - /// Table descriptors, covering 512 MiB windows. - lvl2: [TableDescriptor; NUM_TABLES], +/// The min supported address space size. +pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - /// Index of the next free MMIO page. - cur_l3_mmio_index: usize, - - /// Have the tables been initialized? - initialized: bool, -} +/// The max supported address space size. +pub const MAX_ADDR_SPACE_SIZE: usize = 8 * 1024 * 1024 * 1024; // 8 GiB -pub(in crate::memory::mmu) type ArchTranslationTable = FixedSizeTranslationTable; +/// The supported address space size granule. +pub type AddrSpaceSizeGranule = Granule512MiB; -// Supported translation granules are exported below, so that BSP code can pick between the options. -// This driver only supports 64 KiB at the moment. - -#[derive(Copy, Clone)] -/// 64 KiB translation granule. -pub enum Granule64KiB {} +/// Constants for indexing the MAIR_EL1. +#[allow(dead_code)] +pub mod mair { + pub const DEVICE: u64 = 0; + pub const NORMAL: u64 = 1; +} //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -/// The translation tables. +/// The kernel translation tables. /// /// # Safety /// /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static KERNEL_TABLES: InitStateLock = - InitStateLock::new(ArchTranslationTable::new()); +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new()); static MMU: MemoryManagementUnit = MemoryManagementUnit; @@ -189,311 +71,61 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- -impl mmu::interface::TranslationGranule for Granule512MiB { - const SIZE: usize = 512 * 1024 * 1024; - const SHIFT: usize = 29; // log2(SIZE) -} - -impl BaseAddr for [T; N] { - fn phys_base_addr(&self) -> Address { - // The binary is still identity mapped, so we don't need to convert here. - Address::new(self as *const _ as usize) - } -} - -impl convert::From for TableDescriptor { - fn from(next_lvl_table_addr: usize) -> Self { - let val = InMemoryRegister::::new(0); +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; - val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True - + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + // Attribute 0 - Device. + MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, ); - - TableDescriptor(val.get()) } -} - -/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. -impl convert::From - for register::FieldValue -{ - fn from(attribute_fields: AttributeFields) -> Self { - // Memory attributes. - let mut desc = match attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => { - STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) - } - MemAttributes::Device => { - STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable - + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) - } - }; - - // Access Permissions. - desc += match attribute_fields.acc_perms { - AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, - AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, - }; - - // The execute-never attribute is mapped to PXN in AArch64. - desc += if attribute_fields.execute_never { - STAGE1_PAGE_DESCRIPTOR::PXN::True - } else { - STAGE1_PAGE_DESCRIPTOR::PXN::False - }; - - // Always set unprivileged exectue-never as long as userspace is not implemented yet. - desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; - - desc - } -} -impl PageDescriptor { - /// Create an instance. - fn new(output_addr: *const Page, attribute_fields: &AttributeFields) -> Self { - let val = InMemoryRegister::::new(0); - - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.clone().into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); + let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpaceSize::SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Ignored + + TCR_EL1::IPS.val(ips) + + TCR_EL1::EPD1::DisableTTBR1Walks + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::T0SZ.val(t0sz), ); - - Self(val.get()) - } - - /// Returns the valid bit. - fn is_valid(&self) -> bool { - InMemoryRegister::::new(self.0) - .is_set(STAGE1_PAGE_DESCRIPTOR::VALID) } } -impl FixedSizeTranslationTable<{ NUM_TABLES }> { - // Reserve the last 256 MiB of the address space for MMIO mappings. - const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; - const L3_MMIO_START_INDEX: usize = 8192 / 2; - - /// Create an instance. - pub const fn new() -> Self { - assert!(NUM_TABLES > 0); - - Self { - lvl3: [[PageDescriptor(0); 8192]; NUM_TABLES], - lvl2: [TableDescriptor(0); NUM_TABLES], - cur_l3_mmio_index: 0, - initialized: false, - } - } - - /// The start address of the table's MMIO range. - #[inline(always)] - fn mmio_start_addr(&self) -> Address { - Address::new( - (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) - | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), - ) - } - - /// The inclusive end address of the table's MMIO range. - #[inline(always)] - fn mmio_end_addr_inclusive(&self) -> Address { - Address::new( - (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) - | (8191 << Granule64KiB::SHIFT) - | (Granule64KiB::SIZE - 1), - ) - } - - /// Helper to calculate the lvl2 and lvl3 indices from an address. - #[inline(always)] - fn lvl2_lvl3_index_from( - &self, - addr: *const Page, - ) -> Result<(usize, usize), &'static str> { - let lvl2_index = addr as usize >> Granule512MiB::SHIFT; - let lvl3_index = (addr as usize & Granule512MiB::MASK) >> Granule64KiB::SHIFT; - - if lvl2_index > (NUM_TABLES - 1) { - return Err("Virtual page is out of bounds of translation table"); - } - - Ok((lvl2_index, lvl3_index)) - } - - /// Returns the PageDescriptor corresponding to the supplied Page. - #[inline(always)] - fn page_descriptor_from( - &mut self, - addr: *const Page, - ) -> Result<&mut PageDescriptor, &'static str> { - let (lvl2_index, lvl3_index) = self.lvl2_lvl3_index_from(addr)?; - - Ok(&mut self.lvl3[lvl2_index][lvl3_index]) - } -} - -/// Setup function for the MAIR_EL1 register. -fn set_up_mair() { - // Define the memory types being mapped. - MAIR_EL1.write( - // Attribute 1 - Cacheable normal DRAM. - MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + - MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + - - // Attribute 0 - Device. - MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, - ); -} - -/// Configure various settings of stage 1 of the EL1 translation regime. -fn configure_translation_control() { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - - TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks - + TCR_EL1::TG0::KiB_64 - + TCR_EL1::SH0::Inner - + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable - + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(T0SZ), - ); -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- /// Return a guarded reference to the kernel's translation tables. -pub(in crate::memory::mmu) fn kernel_translation_tables( -) -> &'static InitStateLock { +pub fn kernel_translation_tables() -> &'static InitStateLock { &KERNEL_TABLES } /// Return a reference to the MMU instance. -pub(in crate::memory::mmu) fn mmu() -> &'static impl mmu::interface::MMU { +pub fn mmu() -> &'static impl memory::mmu::interface::MMU { &MMU } //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ -impl mmu::interface::TranslationGranule for Granule64KiB { - const SIZE: usize = 64 * 1024; - const SHIFT: usize = 16; // log2(SIZE) -} - -impl mmu::interface::TranslationTable - for FixedSizeTranslationTable<{ NUM_TABLES }> -{ - unsafe fn init(&mut self) { - if self.initialized { - return; - } - - // Populate the l2 entries. - for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { - *lvl2_entry = self.lvl3[lvl2_nr].phys_base_addr().into_usize().into(); - } - - self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; - self.initialized = true; - } - - fn phys_base_address(&self) -> Address { - self.lvl2.phys_base_addr() - } - - unsafe fn map_pages_at( - &mut self, - phys_pages: &PageSliceDescriptor, - virt_pages: &PageSliceDescriptor, - attr: &AttributeFields, - ) -> Result<(), &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); - - let p = phys_pages.as_slice(); - let v = virt_pages.as_slice(); - - if p.len() != v.len() { - return Err("Tried to map page slices with unequal sizes"); - } - - // No work to do for empty slices. - if p.is_empty() { - return Ok(()); - } - - if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { - return Err("Tried to map outside of physical address space"); - } - let iter = p.iter().zip(v.iter()); - for (phys_page, virt_page) in iter { - let page_descriptor = self.page_descriptor_from(virt_page.as_ptr())?; - if page_descriptor.is_valid() { - return Err("Virtual page is already mapped"); - } - - *page_descriptor = PageDescriptor::new(phys_page.as_ptr(), &attr); - } - - Ok(()) - } - - fn next_mmio_virt_page_slice( - &mut self, - num_pages: usize, - ) -> Result, &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); - - if num_pages == 0 { - return Err("num_pages == 0"); - } - - if (self.cur_l3_mmio_index + num_pages) > 8191 { - return Err("Not enough MMIO space left"); - } - - let addr = (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) - | (self.cur_l3_mmio_index << Granule64KiB::SHIFT); - self.cur_l3_mmio_index += num_pages; - - Ok(PageSliceDescriptor::from_addr( - Address::new(addr), - num_pages, - )) - } - - fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { - let start_addr = virt_pages.start_addr(); - let end_addr_inclusive = virt_pages.end_addr_inclusive(); - - for i in [start_addr, end_addr_inclusive].iter() { - if (*i >= self.mmio_start_addr()) && (*i <= self.mmio_end_addr_inclusive()) { - return true; - } - } - - false - } -} - -impl mmu::interface::MMU for MemoryManagementUnit { +impl memory::mmu::interface::MMU for MemoryManagementUnit { unsafe fn enable( &self, - phys_kernel_table_base_addr: Address, + kernel_table_phys_base_addr: Address, ) -> Result<(), &'static str> { // Fail early if translation granule is not supported. Both RPis support it, though. if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { @@ -501,12 +133,12 @@ impl mmu::interface::MMU for MemoryManagementUnit { } // Prepare the memory attribute indirection register. - set_up_mair(); + self.set_up_mair(); // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(phys_kernel_table_base_addr.into_usize() as u64); + TTBR0_EL1.set_baddr(kernel_table_phys_base_addr.into_usize() as u64); - configure_translation_control(); + self.configure_translation_control(); // Switch the MMU on. // @@ -527,32 +159,11 @@ impl mmu::interface::MMU for MemoryManagementUnit { // Testing //-------------------------------------------------------------------------------------------------- -#[cfg(test)] -pub(in crate::memory::mmu) type MinSizeArchTranslationTable = FixedSizeTranslationTable<1>; - #[cfg(test)] mod tests { use super::*; use test_macros::kernel_test; - /// Check if the size of `struct TableDescriptor` is as expected. - #[kernel_test] - fn size_of_tabledescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - - /// Check if the size of `struct PageDescriptor` is as expected. - #[kernel_test] - fn size_of_pagedescriptor_equals_64_bit() { - assert_eq!( - core::mem::size_of::(), - core::mem::size_of::() - ); - } - /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..f682d6a49 --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,462 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::{ + mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, + }, + Address, AddressType, Physical, Virtual, + }, +}; +use core::convert; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait BaseAddr { + fn phys_base_addr(&self) -> Address; +} + +const NUM_LVL2_TABLES: usize = + bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, hence the "reverse" order of appearance. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], + + /// Index of the next free MMIO page. + cur_l3_mmio_index: usize, + + /// Have the tables been initialized? + initialized: bool, +} + +/// A translation table type for the kernel space. +pub type KernelTranslationTable = FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl BaseAddr for [T; N] { + fn phys_base_addr(&self) -> Address { + // The binary is still identity mapped, so we don't need to convert here. + Address::new(self as *const _ as usize) + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::VALID::True + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr( + output_addr: *const Page, + attribute_fields: &AttributeFields, + ) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::VALID::True + + STAGE1_PAGE_DESCRIPTOR::AF::True + + attribute_fields.clone().into() + + STAGE1_PAGE_DESCRIPTOR::TYPE::Table + + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + ); + + Self { value: val.get() } + } + + /// Returns the valid bit. + fn is_valid(&self) -> bool { + InMemoryRegister::::new(self.value) + .is_set(STAGE1_PAGE_DESCRIPTOR::VALID) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl FixedSizeTranslationTable { + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { + assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); + assert!(NUM_TABLES > 0); + assert!((bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + cur_l3_mmio_index: 0, + initialized: false, + } + } + + /// The start address of the table's MMIO range. + #[inline(always)] + const fn mmio_start_addr(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), + ) + } + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] + const fn mmio_end_addr_inclusive(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) + | (Granule64KiB::SIZE - 1), + ) + } + + /// Helper to calculate the lvl2 and lvl3 indices from an address. + #[inline(always)] + fn lvl2_lvl3_index_from( + &self, + addr: *const Page, + ) -> Result<(usize, usize), &'static str> { + let lvl2_index = addr as usize >> Granule512MiB::SHIFT; + let lvl3_index = (addr as usize & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + + if lvl2_index > (NUM_TABLES - 1) { + return Err("Virtual page is out of bounds of translation table"); + } + + Ok((lvl2_index, lvl3_index)) + } + + /// Returns the PageDescriptor corresponding to the supplied Page. + #[inline(always)] + fn page_descriptor_from( + &mut self, + addr: *const Page, + ) -> Result<&mut PageDescriptor, &'static str> { + let (lvl2_index, lvl3_index) = self.lvl2_lvl3_index_from(addr)?; + + Ok(&mut self.lvl3[lvl2_index][lvl3_index]) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl memory::mmu::translation_table::interface::TranslationTable + for FixedSizeTranslationTable +{ + unsafe fn init(&mut self) { + if self.initialized { + return; + } + + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { + let desc = TableDescriptor::from_next_lvl_table_addr( + self.lvl3[lvl2_nr].phys_base_addr().into_usize(), + ); + *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; + } + + fn phys_base_address(&self) -> Address { + self.lvl2.phys_base_addr() + } + + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + let p = phys_pages.as_slice(); + let v = virt_pages.as_slice(); + + if p.len() != v.len() { + return Err("Tried to map page slices with unequal sizes"); + } + + // No work to do for empty slices. + if p.is_empty() { + return Ok(()); + } + + if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { + return Err("Tried to map outside of physical address space"); + } + + let iter = p.iter().zip(v.iter()); + for (phys_page, virt_page) in iter { + let page_descriptor = self.page_descriptor_from(virt_page.as_ptr())?; + if page_descriptor.is_valid() { + return Err("Virtual page is already mapped"); + } + + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + } + + Ok(()) + } + + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + if num_pages == 0 { + return Err("num_pages == 0"); + } + + if (self.cur_l3_mmio_index + num_pages) > 8191 { + return Err("Not enough MMIO space left"); + } + + let addr = (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (self.cur_l3_mmio_index << Granule64KiB::SHIFT); + self.cur_l3_mmio_index += num_pages; + + Ok(PageSliceDescriptor::from_addr( + Address::new(addr), + num_pages, + )) + } + + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { + let start_addr = virt_pages.start_addr(); + let end_addr_inclusive = virt_pages.end_addr_inclusive(); + + for i in [start_addr, end_addr_inclusive].iter() { + if (*i >= self.mmio_start_addr()) && (*i <= self.mmio_end_addr_inclusive()) { + return true; + } + } + + false + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +pub type MinSizeKernelTranslationTable = FixedSizeTranslationTable<1>; + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct TableDescriptor` is as expected. + #[kernel_test] + fn size_of_tabledescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } + + /// Check if the size of `struct PageDescriptor` is as expected. + #[kernel_test] + fn size_of_pagedescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } +} diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs index a2c57eb09..3a766009d 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp.rs b/15_virtual_mem_part2_mmio_remap/src/bsp.rs index 257502491..c558922f7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs index e7a37697e..4c0e11249 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,7 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, memory, memory::mmu::Physical, synchronization, + bsp, cpu, driver, exception, memory, memory::Physical, synchronization, synchronization::InitStateLock, }; use core::sync::atomic::{AtomicBool, Ordering}; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index c1fa1d14c..45dbe75c6 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -5,7 +5,7 @@ //! GPIO Driver. use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::mmu::Physical, + bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::Physical, synchronization, synchronization::IRQSafeNullLock, }; use core::sync::atomic::{AtomicUsize, Ordering}; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index dfc5b66ed..69c001adf 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -6,7 +6,7 @@ mod peripheral_ic; -use crate::{driver, exception, memory, memory::mmu::Physical}; +use crate::{driver, exception, memory, memory::Physical}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 8fe915b34..654ac3c70 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -8,7 +8,7 @@ use super::{InterruptController, PendingIRQs, PeripheralIRQ}; use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, exception, memory, - memory::mmu::Physical, + memory::Physical, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e404f11bf..d7d83840e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,7 +6,7 @@ use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, - memory::mmu::Physical, synchronization, synchronization::IRQSafeNullLock, + memory::Physical, synchronization, synchronization::IRQSafeNullLock, }; use core::{ fmt, diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs index 9d359cb7c..7b48d7b51 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -37,7 +37,7 @@ pub mod mmu; -use crate::memory::mmu::{Address, Physical, Virtual}; +use crate::memory::{Address, Physical, Virtual}; use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs index fc16c87c2..76b93e47a 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -9,9 +9,10 @@ use crate::{ memory::{ mmu as kernel_mmu, mmu::{ - interface, AccessPermissions, AttributeFields, Granule64KiB, MemAttributes, Page, - PageSliceDescriptor, Physical, Virtual, + AccessPermissions, AddressSpaceSize, AttributeFields, MemAttributes, Page, + PageSliceDescriptor, TranslationGranule, }, + Physical, Virtual, }, }; @@ -21,12 +22,14 @@ use crate::{ /// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to /// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. -pub type KernelGranule = Granule64KiB; +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; + +/// The address space size chosen by this BSP. +pub type KernelVirtAddrSpaceSize = AddressSpaceSize<{ 8 * 1024 * 1024 * 1024 }>; //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -use interface::TranslationGranule; /// Helper function for calculating the number of pages the given parameter spans. const fn size_to_num_pages(size: usize) -> usize { @@ -94,8 +97,8 @@ pub fn phys_addr_space_end_page() -> *const Page { pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { kernel_mmu::kernel_map_pages_at( "Kernel boot-core stack", - &phys_stack_page_desc(), &virt_stack_page_desc(), + &phys_stack_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, acc_perms: AccessPermissions::ReadWrite, @@ -105,8 +108,8 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { kernel_mmu::kernel_map_pages_at( "Kernel code and RO data", - &phys_ro_page_desc(), &virt_ro_page_desc(), + &phys_ro_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, acc_perms: AccessPermissions::ReadOnly, @@ -116,8 +119,8 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { kernel_mmu::kernel_map_pages_at( "Kernel data and bss", - &phys_data_page_desc(), &virt_data_page_desc(), + &phys_data_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, acc_perms: AccessPermissions::ReadWrite, diff --git a/15_virtual_mem_part2_mmio_remap/src/console.rs b/15_virtual_mem_part2_mmio_remap/src/console.rs index 3552823cc..c3154ba23 100644 --- a/15_virtual_mem_part2_mmio_remap/src/console.rs +++ b/15_virtual_mem_part2_mmio_remap/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/cpu.rs index c9e5af72c..9c8eb6f6e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs b/15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs b/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs +++ b/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/15_virtual_mem_part2_mmio_remap/src/exception.rs b/15_virtual_mem_part2_mmio_remap/src/exception.rs index dfa852a87..3c5e7bc83 100644 --- a/15_virtual_mem_part2_mmio_remap/src/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/exception.rs @@ -7,10 +7,14 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/exception.rs"] mod arch_exception; -pub use arch_exception::*; pub mod asynchronous; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- diff --git a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs index c16ce0077..2b9ef05f3 100644 --- a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs @@ -6,11 +6,18 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] -mod arch_exception_async; -pub use arch_exception_async::*; +mod arch_asynchronous; use core::{fmt, marker::PhantomData}; +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::{ + is_local_irq_masked, local_irq_mask, local_irq_mask_save, local_irq_restore, local_irq_unmask, + print_state, +}; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -116,7 +123,7 @@ impl IRQNumber<{ MAX_INCLUSIVE }> { } /// Return the wrapped number. - pub fn get(self) -> usize { + pub const fn get(self) -> usize { self.0 } } diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index e66dd3e86..89df0dd5c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -7,24 +7,7 @@ //! The `kernel` library. //! -//! Used by `main.rs` to compose the final kernel binary. -//! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`bsp::exception::asynchronous::irq_manager()`] - Returns a reference to the kernel's [IRQ -//! Handling interface]. -//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. -//! - [`state::state_manager()`] - Returns a reference to the kernel's [state management] instance. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [IRQ Handling interface]: ../libkernel/exception/asynchronous/interface/trait.IRQManager.html -//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html -//! [state management]: ../libkernel/state/struct.StateManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html +//! Used to compose the final kernel binary. //! //! # Code organization and architecture //! @@ -39,15 +22,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -56,9 +46,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -110,6 +99,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![allow(incomplete_features)] #![feature(asm)] @@ -130,9 +127,6 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(crate::test_runner)] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()` (defined in `main.rs`). - mod panic_wait; mod runtime_init; mod synchronization; diff --git a/15_virtual_mem_part2_mmio_remap/src/memory.rs b/15_virtual_mem_part2_mmio_remap/src/memory.rs index 1ef0285a6..1493b1a94 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory.rs @@ -6,12 +6,85 @@ pub mod mmu; -use core::ops::RangeInclusive; +use crate::common; +use core::{marker::PhantomData, ops::RangeInclusive}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Metadata trait for marking the type of an address. +pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} + +/// Zero-sized type to mark a physical address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Physical {} + +/// Zero-sized type to mark a virtual address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Virtual {} + +/// Generic address type. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub struct Address { + value: usize, + _address_type: PhantomData ATYPE>, +} //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +impl AddressType for Physical {} +impl AddressType for Virtual {} + +impl Address { + /// Create an instance. + pub const fn new(value: usize) -> Self { + Self { + value, + _address_type: PhantomData, + } + } + + /// Align down. + pub const fn align_down(self, alignment: usize) -> Self { + let aligned = common::align_down(self.value, alignment); + + Self { + value: aligned, + _address_type: PhantomData, + } + } + + /// Converts `Address` into an usize. + pub const fn into_usize(self) -> usize { + self.value + } +} + +impl core::ops::Add for Address { + type Output = Self; + + fn add(self, other: usize) -> Self { + Self { + value: self.value + other, + _address_type: PhantomData, + } + } +} + +impl core::ops::Sub for Address { + type Output = Self; + + fn sub(self, other: usize) -> Self { + Self { + value: self.value - other, + _address_type: PhantomData, + } + } +} + /// Zero out an inclusive memory range. /// /// # Safety diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index 9204874a9..b02055937 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -7,12 +7,16 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/memory/mmu.rs"] mod arch_mmu; -pub use arch_mmu::*; mod mapping_record; +mod translation_table; mod types; -use crate::{bsp, synchronization, warn}; +use crate::{ + bsp, + memory::{Address, Physical, Virtual}, + synchronization, warn, +}; pub use types::*; @@ -24,59 +28,6 @@ pub use types::*; pub mod interface { use super::*; - /// Describes the characteristics of a translation granule. - #[allow(missing_docs)] - pub trait TranslationGranule { - const SIZE: usize; - const MASK: usize = Self::SIZE - 1; - const SHIFT: usize; - } - - /// Translation table operations. - pub trait TranslationTable { - /// Anything that needs to run before any of the other provided functions can be used. - /// - /// # Safety - /// - /// - Implementor must ensure that this function can run only once or is harmless if invoked - /// multiple times. - unsafe fn init(&mut self); - - /// The translation table's base address to be used for programming the MMU. - fn phys_base_address(&self) -> Address; - - /// Map the given physical pages to the given virtual pages. - /// - /// # Safety - /// - /// - Using wrong attributes can cause multiple issues of different nature in the system. - /// - It is not required that the architectural implementation prevents aliasing. That is, - /// mapping to the same physical memory using multiple virtual addresses, which would - /// break Rust's ownership assumptions. This should be protected against in this module - /// (the kernel's generic MMU code). - unsafe fn map_pages_at( - &mut self, - phys_pages: &PageSliceDescriptor, - virt_pages: &PageSliceDescriptor, - attr: &AttributeFields, - ) -> Result<(), &'static str>; - - /// Obtain a free virtual page slice in the MMIO region. - /// - /// The "MMIO region" is a distinct region of the implementor's choice, which allows - /// differentiating MMIO addresses from others. This can speed up debugging efforts. - /// Ideally, those MMIO addresses are also standing out visually so that a human eye can - /// identify them. For example, by allocating them from near the end of the virtual address - /// space. - fn next_mmio_virt_page_slice( - &mut self, - num_pages: usize, - ) -> Result, &'static str>; - - /// Check if a virtual page splice is in the "MMIO region". - fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; - } - /// MMU functions. pub trait MMU { /// Turns on the MMU. @@ -87,16 +38,23 @@ pub mod interface { /// - Changes the HW's global state. unsafe fn enable( &self, - phys_kernel_table_base_addr: Address, + kernel_table_phys_base_addr: Address, ) -> Result<(), &'static str>; } } +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes the size of an address space. +pub struct AddressSpaceSize; + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -use interface::{TranslationTable, MMU}; +use interface::MMU; use synchronization::interface::ReadWriteEx; +use translation_table::interface::TranslationTable; /// Map pages in the kernel's translation tables. /// @@ -108,14 +66,14 @@ use synchronization::interface::ReadWriteEx; /// - Does not prevent aliasing. unsafe fn kernel_map_pages_at_unchecked( name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { arch_mmu::kernel_translation_tables() - .write(|tables| tables.map_pages_at(phys_pages, virt_pages, attr))?; + .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; - if let Err(x) = mapping_record::kernel_add(name, phys_pages, virt_pages, attr) { + if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { warn!("{}", x); } @@ -125,7 +83,44 @@ unsafe fn kernel_map_pages_at_unchecked( //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -use interface::TranslationGranule; + +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's mask. + pub const MASK: usize = Self::SIZE - 1; + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpaceSize { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); + assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + + // Must adhere to architectural restrictions. + assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); + assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); + assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + + AS_SIZE + } +} /// Raw mapping of virtual to physical pages in the kernel translation tables. /// @@ -134,11 +129,11 @@ use interface::TranslationGranule; /// # Safety /// /// - See `kernel_map_pages_at_unchecked()`. -/// - Does not prevent aliasing. Currently, we have to trust the callers. +/// - Does not prevent aliasing. Currently, the callers must be trusted. pub unsafe fn kernel_map_pages_at( name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { let is_mmio = arch_mmu::kernel_translation_tables() @@ -147,7 +142,7 @@ pub unsafe fn kernel_map_pages_at( return Err("Attempt to manually map into MMIO region"); } - kernel_map_pages_at_unchecked(name, phys_pages, virt_pages, attr)?; + kernel_map_pages_at_unchecked(name, virt_pages, phys_pages, attr)?; Ok(()) } @@ -179,8 +174,8 @@ pub unsafe fn kernel_map_mmio( kernel_map_pages_at_unchecked( name, - &phys_pages, &virt_pages, + &phys_pages, &AttributeFields { mem_attributes: MemAttributes::Device, acc_perms: AccessPermissions::ReadWrite, @@ -213,38 +208,3 @@ pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> { pub fn kernel_print_mappings() { mapping_record::kernel_print() } - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Sanity checks for the kernel TranslationTable implementation. - #[kernel_test] - fn translationtable_implementation_sanity() { - // Need to take care that `tables` fits into the stack. - let mut tables = MinSizeArchTranslationTable::new(); - - unsafe { tables.init() }; - - let x = tables.next_mmio_virt_page_slice(0); - assert!(x.is_err()); - - let x = tables.next_mmio_virt_page_slice(1_0000_0000); - assert!(x.is_err()); - - let x = tables.next_mmio_virt_page_slice(2).unwrap(); - assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); - - assert_eq!(tables.is_virt_page_slice_mmio(&x), true); - - assert_eq!( - tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), - false - ); - } -} diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs index cd3378089..cd46403fd 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -42,8 +42,8 @@ static KERNEL_MAPPING_RECORD: InitStateLock = impl MappingRecordEntry { pub fn new( name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Self { Self { @@ -97,13 +97,13 @@ impl MappingRecord { pub fn add( &mut self, name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { let x = self.find_next_free()?; - *x = Some(MappingRecordEntry::new(name, phys_pages, virt_pages, attr)); + *x = Some(MappingRecordEntry::new(name, virt_pages, phys_pages, attr)); Ok(()) } @@ -191,11 +191,11 @@ use synchronization::interface::ReadWriteEx; /// Add an entry to the mapping info record. pub fn kernel_add( name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, phys_pages, virt_pages, attr)) + KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, virt_pages, phys_pages, attr)) } pub fn kernel_find_and_insert_mmio_duplicate( diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..7e16e6060 --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +use crate::memory::{ + mmu::{AttributeFields, PageSliceDescriptor}, + Address, Physical, Virtual, +}; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Translation table interfaces. +pub mod interface { + use super::*; + + /// Translation table operations. + pub trait TranslationTable { + /// Anything that needs to run before any of the other provided functions can be used. + /// + /// # Safety + /// + /// - Implementor must ensure that this function can run only once or is harmless if invoked + /// multiple times. + unsafe fn init(&mut self); + + /// The translation table's base address to be used for programming the MMU. + fn phys_base_address(&self) -> Address; + + /// Map the given virtual pages to the given physical pages. + /// + /// # Safety + /// + /// - Using wrong attributes can cause multiple issues of different nature in the system. + /// - It is not required that the architectural implementation prevents aliasing. That is, + /// mapping to the same physical memory using multiple virtual addresses, which would + /// break Rust's ownership assumptions. This should be protected against in the kernel's + /// generic MMU code. + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str>; + + /// Obtain a free virtual page slice in the MMIO region. + /// + /// The "MMIO region" is a distinct region of the implementor's choice, which allows + /// differentiating MMIO addresses from others. This can speed up debugging efforts. + /// Ideally, those MMIO addresses are also standing out visually so that a human eye can + /// identify them. For example, by allocating them from near the end of the virtual address + /// space. + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str>; + + /// Check if a virtual page splice is in the "MMIO region". + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use crate::bsp; + use arch_translation_table::MinSizeKernelTranslationTable; + use interface::TranslationTable; + use test_macros::kernel_test; + + /// Sanity checks for the kernel TranslationTable implementation. + #[kernel_test] + fn translationtable_implementation_sanity() { + // Need to take care that `tables` fits into the stack. + let mut tables = MinSizeKernelTranslationTable::new(); + + unsafe { tables.init() }; + + let x = tables.next_mmio_virt_page_slice(0); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(1_0000_0000); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(2).unwrap(); + assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); + + assert_eq!(tables.is_virt_page_slice_mmio(&x), true); + + assert_eq!( + tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), + false + ); + } +} diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs index cea7bb39c..594315882 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs @@ -2,33 +2,17 @@ // // Copyright (c) 2020-2021 Andre Richter -//! Memory Management Unit Types. +//! Memory Management Unit types. -use crate::{bsp, common}; +use crate::{ + bsp, common, + memory::{Address, AddressType, Physical, Virtual}, +}; use core::{convert::From, marker::PhantomData, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -use super::interface::TranslationGranule; - -/// Metadata trait for marking the type of an address. -pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} - -/// Zero-sized type to mark a physical address. -#[derive(Copy, Clone, PartialOrd, PartialEq)] -pub enum Physical {} - -/// Zero-sized type to mark a virtual address. -#[derive(Copy, Clone, PartialOrd, PartialEq)] -pub enum Virtual {} - -/// Generic address type. -#[derive(Copy, Clone, PartialOrd, PartialEq)] -pub struct Address { - value: usize, - _address_type: PhantomData, -} /// Generic page type. #[repr(C)] @@ -80,60 +64,6 @@ pub struct MMIODescriptor { // Public Code //-------------------------------------------------------------------------------------------------- -impl AddressType for Physical {} -impl AddressType for Virtual {} - -//------------------------------------------------------------------------------ -// Address -//------------------------------------------------------------------------------ - -impl Address { - /// Create an instance. - pub const fn new(value: usize) -> Self { - Self { - value, - _address_type: PhantomData, - } - } - - /// Align down. - pub const fn align_down(self, alignment: usize) -> Self { - let aligned = common::align_down(self.value, alignment); - - Self { - value: aligned, - _address_type: PhantomData, - } - } - - /// Converts `Address` into an usize. - pub const fn into_usize(self) -> usize { - self.value - } -} - -impl core::ops::Add for Address { - type Output = Self; - - fn add(self, other: usize) -> Self { - Self { - value: self.value + other, - _address_type: PhantomData, - } - } -} - -impl core::ops::Sub for Address { - type Output = Self; - - fn sub(self, other: usize) -> Self { - Self { - value: self.value - other, - _address_type: PhantomData, - } - } -} - //------------------------------------------------------------------------------ // Page //------------------------------------------------------------------------------ diff --git a/15_virtual_mem_part2_mmio_remap/src/print.rs b/15_virtual_mem_part2_mmio_remap/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/15_virtual_mem_part2_mmio_remap/src/print.rs +++ b/15_virtual_mem_part2_mmio_remap/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/15_virtual_mem_part2_mmio_remap/src/state.rs b/15_virtual_mem_part2_mmio_remap/src/state.rs index d08e67d6e..c94d04c84 100644 --- a/15_virtual_mem_part2_mmio_remap/src/state.rs +++ b/15_virtual_mem_part2_mmio_remap/src/state.rs @@ -69,7 +69,7 @@ impl StateManager { } } - /// Return if the kernel is still in an init state. + /// Return if the kernel is init state. pub fn is_init(&self) -> bool { self.state() == State::Init } diff --git a/15_virtual_mem_part2_mmio_remap/src/time.rs b/15_virtual_mem_part2_mmio_remap/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/15_virtual_mem_part2_mmio_remap/src/time.rs +++ b/15_virtual_mem_part2_mmio_remap/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/X1_JTAG_boot/Cargo.lock b/X1_JTAG_boot/Cargo.lock index 102e57301..e87b55afe 100644 --- a/X1_JTAG_boot/Cargo.lock +++ b/X1_JTAG_boot/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "4.1.0" +version = "5.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a389514c1229e12a03c0e8de8b357671b71e1d1bdab3a4f5b591abc94169cba8" +checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" dependencies = [ "register", ] @@ -19,15 +19,15 @@ dependencies = [ [[package]] name = "register" -version = "0.5.1" +version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "deaba5b0e477d21f61a57504bb5cef4a1e86de30300b457d38971c1cfc98b815" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.5.0" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70323afdb8082186c0986da0e10f6e4ed103d681c921c00597e98d9806dac20f" +checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" diff --git a/X1_JTAG_boot/Cargo.toml b/X1_JTAG_boot/Cargo.toml index 8a1a364ef..b8906da1f 100644 --- a/X1_JTAG_boot/Cargo.toml +++ b/X1_JTAG_boot/Cargo.toml @@ -20,9 +20,9 @@ bsp_rpi4 = ["register"] [dependencies] # Optional dependencies -register = { version = "0.5.x", optional = true } +register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "4.x.x" } +cortex-a = { version = "5.x.x" } diff --git a/X1_JTAG_boot/Makefile b/X1_JTAG_boot/Makefile index 071167b9e..27944c93d 100644 --- a/X1_JTAG_boot/Makefile +++ b/X1_JTAG_boot/Makefile @@ -40,8 +40,9 @@ export LINKER_FILE RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs +FEATURES = bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features bsp_$(BSP) \ + --features $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 4248095748295740449cb7c013902e8a4ea778fb..6fe154299791b169e5b4fbfa9e85d0b734d27d9b 100755 GIT binary patch delta 74 zcmV-Q0JZ7`g!gH?yG{?*ajevymO20?RQQbN~PV delta 74 zcmV-Q0JZn+a diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 0813b2badedf8feb242081bdb478505ffd4044ee..d5573467fe56c7dfe6f5289a4c6d98bee8e3ed6c 100755 GIT binary patch delta 21 dcmZ2rx4>@01W88o%@ZZxGck5;Zj_nN1OQ#L2p9kW delta 21 dcmZ2rx4>@01W87N%@ZZxGck5 //! Architectural processor code. - -use crate::{bsp, cpu}; -use cortex_a::{asm, regs::*}; - -//-------------------------------------------------------------------------------------------------- -// Boot Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - // Expect the boot core to start in EL2. - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..549b5927a --- /dev/null +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{bsp, cpu}; +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function where it is expected by the target machine. +/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is +/// actually set (`SP.set()`). +#[no_mangle] +pub unsafe fn _start() -> ! { + use crate::runtime_init; + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + SP.set(bsp::memory::boot_core_stack_end() as u64); + runtime_init::runtime_init() + } else { + // If not core0, infinitely wait for events. + cpu::wait_forever() + } +} diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs index c80f7e780..b9fdd0f73 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp use cortex_a::regs::*; diff --git a/X1_JTAG_boot/src/_arch/aarch64/time.rs b/X1_JTAG_boot/src/_arch/aarch64/time.rs index 7f1bc6963..3a766009d 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/time.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/time.rs @@ -3,6 +3,13 @@ // Copyright (c) 2018-2021 Andre Richter //! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time use crate::{time, warn}; use core::time::Duration; @@ -55,7 +62,7 @@ impl time::interface::TimeManager for GenericTimer { } // Calculate the register compare value. - let frq = CNTFRQ_EL0.get() as u64; + let frq = CNTFRQ_EL0.get(); let x = match frq.checked_mul(duration.as_nanos() as u64) { None => { warn!("Spin duration too long, skipping"); diff --git a/X1_JTAG_boot/src/bsp.rs b/X1_JTAG_boot/src/bsp.rs index 257502491..c558922f7 100644 --- a/X1_JTAG_boot/src/bsp.rs +++ b/X1_JTAG_boot/src/bsp.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Conditional re-exporting of Board Support Packages. +//! Conditional reexporting of Board Support Packages. mod device_driver; diff --git a/X1_JTAG_boot/src/console.rs b/X1_JTAG_boot/src/console.rs index 3552823cc..c3154ba23 100644 --- a/X1_JTAG_boot/src/console.rs +++ b/X1_JTAG_boot/src/console.rs @@ -20,8 +20,7 @@ pub mod interface { /// Write a Rust format string. fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - /// Block execution until the last buffered character has been physically put on the TX - /// wire. + /// Block until the last buffered character has been physically put on the TX wire. fn flush(&self); } diff --git a/X1_JTAG_boot/src/cpu.rs b/X1_JTAG_boot/src/cpu.rs index c9e5af72c..3834f183c 100644 --- a/X1_JTAG_boot/src/cpu.rs +++ b/X1_JTAG_boot/src/cpu.rs @@ -7,6 +7,12 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/cpu.rs"] mod arch_cpu; -pub use arch_cpu::*; + +mod boot; pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; diff --git a/X1_JTAG_boot/src/cpu/boot.rs b/X1_JTAG_boot/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/X1_JTAG_boot/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/X1_JTAG_boot/src/cpu/smp.rs b/X1_JTAG_boot/src/cpu/smp.rs index 90ecbdf30..38230ce1e 100644 --- a/X1_JTAG_boot/src/cpu/smp.rs +++ b/X1_JTAG_boot/src/cpu/smp.rs @@ -6,5 +6,9 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_cpu_smp; -pub use arch_cpu_smp::*; +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index 340acd4eb..fa5ba64ad 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -7,16 +7,6 @@ //! The `kernel` binary. //! -//! # TL;DR - Overview of important Kernel entities -//! -//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. -//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. -//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. -//! -//! [console interface]: ../libkernel/console/interface/index.html -//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html -//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html -//! //! # Code organization and architecture //! //! The code is divided into different *modules*, each representing a typical **subsystem** of the @@ -30,15 +20,22 @@ //! `src/_arch`, for example, `src/_arch/aarch64`. //! //! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go -//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in -//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's -//! module organization. That means a public function `foo()` defined in -//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. //! -//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. -//! Rather, it's contents are conditionally pulled into respective files using the `#[path = -//! "_arch/xxx/yyy.rs"]` attribute. +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. //! //! ## BSP code //! @@ -47,9 +44,8 @@ //! or instances of drivers for devices that are featured on the respective board. //! //! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means -//! whatever is provided must be called starting from the `bsp` namespace, e.g. -//! `bsp::driver::driver_manager()`. +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. //! //! ## Kernel interfaces //! @@ -101,6 +97,14 @@ //! //! - `crate::memory::*` //! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] @@ -109,9 +113,6 @@ #![no_main] #![no_std] -// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls -// `runtime_init()`, which jumps to `kernel_init()`. - mod bsp; mod console; mod cpu; diff --git a/X1_JTAG_boot/src/print.rs b/X1_JTAG_boot/src/print.rs index 1ea96b6ad..5a5638113 100644 --- a/X1_JTAG_boot/src/print.rs +++ b/X1_JTAG_boot/src/print.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Printing facilities. +//! Printing. use crate::{bsp, console}; use core::fmt; diff --git a/X1_JTAG_boot/src/time.rs b/X1_JTAG_boot/src/time.rs index 4f2f4e38f..953b6f0df 100644 --- a/X1_JTAG_boot/src/time.rs +++ b/X1_JTAG_boot/src/time.rs @@ -7,7 +7,11 @@ #[cfg(target_arch = "aarch64")] #[path = "_arch/aarch64/time.rs"] mod arch_time; -pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -18,8 +22,6 @@ pub mod interface { use core::time::Duration; /// Time management functions. - /// - /// The `BSP` is supposed to supply one global instance. pub trait TimeManager { /// The timer's resolution. fn resolution(&self) -> Duration; diff --git a/rust-toolchain b/rust-toolchain index b50e4d140..26668a71b 100644 --- a/rust-toolchain +++ b/rust-toolchain @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2021-01-01" +channel = "nightly-2021-01-08" components = ["llvm-tools-preview"] targets = ["aarch64-unknown-none-softfloat"] From c8e9b9713fe016409aba671b908e9d8f5b7a1646 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 23 Jan 2021 22:53:26 +0100 Subject: [PATCH 024/214] update deps --- 04_zero_overhead_abstraction/Cargo.lock | 4 ++-- 05_safe_globals/Cargo.lock | 4 ++-- 06_drivers_gpio_uart/Cargo.lock | 4 ++-- 07_uart_chainloader/Cargo.lock | 4 ++-- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6856 -> 6856 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6728 -> 6728 bytes 08_timestamps/Cargo.lock | 4 ++-- 09_hw_debug_JTAG/Cargo.lock | 4 ++-- 10_privilege_level/Cargo.lock | 4 ++-- .../Cargo.lock | 4 ++-- 12_exceptions_part1_groundwork/Cargo.lock | 4 ++-- 13_integrated_testing/Cargo.lock | 8 ++++---- .../Cargo.lock | 8 ++++---- 15_virtual_mem_part2_mmio_remap/Cargo.lock | 8 ++++---- 14 files changed, 30 insertions(+), 30 deletions(-) diff --git a/04_zero_overhead_abstraction/Cargo.lock b/04_zero_overhead_abstraction/Cargo.lock index dd0cd5773..70708c19a 100644 --- a/04_zero_overhead_abstraction/Cargo.lock +++ b/04_zero_overhead_abstraction/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.1" +version = "5.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" +checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" dependencies = [ "register", ] diff --git a/05_safe_globals/Cargo.lock b/05_safe_globals/Cargo.lock index dd0cd5773..70708c19a 100644 --- a/05_safe_globals/Cargo.lock +++ b/05_safe_globals/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.1" +version = "5.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" +checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" dependencies = [ "register", ] diff --git a/06_drivers_gpio_uart/Cargo.lock b/06_drivers_gpio_uart/Cargo.lock index e2d680a76..e87b55afe 100644 --- a/06_drivers_gpio_uart/Cargo.lock +++ b/06_drivers_gpio_uart/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.1" +version = "5.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" +checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" dependencies = [ "register", ] diff --git a/07_uart_chainloader/Cargo.lock b/07_uart_chainloader/Cargo.lock index e2d680a76..e87b55afe 100644 --- a/07_uart_chainloader/Cargo.lock +++ b/07_uart_chainloader/Cargo.lock @@ -2,9 +2,9 @@ # It is not intended for manual editing. [[package]] name = "cortex-a" -version = "5.1.1" +version = "5.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c9b7504606c9e83a6b35e6f99db890084baf13fbbed67603fcc2a62b2cfe62b" +checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" dependencies = [ "register", ] diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 0719de414d1e2a8495d6c68862e25eec89998ba2..80df7c197febdb8b7a7739c51af927e30915b3c9 100755 GIT binary patch delta 335 zcmX?Mdct&q3d;fJ_-hkY4lqvL_*9*d@!MoxCUZvH&8|$#Ss7n!e#>sa%9y-aj(0hu zV4~4PZU%-4JPZsQUO6*N%yyi7jW3OH)n;4%(~OLolQ{(y7=t!z2?{Yyb`aKKl%1R_ zEG@|N|NnFa28ApC4>Ml;XUy>P;pCaZzZqF4PZfF1cxrN^s5+zT824O0hI1~;u@vZZ9bByQP@lHURC6hl`4 delta 330 zcmX?Mdct&q%0!I=jMFy0RA*%TK3R~-oY8KxC)09P#+REvvKz2625(m4UCt;NXf%QDaRD zVGafmfRG5Q!~ do)BgTD+Nl!)IpWOO)HsfDOoRxTlS*lcK~{oLEr!Y diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index c8d8f9685adcdfcaeeb113672567c4f264e52f06..54fe2b818bced6213d5f094a90562842b864b22b 100755 GIT binary patch delta 395 zcmX?Ma>8VS%0>+qCdP}Kb(xN{GCtV+mEC}qF?q8Z?{Y@LM5Bq^3=9)^7#KFZa%PyA z?Kt@!Um9cgWLN%l#=Dc(^7}9`icbE`Z_Bu0vZa6x6O-uVa)E_>{EQ4!9<(#9{Qm#{ zhwnj~)dbHmvK(NJzc%@cPz0muWM5%vb*}&arzP@!~&YhMzD0{h!Xk+!Vxd zWPi~DHij*nKwGsRicIbo-pMF2*;8acr zi16{w8JcE;97$FqHANI|kMB|v4 rJ_t8VS3d;fJ_-h+gSeO_uY}RBt&dPXy^G9|AR>tDZO1#S%1q+QPax*YY;9+3c z@XDECVs_xhMb%|ffa*Z{9x*en;$U%D`GA>YRp-C| z;tUK9Kg%ckih66RFmNz{0E9%)0Zb4P4XBL7cE;97$Fo;+vZQAaZF4q q43mQ-)Er@kN Date: Thu, 28 Jan 2021 23:25:57 +0100 Subject: [PATCH 025/214] PL011: Use BUSY bit Fixes #100 --- 06_drivers_gpio_uart/README.md | 156 ++++++++++------- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs | 1 + .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 144 +++++++++------- 06_drivers_gpio_uart/src/cpu.rs | 5 +- 07_uart_chainloader/README.md | 17 +- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6856 -> 6808 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6728 -> 6680 bytes 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 1 + .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 144 +++++++++------- 07_uart_chainloader/src/cpu.rs | 5 +- 08_timestamps/README.md | 46 ++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- 14_exceptions_part2_peripheral_IRQs/README.md | 46 ++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 159 +++++++++++------- 15_virtual_mem_part2_mmio_remap/README.md | 22 +-- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 159 +++++++++++------- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8240 -> 8144 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 7968 -> 6848 bytes .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 141 +++++++++------- 24 files changed, 1094 insertions(+), 798 deletions(-) diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 2f23d1b5b..9b526c331 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -188,13 +188,14 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 05_safe_globals/src/_arch/aarch64/cpu.rs +++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -@@ -17,6 +17,16 @@ +@@ -17,6 +17,17 @@ // Public Code //-------------------------------------------------------------------------------------------------- +pub use asm::nop; + +/// Spin for `n` cycles. ++#[cfg(feature = "bsp_rpi3")] +#[inline(always)] +pub fn spin_for_cycles(n: usize) { + for _ in 0..n { @@ -435,12 +436,17 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_g diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -0,0 +1,381 @@ +@@ -0,0 +1,403 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! PL011 UART driver. ++//! ++//! # Resources ++//! ++//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf ++//! - https://developer.arm.com/documentation/ddi0183/latest + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -455,50 +461,63 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + +// PL011 UART registers. +// -+// Descriptions taken from -+// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf ++// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. +register_bitfields! { + u32, + -+ /// Flag Register ++ /// Flag Register. + FR [ + /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the -+ /// Line Control Register, UARTLCR_ LCRH. ++ /// Line Control Register, LCR_H. + /// -+ /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If -+ /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does -+ /// not indicate if there is data in the transmit shift register. ++ /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. ++ /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. ++ /// - This bit does not indicate if there is data in the transmit shift register. + TXFE OFFSET(7) NUMBITS(1) [], + + /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the -+ /// UARTLCR_ LCRH Register. ++ /// LCR_H Register. + /// -+ /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If -+ /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. ++ /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. ++ /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + TXFF OFFSET(5) NUMBITS(1) [], + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the -+ /// UARTLCR_H Register. ++ /// LCR_H Register. + /// + /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If + /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. -+ RXFE OFFSET(4) NUMBITS(1) [] ++ ++ /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the ++ /// LCR_H Register. ++ /// ++ /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. ++ /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. ++ RXFE OFFSET(4) NUMBITS(1) [], ++ ++ /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains ++ /// set until the complete byte, including all the stop bits, has been sent from the shift ++ /// register. ++ /// ++ /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether ++ /// the UART is enabled or not. ++ BUSY OFFSET(3) NUMBITS(1) [] + ], + -+ /// Integer Baud rate divisor ++ /// Integer Baud Rate Divisor. + IBRD [ -+ /// Integer Baud rate divisor -+ IBRD OFFSET(0) NUMBITS(16) [] ++ /// The integer baud rate divisor. ++ BAUD_DIVINT OFFSET(0) NUMBITS(16) [] + ], + -+ /// Fractional Baud rate divisor ++ /// Fractional Baud Rate Divisor. + FBRD [ -+ /// Fractional Baud rate divisor -+ FBRD OFFSET(0) NUMBITS(6) [] ++ /// The fractional baud rate divisor. ++ BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] + ], + -+ /// Line Control register -+ LCRH [ ++ /// Line Control Register. ++ LCR_H [ + /// Word length. These bits indicate the number of data bits transmitted or received in a + /// frame. + WLEN OFFSET(5) NUMBITS(2) [ @@ -511,34 +530,42 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + /// Enable FIFOs: + /// + /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding -+ /// registers ++ /// registers. + /// -+ /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). ++ /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). + FEN OFFSET(4) NUMBITS(1) [ + FifosDisabled = 0, + FifosEnabled = 1 + ] + ], + -+ /// Control Register ++ /// Control Register. + CR [ + /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. -+ /// Data reception occurs for UART signals. When the UART is disabled in the middle of -+ /// reception, it completes the current character before stopping. -+ RXE OFFSET(9) NUMBITS(1) [ ++ /// Data reception occurs for either UART signals or SIR signals depending on the setting of ++ /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the ++ /// current character before stopping. ++ RXE OFFSET(9) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. -+ /// Data transmission occurs for UART signals. When the UART is disabled in the middle of -+ /// transmission, it completes the current character before stopping. -+ TXE OFFSET(8) NUMBITS(1) [ ++ /// Data transmission occurs for either UART signals, or SIR signals depending on the ++ /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it ++ /// completes the current character before stopping. ++ TXE OFFSET(8) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + -+ /// UART enable ++ /// UART enable: ++ /// ++ /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or ++ /// reception, it completes the current character before stopping. ++ /// ++ /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals ++ /// or SIR signals depending on the setting of the SIREN bit + UARTEN OFFSET(0) NUMBITS(1) [ + /// If the UART is disabled in the middle of transmission or reception, it completes the + /// current character before stopping. @@ -547,9 +574,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + ] + ], + -+ /// Interrupt Clear Register ++ /// Interrupt Clear Register. + ICR [ -+ /// Meta field for all pending interrupts ++ /// Meta field for all pending interrupts. + ALL OFFSET(0) NUMBITS(11) [] + ] +} @@ -563,7 +590,7 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + (0x1c => _reserved2), + (0x24 => IBRD: WriteOnly), + (0x28 => FBRD: WriteOnly), -+ (0x2c => LCRH: WriteOnly), ++ (0x2c => LCR_H: WriteOnly), + (0x30 => CR: WriteOnly), + (0x34 => _reserved3), + (0x44 => ICR: WriteOnly), @@ -621,11 +648,18 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + /// This results in 8N1 and 921_600 baud. + /// + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): -+ /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). ++ /// `(48_000_000 / 16) / 921_600 = 3.2552083`. ++ /// ++ /// This means the integer part is `3` and goes into the `IBRD`. ++ /// The fractional part is `0.2552083`. + /// -+ /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will -+ /// give the best approximation we can get. A 5modulo error margin is acceptable for UART and we're -+ /// now at 0.02modulo. ++ /// `FBRD` calculation according to the PL011 Technical Reference Manual: ++ /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. ++ /// ++ /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a ++ /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. ++ /// ++ /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16modulo`. + pub fn init(&mut self) { + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -643,14 +677,18 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + // Clear all pending interrupts. + self.registers.ICR.write(ICR::ALL::CLEAR); + -+ // Set the baud rate. -+ self.registers.IBRD.write(IBRD::IBRD.val(3)); -+ self.registers.FBRD.write(FBRD::FBRD.val(16)); -+ -+ // Set 8N1 + FIFO on. ++ // From the PL011 Technical Reference Manual: ++ // ++ // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is ++ // updated on a single write strobe generated by a LCR_H write. So, to internally update the ++ // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. ++ // ++ // Set the baud rate, 8N1 and FIFO enabled. ++ self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); ++ self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); + self.registers -+ .LCRH -+ .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); ++ .LCR_H ++ .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); + + // Turn the UART on. + self.registers @@ -673,25 +711,10 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { -+ // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per -+ // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = -+ // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. -+ // -+ // Now make an educated guess for a good delay value. According to Wikipedia, the fastest -+ // RPi4 clocks around 1.5 GHz. -+ // -+ // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 -+ // ns would the CPU be clocked at 2 GHz. -+ const CHAR_TIME_SAFE: usize = 24_000; -+ -+ // Spin until TX FIFO empty is set. -+ while !self.registers.FR.matches_all(FR::TXFE::SET) { ++ // Spin until the busy bit is cleared. ++ while self.registers.FR.matches_all(FR::BUSY::SET) { + cpu::nop(); + } -+ -+ // After the last character has been queued for transmission, wait for the time of one -+ // character + some extra time for safety. -+ cpu::spin_for_cycles(CHAR_TIME_SAFE); + } + + /// Retrieve a character. @@ -1217,12 +1240,15 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs diff -uNr 05_safe_globals/src/cpu.rs 06_drivers_gpio_uart/src/cpu.rs --- 05_safe_globals/src/cpu.rs +++ 06_drivers_gpio_uart/src/cpu.rs -@@ -15,4 +15,4 @@ +@@ -15,4 +15,7 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::wait_forever; -+pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; ++pub use arch_cpu::{nop, wait_forever}; ++ ++#[cfg(feature = "bsp_rpi3")] ++pub use arch_cpu::spin_for_cycles; diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs --- 05_safe_globals/src/driver.rs diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs index e16fdecc8..ade53a3ca 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -20,6 +20,7 @@ use cortex_a::asm; pub use asm::nop; /// Spin for `n` cycles. +#[cfg(feature = "bsp_rpi3")] #[inline(always)] pub fn spin_for_cycles(n: usize) { for _ in 0..n { diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 39834723b..ce39752d1 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,25 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - // - // Now make an educated guess for a good delay value. According to Wikipedia, the fastest - // RPi4 clocks around 1.5 GHz. - // - // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 - // ns would the CPU be clocked at 2 GHz. - const CHAR_TIME_SAFE: usize = 24_000; - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - cpu::spin_for_cycles(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/06_drivers_gpio_uart/src/cpu.rs b/06_drivers_gpio_uart/src/cpu.rs index 103f00dba..f0df4791f 100644 --- a/06_drivers_gpio_uart/src/cpu.rs +++ b/06_drivers_gpio_uart/src/cpu.rs @@ -15,4 +15,7 @@ pub mod smp; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "bsp_rpi3")] +pub use arch_cpu::spin_for_cycles; diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 0bd4bce20..1cca2b122 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -204,7 +204,7 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs 07_uart_chainloader diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs --- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs -@@ -34,3 +34,19 @@ +@@ -35,3 +35,19 @@ asm::wfe() } } @@ -241,7 +241,7 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_uart diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -257,7 +257,7 @@ +@@ -279,7 +279,7 @@ } /// Retrieve a character. @@ -250,7 +250,7 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 // If RX FIFO is empty, if self.registers.FR.matches_all(FR::RXFE::SET) { // immediately return in non-blocking mode. -@@ -272,12 +272,7 @@ +@@ -294,12 +294,7 @@ } // Read one character. @@ -264,7 +264,7 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 // Update statistics. self.chars_read += 1; -@@ -357,14 +352,14 @@ +@@ -379,14 +374,14 @@ impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { self.inner @@ -393,12 +393,15 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader diff -uNr 06_drivers_gpio_uart/src/cpu.rs 07_uart_chainloader/src/cpu.rs --- 06_drivers_gpio_uart/src/cpu.rs +++ 07_uart_chainloader/src/cpu.rs -@@ -15,4 +15,4 @@ +@@ -15,7 +15,7 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- --pub use arch_cpu::{nop, spin_for_cycles, wait_forever}; -+pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; +-pub use arch_cpu::{nop, wait_forever}; ++pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; + + #[cfg(feature = "bsp_rpi3")] + pub use arch_cpu::spin_for_cycles; diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 80df7c197febdb8b7a7739c51af927e30915b3c9..83fe7bffab6b25be79d8cd89f4803003add22f0b 100755 GIT binary patch delta 1161 zcmah{T}YEr7(VCw_D$#C{{PImO`UFotX3*U=BJ|NBBBv!(2q+BONy2^g@lU&5ruX- zl64ViMKQ06)rDON5hz}W{dntCg1G{V0-03nJ=+ngn-093?|Gm1=lS0A&eYtiiIm_s z#5RY?^R+5$Oe)YQM(A056%psSnXHJ2JfJS29M{L%1q-%!S;(|_dNCKtDV0^88bX#@ zyF>iIzGu_+IvNqr3PBg$5dGL0YgQb=3{k082$|`e^hyY_%NBdAtjEfri+Cn1vDKVS zQMh4_E#)_3TplYce2Ps@AIV23QUFqtEr|5uePpDmgf0~Y@k83AmWAmOdQ)A71^PtY z;w2`M1flU9Wt53`cG*ZAvJzE_=s^??B&(#T3jChs5~|m*CSC-nM-L>udkMvl4P+-a;Jy81FJ2Cnl+A;pN)>q`T6Gr)gVy=1ZUgd;6!>V&R8mq>p2C=kX?@@>74)NFS4o3mb6vdkeI!x%H?G95BE4qUWkqcq&R z&TDOTIL>(k=i4}j70#T0<9Bj?tE-!1C6BM>n5z)cx|RAZ=d4$YvpU8(ALe+5&RXoM zurbR<=vT`DRgXH0+_cGBhcUfo?eJ~+zHt3o-jCffSe*OS+z)Mq;!C)bhKf6W1HAGp AJOBUy delta 1209 zcmb7DO>7fK6n?Yob?gxEf9y3F?*?LLjS8}9NU7S!Nt9B;0kKL1NC>iusMfH6+gqRXwEMm9 z{mq*<=Q^)7Y)Z6oiN_e%+~OH=8B zeB!?27Wcr1MXzJmuX>yn2QjmReqFT9c)oM0s&TFCio4@W!7>t5u*I047_owUs`g zqdo7?vn0h9H(6wsuE+A=P@ z8Z5M&&(W$JovlTnZm?HDtbDZ_Z1N*a{nwIQ zdWPjVKOQPzwSdehx1+mt*e_9=bNqm`hWto*jE&=ekxXLE$98AvP0>kYRQyHm0sN;Yj}>c%`nqWIe5mq0{QW z*$UI+asAhL!-`AjN=0BCvnxNbqw((X9}^9)ps=Qs4ZL3AjNqe!|6E{PaaaE9KP>q5 zsz(I&3jgB*i!NkHCs{J_q56xcdt2~x0~75_QBZ4=MH-t)+JfZ4ViX^&kX=O-`w+z^ z5!u8FRV-zHP9Q#&q!pS;K_aUU5rh;`3_eIuL&b+|w+}8+A>N1+O~x}bf5gjy<;-_3 z|95Wt*}&TawNCcYd)W?2`y>Ff3}87oukprh;7u9$T>U}GV1=^IZ?Nf!Re5qF45qE5 z&6PJHHPmg$U%TIHPIn1w@-tGUfIIRy%hksMXPApHI3&HO;pf2*rHZSv^>;!Kuu!GI z`xf;2?Y3Q6n$hZ;9b=3Y>iNUhn4UYvI{<2jsF1Cbz|XN^esMH|n@1|_JdQ@p+-inj zf$0hPuxXc}d&4;?t?wTP``0sJ3n&QyuS6b=jV5@j7_eH@Lpxc=Z)LD8+9j3mz+!Z$ ze?8CFsGn95ya=5LH^wOATgi&k_e)YQ50eg8I`Kj@FU@OsJ(|hi>g0XItdZw~rwZQt zPBm!#Rt;G+w;Ih2zl-)9YZiG!PAs)Cm~C8;tR2VM-ibx|?s?$5lVI6bf!k-9^+q0e zg3@N2SsYQjT#`vO%U;1()w?}LwdX`G%g2eabxpRmoPgzg8MJy`TmoA#@&-NElo`Li z@64GYDE%UX?bW`J#YjvabeBm(a6>Lr$b0|>UAMwcdtv>wzAZ3HBg|ZBS5DjGK^mBX zPsENZgf0>~q~PV)J+X3u%ar%@YK{j;zi#GnGnUc!PTBoIbLFkT^u+U8;{N_1tc3SY zVI-~(ReHHXk$t2`SonK|!DFp1C|mww3*4#F zdC>a)9%O;0L=SE6(L=k=fM2WV+BoWWJ%7}TsRXAd&Jm$oH_#vAwS6Z^6J81jOsgS> zmo5?Z^35L3=+N*9E5Wc)H764;xCYG}UoUZETKa;ivkT1X|8=OVsu5pSeQYzFPri0`@Y f!1IFOP(>-OHtAOu_K2`g;Y@l;x|znG(~tiJ1h!Y{ delta 1313 zcmah|O>7%g5T3VdyX&OIUfa8CC$+QdxV4>Dl%z=vYG|BRIp7{r+=>f_+KLbo2qDCw z5eICkh##dwe!NHpRU$tEO-5+J4fw7Vi|4TOODp0ACs*=CnXT9)C!?w?|s`!dvq}WelfvndDJQKvRE_jE9 zewBUe{YEex6|c|uo+CaZMYCnCetXZ3D2%D~)y@-yOx8pDt`fC8Ksy0yd$F<2lR#e| z8=)6TH(PzGOr}|XAa7K==>k-X;=a69f}S<|q;PBZ7+AkeNG4zqEVEAnXF^1wt+dB% z;Y=Nvt}x%twgU-4-_Nw*a3AfZD>z>(4K6}O#F~H+_XIPH|KGlmp%QAzh_my-q)<}X ztRau=hv311g$!+qk;MRkndS$Axoz89+r=Q= zxd3$M9GKP>pw`R8{4fbLf?+cY33fzII7KGpI62SWkPo{R<%7r9>OCUB5h*24Uy0qCzvg(L(X!M1PAgtLgqX$)ZrAH%jtL9mcp-@FHObD};{u(YlD%UW0ucIug=TRK&Q+YLa?TzX!c+HPodl<5r(H zzclTs7GF~$NBg|6Bq`&}4ynUNmiiHyMLkNSpZx^R;(}_Rr>TJY#B!RJG3>xY`;rd& z9gbzE!r6{FA6$3rL+k9_@JNDR9=7g>2+#@@~ULkbjJD^bi|?cD#zUGCJVbU_86jKDh-$2=WMY5r!6yDC*n%q$SwmTs zrP$-%F*dJi$wONE6>r&WK(uIL*3Q9WLXN87L //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,25 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - // - // Now make an educated guess for a good delay value. According to Wikipedia, the fastest - // RPi4 clocks around 1.5 GHz. - // - // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 - // ns would the CPU be clocked at 2 GHz. - const CHAR_TIME_SAFE: usize = 24_000; - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - cpu::spin_for_cycles(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/07_uart_chainloader/src/cpu.rs b/07_uart_chainloader/src/cpu.rs index c543bd988..7de5c29c1 100644 --- a/07_uart_chainloader/src/cpu.rs +++ b/07_uart_chainloader/src/cpu.rs @@ -15,4 +15,7 @@ pub mod smp; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; +pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; + +#[cfg(feature = "bsp_rpi3")] +pub use arch_cpu::spin_for_cycles; diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 432ac091a..83e38ca53 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -5,7 +5,7 @@ - We add abstractions for timer hardware, and implement them for the ARM architectural timer in `_arch/aarch64`. - The new timer functions are used to annotate UART prints with timestamps, and to get rid of the - cycle-based delays in the `GPIO` and `UART` device drivers, which boosts accuracy. + cycle-based delays in the `GPIO` device driver, which boosts accuracy. - A `warn!()` macro is added. ## Test it @@ -135,11 +135,12 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs 08_timestamps/src/_a diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ 08_timestamps/src/_arch/aarch64/cpu.rs -@@ -19,14 +19,6 @@ +@@ -19,15 +19,6 @@ pub use asm::nop; -/// Spin for `n` cycles. +-#[cfg(feature = "bsp_rpi3")] -#[inline(always)] -pub fn spin_for_cycles(n: usize) { - for _ in 0..n { @@ -150,7 +151,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -34,19 +26,3 @@ +@@ -35,19 +26,3 @@ asm::wfe() } } @@ -320,33 +321,7 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 08_times diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -235,16 +235,13 @@ - - /// Block execution until the last buffered character has been physically put on the TX wire. - fn flush(&self) { -+ use crate::{time, time::interface::TimeManager}; -+ use core::time::Duration; -+ - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. -- // -- // Now make an educated guess for a good delay value. According to Wikipedia, the fastest -- // RPi4 clocks around 1.5 GHz. -- // -- // So lets try to be on the safe side and default to 24_000 cycles, which would equal 12_000 -- // ns would the CPU be clocked at 2 GHz. -- const CHAR_TIME_SAFE: usize = 24_000; -+ const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { -@@ -253,11 +250,11 @@ - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. -- cpu::spin_for_cycles(CHAR_TIME_SAFE); -+ time::time_manager().spin_for(CHAR_TIME_SAFE); +@@ -279,7 +279,7 @@ } /// Retrieve a character. @@ -355,7 +330,7 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08 // If RX FIFO is empty, if self.registers.FR.matches_all(FR::RXFE::SET) { // immediately return in non-blocking mode. -@@ -272,7 +269,12 @@ +@@ -294,7 +294,12 @@ } // Read one character. @@ -369,7 +344,7 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08 // Update statistics. self.chars_read += 1; -@@ -352,14 +354,14 @@ +@@ -374,14 +379,14 @@ impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { self.inner @@ -498,11 +473,14 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs --- 07_uart_chainloader/src/cpu.rs +++ 08_timestamps/src/cpu.rs -@@ -15,4 +15,4 @@ +@@ -15,7 +15,4 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- --pub use arch_cpu::{branch_to_raw_addr, nop, spin_for_cycles, wait_forever}; +-pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; +- +-#[cfg(feature = "bsp_rpi3")] +-pub use arch_cpu::spin_for_cycles; +pub use arch_cpu::{nop, wait_forever}; diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 2c74961d9..e6dd762ec 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -1775,8 +1775,8 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -5,8 +5,8 @@ - //! PL011 UART driver. +@@ -10,8 +10,8 @@ + //! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ - bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -1786,11 +1786,11 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs }; use core::fmt; use register::{mmio::*, register_bitfields, register_structs}; -@@ -109,6 +109,48 @@ +@@ -135,6 +135,52 @@ ] ], -+ /// Interrupt FIFO Level Select Register ++ /// Interrupt FIFO Level Select Register. + IFLS [ + /// Receive interrupt FIFO level select. The trigger points for the receive interrupt are as + /// follows. @@ -1803,25 +1803,29 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs + ] + ], + -+ /// Interrupt Mask Set Clear Register ++ /// Interrupt Mask Set/Clear Register. + IMSC [ + /// Receive timeout interrupt mask. A read returns the current mask for the UARTRTINTR -+ /// interrupt. On a write of 1, the mask of the interrupt is set. A write of 0 clears the -+ /// mask. ++ /// interrupt. ++ /// ++ /// - On a write of 1, the mask of the UARTRTINTR interrupt is set. ++ /// - A write of 0 clears the mask. + RTIM OFFSET(6) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + -+ /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. On -+ /// a write of 1, the mask of the interrupt is set. A write of 0 clears the mask. ++ /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. ++ /// ++ /// - On a write of 1, the mask of the UARTRXINTR interrupt is set. ++ /// - A write of 0 clears the mask. + RXIM OFFSET(4) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ] + ], + -+ /// Masked Interrupt Status Register ++ /// Masked Interrupt Status Register. + MIS [ + /// Receive timeout masked interrupt status. Returns the masked interrupt state of the + /// UARTRTINTR interrupt. @@ -1832,12 +1836,12 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs + RXMIS OFFSET(4) NUMBITS(1) [] + ], + - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts -@@ -127,7 +169,10 @@ + /// Meta field for all pending interrupts. +@@ -153,7 +199,10 @@ (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), - (0x34 => _reserved3), + (0x34 => IFLS: ReadWrite), @@ -1847,7 +1851,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (0x44 => ICR: WriteOnly), (0x48 => @END), } -@@ -157,7 +202,8 @@ +@@ -183,7 +232,8 @@ /// Representation of the UART. pub struct PL011Uart { @@ -1857,9 +1861,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } //-------------------------------------------------------------------------------------------------- -@@ -214,6 +260,14 @@ - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); +@@ -251,6 +301,14 @@ + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); + // Set RX FIFO fill level at 1/8. + self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); @@ -1872,7 +1876,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs // Turn the UART on. self.registers .CR -@@ -308,9 +362,13 @@ +@@ -333,9 +391,13 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -1888,7 +1892,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } } } -@@ -330,6 +388,21 @@ +@@ -355,6 +417,21 @@ Ok(()) } @@ -1910,7 +1914,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } impl console::interface::Write for PL011Uart { -@@ -376,3 +449,24 @@ +@@ -401,3 +478,24 @@ self.inner.lock(|inner| inner.chars_read) } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 7b3a85cf0..1dd0a14c2 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,7 +135,7 @@ register_bitfields! { ] ], - /// Interrupt FIFO Level Select Register + /// Interrupt FIFO Level Select Register. IFLS [ /// Receive interrupt FIFO level select. The trigger points for the receive interrupt are as /// follows. @@ -122,25 +148,29 @@ register_bitfields! { ] ], - /// Interrupt Mask Set Clear Register + /// Interrupt Mask Set/Clear Register. IMSC [ /// Receive timeout interrupt mask. A read returns the current mask for the UARTRTINTR - /// interrupt. On a write of 1, the mask of the interrupt is set. A write of 0 clears the - /// mask. + /// interrupt. + /// + /// - On a write of 1, the mask of the UARTRTINTR interrupt is set. + /// - A write of 0 clears the mask. RTIM OFFSET(6) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. On - /// a write of 1, the mask of the interrupt is set. A write of 0 clears the mask. + /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. + /// + /// - On a write of 1, the mask of the UARTRXINTR interrupt is set. + /// - A write of 0 clears the mask. RXIM OFFSET(4) NUMBITS(1) [ Disabled = 0, Enabled = 1 ] ], - /// Masked Interrupt Status Register + /// Masked Interrupt Status Register. MIS [ /// Receive timeout masked interrupt status. Returns the masked interrupt state of the /// UARTRTINTR interrupt. @@ -151,9 +181,9 @@ register_bitfields! { RXMIS OFFSET(4) NUMBITS(1) [] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -167,7 +197,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => IFLS: ReadWrite), (0x38 => IMSC: ReadWrite), @@ -229,11 +259,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. + /// + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -251,14 +288,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Set RX FIFO fill level at 1/8. self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); @@ -289,22 +330,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index beb2cffc6..b326d589c 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1189,8 +1189,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -5,10 +5,13 @@ - //! PL011 UART driver. +@@ -10,10 +10,13 @@ + //! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, @@ -1206,7 +1206,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ use register::{mmio::*, register_bitfields, register_structs}; //-------------------------------------------------------------------------------------------------- -@@ -202,6 +205,8 @@ +@@ -232,6 +235,8 @@ /// Representation of the UART. pub struct PL011Uart { @@ -1215,10 +1215,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ inner: IRQSafeNullLock, irq_number: bsp::device_driver::IRQNumber, } -@@ -234,7 +239,15 @@ - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5modulo error margin is acceptable for UART and we're - /// now at 0.02modulo. +@@ -271,7 +276,15 @@ + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16modulo`. - pub fn init(&mut self) { + /// + /// # Safety @@ -1232,7 +1232,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, // those queued characters would be lost. -@@ -272,6 +285,8 @@ +@@ -313,6 +326,8 @@ self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -1241,7 +1241,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } /// Send a character. -@@ -361,13 +376,18 @@ +@@ -390,13 +405,18 @@ /// /// # Safety /// @@ -1263,7 +1263,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ irq_number, } } -@@ -384,7 +404,14 @@ +@@ -413,7 +433,14 @@ } unsafe fn init(&self) -> Result<(), &'static str> { @@ -1279,7 +1279,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ Ok(()) } -@@ -403,6 +430,16 @@ +@@ -432,6 +459,16 @@ Ok(()) } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d7d83840e..e279c2062 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, @@ -20,50 +25,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -76,34 +94,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -112,7 +138,7 @@ register_bitfields! { ] ], - /// Interrupt FIFO Level Select Register + /// Interrupt FIFO Level Select Register. IFLS [ /// Receive interrupt FIFO level select. The trigger points for the receive interrupt are as /// follows. @@ -125,25 +151,29 @@ register_bitfields! { ] ], - /// Interrupt Mask Set Clear Register + /// Interrupt Mask Set/Clear Register. IMSC [ /// Receive timeout interrupt mask. A read returns the current mask for the UARTRTINTR - /// interrupt. On a write of 1, the mask of the interrupt is set. A write of 0 clears the - /// mask. + /// interrupt. + /// + /// - On a write of 1, the mask of the UARTRTINTR interrupt is set. + /// - A write of 0 clears the mask. RTIM OFFSET(6) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. On - /// a write of 1, the mask of the interrupt is set. A write of 0 clears the mask. + /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. + /// + /// - On a write of 1, the mask of the UARTRXINTR interrupt is set. + /// - A write of 0 clears the mask. RXIM OFFSET(4) NUMBITS(1) [ Disabled = 0, Enabled = 1 ] ], - /// Masked Interrupt Status Register + /// Masked Interrupt Status Register. MIS [ /// Receive timeout masked interrupt status. Returns the masked interrupt state of the /// UARTRTINTR interrupt. @@ -154,9 +184,9 @@ register_bitfields! { RXMIS OFFSET(4) NUMBITS(1) [] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -170,7 +200,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => IFLS: ReadWrite), (0x38 => IMSC: ReadWrite), @@ -234,11 +264,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. + /// + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. /// /// # Safety /// @@ -264,14 +301,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Set RX FIFO fill level at 1/8. self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); @@ -304,22 +345,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 6fe154299791b169e5b4fbfa9e85d0b734d27d9b..9f8ada21812d7d7772028c6bfed45c182479b457 100755 GIT binary patch delta 2277 zcmb7FeN0nV6uB5>*6DKj8*F?@(Us_mk-QT*tZaRp*Y)e3L2}qPAITyYv5Q6+ zCq>6kAlcXu8GFEAg#GkKUe7g_QiU(&I+nW2R92kYSW1j|De)D2CV679dy23W#YVBA@b~p94PmE&e%KYnNZQ%cnEzt#XI*qnL z7n5@U>_Imt^~r)ZIEsnKfQYvV5#xOOSfB)jf^J^&44(N2x)$>bnJxe|p$ zu-5|eWN!tTxS*2SfQeRy99Y-|$UlOhEVk1wqyBMn8tjSS-#H*?Er51lyofMn0y~hr z5Bz3(V>ki9P=`WZpek@ukajEpBXi0c1RHtiD(I-8lr(u(kw&${ z`fkH&;yuX`FZku6-0ShfUmmF4^XqIqjt}~AgsIDnsh)o`75GCr$Ij%~dFDc*=N;C{ z;ld=6bT@KPGbC9J{X$c1ue%^=z-yn%VQs#(DI}`Ra!p$5YEg}3HrE|O@T{<@Q8R~VDLWHeVKoZ9MJ1o#X|>MySJvhy&KJ7LUh$6?$Iex1?k%f=U> z`Zlw>pvu~_kQ|p}hrtRYGd-GKBz*G`T ze5Q@h00G0g5A^enBC-V-b;R6bl!0S~^hvdD8jQAQZ*_rWicBj!-zg0fgx zYL>h_?)UHlXfO6>fbDaHhCtqaW2$+5r3%f6X^7s*NTJD@o3WENW-i7D>06oKrW{!U zQQX(I(QR2JxSEozl474WvIeCk+-jq~pkTN?xHq4MvUHV?x+@sH1*&Se-O;*V(LRqe z&*&nzH)lt_=c98c9m&>-Z>C4Aoz$PbDei7^WHy~mOLLa#t8yaK^yrb>qI0Nc;kKT3 z-G(diGi3{ThtG26A#+NJJfQqrg~M_KC`!^)>zw4GTC`~ zcINs0pP6Sa+fO<=v$2WfT*MQnf(6+0nE~y^2%Yr*jEKdPM<#hhKBf7b6?=zjxOD7x zRJ*L{1p`Th^O+4D$#5uxmB9F_@oBJoA znoE$v#>_8AzG6O7M9$-4W1aVztNu19AYxQ4u; z2eFkH?;fg%@55;&OGqq2opHcERfWjbulQ^D)Km^im5k-?6$ z&4uFVArbbH8W8zaRYSLnjq+PxcwNr8qs^BZ%1O|O!s1NLxYTm7dwPf?p+>CvC(W!L z-)bo)g_gLiZI`=U(>6ig*63>v*+hEL^o)BL%Ui*@4!oQ=XWkeOmZ7loOhQ)hJl4YL zw<0pxh=?%LJbvp0CkIX-Wojc*f+u*ne^%`p5m4#uI!73wv@lC8Lbu>J=&Ih)*KEha zO-@AXV?d@L?BRT25t-^kvIY&1UJ;#kJK=}ek&j#hz<3L|l7O)Tm}|tbLvI7$G;kzs zLgZE8oPv3#gA;f|;An_qGChQ4#w+H>W^D()?Tl@JlcxuGIkv^)TGVp9JMbI|OD(6m zeGC*J6TqSNmiX>?gxsCS8x|1>=otN_wN~Oh6`K{@0tX9>LO?I|EGD5rd>z9%SgU9uue<_*l z<%l2ba#3yWm6(wWpE!5ixMRmLH-k7L*BFz_@^`t$epk=1MLD+Mh(s*B$5rVlB5@Dd zLb+F(oO`~P=&_V)cjAKkcWben3R1R5d0Z(VgP%_+YKws@V{j-nrqm^(0V%9+-iMSG zs@kQ^p#z$ocn~RN`WTxxL;r5=N_o~;d7oV=Rs~p_9Y^6DCgMF270W>$#J2MC9QPoVOx5}ScumkF*M#mjnK@iWKQ~pR%^mnG{lJ445KbDK zmf`N1QM()%mDWLiQdM>SlT_cRGd(rT;Ax5$=G-gH4r7lCy=T)d%3OY9Vyab;5jGg* zyS0LBo>kplM~vw7KEP+_reFG+16EME#7g?_*omrgvKG-^1ZgX!C^&#+F9_}997-y` z%ascDg%DvMhm_$@kg^LL5cJ6U_9Nv3=(F)*Xt%*XZF(YR?NKceyIWoFfKd~305a|2 zOr*p>mOcC|Qdo@I5|C0@?{RtOK^oYNNP8Mm*hOdqI~KE`88VR;vvP=;L1r&Q&>AHN zssIA!0|n9YBq&o2D(4#iQkelWIFlO)G$Np#JG*maffN*Gq>mChA0u4#uY|3x@}~g0 zpGH{72xE1^9-DSr86_hWfLef;CE$(_CNaYB6H>vE6_HesTMq9fCVQA2++YOnA%X9`2#u k1sh7{HtaM}Cltjb+W-In diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index d5573467fe56c7dfe6f5289a4c6d98bee8e3ed6c..e9d7e7ddde04a9b940c72fac1745384b4231845f 100755 GIT binary patch delta 1548 zcmb7D-D@0G6hC)nXET%7B&)kS`;q-(lkCo{B261(sct5QNRt=oQl$vm#2{!D3i}W2 z80tf@4|}n|kVP%-z@t!2=dS6e0cvpt0Hl?yf`E@(6B~Owd+g4&MseDR; z>)T3HLb207P6k2%+AWD{qXqLB+EW~VoHzFYdDd3#OTgL;;uCm&5yzIDYQFn$hKWLcK#fUIYzfQMc%vNR={g*uIOXf@Btc z$67Whcl}USPS138$*6tcF$?Dd{$sj?>qMVYg4KfZGt}(FgJhZHsD0%c%1I*cDFn_q z4LzC*rlQdWEEMHn9^}3cf7vYq<`0Q z7;wizz$$eAKj6}TMBKv>FW^h!fSn2OWjVl0hXCUcHsE`IBjCTR;}3CVe#GTUbHvG6 z(2*?faCa2&jV)Wc6^zR+Z(EFqXf zAHEOgTb9`Q(CxWw_~Gyg)wPk?&NovmUMQlL=Kdh}KbjX(S#c2@af!KcwfFv@ z>y^Vy9~u=JVfIZdPn_7;L9EWN!N(tb#vGSJVFJ0}xRb5$uAgrrqok zwZX!VxsHw2Z4)0!?O<)S+QjW6h1xbHZLF3`f3%4U*0{TfwAi$6M5bpR>+6rb$<00Y z%z1w2+%tWP4_64YX@l{!%}meQYJ~k1z$u4|C*i9=Mi?M3F=s019BR3Ip1w4-)-2s( zpqB+#oXd8F_Ci^yoo6m9^nhHsgbkNp=6>JfHv0yGGGiVMC&7A!=fu4_Qyj`D@TUO? z7fxG=(`OFGc2Q(}1w~??o1}d1`$OE_>OrUPhoJNBn!$WWiq(vv)Er^SXgftxknV9j zLs0+Jh2gnm^hA<0eu9;nF9zMs4Ig-KFc9yCfcs_G?;%R*&0~~p~x|cL<$+{9uA1X$O<4o(y%|A z3X7Hl%;0=nkafM|Ufq7&TS>8_BP^_sqFpuM_&gGu2NH)l7vT1AaJI(B21hZ`6!2pf zn9*1RMaD{KshS%&gi1mUBBCIWiV>gew~@5IH{Bogob#sReH^pq^3trMg_P9%*;Mu@0 zAO<{lA6`I48fa2)47iO9?B{|1#oYJZzMA&y6P1*C)E}i_{Xhb$;fJA${v&^?8OHgU>{p>{4`0K=8Hax*%tFhQG?1=Sp{z&;{*68{qU#BCo zWr@m*`}ZllCfenX+lwKDMBKhoJ3SZ-m#5JjjkHsz(~y2jqn&j~qu$io8VTP4>0@e# z)Ih;O`l*0hxlg6rZx_k(fiPq59mYMCRCQb7+wrZS>)dPC3Nd z^tZ)xhA&N>rNmj~5MMXGqI~`W|JEA~S*=;C!`$U}p&UR)iQ9aLaQn=^1`fypvXE5S;yvGqvfB6qO== zJFUu}M~-6ROsXqh2lgQ)Q0x(b>s%EtFuFqBhJd?y=8Td8BZOGwanS4Lu0cqVOUSaF z*q5l|#8Y_$999++6(F!osh+T$FDLbA=isD#Kxu<}vU}imWg#jI$nts=RC`2TfzzcN z;(xDYCDlmX(}W5DxrT^}3o@!R@lizP86+i5^}$pXi`^Nv>3zi{jG6wV`#7hFf00p5 zpSFrgnR&_t5!4BC6_g1g=o3V^(7#Yzm03Xdmx%3|rG91FVQ(`O**GO1a8a3HF<>yxd=vBzByVWSN#Y$`yJG(W}NPAhIE~0>coxwF^iXcitWOfKoBmv6NJ! z{;gw-6qOO671#%Omya2xXsP;mh5O^2BJ(i_Fx*eD3uXNY=TSc`$yOYkk8y57=eLs8 zu_CZl*t}>&g&G@KsgmqBN-@7lnW36<3`#*aRY7-BRwf6wiH@ABtV|gl!tTB@RY-xZ zONS2d@x-TO3o7@r2A33O*j{6zEOpDowKI#e(U`iNFc(U3DwzNW60S*+%&g4W!6Hfz zEDT9qZ?Ix)=7qfQoQFS4v$Z?+rMgyny=`7`EIXGzm@Q`J6!~?2nN^e1Tmb}iI|=iT z+TE_p=j!?!JexN+dfT_wx3n~Rk@tV?Idpv}*9}h78Ef==fw%EbTl?}AiJeyC*Qt(+qO1uN8`r% zeESy9Hq_W`epIugt&MMKYu)^S2-r1X&S~*v0;@B^dVoqP4HSD?eJquGu1OsaSYWuTcVu$JOlJIilBc zliq6+bFwQa)$_uH=N&dJt_3sZgtLrtJcr^Q2u_|XC17Vxkob&vDwAcAX diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 876aa461d..ce39752d1 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -3,6 +3,11 @@ // Copyright (c) 2018-2021 Andre Richter //! PL011 UART driver. +//! +//! # Resources +//! +//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +//! - https://developer.arm.com/documentation/ddi0183/latest use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -17,50 +22,63 @@ use register::{mmio::*, register_bitfields, register_structs}; // PL011 UART registers. // -// Descriptions taken from -// https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. register_bitfields! { u32, - /// Flag Register + /// Flag Register. FR [ /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// Line Control Register, UARTLCR_ LCRH. + /// Line Control Register, LCR_H. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is empty. If - /// the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. This bit does - /// not indicate if there is data in the transmit shift register. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. TXFE OFFSET(7) NUMBITS(1) [], /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_ LCRH Register. + /// LCR_H Register. /// - /// If the FIFO is disabled, this bit is set when the transmit holding register is full. If - /// the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. TXFF OFFSET(5) NUMBITS(1) [], /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the - /// UARTLCR_H Register. + /// LCR_H Register. /// /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. - RXFE OFFSET(4) NUMBITS(1) [] + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] ], - /// Integer Baud rate divisor + /// Integer Baud Rate Divisor. IBRD [ - /// Integer Baud rate divisor - IBRD OFFSET(0) NUMBITS(16) [] + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] ], - /// Fractional Baud rate divisor + /// Fractional Baud Rate Divisor. FBRD [ - /// Fractional Baud rate divisor - FBRD OFFSET(0) NUMBITS(6) [] + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] ], - /// Line Control register - LCRH [ + /// Line Control Register. + LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. WLEN OFFSET(5) NUMBITS(2) [ @@ -73,34 +91,42 @@ register_bitfields! { /// Enable FIFOs: /// /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding - /// registers + /// registers. /// - /// 1 = transmit and receive FIFO buffers are enabled (FIFO mode). + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). FEN OFFSET(4) NUMBITS(1) [ FifosDisabled = 0, FifosEnabled = 1 ] ], - /// Control Register + /// Control Register. CR [ /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. - /// Data reception occurs for UART signals. When the UART is disabled in the middle of - /// reception, it completes the current character before stopping. - RXE OFFSET(9) NUMBITS(1) [ + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. - /// Data transmission occurs for UART signals. When the UART is disabled in the middle of - /// transmission, it completes the current character before stopping. - TXE OFFSET(8) NUMBITS(1) [ + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ Disabled = 0, Enabled = 1 ], - /// UART enable + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit UARTEN OFFSET(0) NUMBITS(1) [ /// If the UART is disabled in the middle of transmission or reception, it completes the /// current character before stopping. @@ -109,9 +135,9 @@ register_bitfields! { ] ], - /// Interrupt Clear Register + /// Interrupt Clear Register. ICR [ - /// Meta field for all pending interrupts + /// Meta field for all pending interrupts. ALL OFFSET(0) NUMBITS(11) [] ] } @@ -125,7 +151,7 @@ register_structs! { (0x1c => _reserved2), (0x24 => IBRD: WriteOnly), (0x28 => FBRD: WriteOnly), - (0x2c => LCRH: WriteOnly), + (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), (0x34 => _reserved3), (0x44 => ICR: WriteOnly), @@ -183,11 +209,18 @@ impl PL011UartInner { /// This results in 8N1 and 921_600 baud. /// /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): - /// `(48_000_000 / 16) / 921_600 = 3.2552083`. `3` goes to the `IBRD` (integer field). + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. /// - /// The `FBRD` (fractional field) is only 6 bits so `0.2552083 * 64 = 16.3 rounded to 16` will - /// give the best approximation we can get. A 5% error margin is acceptable for UART and we're - /// now at 0.02%. + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. pub fn init(&mut self) { // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, @@ -205,14 +238,18 @@ impl PL011UartInner { // Clear all pending interrupts. self.registers.ICR.write(ICR::ALL::CLEAR); - // Set the baud rate. - self.registers.IBRD.write(IBRD::IBRD.val(3)); - self.registers.FBRD.write(FBRD::FBRD.val(16)); - - // Set 8N1 + FIFO on. + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); self.registers - .LCRH - .write(LCRH::WLEN::EightBit + LCRH::FEN::FifosEnabled); + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); // Turn the UART on. self.registers @@ -235,22 +272,10 @@ impl PL011UartInner { /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { - use crate::{time, time::interface::TimeManager}; - use core::time::Duration; - - // The bit time for 921_600 baud is 1 / 921_600 = 1.09 µs. 8N1 has a total of 10 bits per - // symbol (start bit, 8 data bits, stop bit), so one symbol takes round about 10 * 1.09 = - // 10.9 µs, or 10_900 ns. Round it up to 12_000 ns to be on the safe side. - const CHAR_TIME_SAFE: Duration = Duration::from_nanos(12_000); - - // Spin until TX FIFO empty is set. - while !self.registers.FR.matches_all(FR::TXFE::SET) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { cpu::nop(); } - - // After the last character has been queued for transmission, wait for the time of one - // character + some extra time for safety. - time::time_manager().spin_for(CHAR_TIME_SAFE); } /// Retrieve a character. From dc9b3c0f381f1747eb8020e069649033606342b9 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 29 Jan 2021 22:26:45 +0100 Subject: [PATCH 026/214] Add isb before timer read --- 08_timestamps/README.md | 19 +++++++++++++++--- 08_timestamps/src/_arch/aarch64/time.rs | 17 ++++++++++++++-- 09_hw_debug_JTAG/src/_arch/aarch64/time.rs | 17 ++++++++++++++-- 10_privilege_level/src/_arch/aarch64/time.rs | 17 ++++++++++++++-- .../src/_arch/aarch64/time.rs | 17 ++++++++++++++-- .../src/_arch/aarch64/time.rs | 17 ++++++++++++++-- .../src/_arch/aarch64/time.rs | 17 ++++++++++++++-- .../src/_arch/aarch64/time.rs | 17 ++++++++++++++-- .../src/_arch/aarch64/time.rs | 17 ++++++++++++++-- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8144 -> 8144 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6848 -> 6864 bytes X1_JTAG_boot/src/_arch/aarch64/time.rs | 17 ++++++++++++++-- 12 files changed, 151 insertions(+), 21 deletions(-) diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 83e38ca53..0f8ec5d2a 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -175,7 +175,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/aarch64/time.rs --- 07_uart_chainloader/src/_arch/aarch64/time.rs +++ 08_timestamps/src/_arch/aarch64/time.rs -@@ -0,0 +1,105 @@ +@@ -0,0 +1,118 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -191,7 +191,7 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ + +use crate::{time, warn}; +use core::time::Duration; -+use cortex_a::regs::*; ++use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -209,6 +209,19 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ +static TIME_MANAGER: GenericTimer = GenericTimer; + +//-------------------------------------------------------------------------------------------------- ++// Private Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl GenericTimer { ++ #[inline(always)] ++ fn read_cntpct(&self) -> u64 { ++ // Prevent that the counter is read ahead of time due to out-of-order execution. ++ unsafe { barrier::isb(barrier::SY) }; ++ CNTPCT_EL0.get() ++ } ++} ++ ++//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + @@ -227,8 +240,8 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ + } + + fn uptime(&self) -> Duration { ++ let current_count: u64 = self.read_cntpct() * NS_PER_S; + let frq: u64 = CNTFRQ_EL0.get() as u64; -+ let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; + + Duration::from_nanos(current_count / frq) + } diff --git a/08_timestamps/src/_arch/aarch64/time.rs b/08_timestamps/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/08_timestamps/src/_arch/aarch64/time.rs +++ b/08_timestamps/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/10_privilege_level/src/_arch/aarch64/time.rs b/10_privilege_level/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/10_privilege_level/src/_arch/aarch64/time.rs +++ b/10_privilege_level/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/13_integrated_testing/src/_arch/aarch64/time.rs b/13_integrated_testing/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/13_integrated_testing/src/_arch/aarch64/time.rs +++ b/13_integrated_testing/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 9f8ada21812d7d7772028c6bfed45c182479b457..4cd4401535b2b3cb1f5cbf807c72b1f69637cedd 100755 GIT binary patch delta 236 zcmca$f5Co&3?utQ+0%@dH@=K#WUSuo$@G{{NP&YPG2yZHI1ct-BmoSYE(Qy}?; zS`%NdW>^U_Q5$F)7Z7K0O^y;e#}*4TvEkcfJz*uL8~-NH5)P7K02;~!#0(A^6BU^o zf}9R$O#Jiv|8xZgg)5Wa2&)J>{r^8b$pK=rzKfYAJ_7O;7!A2>G2yZHI1ct-BmFBu`m zoB|oM5Y?Dh-~Ml909wHW#0(A^6BU^of}9R$Oyv7Bxk@-_@(W=VLC639r)S)9|H;WP tBZzY*!%o}DvLfz`w_4nAa^|8CuL6s>lEk~YYi+f16 zC;^>fw{548`PLREV{F5|mqqOl$WU&>-eMyYx$RnRv4mUL&$b)^_vms!De-BIeMb?E+bh#hO}Aww+HIGEO&Y4IbWA0$Gjt9(u+FpvyM z>zs2hAQzyEV$jSHt&4^wEpb>RGtP<~RKl~1T06C(o`DV{&V)@%e8*UNA$ckJM#eM8 z*^x@OOuZt0H{6N%adYdT#V#3Cb4cVRT5WC^s3@-&tuZ&lO^S*s^T_+ooWA}Q{T?odg zmre%5wT&^Mc!AV|Jt!q=j2D$eJeWv|MuU1FUW`TU`gV6T%1d_Nyf@$Xe&=O(sXNn5 zo-Vc2+G<|dVU_?ahSb7ATbvMJ=Y(q{$36u<69_6P1bsx*2(xFYhAx zvH(roHaSSBuwAR$Y#V;|qwYKgDkSU235l{NdsH&Y zoKis4LX>Dgr`ZANl(gZ-(DwW$hOGg}3{J5dQXmS=w0sQc)=zyd2*Fpw0JApyjI%D- zR;#^G5zD=63R{yVT)7ynL+W$F1{<&scB!E!tv1r~dqUp{gw9m;Ih8N-pb~Sa z#hZ35ana-Mj5(kvqA%@&g@o$^6b;}Xut?(6g(L-a9}&N}2n?AC?TS@QR-t1UsEHTz z7kd@H8~A6sJ~Xad1?Z2+YPAz8pRpiOm`;d~ms3F}+%q`w6*s%P_X_D}F?V3MF^Sta ziAO+N*syyhJ%$SfEwLF9@UBC?Q&cjdQHC$GttQ!+)G;?6UvtZ#~ur2LJ#7 diff --git a/X1_JTAG_boot/src/_arch/aarch64/time.rs b/X1_JTAG_boot/src/_arch/aarch64/time.rs index 3a766009d..63017305b 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/time.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/time.rs @@ -13,7 +13,7 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::regs::*; +use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -30,6 +30,19 @@ struct GenericTimer; static TIME_MANAGER: GenericTimer = GenericTimer; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -49,8 +62,8 @@ impl time::interface::TimeManager for GenericTimer { } fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; let frq: u64 = CNTFRQ_EL0.get() as u64; - let current_count: u64 = CNTPCT_EL0.get() * NS_PER_S; Duration::from_nanos(current_count / frq) } From a5884321a717f0f4f9252cbcc0f7df84b92bc119 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 29 Jan 2021 22:30:02 +0100 Subject: [PATCH 027/214] Fix rustdoc warnings --- 01_wait_forever/src/main.rs | 4 +++- 02_runtime_init/README.md | 9 ++++++--- 02_runtime_init/src/main.rs | 5 ++++- 03_hacky_hello_world/README.md | 16 +++++++-------- 03_hacky_hello_world/src/main.rs | 1 + 03_hacky_hello_world/src/print.rs | 4 ++-- 04_zero_overhead_abstraction/README.md | 6 +++--- 04_zero_overhead_abstraction/src/main.rs | 1 + 04_zero_overhead_abstraction/src/print.rs | 4 ++-- 05_safe_globals/README.md | 17 ++++++++-------- 05_safe_globals/src/main.rs | 1 + 05_safe_globals/src/print.rs | 4 ++-- 05_safe_globals/src/synchronization.rs | 9 +++++---- 06_drivers_gpio_uart/README.md | 14 ++++++------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 06_drivers_gpio_uart/src/main.rs | 1 + 06_drivers_gpio_uart/src/panic_wait.rs | 2 +- 06_drivers_gpio_uart/src/print.rs | 4 ++-- 06_drivers_gpio_uart/src/synchronization.rs | 9 +++++---- 07_uart_chainloader/README.md | 8 +++++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 07_uart_chainloader/src/main.rs | 2 ++ 07_uart_chainloader/src/panic_wait.rs | 2 +- 07_uart_chainloader/src/print.rs | 4 ++-- 07_uart_chainloader/src/synchronization.rs | 9 +++++---- 08_timestamps/README.md | 8 +++++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 08_timestamps/src/main.rs | 1 + 08_timestamps/src/panic_wait.rs | 2 +- 08_timestamps/src/print.rs | 4 ++-- 08_timestamps/src/synchronization.rs | 9 +++++---- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 09_hw_debug_JTAG/src/main.rs | 1 + 09_hw_debug_JTAG/src/panic_wait.rs | 2 +- 09_hw_debug_JTAG/src/print.rs | 4 ++-- 09_hw_debug_JTAG/src/synchronization.rs | 9 +++++---- 10_privilege_level/README.md | 6 +++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 10_privilege_level/src/main.rs | 1 + 10_privilege_level/src/panic_wait.rs | 2 +- 10_privilege_level/src/print.rs | 4 ++-- 10_privilege_level/src/synchronization.rs | 9 +++++---- .../README.md | 10 +++++----- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- .../src/main.rs | 1 + .../src/panic_wait.rs | 2 +- .../src/print.rs | 4 ++-- .../src/synchronization.rs | 9 +++++---- 12_exceptions_part1_groundwork/README.md | 6 +++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 12_exceptions_part1_groundwork/src/main.rs | 1 + .../src/panic_wait.rs | 2 +- 12_exceptions_part1_groundwork/src/print.rs | 4 ++-- .../src/synchronization.rs | 9 +++++---- 13_integrated_testing/README.md | 14 +++++++------ .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- 13_integrated_testing/src/lib.rs | 1 + 13_integrated_testing/src/panic_wait.rs | 2 +- 13_integrated_testing/src/print.rs | 4 ++-- 13_integrated_testing/src/synchronization.rs | 9 +++++---- 14_exceptions_part2_peripheral_IRQs/README.md | 20 +++++++++---------- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- .../src/exception/asynchronous.rs | 2 +- .../src/lib.rs | 1 + .../src/panic_wait.rs | 2 +- .../src/print.rs | 4 ++-- .../src/synchronization.rs | 9 +++++---- 15_virtual_mem_part2_mmio_remap/README.md | 6 +++--- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- .../src/exception/asynchronous.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 1 + .../src/panic_wait.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/print.rs | 4 ++-- .../src/synchronization.rs | 9 +++++---- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 4 ++-- X1_JTAG_boot/src/main.rs | 1 + X1_JTAG_boot/src/panic_wait.rs | 2 +- X1_JTAG_boot/src/print.rs | 4 ++-- X1_JTAG_boot/src/synchronization.rs | 9 +++++---- 79 files changed, 220 insertions(+), 178 deletions(-) diff --git a/01_wait_forever/src/main.rs b/01_wait_forever/src/main.rs index 10cf6bb42..8cd4b1441 100644 --- a/01_wait_forever/src/main.rs +++ b/01_wait_forever/src/main.rs @@ -100,8 +100,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. +//! +//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html #![feature(asm)] #![feature(global_asm)] diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 7948aae4a..50cce8798 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -199,15 +199,18 @@ diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs --- 01_wait_forever/src/main.rs +++ 02_runtime_init/src/main.rs -@@ -102,6 +102,7 @@ +@@ -102,8 +102,10 @@ //! - //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. + //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. + //! + //! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html ++//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(asm)] #![feature(global_asm)] -@@ -110,6 +111,15 @@ +@@ -112,6 +114,15 @@ mod bsp; mod cpu; diff --git a/02_runtime_init/src/main.rs b/02_runtime_init/src/main.rs index ba5a877c7..fd9c36bc3 100644 --- a/02_runtime_init/src/main.rs +++ b/02_runtime_init/src/main.rs @@ -100,9 +100,12 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(asm)] #![feature(global_asm)] diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 11c804bff..c8dc66cc7 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -139,17 +139,17 @@ diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs --- 02_runtime_init/src/main.rs +++ 03_hacky_hello_world/src/main.rs -@@ -100,19 +100,25 @@ - //! +@@ -101,21 +101,25 @@ //! # Boot flow //! --//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. + //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. -+//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -+//! + //! +-//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(asm)] +#![feature(format_args_nl)] @@ -167,7 +167,7 @@ diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs mod runtime_init; /// Early init code. -@@ -121,5 +127,7 @@ +@@ -124,5 +128,7 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { @@ -226,7 +226,7 @@ diff -uNr 02_runtime_init/src/print.rs 03_hacky_hello_world/src/print.rs + +/// Prints without a newline. +/// -+/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html ++/// Carbon copy from +#[macro_export] +macro_rules! print { + ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -234,7 +234,7 @@ diff -uNr 02_runtime_init/src/print.rs 03_hacky_hello_world/src/print.rs + +/// Prints with a newline. +/// -+/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html ++/// Carbon copy from +#[macro_export] +macro_rules! println { + () => ($crate::print!("\n")); diff --git a/03_hacky_hello_world/src/main.rs b/03_hacky_hello_world/src/main.rs index 378e0bafb..0815d5e21 100644 --- a/03_hacky_hello_world/src/main.rs +++ b/03_hacky_hello_world/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(asm)] #![feature(format_args_nl)] diff --git a/03_hacky_hello_world/src/print.rs b/03_hacky_hello_world/src/print.rs index fa6451c75..8ea5db240 100644 --- a/03_hacky_hello_world/src/print.rs +++ b/03_hacky_hello_world/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index 6c9ad70c4..d20758d1c 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -242,9 +242,9 @@ diff -uNr 03_hacky_hello_world/src/cpu.rs 04_zero_overhead_abstraction/src/cpu.r diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main.rs --- 03_hacky_hello_world/src/main.rs +++ 04_zero_overhead_abstraction/src/main.rs -@@ -106,9 +106,7 @@ - //! +@@ -107,9 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![feature(asm)] #![feature(format_args_nl)] @@ -252,7 +252,7 @@ diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main #![feature(panic_info_message)] #![no_main] #![no_std] -@@ -127,7 +125,8 @@ +@@ -128,7 +126,8 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/04_zero_overhead_abstraction/src/main.rs b/04_zero_overhead_abstraction/src/main.rs index 7de836fe5..9dfc7d007 100644 --- a/04_zero_overhead_abstraction/src/main.rs +++ b/04_zero_overhead_abstraction/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/04_zero_overhead_abstraction/src/print.rs b/04_zero_overhead_abstraction/src/print.rs index fa6451c75..8ea5db240 100644 --- a/04_zero_overhead_abstraction/src/print.rs +++ b/04_zero_overhead_abstraction/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/05_safe_globals/README.md b/05_safe_globals/README.md index 3db54d6be..ea61489e2 100644 --- a/05_safe_globals/README.md +++ b/05_safe_globals/README.md @@ -211,7 +211,7 @@ diff -uNr 04_zero_overhead_abstraction/src/console.rs 05_safe_globals/src/consol diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs --- 04_zero_overhead_abstraction/src/main.rs +++ 05_safe_globals/src/main.rs -@@ -108,6 +108,7 @@ +@@ -109,6 +109,7 @@ #![feature(format_args_nl)] #![feature(panic_info_message)] @@ -219,7 +219,7 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs #![no_main] #![no_std] -@@ -118,6 +119,7 @@ +@@ -119,6 +120,7 @@ mod panic_wait; mod print; mod runtime_init; @@ -227,7 +227,7 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs /// Early init code. /// -@@ -125,8 +127,15 @@ +@@ -126,8 +128,15 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { @@ -248,17 +248,18 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs diff -uNr 04_zero_overhead_abstraction/src/synchronization.rs 05_safe_globals/src/synchronization.rs --- 04_zero_overhead_abstraction/src/synchronization.rs +++ 05_safe_globals/src/synchronization.rs -@@ -0,0 +1,76 @@ +@@ -0,0 +1,77 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Synchronization primitives. +//! -+//! Suggested literature: -+//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -+//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -+//! - https://doc.rust-lang.org/std/cell/index.html ++//! # Resources ++//! ++//! - ++//! - ++//! - + +use core::cell::UnsafeCell; + diff --git a/05_safe_globals/src/main.rs b/05_safe_globals/src/main.rs index bb6abcb82..839558158 100644 --- a/05_safe_globals/src/main.rs +++ b/05_safe_globals/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/05_safe_globals/src/print.rs b/05_safe_globals/src/print.rs index fa6451c75..8ea5db240 100644 --- a/05_safe_globals/src/print.rs +++ b/05_safe_globals/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/05_safe_globals/src/synchronization.rs b/05_safe_globals/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/05_safe_globals/src/synchronization.rs +++ b/05_safe_globals/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 9b526c331..9bb29be51 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -445,8 +445,8 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri +//! +//! # Resources +//! -+//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -+//! - https://developer.arm.com/documentation/ddi0183/latest ++//! - ++//! - + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -1302,15 +1302,15 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs --- 05_safe_globals/src/main.rs +++ 06_drivers_gpio_uart/src/main.rs -@@ -106,6 +106,7 @@ - //! +@@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -115,6 +116,7 @@ +@@ -116,6 +117,7 @@ mod bsp; mod console; mod cpu; @@ -1318,7 +1318,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -126,16 +128,49 @@ +@@ -127,16 +129,49 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1398,7 +1398,7 @@ diff -uNr 05_safe_globals/src/panic_wait.rs 06_drivers_gpio_uart/src/panic_wait. + +/// Prints with a newline - only use from the panic handler. +/// -+/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html ++/// Carbon copy from +#[macro_export] +macro_rules! panic_println { + ($($arg:tt)*) => ({ diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index a16c0de7b..46a890312 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/06_drivers_gpio_uart/src/panic_wait.rs b/06_drivers_gpio_uart/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/06_drivers_gpio_uart/src/panic_wait.rs +++ b/06_drivers_gpio_uart/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/06_drivers_gpio_uart/src/print.rs b/06_drivers_gpio_uart/src/print.rs index fa6451c75..8ea5db240 100644 --- a/06_drivers_gpio_uart/src/print.rs +++ b/06_drivers_gpio_uart/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/06_drivers_gpio_uart/src/synchronization.rs b/06_drivers_gpio_uart/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/06_drivers_gpio_uart/src/synchronization.rs +++ b/06_drivers_gpio_uart/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 1cca2b122..4cf6d918d 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -406,7 +406,7 @@ diff -uNr 06_drivers_gpio_uart/src/cpu.rs 07_uart_chainloader/src/cpu.rs diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs +++ 07_uart_chainloader/src/main.rs -@@ -102,11 +102,14 @@ +@@ -102,12 +102,16 @@ //! //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. @@ -415,6 +415,8 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs +//! 3. Finally, [`runtime_init::runtime_init()`] is called. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html ++//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -422,7 +424,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -120,6 +123,7 @@ +@@ -121,6 +125,7 @@ mod memory; mod panic_wait; mod print; @@ -430,7 +432,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -148,29 +152,49 @@ +@@ -149,29 +154,49 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f21395d36..17ca3543b 100644 --- a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index fdb787b56..82a6ec589 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -106,6 +106,8 @@ //! 3. Finally, [`runtime_init::runtime_init()`] is called. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] diff --git a/07_uart_chainloader/src/panic_wait.rs b/07_uart_chainloader/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/07_uart_chainloader/src/panic_wait.rs +++ b/07_uart_chainloader/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/07_uart_chainloader/src/print.rs b/07_uart_chainloader/src/print.rs index fa6451c75..8ea5db240 100644 --- a/07_uart_chainloader/src/print.rs +++ b/07_uart_chainloader/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/07_uart_chainloader/src/synchronization.rs b/07_uart_chainloader/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/07_uart_chainloader/src/synchronization.rs +++ b/07_uart_chainloader/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 0f8ec5d2a..af4b80559 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -499,7 +499,7 @@ diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs --- 07_uart_chainloader/src/main.rs +++ 08_timestamps/src/main.rs -@@ -102,14 +102,11 @@ +@@ -102,16 +102,12 @@ //! //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. @@ -508,6 +508,8 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +-//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -515,7 +517,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -123,9 +120,9 @@ +@@ -125,9 +121,9 @@ mod memory; mod panic_wait; mod print; @@ -526,7 +528,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs /// Early init code. /// -@@ -150,51 +147,31 @@ +@@ -152,51 +148,31 @@ /// The main function running after the early init. fn kernel_main() -> ! { diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index 5a7b9f550..efcd98807 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/08_timestamps/src/panic_wait.rs b/08_timestamps/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/08_timestamps/src/panic_wait.rs +++ b/08_timestamps/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/08_timestamps/src/print.rs b/08_timestamps/src/print.rs index 5a5638113..89fa66943 100644 --- a/08_timestamps/src/print.rs +++ b/08_timestamps/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/08_timestamps/src/synchronization.rs b/08_timestamps/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/08_timestamps/src/synchronization.rs +++ b/08_timestamps/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index 5a7b9f550..efcd98807 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/09_hw_debug_JTAG/src/panic_wait.rs b/09_hw_debug_JTAG/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/09_hw_debug_JTAG/src/panic_wait.rs +++ b/09_hw_debug_JTAG/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/09_hw_debug_JTAG/src/print.rs b/09_hw_debug_JTAG/src/print.rs index 5a5638113..89fa66943 100644 --- a/09_hw_debug_JTAG/src/print.rs +++ b/09_hw_debug_JTAG/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/09_hw_debug_JTAG/src/synchronization.rs b/09_hw_debug_JTAG/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/09_hw_debug_JTAG/src/synchronization.rs +++ b/09_hw_debug_JTAG/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 2556ec993..2e88c083b 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -484,7 +484,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs --- 09_hw_debug_JTAG/src/main.rs +++ 10_privilege_level/src/main.rs -@@ -117,6 +117,7 @@ +@@ -118,6 +118,7 @@ mod console; mod cpu; mod driver; @@ -492,7 +492,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -147,12 +148,20 @@ +@@ -148,12 +149,20 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -513,7 +513,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -167,11 +176,15 @@ +@@ -168,11 +177,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index 741883437..193ff7a63 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/10_privilege_level/src/panic_wait.rs b/10_privilege_level/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/10_privilege_level/src/panic_wait.rs +++ b/10_privilege_level/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/10_privilege_level/src/print.rs b/10_privilege_level/src/print.rs index 5a5638113..89fa66943 100644 --- a/10_privilege_level/src/print.rs +++ b/10_privilege_level/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/10_privilege_level/src/synchronization.rs b/10_privilege_level/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/10_privilege_level/src/synchronization.rs +++ b/10_privilege_level/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 0d24f17fb..726012e13 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -1006,9 +1006,9 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs --- 10_privilege_level/src/main.rs +++ 11_virtual_mem_part1_identity_mapping/src/main.rs -@@ -106,7 +106,10 @@ - //! +@@ -107,7 +107,10 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] @@ -1017,7 +1017,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -130,9 +133,18 @@ +@@ -131,9 +134,18 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1037,7 +1037,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -156,6 +168,9 @@ +@@ -157,6 +169,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -1047,7 +1047,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -179,6 +194,13 @@ +@@ -180,6 +195,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 83ee1de57..bfc7d5e42 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] diff --git a/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs b/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs +++ b/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/11_virtual_mem_part1_identity_mapping/src/print.rs b/11_virtual_mem_part1_identity_mapping/src/print.rs index 5a5638113..89fa66943 100644 --- a/11_virtual_mem_part1_identity_mapping/src/print.rs +++ b/11_virtual_mem_part1_identity_mapping/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/11_virtual_mem_part1_identity_mapping/src/synchronization.rs b/11_virtual_mem_part1_identity_mapping/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/11_virtual_mem_part1_identity_mapping/src/synchronization.rs +++ b/11_virtual_mem_part1_identity_mapping/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 829ed0920..c5d0f9f1c 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -967,7 +967,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_p diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs --- 11_virtual_mem_part1_identity_mapping/src/main.rs +++ 12_exceptions_part1_groundwork/src/main.rs -@@ -111,6 +111,7 @@ +@@ -112,6 +112,7 @@ #![feature(const_generics)] #![feature(const_panic)] #![feature(format_args_nl)] @@ -975,7 +975,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] -@@ -142,6 +143,8 @@ +@@ -143,6 +144,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -984,7 +984,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } -@@ -194,13 +197,28 @@ +@@ -195,13 +198,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 80706a248..4aa47c6e7 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] diff --git a/12_exceptions_part1_groundwork/src/panic_wait.rs b/12_exceptions_part1_groundwork/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/12_exceptions_part1_groundwork/src/panic_wait.rs +++ b/12_exceptions_part1_groundwork/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/12_exceptions_part1_groundwork/src/print.rs b/12_exceptions_part1_groundwork/src/print.rs index 5a5638113..89fa66943 100644 --- a/12_exceptions_part1_groundwork/src/print.rs +++ b/12_exceptions_part1_groundwork/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/12_exceptions_part1_groundwork/src/synchronization.rs b/12_exceptions_part1_groundwork/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/12_exceptions_part1_groundwork/src/synchronization.rs +++ b/12_exceptions_part1_groundwork/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 9a99cbdf5..cd03c27e9 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1157,7 +1157,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,168 @@ +@@ -0,0 +1,169 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1267,6 +1267,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html ++//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + +#![allow(incomplete_features)] +#![feature(const_fn_fn_ptr_basics)] @@ -1330,7 +1331,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,128 +6,12 @@ +@@ -6,129 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1433,6 +1434,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +-//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - -#![allow(incomplete_features)] -#![feature(const_fn_fn_ptr_basics)] @@ -1461,7 +1463,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -139,6 +23,7 @@ +@@ -140,6 +23,7 @@ /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -1469,7 +1471,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -165,9 +50,7 @@ +@@ -166,9 +50,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1479,7 +1481,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -194,31 +77,6 @@ +@@ -195,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1605,7 +1607,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/panic_wait.rs 13_integrated_testing + /// Prints with a newline - only use from the panic handler. /// - /// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html + /// Carbon copy from @@ -35,5 +52,16 @@ panic_println!("\nKernel panic!"); } diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index 0bc8d3292..d427375c3 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -107,6 +107,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] diff --git a/13_integrated_testing/src/panic_wait.rs b/13_integrated_testing/src/panic_wait.rs index 76438a9bf..61536ba80 100644 --- a/13_integrated_testing/src/panic_wait.rs +++ b/13_integrated_testing/src/panic_wait.rs @@ -36,7 +36,7 @@ fn _panic_exit() -> ! { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/13_integrated_testing/src/print.rs b/13_integrated_testing/src/print.rs index 5a5638113..89fa66943 100644 --- a/13_integrated_testing/src/print.rs +++ b/13_integrated_testing/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/13_integrated_testing/src/synchronization.rs b/13_integrated_testing/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/13_integrated_testing/src/synchronization.rs +++ b/13_integrated_testing/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index e6dd762ec..060a58e2b 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -1776,7 +1776,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,8 +10,8 @@ - //! - https://developer.arm.com/documentation/ddi0183/latest + //! - use crate::{ - bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, @@ -2180,7 +2180,7 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part +/// context, aka executing an interrupt vector or subcalls of it. +/// +/// Concept and implementation derived from the `CriticalSection` introduced in -+/// https://github.com/rust-embedded/bare-metal ++/// +#[derive(Clone, Copy)] +pub struct IRQContext<'irq_context> { + _0: PhantomData<&'irq_context ()>, @@ -2296,8 +2296,8 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs --- 13_integrated_testing/src/lib.rs +++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -109,9 +109,11 @@ - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +@@ -110,9 +110,11 @@ + //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] +#![feature(asm)] @@ -2308,7 +2308,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -@@ -135,6 +137,7 @@ +@@ -136,6 +138,7 @@ pub mod exception; pub mod memory; pub mod print; @@ -2510,7 +2510,7 @@ diff -uNr 13_integrated_testing/src/state.rs 14_exceptions_part2_peripheral_IRQs diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_peripheral_IRQs/src/synchronization.rs --- 13_integrated_testing/src/synchronization.rs +++ 14_exceptions_part2_peripheral_IRQs/src/synchronization.rs -@@ -27,6 +27,21 @@ +@@ -28,6 +28,21 @@ /// Locks the mutex and grants the closure temporary mutable access to the wrapped data. fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; } @@ -2532,7 +2532,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip } /// A pseudo-lock for teaching purposes. -@@ -35,8 +50,18 @@ +@@ -36,8 +51,18 @@ /// other cores to the contained data. This part is preserved for later lessons. /// /// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is @@ -2553,7 +2553,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip where T: ?Sized, { -@@ -47,10 +72,22 @@ +@@ -48,10 +73,22 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -2579,7 +2579,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip /// Create an instance. pub const fn new(data: T) -> Self { Self { -@@ -62,8 +99,9 @@ +@@ -63,8 +100,9 @@ //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ @@ -2590,7 +2590,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip type Data = T; fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { -@@ -71,6 +109,32 @@ +@@ -72,6 +110,32 @@ // mutable reference will ever only be given out once at a time. let data = unsafe { &mut *self.data.get() }; diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 1dd0a14c2..c8026dcf5 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs index 2b9ef05f3..e9aace357 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs @@ -38,7 +38,7 @@ pub struct IRQDescriptor { /// context, aka executing an interrupt vector or subcalls of it. /// /// Concept and implementation derived from the `CriticalSection` introduced in -/// https://github.com/rust-embedded/bare-metal +/// #[derive(Clone, Copy)] pub struct IRQContext<'irq_context> { _0: PhantomData<&'irq_context ()>, diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index bc50cabd8..2d12290f6 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -107,6 +107,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] #![feature(asm)] diff --git a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs index 0b724abd7..8c7770b6e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs @@ -36,7 +36,7 @@ fn _panic_exit() -> ! { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/14_exceptions_part2_peripheral_IRQs/src/print.rs b/14_exceptions_part2_peripheral_IRQs/src/print.rs index 5a5638113..89fa66943 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/print.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs index 6133826eb..fe9d454a8 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index b326d589c..9adbe9c2e 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1190,7 +1190,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,10 +10,13 @@ - //! - https://developer.arm.com/documentation/ddi0183/latest + //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, @@ -1939,7 +1939,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs --- 14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 15_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -110,6 +110,7 @@ +@@ -111,6 +111,7 @@ #![allow(incomplete_features)] #![feature(asm)] @@ -1947,7 +1947,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -131,6 +132,7 @@ +@@ -132,6 +133,7 @@ mod synchronization; pub mod bsp; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index e279c2062..4b1e9c937 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, diff --git a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs index 2b9ef05f3..e9aace357 100644 --- a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs +++ b/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs @@ -38,7 +38,7 @@ pub struct IRQDescriptor { /// context, aka executing an interrupt vector or subcalls of it. /// /// Concept and implementation derived from the `CriticalSection` introduced in -/// https://github.com/rust-embedded/bare-metal +/// #[derive(Clone, Copy)] pub struct IRQContext<'irq_context> { _0: PhantomData<&'irq_context ()>, diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index 89df0dd5c..a509d86f4 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -107,6 +107,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(incomplete_features)] #![feature(asm)] diff --git a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs index 0b724abd7..8c7770b6e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs +++ b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs @@ -36,7 +36,7 @@ fn _panic_exit() -> ! { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/15_virtual_mem_part2_mmio_remap/src/print.rs b/15_virtual_mem_part2_mmio_remap/src/print.rs index 5a5638113..89fa66943 100644 --- a/15_virtual_mem_part2_mmio_remap/src/print.rs +++ b/15_virtual_mem_part2_mmio_remap/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs index 6133826eb..fe9d454a8 100644 --- a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs +++ b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index ce39752d1..d2aae4d24 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -6,8 +6,8 @@ //! //! # Resources //! -//! - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf -//! - https://developer.arm.com/documentation/ddi0183/latest +//! - +//! - use crate::{ bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index fa5ba64ad..4f45d955e 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -105,6 +105,7 @@ //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/X1_JTAG_boot/src/panic_wait.rs b/X1_JTAG_boot/src/panic_wait.rs index 7980f5de5..66fbe6b40 100644 --- a/X1_JTAG_boot/src/panic_wait.rs +++ b/X1_JTAG_boot/src/panic_wait.rs @@ -19,7 +19,7 @@ fn _panic_print(args: fmt::Arguments) { /// Prints with a newline - only use from the panic handler. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! panic_println { ($($arg:tt)*) => ({ diff --git a/X1_JTAG_boot/src/print.rs b/X1_JTAG_boot/src/print.rs index 5a5638113..89fa66943 100644 --- a/X1_JTAG_boot/src/print.rs +++ b/X1_JTAG_boot/src/print.rs @@ -20,7 +20,7 @@ pub fn _print(args: fmt::Arguments) { /// Prints without a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); @@ -28,7 +28,7 @@ macro_rules! print { /// Prints with a newline. /// -/// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html +/// Carbon copy from #[macro_export] macro_rules! println { () => ($crate::print!("\n")); diff --git a/X1_JTAG_boot/src/synchronization.rs b/X1_JTAG_boot/src/synchronization.rs index f956b5cdd..7bb2938be 100644 --- a/X1_JTAG_boot/src/synchronization.rs +++ b/X1_JTAG_boot/src/synchronization.rs @@ -4,10 +4,11 @@ //! Synchronization primitives. //! -//! Suggested literature: -//! - https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html -//! - https://stackoverflow.com/questions/59428096/understanding-the-send-trait -//! - https://doc.rust-lang.org/std/cell/index.html +//! # Resources +//! +//! - +//! - +//! - use core::cell::UnsafeCell; From 8ee50a2f8b483d1103f228f2372d0e6f7446ad56 Mon Sep 17 00:00:00 2001 From: Kaifoon Date: Fri, 29 Jan 2021 07:48:04 +0800 Subject: [PATCH 028/214] Update README.CN.md update filename `memory.ts` to `memory.rs` --- 00_before_we_start/README.CN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/00_before_we_start/README.CN.md b/00_before_we_start/README.CN.md index 58f0af400..3d7e19576 100644 --- a/00_before_we_start/README.CN.md +++ b/00_before_we_start/README.CN.md @@ -14,7 +14,7 @@ 有的内核的子系统是基于特定目标架构的处理器的低级代码。对于每种支持的处理器架构,都有一个子文件夹在`src/_arch`中。例如,`src/_arch/aarch64` -子模块系统在每个处理器架构的`src`文件夹中。例如,属于内核内存子系统(`src/memory.rs`)的代码将会被放在`src/_arch/aarch64/memory.rs`.后一个文件将被直接包含和重新导出到`src/memory.ts`中,以便架构代码部分是明显的遵循代码模块话。这意味着在`src/_arch/aarch64/memory.rs`中定义的公共函数foo()。仅可通过`crate :: memory :: foo()`访问。 +子模块系统在每个处理器架构的`src`文件夹中。例如,属于内核内存子系统(`src/memory.rs`)的代码将会被放在`src/_arch/aarch64/memory.rs`.后一个文件将被直接包含和重新导出到`src/memory.rs`中,以便架构代码部分是明显的遵循代码模块话。这意味着在`src/_arch/aarch64/memory.rs`中定义的公共函数foo()。仅可通过`crate :: memory :: foo()`访问。 `_arch`中的`_`表示此文件夹不属于标准模块层次结构。而是使用`#[path =“ _ arch/xxx/yyy.rs”]`属性将其内容有条件地引入各自的文件中。 From 9d74e021efb2473a6c472b9c167af1df461fb817 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 7 Mar 2021 21:01:02 +0100 Subject: [PATCH 029/214] Docker utils: Bump various versions --- docker/rustembedded-osdev-utils/Dockerfile | 71 ++++++++++++---------- 1 file changed, 38 insertions(+), 33 deletions(-) diff --git a/docker/rustembedded-osdev-utils/Dockerfile b/docker/rustembedded-osdev-utils/Dockerfile index 7127bd14b..47f69e438 100644 --- a/docker/rustembedded-osdev-utils/Dockerfile +++ b/docker/rustembedded-osdev-utils/Dockerfile @@ -14,38 +14,39 @@ LABEL maintainer="The resources team , Andre # Ruby gems COPY Gemfile . -RUN set -ex; \ - tempPkgs=' \ - automake \ - bison \ - build-essential \ - flex \ - git \ - libtool \ - pkg-config \ - wget \ - '; \ - apt-get update; \ - apt-get install -q -y --no-install-recommends \ - $tempPkgs \ +RUN set -ex; \ + tempPkgs=' \ + automake \ + bison \ + build-essential \ + flex \ + git \ + libtool \ + ninja-build \ + pkg-config \ + wget \ + '; \ + apt-get update; \ + apt-get install -q -y --no-install-recommends \ + $tempPkgs \ # persistent packages - ca-certificates \ - gdb-multiarch \ - libpixman-1-dev \ - libglib2.0-dev \ - libusb-1.0.0-dev \ - locales \ - python3 \ - ruby \ - ruby-dev \ - ; \ + ca-certificates \ + gdb-multiarch \ + libpixman-1-dev \ + libglib2.0-dev \ + libusb-1.0.0-dev \ + locales \ + python3 \ + ruby \ + ruby-dev \ + ; \ # Ruby dependencies gem install bundler; \ bundle install --retry 3 --without development; \ # QEMU git clone git://git.qemu.org/qemu.git; \ cd qemu; \ - git checkout tags/v5.1.0; \ + git checkout tags/v5.2.0; \ ./configure --target-list=aarch64-softmmu --enable-modules \ --enable-tcg-interpreter --enable-debug-tcg \ --python=/usr/bin/python3; \ @@ -61,16 +62,20 @@ RUN set -ex; \ make -j8; \ make install; \ # GDB - wget -P ~ git.io/.gdbinit; \ + wget -P ~ git.io/.gdbinit; \ # GCC AArch64 tools - wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf.tar.xz; \ - tar -xf gcc-arm-9*; \ - cp gcc-arm-9*/bin/aarch64-none-elf-objdump gcc-arm-9*/bin/aarch64-none-elf-nm /usr/local/bin/; \ - rm -rf gcc-arm-9*; \ + wget https://developer.arm.com/-/media/Files/downloads/gnu-a/10.2-2020.11/binrel/gcc-arm-10.2-2020.11-x86_64-aarch64-none-elf.tar.xz; \ + tar -xf gcc-arm-10*; \ + cp \ + gcc-arm-10*/bin/aarch64-none-elf-objdump \ + gcc-arm-10*/bin/aarch64-none-elf-readelf \ + gcc-arm-10*/bin/aarch64-none-elf-nm \ + /usr/local/bin/; \ + rm -rf gcc-arm-10*; \ # Cleanup - apt-get purge -y --auto-remove $tempPkgs; \ - apt-get autoremove -q -y; \ - apt-get clean -q -y; \ + apt-get purge -y --auto-remove $tempPkgs; \ + apt-get autoremove -q -y; \ + apt-get clean -q -y; \ rm -rf /var/lib/apt/lists/* # Locales From 6db8b2bc728aa9552e56116c0f01824e4205a6d6 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 7 Mar 2021 21:01:44 +0100 Subject: [PATCH 030/214] Update toolchain --- .rustfmt.toml | 2 +- 06_drivers_gpio_uart/README.md | 7 ++++--- 06_drivers_gpio_uart/src/main.rs | 1 + 07_uart_chainloader/README.md | 7 ++++--- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6808 -> 6800 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6680 -> 6656 bytes 07_uart_chainloader/src/main.rs | 1 + 08_timestamps/README.md | 7 ++++--- 08_timestamps/src/main.rs | 1 + 09_hw_debug_JTAG/src/main.rs | 1 + 10_privilege_level/README.md | 6 +++--- 10_privilege_level/src/main.rs | 1 + .../README.md | 10 +++++----- .../src/main.rs | 1 + 12_exceptions_part1_groundwork/README.md | 6 +++--- 12_exceptions_part1_groundwork/src/main.rs | 1 + 13_integrated_testing/README.md | 12 +++++++----- 13_integrated_testing/src/lib.rs | 1 + 14_exceptions_part2_peripheral_IRQs/README.md | 6 +++--- .../src/lib.rs | 1 + 15_virtual_mem_part2_mmio_remap/Makefile | 2 +- 15_virtual_mem_part2_mmio_remap/README.md | 6 +++--- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 1 + X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8144 -> 8128 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6864 -> 6832 bytes X1_JTAG_boot/src/main.rs | 1 + rust-toolchain | 2 +- 27 files changed, 50 insertions(+), 34 deletions(-) diff --git a/.rustfmt.toml b/.rustfmt.toml index 25a745377..5e98361e6 100644 --- a/.rustfmt.toml +++ b/.rustfmt.toml @@ -1,6 +1,6 @@ newline_style = "Unix" edition = "2018" -merge_imports = true +imports_granularity = "Crate" format_code_in_doc_comments = true normalize_comments = true wrap_comments = true diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 9bb29be51..a4176ca51 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -1302,15 +1302,16 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs --- 05_safe_globals/src/main.rs +++ 06_drivers_gpio_uart/src/main.rs -@@ -107,6 +107,7 @@ +@@ -107,6 +107,8 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html ++#![allow(clippy::clippy::upper_case_acronyms)] +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -116,6 +117,7 @@ +@@ -116,6 +118,7 @@ mod bsp; mod console; mod cpu; @@ -1318,7 +1319,7 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -127,16 +129,49 @@ +@@ -127,16 +130,49 @@ /// # Safety /// /// - Only a single core must be active and running this function. diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index 46a890312..627b5b59b 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 4cf6d918d..91242143d 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -406,7 +406,7 @@ diff -uNr 06_drivers_gpio_uart/src/cpu.rs 07_uart_chainloader/src/cpu.rs diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs +++ 07_uart_chainloader/src/main.rs -@@ -102,12 +102,16 @@ +@@ -102,13 +102,17 @@ //! //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. @@ -418,13 +418,14 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs +//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + #![allow(clippy::clippy::upper_case_acronyms)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -121,6 +125,7 @@ +@@ -122,6 +126,7 @@ mod memory; mod panic_wait; mod print; @@ -432,7 +433,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs mod runtime_init; mod synchronization; -@@ -149,29 +154,49 @@ +@@ -150,29 +155,49 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 83fe7bffab6b25be79d8cd89f4803003add22f0b..13cf8aa1ed21d4896c05b5c25caf8d17c69289fa 100755 GIT binary patch delta 2749 zcmZ`)TWl298UD}AUS<~G?Y$aUJBu;v1SmCTair9ou=F zC6oSB-XMiop*Z|y4*GRY{m-=P&wO6tHarsntIoRxi%?kTM`SW3N$ z`)ok5)4@?Y#e#O~oy1;{gQ?FE)nw%A)THcfnvy-K*LcC+^X$t+gQE|fk7)N<+UKy( zLwg;x3vzAh0sd|H?{<>@T{AlFe;6G37UiPo$baw7&j$fUPE3k{L8qf$p?icW{X zwVna~tn5z(nLmH8@oa$Otf0&)J_Yp%M}2y>@jWBsWQHi@UnLr>qq;wh%3T!`TIE}R z)N;SLGkV~6cSfI_;}4uaYyTP|XhY$SS5R!AOe>_M>VX^p_Bd!{Ms}ylz}MZ{<_#XI zx~-P&xazKHF82hQZ?VdzkE2iblFI+0coOe3SH4c6*iE@K^@e+N^X&z5(^=36F}`cp zT7RjM{*fCdR#26RZoTo)j5%QX7Gk?8yHZQ^5a>d0lqi;j+X}cf$K>9n~B)0#TZ9IZUVP3GAvUpFPEea0MG$;FQPIUX=Bwz<6@U2ilz4FHg_?$ z;T-^-hF&S0B%u2afDTtt>~(4@TXr2>_`Co9?)mtXBv4Dx$0u&GxS@!sII0+m=;uHo)Vd|AG49~QDDzFv9GmHdvVR*){i}rT{_q?p6viKM6ui8mh z(1j6v9{zAykSTmcNv#cu-NfT1HM@A{Vw;iI-2HOYSm?TZ0i88*6Um=J6-9DR8Ms{+T;(^waRGTj6Dq= z?=N{SF|W`_H@fcdPR$51cm7we_76=gqmh1>W`s+$!XzT96Z2L=kwXCJBT5eK!7zu) z^#ZPlY?GGGMkt;AKx@t3pw{d^mHmgZ}lSj*|Lm6kKc%JS3g3 z=V!_rzM658p)@3nFhFIjnuseWdxGiV7d-m30O52jYSIcl16SEboAC^CGu>zxa4v?A zFV{`HB_dZjnd!NZY|LP->-aZ<@>^oLXmFF zt$-IETd51ttq>JRDpv>Ig^1>$h+qqrKogDzg@uL6K+`#}hi%k?i#w@4)lm?8lV3|u zQ(dHuX=ztSM*%agyE=G#)p|EhJ-CfdgOX^W-Y>C%KuHJxVO6A+8uY#KkiOgA*ZG4? zs2G;ZTr8@vIbiGSR9ia=O?EB4ANO{Xozm+4zD}i;->urxyrH8IZt1{GD0vkvNzlNc zCyCbRyP?@WyhaZq6{1CoaYB~AT)jLn3;6wrNcrULgMCgKaX~4E{A@kP&sU$VLXOw? z;Rw`cMBL|A^Bpzy%*9`+!E5(#W-%#atN41irvxuoHeolWE|ZQLxNEkH$VM9$Bhyyv zf)R(?V!OgHV||^JY1GB#lfk%6;uHVMD$PJ~F(3+PYHB|^nn)`|`6YLV5Bq#S_N?)L z;zV*jvrwVp%JQh^YNh7?Pwp{Z@AYojD_&VF^k;D$v(89Zaec))W7djmfba9JPrT?` z)T*$k*Dc#i9-|f?9a~)MtTU#exc*z-XYrR%R9I)cq{Zd2&K4?(u5-Wd8~$$x7Gd;T z?4#ECB7e&lRFAqXE+6xKQH_^dka?v)BIW#itAFQ)CE}kg{FRnJv1|B#Z_&#w`gMNI U|1GBRXdqIWw44=p{z@S9PvL!Hn*aa+ delta 2883 zcmaJ@ZERE58Gg^bcI<1M?>Mmo#CA+boYHn9Bu=K$+=Pz}SjU!nSy9L0u%RYItPQYg z-9~NbCV@|A8H<^|HEM59Pxr3@gp-VUF=r5`}jZskwua5`T#)Q$ftnSlpq(NwvcP zN*jl3w6yF^b{5lc&~>slXjPBALsa&S!NK++#TlQI7bhdkdt(lmG`?Gis#0=gazd_Y zo|LPT`-zT5h}wAC`orkUn2bHL{YQBJ{}(=M3ZFHFH$wQJjkE!y>w5d6uInBvX&pA2 zNfCu3_1$`Zn>uPQS98QVZ>WZ?ZQL+wUdNhAs5sAPJxGkq*+TrTwd%RltSz+Xo87t# zQrjukbH0dXMl2K_s8Dk$xe8`@nhVm5ReyrC1??edueS1MY<27mf8Mq@>bnm?z1{k3 zQ9%&iGd>1iJ{DAM3bM8R7J}ff=HHm_*V}0Hr<2ri9CJH-uHdmFUcpmtE#3Hmp0Y7r z1UfCzQ6KsKIVShrNOkjTzUPwnA{YK~kNQ)5*7kPvHMpk}arIYGI9{UWl2TI$we`kotfKj5*V^HTlfrMY;_)HK(U}sz zB|DSCA$_cPDz=Ei({ge0uxqUS_Po({k~yZXETsy9o$p)KXvez>X)`!*2MTWPlZP)q zGQw6Uy#r%sWJgl01+dmBIi^I+1et_M=H^9tc+KYp{M`jS)_(U{0pInh5x$4@>u}OE zjOk;}DY8*`3i@_}(BCZE=h~uR^p45suvP7N3xqy~ff&xyCdCwFAp`xyK3 z5Wce@DdRj(DQMt-K*dp^K~_25#O(1bgho-J8983N2-zd5bZB3V_AV-Q zCo1)jQXHR^?a3G_RTNVe-xhFlAdT%Pq8hw=`HvTQqr&1CO6k`qn+aGjiCVT7nCbV>(2mX-Vx!e?lGUOPIINXy)`A`UVj>1%(d&gBA8Typ`)@ z?kPX!_FF&Setlraz?_|_xgZS51W3=E-m8B^biM@65%F7CTM(o^A?PB)4?+JWM|yk~ zom$#D$IliItUeCLyX#4t1>?x+s6K%z7wsk@QD~uqqypVKi$`_$8eqFgTZl%Cf2U+~ zAY~iX(JvBdoDBL`YKVTqi44dBJ;c7M9qL-bD@%imTjq^pKJ*Uqsk(%Cs^q;vb~T6C z@b^oWcAsW+R9riSR#}zoHH>5TH?P&_7ppmA4bwyn)A?9YGKC9g5;5pP%zQqugFgQl z-%%Qnq4{sg?rUbfZQ8Ivk#D zvyV(eTt-QYO10l#xp#c=F&>8f5HKB7O@IqJl^6$R~ zh+0G;vHh6NDR2I&7kzya*8NMSTqa=Aji^=4VExE)P2iq4trs}PVm+ib<+D45!k)k*GTlR@=C&fe?aNdh;X%sV zv8f~n_3zI-musGy@|0->{!j`m`Kad=_973~hWM+szJCLXxuvoI diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index f2dec5cba4e85d94d9b3a61918b90cca5c7be02b..a0b618f8d63c28f5a04c2702fecaa5f14a043cac 100755 GIT binary patch delta 3803 zcmcImeQaCR6+ibq+j(}J@7Qsif}NzLiBk&2rEWkhFKG}ZW3)>R6RTL#R%mIv4y8qm zZPo0i4AQ1Ro^CcqwT)0T(PyU7N$r^T$8cIEjSo?EW%&nbIPLzh<9-lmo0t;6{qFNi z*fwThJ?`?@Jw4g3%ZWa8`02%RkdJpnM^J)Wx&+xEZ=bVjs9>8 zQ$m5x?!h?1D3nN`2ZHinL-lI%P2;9W3!~(KL(MC8(gqxq7oa!I8PPO!6xMa>#$H}7 zHOI)KrO%RfMkej&5{(rakX)H*BH@K~ujv`U771@8X(TNxiMIbbhF>1S*#A9-Umn7> z|3eHX4vQ!e(Gf~{c%S@)Z=y*{BalpSLCM;8g`m_gt^|DLO!=4Z~G+Y$VTSOZN)kx`yvBp)b>Cb4Jw}J z{mALglpP1rPJ>gJ^Ku{|PHtMRNleLgp);~K@g&dNx8MC8qOsI`$>*fwmh=mlXCb{D z(s{WdaTWh<_&-oW+7FG+)AjS6rys+AA5S_O{O=e>-D9iNRd_W zk$b!J%TUrGdvfHU=s>laPs_D{4kF`Y4P}I`|(P ze)baIRMHTmOHh7)mp%vOhUCksh|A$?%Do{U1QwJlC3Gsl>T+8?G@{@SeSZ}te{hD{ zp9TJmT%QQA`rO>o=lvXO1^IL0Q;?5V=G8Jw|EQ-+m@X3e9xB3Wf0dJJ)pLJQv%mT< zwd3^S4)1;^-XxtfyC{pr07VwnomeIBUu34?x3URWluu1EzP5@Ug4$cizwWa zp4xDQ*B`#XszUFhc*)8Crg&qoF;_0AQ1qN!mUsqT>0-fH^*r_nKK{n8w*T5m+B;|_ z7tyT@G@-L-4i2ck0N>8ZuEfHAGxmjC3T=55W-DRVKx5kSi4nDf{{xF(?47t;WAqMc9v40+R(D4!?R(-63W{yew5AKro9mI*Ad4cb6 z1)PU%bX1V<&XB+|QL-9;t7RSE?xM$Jc&|uBu??ls7a0(;%ynMbi zz!ZL+yTOLQuHyaf+Ks%oc+3sSlMBN6Ve=%4>hpf?n2XMwk?U~k>T+qEx=ZW2^ef-! z)C+wl%nou6!9R~Iis0;ga^(dfa=DCXu0EsqwF38_ik z%E`=ewLNndm*;YeRC$+fJGhk=k0VjI`R-dKmGiq#4{RHlQ<$1>6_<1~dL^zf{`<0lJ4AiY0^ES)8=)nt zK7~3IPT)q4%sOaBL_6KwrSAj1(b=i{)*w}7YF^ZSKYy@%T{{I;UBZq^fk|TxJ5^|) z{H1j^J-oh4XiJF7mK^ckpE4Q~_g=A^)h52nJ1c6M(W56}^EJ0AF8G`>n2mQ5Ot$=?u(`LBDngp2>jGZ^DpH6>b+W z6#XMOh*!apvCRQ3HbgUk6CFn1phi_!s)~;6>VngTaTHG#y-4HlRMe>xHE2&ZT*{>3 zIdG(>(~q@XUkgjqh@pYICX35E+bkDRr2JYMurJIPp}xYGRJMdxG-+?Jc(1segd#8e zn>-H1?@4&t(de#2OEvVwQDvfmf3LFX(+2|6m|;8{ucTW)k|}SpbS~0F{ybnmcD%{N zX2k6J<(Na*A8o|r;wntc0h0*&8O$FbZ2jeQYYXHuHIzprM#qMhf?k_&AR*}|C4JT~ zKA5ep)IQ!z8uwLsSQlSj6$oB?sxa+cXi2*$vJqOYLW@XYobRYw_7Iq^oJCIY33Dl7 zPNB#E+QgMU(m$%ByvT0ro<&-Zfbf#6Pa8JfIBnNWND;~}Ku^C|)bR>*s1Ax;bDQx( zRoyB9M=o+40>`+GZWTdsUhX!7Bi>=t6pVB*HR+n~&ZF&Uu6Y~(YV~bmc>qf{md{i# zza!P8$&lOZ>nrFK6A-kF!vBi70;UWpyDXGlxLUrBN zYJ97e0Y~EV?cx(@DP!ZFmvR*tv zO_~wIamA6i7Sm=#3<8&NB*#BCRu)lO{>Ph({?lTUNBBPk6vBVxGNIU1+{#K~424E` zlWsR>csagAV>r=#nP=Gpi}7F=qxk#jyizLev{;l+SaIF8=4-+A_uRZ0WcY;V@T~Wy z*q_8>1f$aYCAzD=xL&~UTZ@jGF~={hHOkf;pXDF6=nE*~)*4R-b39?KE%XxI#hVMO(|`AFg37e}rz4x;20_mC>>?R@7;la8gZk?9dY0lDIUv`fVof;x?wUc+KlTpO~T=7GW@>qAkMM2*d&s`js2~l_hu;)ZN^#sRC($EnD>ol+> zz^`m64jMB3iw&7pk1I--+`e2xyl6rCl^}p47L;mx78N}!&8vUv{QHw zWH;DpBoicM1GC5ZxY0t&%6h6gPO~A5l^M~QbTTOyg#K5VJ$p!?$o&>6++~+Z8FW)+ zOF31p0`RTp#i2de>@h(PxTUt=V1Jw26Xc5I)8f!kb3etEV4usop*Slx1TvyM`BhzS z&*_nPg(t`lcL8(N#4PCJ3XcG^x55?7ilqS#bePJ9?PU+0Smg=ESxG?dSU=>&fqY-U zeuGzhe{Y>M`mTOQD9&iK#3JPe;dP0Ha@_bags+21;Q|xc8NKz4uGges079*di9MHf zkzIUIsFM%5mDGo%Oo*f$5NQAWVq}5H3`nDmMtD~ENr-1fmfY$dYx~6#svHKf<6=p& z1m>)UIWmrn!n~D5QrKrBWg9rTzk(`zz~=_6x05{^!`R}I+J>=r0(|zs^%t-v^UX2* z2w<}dn+2lAL1saTZC;OT1CbeU#1E~)p;PdMV+zX7fXA)i5z#ggWxd^9tW5s z&;uT0NK3B6h;)ERZdmWE9^s^M!ieZ3MXS{w@#nR@O-c%c zjf+B3mzkDsCM_ci+I4kKTqF2u)L>Dj$WPOMQhk=F4DfH+u&047-El+QcTl&`^wZ37 z{xF;#D3)@21bh5ZFzCnpSW#`Le?khw_>)~jZY3=?B-=}A$nSpD8hSX~o*WIZZYv}23c4}^DZ9^5GDyHc}MHK0)lyWK21vlyfd2RZdH=(U? zdI+aiTh!aFOW85?0qbJf`zaJ{?$l(rZ4XRN=L ze*}8k5m%p!BJnaQmlUevlb|kMOe0rCcXAQ-TdJkD7m?soVny--lJXWB%#xKNS)8{G2EALfWwup@MaNBx_My4%g+cG&fZ;8khh!Xn z28Y(eAYQgtVGv&vMT)!yxiRd$hF*FBc-|5@wgT9X3RrFoBXyH=RSTg??2Ja*S(PEj zM~+VpOt$qfcIO~|XJDiVc{OFPWvA7gy}|x=5smPG+^hN=Eq3>wgIAa^)a6z;J62X< z*28SsUR{suMm_cm_Va4Ov4x{_pa$LQykq5eG2=dRR2&uRXI1A{Fk2igCMlyrC&jp@ z1lhx@ba1CzIe|*uhDsf_m&RvATQY`9<;9f2?;_adKpNXnL^T*Ut3NDhkh#Y(l+rIz zHskPM0<~-_HA{+7NbT;_K0}W4LWv^oyN&$j^qe}APd4z_abtTyYz+;;W3EtmQR`etqD&ivsq*+kmpk>|B@M5oF?4v$}9t)NJ{ zMbT-5ABz4%kFX$%PAzxfF5LDSB5)0YSkkD;^4e~z>D5N zUP%)WPl*~XlTFGYHtOqTOFCa*w4dL;`Bv#lZr1e!pKe;O88`8|{7sxTVnxXmE}RL( zpaU`UdSM;T^Npzki##&nCy`fSF;f14M*9gwWP!4T7UGg%{4#_P9Xe%(5N1Q2E190x zN77c(VvVv^a4`J>Q&QGq?%qj%hYPK+NaJoA*V4ELbBlH#B9v)jnvQ#KQe+-wvPtCj zBz)1uN(vlg=JI}@R4d9?1THoz=UA*ejhm8dss`|KCp7 zdE?fFR|!+*z+bGK)_x*V&VY>9D?B~$>Pnn?tjRD=mo3_7a&i8A$lk;evv|eu-aUbF z3Ob@?v~~sL(pAJ?V~PC}&DeJ(t?G373O1p7DjK|BIrcq>q#P7_6e2ev!o%~d`bb4f z0H7NmB3}68y5HowZ4~N5@3_%J+9x%X<9XrtEm2>uSZX}5j^?4q0hNj=wBX9&C59`U z-&&S4xmw>}rJNTczh zSoZ*Ktbm0^ZCJBYvfKVo8s)eG*4Dd1F+9iEj*eM2os&utn@<9s@uNE zAF=@c8O?Hmb7-y(q(3hg{VcmUAEex*`e2od`P3(>WP#nL{;O)KlSk<&Qzp)mQm~VW3)dxmzCe({fr%cTq8cT70QWn1V0=|I2ZskY8D5 z?zqf(=eh^H-`cqP&O5yicXWfcw7Sz>!=}`i-G6@MgW7@?LnT%4#|l1>YJB(X*s*;F ze%-9jucNw*?^fJ3b04kB`1Yu$tM8Dnn%c_E)$Mh8EuhZ${$}o@XpHZV<{Wive7E9H zn)?MSj+ygf6aShyH?0?FyE;_!WY?^DtheB-CvV1m=KQ}A>UHOFtIaW0sW#Rw_Z8sg zp;YjuZ&4n8Ozp1?v7PGS+V=m8{ejBrYAT|pfq2Xr-*(ksx1D{UzFgO)j@LE(8&O-i AnE(I) diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index 82a6ec589..4fc6449af 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -109,6 +109,7 @@ //! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(core_intrinsics)] diff --git a/08_timestamps/README.md b/08_timestamps/README.md index af4b80559..43c165386 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -499,7 +499,7 @@ diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs --- 07_uart_chainloader/src/main.rs +++ 08_timestamps/src/main.rs -@@ -102,16 +102,12 @@ +@@ -102,17 +102,13 @@ //! //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. @@ -511,13 +511,14 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs -//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + #![allow(clippy::clippy::upper_case_acronyms)] -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] -#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -125,9 +121,9 @@ +@@ -126,9 +122,9 @@ mod memory; mod panic_wait; mod print; @@ -528,7 +529,7 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs /// Early init code. /// -@@ -152,51 +148,31 @@ +@@ -153,51 +149,31 @@ /// The main function running after the early init. fn kernel_main() -> ! { diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index efcd98807..14e1d0ab3 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index efcd98807..14e1d0ab3 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 2e88c083b..0d58e281a 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -484,7 +484,7 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs --- 09_hw_debug_JTAG/src/main.rs +++ 10_privilege_level/src/main.rs -@@ -118,6 +118,7 @@ +@@ -119,6 +119,7 @@ mod console; mod cpu; mod driver; @@ -492,7 +492,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -148,12 +149,20 @@ +@@ -149,12 +150,20 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -513,7 +513,7 @@ diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -168,11 +177,15 @@ +@@ -169,11 +178,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index 193ff7a63..181a9aa5b 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(panic_info_message)] diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 726012e13..7b822f68c 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -1006,10 +1006,10 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs --- 10_privilege_level/src/main.rs +++ 11_virtual_mem_part1_identity_mapping/src/main.rs -@@ -107,7 +107,10 @@ - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +@@ -108,7 +108,10 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + #![allow(clippy::clippy::upper_case_acronyms)] +#![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] @@ -1017,7 +1017,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -131,9 +134,18 @@ +@@ -132,9 +135,18 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1037,7 +1037,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -157,6 +169,9 @@ +@@ -158,6 +170,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -1047,7 +1047,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -180,6 +195,13 @@ +@@ -181,6 +196,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index bfc7d5e42..9f755c9ba 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index c5d0f9f1c..53c953df0 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -967,7 +967,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_p diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs --- 11_virtual_mem_part1_identity_mapping/src/main.rs +++ 12_exceptions_part1_groundwork/src/main.rs -@@ -112,6 +112,7 @@ +@@ -113,6 +113,7 @@ #![feature(const_generics)] #![feature(const_panic)] #![feature(format_args_nl)] @@ -975,7 +975,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] -@@ -143,6 +144,8 @@ +@@ -144,6 +145,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -984,7 +984,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } -@@ -195,13 +198,28 @@ +@@ -196,13 +199,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 4aa47c6e7..9dee3a44c 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -107,6 +107,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index cd03c27e9..383b0a0dd 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1157,7 +1157,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,169 @@ +@@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1269,6 +1269,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + ++#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(incomplete_features)] +#![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] @@ -1331,7 +1332,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,129 +6,12 @@ +@@ -6,130 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1436,6 +1437,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - +-#![allow(clippy::clippy::upper_case_acronyms)] -#![allow(incomplete_features)] -#![feature(const_fn_fn_ptr_basics)] -#![feature(const_generics)] @@ -1463,7 +1465,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -140,6 +23,7 @@ +@@ -141,6 +23,7 @@ /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -1471,7 +1473,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -166,9 +50,7 @@ +@@ -167,9 +50,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1481,7 +1483,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -195,31 +77,6 @@ +@@ -196,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index d427375c3..3e0a9eea5 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -109,6 +109,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 060a58e2b..c04cc9ae7 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2296,9 +2296,9 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs --- 13_integrated_testing/src/lib.rs +++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -110,9 +110,11 @@ - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -111,9 +111,11 @@ + #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] @@ -2308,7 +2308,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] -@@ -136,6 +138,7 @@ +@@ -137,6 +139,7 @@ pub mod exception; pub mod memory; pub mod print; diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index 2d12290f6..fd4bdde51 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -109,6 +109,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 4f7561dbd..206b3d970 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -170,7 +170,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 9adbe9c2e..65127bd6a 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1939,15 +1939,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs --- 14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 15_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -111,6 +111,7 @@ - +@@ -112,6 +112,7 @@ + #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] +#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -132,6 +133,7 @@ +@@ -133,6 +134,7 @@ mod synchronization; pub mod bsp; diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index a509d86f4..608d4b07b 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -109,6 +109,7 @@ //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_fn)] diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 4cd4401535b2b3cb1f5cbf807c72b1f69637cedd..d4148b720f230a78479ea44cdc8791b9c28bc753 100755 GIT binary patch delta 2920 zcmb7GZERE58Gg^bzVUUwZibe&nNFb9NXMP(>0{!8N6wL|}7&Y|hw^RWMeX4QZ;T+F{b{m=MBPiA$5a_t=*e z!6t22x;poq^L{_i``RD*BmTh!M53-sB6=NY zMfV~)Ap$+cDg_TY@TkI7VUVb7yLeC-^Rd5+s|jH*6dY#1GHo_h!pvn>B<)->=A+&o z4~oE!?mtDu_G@a<(T~-{JGUcyes?>v`VKoO)pKK?NUr7>fxxaT zHRUC?fo~B@@0IOZEBNhX z73T8bWnC|MrCtp`=*N#9BD3a(ZHt<;m}n18h_;ZNr>21TuK}L#f4WKgCtCH3_9TzF z)2gPTC=(N%HRrQ{ayhrIuRNUvSH0-@?th4)1f%$Y8Q2-Ki@d`=G?#CV`~xjN*QCv& zWnFM4RjktKe_M9cI5Fn?mSr&tU6Nk`kWrfLT5KIE9}&$(g*ATWr56k;IynAo-B3q2sA z{4GPCpIy9{k?r|h56l?ndCip3#sw?beF?XvD{CmCi(vlYu}6O8h@eT|J}KW35o zj(tjmt`>=XmR(zbyLKWcJ9ItJi&SSTbH2kRbd=aFv5}nGO}OJOxI{k^UnZ7I?MVJf zvA?VZ$v=d-wju_&3h@de zh6j^+8hDP?dv=wXv-65KW1TprX&J=GGpVzx?8R~fW!Li>j^=I(-j!jr6WO=_IaO1L zJ$d2By~nQQz-Vu_s(EwOR1A}K^UleIhL)?wo!TkydCq^2z1VM}~FvE-ZqE#O7N@~haoN3HCVhIm)Xo(SYMBsceW-CwqXuNY* z=d4U{EMfIfAbsEWUkq(S!`u*O0Y@G(Z%!|9yPGup6&b7;s$%A%ecs_G=(MiinvSidx6TQ4 zW(T1+mzuRnGo)(fn#WsSyL=p@7)0+QMYZVgN|7RTFJjY0i^&=0vMwV&_Ow+Y73?MJ z8WLk6Yps2_g5G>z)DlY}#S?n!wX??yN7$rQ**IKF-<|vJI5y{C?^(ILWp3a2*^U~@ zV<+o#ML|86ulyXsO^5>j zw!m9$mZK1U!r1dW1^4rhsL$riQXu>0E4(77&v?G`|5zMJt&~|;$*Vd9dvhPZa z9kwe7+y%SeLF#D>)<01Wsh0EkUh74`I9c$Q@5*OLZzF*?cYkxpxD_2=*s8n+mBx}0ZJw6)s02arD#St|Q znvDN);z3E9xQK1V8lARhdBEMPGrWN{lwVvhnu;ezNOv0yJ)Y@F?EM(NX$E{n)h4i^ zIBGG`5-OYLMN(5o=2*XT;kk&jniTC_h%O&^d@NPkK&n|?Swr3L%QJUaxb#zN$RhdE z&%?SBJ6fhR{=Zt63@?R-)lbf)%34Y3kdkq}aY1H1ZS425js;5gCTHe!tCRIFc+zsG zFth!L{bfN_oyV2gISh+FV~?y)KTczh^D6zk&gxuC>#B|H{?}|DW7IW;!lkTl;g-7l z5y(cEMTTv@s`$X3a?`r6yEi=3+{&I&s>xnLD@puC&$`G->q!k9CIII{w}BQw z5gq4%4zOZX5qam|6>g~d$PnAXy{l@gXLq=IvY#DEJHjrhx2l~m^E+nNy#8EUJ#~_2 zXt?CpVn|2C5aKnc;jjZ7e;ndkMnhiq2gh3Je(R^#??YRNozc`NU^g}SZKqV=+o=^| zPGSjso$yg7?!)RVSI4+eXFwAhZxqrnXP4Y2=yF<5<~TLNp^pKUZSLr(?c|MtIWAXf zCHB!dlnLYemjEmwE<+mQGRwv}oAe##(eB;)GQi>fdEGxzhyOe3Y!Y=giCTiF9Xg=x zvS*}r!84Mr0b0ivw7J=z%W~$v8X@>3rwiFyS*0%1+dr#wz4>LYXhq#x=xM&Fg_(Xe z_}bHjSctQrmDy!L+@pdlS6@J+B@P#I*pOpX%NDgcZ9l=Q)S%bgu9aVDA`Lt^llGp1`1+WJHHhUwJAx(#}Y23n7)A_&$OB9O^Tp zA&U&NVuPLktr`ZESg&VW3@do+`^Rq)Rac{x9Waz<;I4(}qiwS6>%~s#Q1MsUuMFQ* zq-2!pw6e04@>g)Dn6F3$uy>p!<7z3<2PP2IkG&J&0y!zzPfGuEjuq0g*h9Oqhd$s< zfmu#3^I;(29#OA&KoydGkF+n#}a#Un;_bWHTvm+-s2CM z*_7cIjNsafcv%2_vzLJ$-vKnMD2>Ljm%|5IE`?nX46B5?@NK9IU&ehE1S6-vFK$#~ z&SQ6>Jss)#0QwnoldNa~?)6jO8)@I!KBp%*XRr$+MIc5_cZ;6`Tuen@mB1&@);tD6 zNB;$A7!&t2km|8A5yPri=h$CN?Q4#q#d`~Z&Z5Pwr~Tp>4nw6;TUFp%YR*uRS0n+y znAL!?UZ6|O+{5;#HWh|+L6JxQ!>H-J@jS)6^%*gD6DZyJ^B3p`r`X!GGRvU_`KSXY zEI0&Fh0{O)`zZT%T5)_pN0)~4rnGe^x0ox$WR6hG0K1u*-*BJ6VP)zk#+F5zUzCqN zXj&)6CS?j>2RRoXikBnl*!BX@MCM~U0J?&m$0|Zc5;dx zHb3 zwRG|`wYcm0iRR9QH%4~7Hn3-Rx9P+$FlCM1XHPf_R?xF}ZbJFqbNbxIg*_wZ#>%K- z&&0PNco608|5PX9KN>5~$alVqKJ`45s{!w^-TJVs`=GU+z7~3(wrP5r-)pU+<7ZU# zPpm8Bl`TRkE#P{Zy+fsR{!t>Zi%p-28K`tn-P7Fu=NDtVMhI@ifyZkgsO;?@LC`D9 zN8@*JteO821lPv*EXQK6^0=uJe{SquoCrR}3@n&F5Z2H(MD&K%(3?pnB?O8r+huX8 z3M<)BOYwGnv0rpzT0=U(KUBTsz#58I#k+($F?Ukahy1iuyt)nn`@D>P!bT<45bXvG1RZ_6@15yKtKW3#M# zI@+k2gIS4JExJrdPczKbDOEX&vjVHlE-CLar@FCN1+fQBI`LW7ZBwS?5!-X70P5;7 z(7i_>*voF%Hdmub?Ku!se2%yhrQ{?ppwDVCbSB|%xCpxl@530A@hd=m?5mk>)5}jl zZB?NBNvKiCt9Ae(Qw&38(_>tfAO{s4&OsM1}RQrV8bcmpT=M zm}ybY)ITGL7)9(XJ^IW`S$I~B{}O}P@siUb)Oyu0!&{;D6i#y`AXq!`<%DO!1=XMh zENEO8`m;)j`Pgza?1{CVeUa@V8<;uART<7r+$H1zHYH`TY2xc?-D0*c$L{%mwW3Lu zWl7XW7rw}8BH8Ed3G(*5#Pd>e-^V<8cEfjViPK&i+nKkXpI?^P-ebYMlFEks#LkvP z+@9QH2gO%ya*wkyz78^TensVmq<8BW*tGmJBxGpHOCTCat3@)VT{l%p%2pWys}icI z?LbZf>k^W7d3tON_&`ck!84_zQaUyLFdXO?+NvwfVA8(ez&6=2fdXlX3(4E>K0DCC z+G(0*>G{5Q@9&&@&hMOi&i!o7sWn3jLT55)zl&UdjtH$6^*wZcu7znuBq#Me8m%K! zXD?A;!bi1L)a_qFR9r`{&P9v@Rl71biDt*D7}mu6ZhuuV`C>IxyAil@j+5oiUf}qe zgvYxpdsDMizRL4hrf(85abO`6^2OFsZ4g4a5USc$xQT-tUF!81#g)=`jNi8ri#&MFjG#7W>SKgl;lRY&**_Dbj-EXYx*8F z*HjUT*p%oMQZ5Um95d1WxpD+oUT;BSD9P=hp^66U#hm|{Z@THV{^t5y77rxVE*Yc_hQ8}R-(iz z|ENVROpAiAb4vORjJF}Z4AM!VJbVZ58F&v`V6R?3UN&1l{u18ps@qZ?`b5`DU#gZz zzqwBxc8JVaYT6bxXcK}hd|4DCUHU|skWE7YQ!R~!8 zlN63~>H6y9nRMQRop1jeD@tL-FD#^dbVb zC!YcXxpFdWDjJAmmyS#tnSy`3Fex^9?aC4=ypE&!CtI;Euf*ZI#tOWba6Cz(z-uh$ z;|}Q1m^3gYWQT_{C~%rt6E2YgSB0GLZ%~M?&*{}iFh{Wb9kX2fwnWOOHVPG9M}^W+ zc&?xzWMZ>xF!`!r4bRV4V4e#`P>P3ONIneFQH-|S)dObWzewZx+}S%RY|qzv;6{3W z9ByBQBV5O@K!F4d+yR4ju$dZ(t)P)QR%9sWqY0HQye+=Am9dp4VCf_TbFfJW!Y5$p zaH0CCR3biCNP#KnRlBWr_J(@U>XeRW&~l0D`G?_qE0TWUp-x~&9$C2&a~=JC8=(* zC0uZ`&qyVHm`WZ>QeJmsLMhUwG!aGGJgDm`c%DMMLWtp>gl+_$$LbxMtXix&MVrAB zkEww{j4CE|wbJ^y7y)(z=WrzZ$vkhn~YZt;kc7IQ=oNJc|B18apG`#;%}EF2lEvM&n28wKzkJCAz^$ z+*&9Oc+7ILj@gZv^N7VOs9lR6#YT+Y3XV`nQ~*ji27zur_G256}`B^@9|b+MZiv>&J{+s@Lr65y*5kX;b@iTe8R{ zYbMOdEPz*TF*eWFp@7XBh zlhEE#zfKu!cc^!AJz~$Se!zv>kGeK*Nvf((&R6s&Le*IOX1C;>5(bB(f(bHOR}N=~xn)f( zNEObV4Io0-M{sk|Qfr|~ZJvkN9;QSsnTAK6&SLbK476=haA6o~X6n8IXQ;beiNd$( zyD5J*qedfbjhme=q)fR~CtRqCuDc-N>9^2`RX*r%-g&G`+-*H9O6Lx3Z65f1YthKf zxcM(P`^D%@>ul3_)9!PJt&vy#;lcO2JhM-XSGld_lV?du==v8O1^LS4a?-Ya(C-g5 zXCLiKTp{Hu76vE2uOxg-%i0}2JavndDW6H}n0!=_bMnSLnnI{C|Bz!IS9_C zd>PvI&kp;yZ#mu-FQ=rq`|ZQIvt8ps7$)t;ZIHX!vk>R|`SI<*Px{Q7cv33AR$=jQ z8e+c8P$~$eAvzvN;WJf(8W2t!^&f8=9z4_%V)E3YuXS z?kai+ITSMA=dA))LH$xXY8xtTgqF_!+;( zuC5ol@Ft|^OVaZX;$})xa;N~tZ%}oYE?+6yQ?rP;@ykko<0u=y2Y1g_drGf24>?j& zZE5uH)8BpR?{9U{mXh0$9v74Pf&I}%^YHYZG{5CLqP|=90-B@S@ic2wUv|{woJxCPjOs~8#XkYGY;9iv delta 3619 zcmai0du&tJ8ULc}vytS+Wn_Ds=6x8;)4U~N;iRk=`rqQj%!q+Pn~ajbLyVBz>!9+c zrBr?ofbR|R{rj*P)dzo=NFjsWUZGBAPC_t}S?cNMu>R~{Wu53tL^61ffs=YDJ?1{8)?03N++c#jp z*2Q0Vtx6jIP`lR`<7nJ$l(M~u+H9mObAAHH*UCx$0uz}zt@)IuIg@YzM=ekA;fos2 zo!@V$ibN~qIw=1Cqk(1FDoWC73RAj<_BV_D%QLoCB{iRT;>EwA21_B zaOXa_6M@N2xYt-D`RfFd`@qR2E0u@AXFJvdBxa%*o9t4{5cZCO&oDxN32PCyIe{M@ zY<6I?K-3h-EGQ8cwNM|3OoJn4)CQw7h=pMa%1(jD=fERQJBMLBrh(}Nk1_Dr!zda>j5J0V5gjARsf77gxGCkI{5k_y zsk~YChH|3lu_8Ck{!y_SpZ>_hreCSdMi(QX!}K+C-p~`rVGV zs@+Cu|Jy`MH+Ofp^oo|)JYSROGLZR2V(laQ_Srv`x3YriLILwu1qGk%s@TG~@QBPMF zwdS`tQ|2t{is0?9j8epZR3v#80@q;R(r&*0G;-Srqu1c_EN@D%wR)@t45myHwa%Q0 z>&{$4VQsjT$6qhtv3B_1d3@^^TIdGWClUG#f(dh$43$tQ1ABdN=))q*d~*aIy`A$& zxhhsTPr=4A*k?6&1@};_X^e6GEoFXo>6{hdWNAipn4sO$K3cNX3TwtilKJgob5~lC&bVT5)stSG#{%rqn=p( z^mbLqM^Qt4sG%XTG&aWzi70A_WqTUGi{P6H87LqFofx+%hs?E{RXJ&{X^=SbDZw`N zGo%~K=1v^m-dw7&nk%H5wnj;s?I^@r&!=14)XyJpRTsmjkFZvt9J6e8IB|DvM~c0$ zO;T$)N9NrQTZ()je~kJWa@xrpk5Ki%a*3~^=yTxF&XbK%Oa$?ZMc=LPgXcC}B}lR7;*sD40n zdGt(MgU36TlRO8HJ4Ykx6nY^m zR9qFlLUAVR+f*(VQ7bn9+eUJsn4`+>C0py0CF81y_*0N{pMOBcxy3CwyEM?fzkX02 zs;f}~Wp4Za1?^22o-yQ-R0G<9M70-{@0YF2cj$DXJ8vr62;dzzobvZ2cWrQUbb#G{ z`Nm~W+*;J${B-L^HHX`fzX?;g{osBg$s5pfgv7rVG^fi2*WpmzNR6T`t!l&l5T}Vc znTwMA5*pu4$V3d3AymCxLTD=_)oo?%PDr>w1A>A+dm^={jipLRjV@OnwmPk|c3d7s zDhnBHKi+Xt*=enfNXz6Tg1&YuALusd2j}^>8*ezjxQ?=y0{jJca@_?1e|t-4%q0Ch z*sAx?FfiG}var#p3WM(q z3k%Oqii4%YZj23w%r!IhC2mGTsBFj0z@GTTLs^ee8jlr4 zgFWc$9%F>Y1uTk`2#WupaRw}ah1NznA626qKz9;RE;)l6-Fat{Zn+DJs&!{>V}C;P z43B#=^t6_DKs(KyU-VLT8V(8ko}loD_Xb?JbVM}&c;k6xzb(?VdzH*y+NssNszs0| z^(c+p-1sF&hrtm}a+$K;{&M6HUL?8z3r@fbB_|lULjX#cz;%?z$QRn0uMjq}GEbOm zRSv@l>(vP(JPtoz)SNhwgis{mwIi@}&L76aYMJArK}B-h$7&<@xoH*oXblw;&H|ss z2AA;V|9?4dHsn`U=sR|Ua?r8(L`zixn3JM*XMT=D&wURK3_L%8UtWE^LSO$nu0h{N zKgoTsDzg>$-rrK0*OqsUXq`Ssk>|dzTYpe_#CfgbSan`1uA@)r^Aq~~zlP|@%RQ-& z7Mt>F<*MZcxNegQ#&0dn!z-(lk1AK;&6uqWwEsWwIh`-u(YcR}68 Date: Sun, 7 Mar 2021 21:06:48 +0100 Subject: [PATCH 031/214] Bump dependencies --- 04_zero_overhead_abstraction/Cargo.lock | 6 ++++-- 05_safe_globals/Cargo.lock | 6 ++++-- 06_drivers_gpio_uart/Cargo.lock | 6 ++++-- 07_uart_chainloader/Cargo.lock | 6 ++++-- 08_timestamps/Cargo.lock | 6 ++++-- 09_hw_debug_JTAG/Cargo.lock | 6 ++++-- 10_privilege_level/Cargo.lock | 6 ++++-- 11_virtual_mem_part1_identity_mapping/Cargo.lock | 6 ++++-- 12_exceptions_part1_groundwork/Cargo.lock | 6 ++++-- 13_integrated_testing/Cargo.lock | 14 ++++++++------ 14_exceptions_part2_peripheral_IRQs/Cargo.lock | 14 ++++++++------ 15_virtual_mem_part2_mmio_remap/Cargo.lock | 14 ++++++++------ X1_JTAG_boot/Cargo.lock | 6 ++++-- 13 files changed, 64 insertions(+), 38 deletions(-) diff --git a/04_zero_overhead_abstraction/Cargo.lock b/04_zero_overhead_abstraction/Cargo.lock index 70708c19a..3b926a347 100644 --- a/04_zero_overhead_abstraction/Cargo.lock +++ b/04_zero_overhead_abstraction/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/05_safe_globals/Cargo.lock b/05_safe_globals/Cargo.lock index 70708c19a..3b926a347 100644 --- a/05_safe_globals/Cargo.lock +++ b/05_safe_globals/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/06_drivers_gpio_uart/Cargo.lock b/06_drivers_gpio_uart/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/06_drivers_gpio_uart/Cargo.lock +++ b/06_drivers_gpio_uart/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/07_uart_chainloader/Cargo.lock b/07_uart_chainloader/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/07_uart_chainloader/Cargo.lock +++ b/07_uart_chainloader/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/08_timestamps/Cargo.lock b/08_timestamps/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/08_timestamps/Cargo.lock +++ b/08_timestamps/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/09_hw_debug_JTAG/Cargo.lock b/09_hw_debug_JTAG/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/09_hw_debug_JTAG/Cargo.lock +++ b/09_hw_debug_JTAG/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/10_privilege_level/Cargo.lock b/10_privilege_level/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/10_privilege_level/Cargo.lock +++ b/10_privilege_level/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/11_virtual_mem_part1_identity_mapping/Cargo.lock b/11_virtual_mem_part1_identity_mapping/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/11_virtual_mem_part1_identity_mapping/Cargo.lock +++ b/11_virtual_mem_part1_identity_mapping/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/12_exceptions_part1_groundwork/Cargo.lock b/12_exceptions_part1_groundwork/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/12_exceptions_part1_groundwork/Cargo.lock +++ b/12_exceptions_part1_groundwork/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] diff --git a/13_integrated_testing/Cargo.lock b/13_integrated_testing/Cargo.lock index b5c774991..069559203 100644 --- a/13_integrated_testing/Cargo.lock +++ b/13_integrated_testing/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] @@ -37,9 +39,9 @@ checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" [[package]] name = "quote" -version = "1.0.8" +version = "1.0.9" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" +checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7" dependencies = [ "proc-macro2", ] @@ -55,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.59" +version = "1.0.62" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "07cb8b1b4ebf86a89ee88cbd201b022b94138c623644d035185c84d3f41b7e66" +checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" dependencies = [ "proc-macro2", "quote", diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.lock b/14_exceptions_part2_peripheral_IRQs/Cargo.lock index b5c774991..069559203 100644 --- a/14_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/14_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] @@ -37,9 +39,9 @@ checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" [[package]] name = "quote" -version = "1.0.8" +version = "1.0.9" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" +checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7" dependencies = [ "proc-macro2", ] @@ -55,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.59" +version = "1.0.62" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "07cb8b1b4ebf86a89ee88cbd201b022b94138c623644d035185c84d3f41b7e66" +checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" dependencies = [ "proc-macro2", "quote", diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.lock b/15_virtual_mem_part2_mmio_remap/Cargo.lock index b5c774991..069559203 100644 --- a/15_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/15_virtual_mem_part2_mmio_remap/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] @@ -37,9 +39,9 @@ checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" [[package]] name = "quote" -version = "1.0.8" +version = "1.0.9" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "991431c3519a3f36861882da93630ce66b52918dcf1b8e2fd66b397fc96f28df" +checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7" dependencies = [ "proc-macro2", ] @@ -55,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.59" +version = "1.0.62" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "07cb8b1b4ebf86a89ee88cbd201b022b94138c623644d035185c84d3f41b7e66" +checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" dependencies = [ "proc-macro2", "quote", diff --git a/X1_JTAG_boot/Cargo.lock b/X1_JTAG_boot/Cargo.lock index e87b55afe..a3d8bb9ae 100644 --- a/X1_JTAG_boot/Cargo.lock +++ b/X1_JTAG_boot/Cargo.lock @@ -1,10 +1,12 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] name = "cortex-a" -version = "5.1.2" +version = "5.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d33a61602f83c3fff3e092967ad4d5d820a9fea7bc418e119616872085fe731d" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" dependencies = [ "register", ] From e451d9d62c5a076d718a259770db6df1af6676c7 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 7 Mar 2021 21:14:15 +0100 Subject: [PATCH 032/214] Fix accidentally commited temp change --- 15_virtual_mem_part2_mmio_remap/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 206b3d970..4f7561dbd 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -170,7 +170,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - $(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: From 5e65a801457dfbd7aece97f5d4723abdfce85274 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 11 Mar 2021 23:14:08 +0100 Subject: [PATCH 033/214] Rework conditional compilation for lib.rs for tests --- 13_integrated_testing/Cargo.toml | 4 +- 13_integrated_testing/Makefile | 9 +- 13_integrated_testing/README.md | 237 ++++++++++-------- .../src/_arch/aarch64/cpu.rs | 4 + 13_integrated_testing/src/cpu.rs | 5 +- 13_integrated_testing/src/lib.rs | 1 + 13_integrated_testing/src/panic_wait.rs | 33 +-- .../tests/00_console_sanity.rs | 9 +- .../tests/01_timer_sanity.rs | 5 +- .../tests/02_exception_sync_page_fault.rs | 9 +- .../tests/panic_exit_failure/mod.rs | 9 - .../Cargo.toml | 4 +- 14_exceptions_part2_peripheral_IRQs/Makefile | 9 +- 14_exceptions_part2_peripheral_IRQs/README.md | 4 +- .../src/_arch/aarch64/cpu.rs | 4 + .../src/cpu.rs | 5 +- .../src/lib.rs | 1 + .../src/panic_wait.rs | 33 +-- .../tests/00_console_sanity.rs | 9 +- .../tests/01_timer_sanity.rs | 5 +- .../tests/02_exception_sync_page_fault.rs | 9 +- .../tests/03_exception_irq_sanity.rs | 2 - .../tests/panic_exit_failure/mod.rs | 9 - 15_virtual_mem_part2_mmio_remap/Cargo.toml | 4 +- 15_virtual_mem_part2_mmio_remap/Makefile | 9 +- 15_virtual_mem_part2_mmio_remap/README.md | 8 +- .../src/_arch/aarch64/cpu.rs | 4 + 15_virtual_mem_part2_mmio_remap/src/cpu.rs | 5 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 1 + .../src/panic_wait.rs | 33 +-- .../tests/00_console_sanity.rs | 9 +- .../tests/01_timer_sanity.rs | 5 +- .../tests/02_exception_sync_page_fault.rs | 9 +- .../tests/03_exception_irq_sanity.rs | 2 - .../tests/panic_exit_failure/mod.rs | 9 - X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8128 -> 8144 bytes 36 files changed, 249 insertions(+), 268 deletions(-) delete mode 100644 13_integrated_testing/tests/panic_exit_failure/mod.rs delete mode 100644 14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs delete mode 100644 15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs diff --git a/13_integrated_testing/Cargo.toml b/13_integrated_testing/Cargo.toml index 5d57cf357..206dfc878 100644 --- a/13_integrated_testing/Cargo.toml +++ b/13_integrated_testing/Cargo.toml @@ -7,22 +7,22 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- [dependencies] -qemu-exit = "1.x.x" test-types = { path = "test-types" } # Optional dependencies register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } +qemu-exit = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] diff --git a/13_integrated_testing/Makefile b/13_integrated_testing/Makefile index 4f7561dbd..74e67da50 100644 --- a/13_integrated_testing/Makefile +++ b/13_integrated_testing/Makefile @@ -57,9 +57,9 @@ QEMU_MISSING_STRING = "This board is not yet supported for QEMU." RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -124,12 +124,15 @@ qemu: $(KERNEL_BIN) define KERNEL_TEST_RUNNER #!/usr/bin/env bash - $(OBJCOPY_CMD) $$1 $$1.img + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build test: @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 383b0a0dd..3f62ef6c7 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -210,13 +210,13 @@ The convetion is that as long as the test function does not `panic!`, the test w The last of the attributes we added is `#![reexport_test_harness_main = "test_main"]`. Remember that our kernel uses the `no_main` attribute, and that we also set it for the test compilation. We did -that because we wrote our own `_start()` function (in `aarch64.rs`), which kicks off the following -call chain during kernel boot: +that because we wrote our own `_start()` function, which kicks off the following call chain during +kernel boot: | | Function | File | | - | - | - | | 1. | `_start()` | `lib.rs` | -| 2. | (some more arch64 code) | `lib.rs` | +| 2. | (some more aarch64 code) | `lib.rs` | | 3. | `runtime_init()` | `lib.rs` | | 4. | `kernel_init()` | `main.rs` | | 5. | `kernel_main()` | `main.rs` | @@ -237,6 +237,7 @@ implementation in `lib.rs`: #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); test_main(); @@ -245,12 +246,12 @@ unsafe fn kernel_init() -> ! { } ``` -Note that we first call `bsp::console::qemu_bring_up_console()`. Since we are running all our tests -inside `QEMU`, we need to ensure that whatever peripheral implements the kernel's `console` -interface is initialized, so that we can print from our tests. If you recall [tutorial 03], bringing -up peripherals in `QEMU` might not need the full initialization as is needed on real hardware -(setting clocks, config registers, etc...) due to the abstractions in `QEMU`'s emulation code. So -this is an opportunity to cut down on setup code. +Note the call to `bsp::console::qemu_bring_up_console()`. Since we are running all our tests inside +`QEMU`, we need to ensure that whatever peripheral implements the kernel's `console` interface is +initialized, so that we can print from our tests. If you recall [tutorial 03], bringing up +peripherals in `QEMU` might not need the full initialization as is needed on real hardware (setting +clocks, config registers, etc...) due to the abstractions in `QEMU`'s emulation code. So this is an +opportunity to cut down on setup code. [tutorial 03]: ../03_hacky_hello_world @@ -290,16 +291,20 @@ following exit calls for the kernel: //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] use qemu_exit::QEMUExit; +#[cfg(feature = "test_build")] const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); /// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_failure() -> ! { QEMU_EXIT_HANDLE.exit_failure() } /// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_success() -> ! { QEMU_EXIT_HANDLE.exit_success() } @@ -308,6 +313,10 @@ pub fn qemu_exit_success() -> ! { [Click here] in case you are interested in the implementation. Note that for the functions to work, the `-semihosting` flag must be added to the `QEMU` invocation. +You might have also noted the `#[cfg(feature = "test_build")]`. In the `Makefile`, we ensure that +this feature is only enabled when `cargo test` runs. This way, it is ensured that testing-specific +code is conditionally compiled only for testing. + [exit status]: https://en.wikipedia.org/wiki/Exit_status [@phil-opp]: https://github.com/phil-opp [learned how to do this]: https://os.phil-opp.com/testing/#exiting-qemu @@ -322,19 +331,29 @@ Unit test failure shall be triggered by the `panic!` macro, either directly or b safely park the panicked CPU core in a busy loop. This can't be used for the unit tests, because `cargo` would wait forever for `QEMU` to exit and stall the whole test run. Again, conditional compilation is used to differentiate between a release and testing version of how a `panic!` -concludes. Here is the new testing version: +concludes: ```rust -/// The point of exit when the library is compiled for testing. -#[cfg(test)] +/// The point of exit for `libkernel`. +/// +/// It is linked weakly, so that the integration tests can overload its standard behavior. +#[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - cpu::qemu_exit_failure() + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } } ``` -In case none of the unit tests panicked, `lib.rs`'s `kernel_init()` calls -`cpu::qemu_exit_success()` to successfully conclude the unit test run. +In case none of the unit tests panicked, `lib.rs`'s `kernel_init()` calls `cpu::qemu_exit_success()` +to successfully conclude the unit test run. ### Controlling Test Kernel Execution @@ -363,12 +382,15 @@ The file `kernel_test_runner.sh` does not exist by default. We generate it on de define KERNEL_TEST_RUNNER #!/usr/bin/env bash - $(OBJCOPY_CMD) $$1 $$1.img + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build test: @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh @@ -525,14 +547,13 @@ your test code into individual chunks. For example, take a look at `tests/01_tim #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use core::time::Duration; -use libkernel::{bsp, cpu, time, time::interface::TimeManager}; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. @@ -579,30 +600,34 @@ harness = false #### Overriding Panic Behavior -It is also important to understand that the `libkernel` made available to the integration tests is -the _release_ version. Therefore, it won't contain any code attributed with `#[cfg(test)]`! - -One of the implications of this is that the `panic handler` provided by `libkernel` will be the -version from the release kernel that spins forever, and not the test version that exits `QEMU`. - -One way to navigate around this is to declare the _release version of the panic exit function_ in -`lib.rs` as a [weak symbol]: +Did you notice the `#[linkage = "weak"]` attribute some chapters earlier at the `_panic_exit()` +function? This marks the function in `lib.rs` as a [weak symbol]. Let's look at it again: ```rust -#[cfg(not(test))] +/// The point of exit for `libkernel`. +/// +/// It is linked weakly, so that the integration tests can overload its standard behavior. #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - cpu::wait_forever() + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } } ``` [weak symbol]: https://en.wikipedia.org/wiki/Weak_symbol -Integration tests in `$CRATE/tests/` can now override it according to their needs, because depending -on the kind of test, a `panic!` could mean success or failure. For example, -`tests/02_exception_sync_page_fault.rs` is intentionally causing a page fault, so the wanted outcome -is a `panic!`. Here is the whole test (minus some inline comments): +This enables integration tests in `$CRATE/tests/` to override this function according to their +needs. This is useful because depending on the kind of test, a `panic!` could mean success or +failure. For example, `tests/02_exception_sync_page_fault.rs` is intentionally causing a page fault, +so the wanted outcome is a `panic!`. Here is the whole test (minus some inline comments): ```rust //! Page faults must result in synchronous exceptions. @@ -619,13 +644,12 @@ use libkernel::{bsp, cpu, exception, memory, println}; unsafe fn kernel_init() -> ! { use memory::mmu::interface::MMU; + exception::handling_init(); bsp::console::qemu_bring_up_console(); println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { println!("MMU: {}", string); cpu::qemu_exit_failure() @@ -638,11 +662,11 @@ unsafe fn kernel_init() -> ! { // If execution reaches here, the memory access above did not cause a page fault exception. cpu::qemu_exit_failure() } + ``` The `_panic_exit()` version that makes `QEMU` return `0` (indicating test success) is pulled in by -`mod panic_exit_success;`. The counterpart would be `mod panic_exit_failure;`. We provide both in -the `tests` folder, so each integration test can import the one that it needs. +`mod panic_exit_success;`, and it will take precedence over the `weak` version from `lib.rs`. ### Console Tests @@ -685,16 +709,15 @@ The subtest first sends `"ABC"` over the console to the kernel, and then expects #![no_main] #![no_std] -mod panic_exit_failure; - -use libkernel::{bsp, console, print}; +use libkernel::{bsp, console, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { - use bsp::console::{console, qemu_bring_up_console}; + use bsp::console::console; use console::interface::*; - qemu_bring_up_console(); + exception::handling_init(); + bsp::console::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); @@ -796,16 +819,28 @@ diff -uNr 12_exceptions_part1_groundwork/.cargo/config.toml 13_integrated_testin diff -uNr 12_exceptions_part1_groundwork/Cargo.toml 13_integrated_testing/Cargo.toml --- 12_exceptions_part1_groundwork/Cargo.toml +++ 13_integrated_testing/Cargo.toml -@@ -18,11 +18,38 @@ +@@ -7,22 +7,49 @@ + [profile.release] + lto = true + +-# The features section is used to select the target board. + [features] + default = [] + bsp_rpi3 = ["register"] + bsp_rpi4 = ["register"] ++test_build = ["qemu-exit"] + + ##-------------------------------------------------------------------------------------------------- + ## Dependencies ##-------------------------------------------------------------------------------------------------- [dependencies] -+qemu-exit = "1.x.x" +test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", optional = true } +register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } ++qemu-exit = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] @@ -856,7 +891,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg -@@ -41,6 +43,17 @@ +@@ -41,18 +43,30 @@ # Export for build.rs export LINKER_FILE @@ -874,7 +909,14 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -@@ -53,6 +66,7 @@ +-FEATURES = bsp_$(BSP) ++FEATURES = --features bsp_$(BSP) + COMPILER_ARGS = --target=$(TARGET) \ +- --features $(FEATURES) \ ++ $(FEATURES) \ + --release + + RUSTC_CMD = cargo rustc $(COMPILER_ARGS) DOC_CMD = cargo doc $(COMPILER_ARGS) CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) CHECK_CMD = cargo check $(COMPILER_ARGS) @@ -901,7 +943,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile all: $(KERNEL_BIN) -@@ -100,11 +115,26 @@ +@@ -100,11 +115,29 @@ $(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) @@ -916,12 +958,15 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + -+ $(OBJCOPY_CMD) $$1 $$1.img ++ TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') ++ ++ $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER ++test: FEATURES += --features test_build +test: + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh @@ -934,7 +979,7 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_testing/src/_arch/aarch64/cpu.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs +++ 13_integrated_testing/src/_arch/aarch64/cpu.rs -@@ -26,3 +26,20 @@ +@@ -26,3 +26,24 @@ asm::wfe() } } @@ -942,16 +987,20 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_ +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- ++#[cfg(feature = "test_build")] +use qemu_exit::QEMUExit; + ++#[cfg(feature = "test_build")] +const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); + +/// Make the host QEMU binary execute `exit(1)`. ++#[cfg(feature = "test_build")] +pub fn qemu_exit_failure() -> ! { + QEMU_EXIT_HANDLE.exit_failure() +} + +/// Make the host QEMU binary execute `exit(0)`. ++#[cfg(feature = "test_build")] +pub fn qemu_exit_success() -> ! { + QEMU_EXIT_HANDLE.exit_success() +} @@ -1121,12 +1170,13 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_in diff -uNr 12_exceptions_part1_groundwork/src/cpu.rs 13_integrated_testing/src/cpu.rs --- 12_exceptions_part1_groundwork/src/cpu.rs +++ 13_integrated_testing/src/cpu.rs -@@ -15,4 +15,4 @@ - //-------------------------------------------------------------------------------------------------- +@@ -16,3 +16,6 @@ // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- --pub use arch_cpu::{nop, wait_forever}; -+pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; + pub use arch_cpu::{nop, wait_forever}; ++ ++#[cfg(feature = "test_build")] ++pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/src/exception.rs --- 12_exceptions_part1_groundwork/src/exception.rs @@ -1157,7 +1207,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,170 @@ +@@ -0,0 +1,171 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1322,6 +1372,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +#[cfg(test)] +#[no_mangle] +unsafe fn kernel_init() -> ! { ++ exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + test_main(); @@ -1590,43 +1641,32 @@ diff -uNr 12_exceptions_part1_groundwork/src/panic_wait.rs 13_integrated_testing unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } -+/// The point of exit for the "standard" (non-testing) `libkernel`. -+/// -+/// This code will be used by the release kernel binary and the `integration tests`. It is linked -+/// weakly, so that the integration tests can overload it to exit `QEMU` instead of spinning -+/// forever. ++/// The point of exit for `libkernel`. +/// -+/// This is one possible approach to solve the problem that `cargo` can not know who the consumer of -+/// the library will be: -+/// - The release kernel binary that should safely park the paniced core, -+/// - or an `integration test` that is executed in QEMU, which should just exit QEMU. -+#[cfg(not(test))] ++/// It is linked weakly, so that the integration tests can overload its standard behavior. +#[linkage = "weak"] +#[no_mangle] +fn _panic_exit() -> ! { -+ cpu::wait_forever() ++ #[cfg(not(test_build))] ++ { ++ cpu::wait_forever() ++ } ++ ++ #[cfg(test_build)] ++ { ++ cpu::qemu_exit_failure() ++ } +} + /// Prints with a newline - only use from the panic handler. /// /// Carbon copy from -@@ -35,5 +52,16 @@ +@@ -35,5 +52,5 @@ panic_println!("\nKernel panic!"); } - cpu::wait_forever() + _panic_exit() -+} -+ -+//-------------------------------------------------------------------------------------------------- -+// Testing -+//-------------------------------------------------------------------------------------------------- -+ -+/// The point of exit when the library is compiled for testing. -+#[cfg(test)] -+#[no_mangle] -+fn _panic_exit() -> ! { -+ cpu::qemu_exit_failure() } diff -uNr 12_exceptions_part1_groundwork/src/runtime_init.rs 13_integrated_testing/src/runtime_init.rs @@ -1757,7 +1797,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rb 13_integrate diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrated_testing/tests/00_console_sanity.rs --- 12_exceptions_part1_groundwork/tests/00_console_sanity.rs +++ 13_integrated_testing/tests/00_console_sanity.rs -@@ -0,0 +1,36 @@ +@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter @@ -1768,16 +1808,15 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate +#![no_main] +#![no_std] + -+mod panic_exit_failure; -+ -+use libkernel::{bsp, console, print}; ++use libkernel::{bsp, console, exception, print}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { -+ use bsp::console::{console, qemu_bring_up_console}; ++ use bsp::console::console; + use console::interface::*; + -+ qemu_bring_up_console(); ++ exception::handling_init(); ++ bsp::console::qemu_bring_up_console(); + + // Handshake + assert_eq!(console().read_char(), 'A'); @@ -1798,7 +1837,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_testing/tests/01_timer_sanity.rs --- 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs +++ 13_integrated_testing/tests/01_timer_sanity.rs -@@ -0,0 +1,50 @@ +@@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter @@ -1811,14 +1850,13 @@ diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_ +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + -+mod panic_exit_failure; -+ +use core::time::Duration; -+use libkernel::{bsp, cpu, time, time::interface::TimeManager}; ++use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { ++ exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. @@ -1853,7 +1891,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 13_integrated_testing/tests/02_exception_sync_page_fault.rs --- 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs +++ 13_integrated_testing/tests/02_exception_sync_page_fault.rs -@@ -0,0 +1,44 @@ +@@ -0,0 +1,43 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter @@ -1864,10 +1902,10 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 +#![no_main] +#![no_std] + -+/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. ++/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. +/// -+/// Reaching this code is a success, because it is called from the synchronous exception handler, -+/// which is what this test wants to achieve. ++/// In this test, teaching the panic is a success, because it is called from the synchronous ++/// exception handler, which is what this test wants to achieve. +/// +/// It also means that this integration test can not use any other code that calls panic!() directly +/// or indirectly. @@ -1879,13 +1917,12 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 +unsafe fn kernel_init() -> ! { + use memory::mmu::interface::MMU; + ++ exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + -+ exception::handling_init(); -+ + if let Err(string) = memory::mmu::mmu().init() { + println!("MMU: {}", string); + cpu::qemu_exit_failure() @@ -1899,20 +1936,6 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 + cpu::qemu_exit_failure() +} -diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_failure/mod.rs 13_integrated_testing/tests/panic_exit_failure/mod.rs ---- 12_exceptions_part1_groundwork/tests/panic_exit_failure/mod.rs -+++ 13_integrated_testing/tests/panic_exit_failure/mod.rs -@@ -0,0 +1,9 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2019-2021 Andre Richter -+ -+/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. -+#[no_mangle] -+fn _panic_exit() -> ! { -+ libkernel::cpu::qemu_exit_failure() -+} - diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 13_integrated_testing/tests/panic_exit_success/mod.rs --- 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs +++ 13_integrated_testing/tests/panic_exit_success/mod.rs diff --git a/13_integrated_testing/src/_arch/aarch64/cpu.rs b/13_integrated_testing/src/_arch/aarch64/cpu.rs index 948bad747..56443865a 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu.rs @@ -30,16 +30,20 @@ pub fn wait_forever() -> ! { //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] use qemu_exit::QEMUExit; +#[cfg(feature = "test_build")] const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); /// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_failure() -> ! { QEMU_EXIT_HANDLE.exit_failure() } /// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_success() -> ! { QEMU_EXIT_HANDLE.exit_success() } diff --git a/13_integrated_testing/src/cpu.rs b/13_integrated_testing/src/cpu.rs index 9c8eb6f6e..36b6a9d4c 100644 --- a/13_integrated_testing/src/cpu.rs +++ b/13_integrated_testing/src/cpu.rs @@ -15,4 +15,7 @@ pub mod smp; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index 3e0a9eea5..0fa216a98 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -162,6 +162,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); test_main(); diff --git a/13_integrated_testing/src/panic_wait.rs b/13_integrated_testing/src/panic_wait.rs index 61536ba80..20493a912 100644 --- a/13_integrated_testing/src/panic_wait.rs +++ b/13_integrated_testing/src/panic_wait.rs @@ -17,21 +17,21 @@ fn _panic_print(args: fmt::Arguments) { unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } -/// The point of exit for the "standard" (non-testing) `libkernel`. +/// The point of exit for `libkernel`. /// -/// This code will be used by the release kernel binary and the `integration tests`. It is linked -/// weakly, so that the integration tests can overload it to exit `QEMU` instead of spinning -/// forever. -/// -/// This is one possible approach to solve the problem that `cargo` can not know who the consumer of -/// the library will be: -/// - The release kernel binary that should safely park the paniced core, -/// - or an `integration test` that is executed in QEMU, which should just exit QEMU. -#[cfg(not(test))] +/// It is linked weakly, so that the integration tests can overload its standard behavior. #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - cpu::wait_forever() + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } } /// Prints with a newline - only use from the panic handler. @@ -54,14 +54,3 @@ fn panic(info: &PanicInfo) -> ! { _panic_exit() } - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -/// The point of exit when the library is compiled for testing. -#[cfg(test)] -#[no_mangle] -fn _panic_exit() -> ! { - cpu::qemu_exit_failure() -} diff --git a/13_integrated_testing/tests/00_console_sanity.rs b/13_integrated_testing/tests/00_console_sanity.rs index 08b654aef..c3754aa9f 100644 --- a/13_integrated_testing/tests/00_console_sanity.rs +++ b/13_integrated_testing/tests/00_console_sanity.rs @@ -8,16 +8,15 @@ #![no_main] #![no_std] -mod panic_exit_failure; - -use libkernel::{bsp, console, print}; +use libkernel::{bsp, console, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { - use bsp::console::{console, qemu_bring_up_console}; + use bsp::console::console; use console::interface::*; - qemu_bring_up_console(); + exception::handling_init(); + bsp::console::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/13_integrated_testing/tests/01_timer_sanity.rs b/13_integrated_testing/tests/01_timer_sanity.rs index 52dca0fca..f39d83845 100644 --- a/13_integrated_testing/tests/01_timer_sanity.rs +++ b/13_integrated_testing/tests/01_timer_sanity.rs @@ -10,14 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use core::time::Duration; -use libkernel::{bsp, cpu, time, time::interface::TimeManager}; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/13_integrated_testing/tests/02_exception_sync_page_fault.rs b/13_integrated_testing/tests/02_exception_sync_page_fault.rs index bf2ba29f1..185089a3d 100644 --- a/13_integrated_testing/tests/02_exception_sync_page_fault.rs +++ b/13_integrated_testing/tests/02_exception_sync_page_fault.rs @@ -8,10 +8,10 @@ #![no_main] #![no_std] -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. /// -/// Reaching this code is a success, because it is called from the synchronous exception handler, -/// which is what this test wants to achieve. +/// In this test, teaching the panic is a success, because it is called from the synchronous +/// exception handler, which is what this test wants to achieve. /// /// It also means that this integration test can not use any other code that calls panic!() directly /// or indirectly. @@ -23,13 +23,12 @@ use libkernel::{bsp, cpu, exception, memory, println}; unsafe fn kernel_init() -> ! { use memory::mmu::interface::MMU; + exception::handling_init(); bsp::console::qemu_bring_up_console(); println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { println!("MMU: {}", string); cpu::qemu_exit_failure() diff --git a/13_integrated_testing/tests/panic_exit_failure/mod.rs b/13_integrated_testing/tests/panic_exit_failure/mod.rs deleted file mode 100644 index af2ba3781..000000000 --- a/13_integrated_testing/tests/panic_exit_failure/mod.rs +++ /dev/null @@ -1,9 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2019-2021 Andre Richter - -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. -#[no_mangle] -fn _panic_exit() -> ! { - libkernel::cpu::qemu_exit_failure() -} diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.toml b/14_exceptions_part2_peripheral_IRQs/Cargo.toml index 5d57cf357..206dfc878 100644 --- a/14_exceptions_part2_peripheral_IRQs/Cargo.toml +++ b/14_exceptions_part2_peripheral_IRQs/Cargo.toml @@ -7,22 +7,22 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- [dependencies] -qemu-exit = "1.x.x" test-types = { path = "test-types" } # Optional dependencies register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } +qemu-exit = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] diff --git a/14_exceptions_part2_peripheral_IRQs/Makefile b/14_exceptions_part2_peripheral_IRQs/Makefile index 4f7561dbd..74e67da50 100644 --- a/14_exceptions_part2_peripheral_IRQs/Makefile +++ b/14_exceptions_part2_peripheral_IRQs/Makefile @@ -57,9 +57,9 @@ QEMU_MISSING_STRING = "This board is not yet supported for QEMU." RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -124,12 +124,15 @@ qemu: $(KERNEL_BIN) define KERNEL_TEST_RUNNER #!/usr/bin/env bash - $(OBJCOPY_CMD) $$1 $$1.img + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build test: @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index c04cc9ae7..1c3cb7f0c 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2627,7 +2627,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip diff -uNr 13_integrated_testing/tests/03_exception_irq_sanity.rs 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs --- 13_integrated_testing/tests/03_exception_irq_sanity.rs +++ 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs -@@ -0,0 +1,68 @@ +@@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -2640,8 +2640,6 @@ diff -uNr 13_integrated_testing/tests/03_exception_irq_sanity.rs 14_exceptions_p +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + -+mod panic_exit_failure; -+ +use libkernel::{bsp, cpu, exception}; +use test_macros::kernel_test; + diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs index 948bad747..56443865a 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs @@ -30,16 +30,20 @@ pub fn wait_forever() -> ! { //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] use qemu_exit::QEMUExit; +#[cfg(feature = "test_build")] const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); /// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_failure() -> ! { QEMU_EXIT_HANDLE.exit_failure() } /// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_success() -> ! { QEMU_EXIT_HANDLE.exit_success() } diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs index 9c8eb6f6e..36b6a9d4c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/cpu.rs @@ -15,4 +15,7 @@ pub mod smp; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index fd4bdde51..4ea416fd5 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -165,6 +165,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); test_main(); diff --git a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs index 8c7770b6e..e3a9ed8ab 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs @@ -17,21 +17,21 @@ fn _panic_print(args: fmt::Arguments) { unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } -/// The point of exit for the "standard" (non-testing) `libkernel`. +/// The point of exit for `libkernel`. /// -/// This code will be used by the release kernel binary and the `integration tests`. It is linked -/// weakly, so that the integration tests can overload it to exit `QEMU` instead of spinning -/// forever. -/// -/// This is one possible approach to solve the problem that `cargo` can not know who the consumer of -/// the library will be: -/// - The release kernel binary that should safely park the paniced core, -/// - or an `integration test` that is executed in QEMU, which should just exit QEMU. -#[cfg(not(test))] +/// It is linked weakly, so that the integration tests can overload its standard behavior. #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - cpu::wait_forever() + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } } /// Prints with a newline - only use from the panic handler. @@ -56,14 +56,3 @@ fn panic(info: &PanicInfo) -> ! { _panic_exit() } - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -/// The point of exit when the library is compiled for testing. -#[cfg(test)] -#[no_mangle] -fn _panic_exit() -> ! { - cpu::qemu_exit_failure() -} diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs index 08b654aef..c3754aa9f 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs @@ -8,16 +8,15 @@ #![no_main] #![no_std] -mod panic_exit_failure; - -use libkernel::{bsp, console, print}; +use libkernel::{bsp, console, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { - use bsp::console::{console, qemu_bring_up_console}; + use bsp::console::console; use console::interface::*; - qemu_bring_up_console(); + exception::handling_init(); + bsp::console::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs index 52dca0fca..f39d83845 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs @@ -10,14 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use core::time::Duration; -use libkernel::{bsp, cpu, time, time::interface::TimeManager}; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs index bf2ba29f1..185089a3d 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs @@ -8,10 +8,10 @@ #![no_main] #![no_std] -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. /// -/// Reaching this code is a success, because it is called from the synchronous exception handler, -/// which is what this test wants to achieve. +/// In this test, teaching the panic is a success, because it is called from the synchronous +/// exception handler, which is what this test wants to achieve. /// /// It also means that this integration test can not use any other code that calls panic!() directly /// or indirectly. @@ -23,13 +23,12 @@ use libkernel::{bsp, cpu, exception, memory, println}; unsafe fn kernel_init() -> ! { use memory::mmu::interface::MMU; + exception::handling_init(); bsp::console::qemu_bring_up_console(); println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { println!("MMU: {}", string); cpu::qemu_exit_failure() diff --git a/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs index 1f3bb7702..ac25d01a8 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs @@ -10,8 +10,6 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use libkernel::{bsp, cpu, exception}; use test_macros::kernel_test; diff --git a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs b/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs deleted file mode 100644 index af2ba3781..000000000 --- a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_failure/mod.rs +++ /dev/null @@ -1,9 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2019-2021 Andre Richter - -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. -#[no_mangle] -fn _panic_exit() -> ! { - libkernel::cpu::qemu_exit_failure() -} diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.toml b/15_virtual_mem_part2_mmio_remap/Cargo.toml index 5d57cf357..206dfc878 100644 --- a/15_virtual_mem_part2_mmio_remap/Cargo.toml +++ b/15_virtual_mem_part2_mmio_remap/Cargo.toml @@ -7,22 +7,22 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- [dependencies] -qemu-exit = "1.x.x" test-types = { path = "test-types" } # Optional dependencies register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } +qemu-exit = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 4f7561dbd..74e67da50 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -57,9 +57,9 @@ QEMU_MISSING_STRING = "This board is not yet supported for QEMU." RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -124,12 +124,15 @@ qemu: $(KERNEL_BIN) define KERNEL_TEST_RUNNER #!/usr/bin/env bash - $(OBJCOPY_CMD) $$1 $$1.img + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build test: @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 65127bd6a..53eb8047f 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -3022,11 +3022,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault - use memory::mmu::interface::MMU; + use libkernel::driver::interface::DriverManager; - bsp::console::qemu_bring_up_console(); - -@@ -30,10 +30,22 @@ - exception::handling_init(); + bsp::console::qemu_bring_up_console(); +@@ -29,10 +29,22 @@ + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::mmu().init() { - println!("MMU: {}", string); diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs index 948bad747..56443865a 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs @@ -30,16 +30,20 @@ pub fn wait_forever() -> ! { //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] use qemu_exit::QEMUExit; +#[cfg(feature = "test_build")] const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); /// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_failure() -> ! { QEMU_EXIT_HANDLE.exit_failure() } /// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] pub fn qemu_exit_success() -> ! { QEMU_EXIT_HANDLE.exit_success() } diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/cpu.rs index 9c8eb6f6e..36b6a9d4c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/cpu.rs @@ -15,4 +15,7 @@ pub mod smp; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{nop, qemu_exit_failure, qemu_exit_success, wait_forever}; +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index 608d4b07b..dad5e903c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -167,6 +167,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); test_main(); diff --git a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs index 8c7770b6e..e3a9ed8ab 100644 --- a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs +++ b/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs @@ -17,21 +17,21 @@ fn _panic_print(args: fmt::Arguments) { unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } -/// The point of exit for the "standard" (non-testing) `libkernel`. +/// The point of exit for `libkernel`. /// -/// This code will be used by the release kernel binary and the `integration tests`. It is linked -/// weakly, so that the integration tests can overload it to exit `QEMU` instead of spinning -/// forever. -/// -/// This is one possible approach to solve the problem that `cargo` can not know who the consumer of -/// the library will be: -/// - The release kernel binary that should safely park the paniced core, -/// - or an `integration test` that is executed in QEMU, which should just exit QEMU. -#[cfg(not(test))] +/// It is linked weakly, so that the integration tests can overload its standard behavior. #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - cpu::wait_forever() + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } } /// Prints with a newline - only use from the panic handler. @@ -56,14 +56,3 @@ fn panic(info: &PanicInfo) -> ! { _panic_exit() } - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -/// The point of exit when the library is compiled for testing. -#[cfg(test)] -#[no_mangle] -fn _panic_exit() -> ! { - cpu::qemu_exit_failure() -} diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs index 08b654aef..c3754aa9f 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs @@ -8,16 +8,15 @@ #![no_main] #![no_std] -mod panic_exit_failure; - -use libkernel::{bsp, console, print}; +use libkernel::{bsp, console, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { - use bsp::console::{console, qemu_bring_up_console}; + use bsp::console::console; use console::interface::*; - qemu_bring_up_console(); + exception::handling_init(); + bsp::console::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs index 52dca0fca..f39d83845 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs @@ -10,14 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use core::time::Duration; -use libkernel::{bsp, cpu, time, time::interface::TimeManager}; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { + exception::handling_init(); bsp::console::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs index 858357a4d..45f12a1f7 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs @@ -8,10 +8,10 @@ #![no_main] #![no_std] -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. /// -/// Reaching this code is a success, because it is called from the synchronous exception handler, -/// which is what this test wants to achieve. +/// In this test, teaching the panic is a success, because it is called from the synchronous +/// exception handler, which is what this test wants to achieve. /// /// It also means that this integration test can not use any other code that calls panic!() directly /// or indirectly. @@ -23,13 +23,12 @@ use libkernel::{bsp, cpu, exception, memory, println}; unsafe fn kernel_init() -> ! { use libkernel::driver::interface::DriverManager; + exception::handling_init(); bsp::console::qemu_bring_up_console(); println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - exception::handling_init(); - if let Err(string) = memory::mmu::kernel_map_binary_and_enable_mmu() { println!("Enabling MMU failed: {}", string); cpu::qemu_exit_failure() diff --git a/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs index 1f3bb7702..ac25d01a8 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs @@ -10,8 +10,6 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -mod panic_exit_failure; - use libkernel::{bsp, cpu, exception}; use test_macros::kernel_test; diff --git a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs b/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs deleted file mode 100644 index af2ba3781..000000000 --- a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_failure/mod.rs +++ /dev/null @@ -1,9 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2019-2021 Andre Richter - -/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. -#[no_mangle] -fn _panic_exit() -> ! { - libkernel::cpu::qemu_exit_failure() -} diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index d4148b720f230a78479ea44cdc8791b9c28bc753..de6648804151541fd045c79f2f5dcf824f6094db 100755 GIT binary patch delta 2333 zcmb7FU2GFq7CzTwC!WOlv9S~5II$gemDq-oU_|1i;4wjEV?d;iTXw5fX{gFiejbuH zDv-SeRfMEft}YD)9vbLFJ65u-7Lf2tEeVyXcD3ClYN`68st#3K|Ae;e_5rIK&z?IT zD(F*pq{-yY`ObIF{l0U@3!xjKX=l950H@vH{&+zc_(bV}B_l|M5a`B~8c0145c5JJ zI$?k-;{etWxX1Pre0Q|0-}kIzQ)O);+KX|&6#~mbg)bL?V7>!_q8_ds;CM+IyRPC* z{l3EuD&B-S{`F;n^_YYBcU#Fi9$X@$il@ zz=E7ehjEu%xcTB*DfOYUv15`ziZ@(okP5@N>q-L@X3@ehk%C+WayPW)T}7!+V{x>a za?9h2Kvd7u0#IF9B3fK0$Y*gT2*6wEuYr0Nn&~Z!UC_blVuMsbyqef-gv)$Zyr7%R z3nV#WTj)I12$m_h+p#QECM&VQHf%7C4Lyww`}I=LV+1;iD(y0ZWdIfK(Lp>G0x_RJ zzi$WTc8FB?22k1rPAdo9lXe89{Rc`rgwpb6Dp1-8N?C!@4&GH_ zTuLq@pjwM05rqWXFC5o-y4ZI@7YVbOni+Og|F+fBMZJsbXr|>IIbBXC_wFhmU zTG}gqa1O_0I?S&BNJLM#18jKB{yor2JwhV7J<&>wKPzYW$EdjXnv6b9pVB9yhd$p~5DikQ z*Jcff21xO>>t#r7SClhq3|@(fXCbvsT{DN()xNZ|;JvxC5F*CBh=!s$(`e{4z*ahl zjN50o(l512e4E~zY{=Ri=P*dOQIY`q4U+u_r#h~KJm?{2{GB9QC7@ojR7eZPq??33 z$q0~43)W<&s9esJgPdrN%f>3H02t%6nd5|dQPCWloh~nvxgs8EGfktVnHIhL2UI54 zVvyAq76T5d#C39Z9g(wm#Ikq{t^X~l@g{4u?f$*yTC4c}IW+?v{~q1D?FV`2Zxhjo z>Eif5@35C_F7*8`woqGm3yq*J*#;;?u2eTY{qgoEqG^TSpuBj25HsWc_#7k7}Rprb=LD|eV zBjBya8~CQvU8Ny?x|{Nn^N8b<$p-=DL*tFOSutrZD)oS$!IxK+dUnF@#C!lYYqxv* zbXW_oRwzB~+IoA>9(UjFw)S@SW8o;y=4rNUuP3jtg8g6SlT9V8nuvdT*=K7zNGCHp zI@&*ND8;Z#_3GBJI9BTQ>s%W<;&6rk->fgRDzzGG{?ccTy=3iMu2S(oI!n*1+B(6) zPFKazy3*UJIyU5N5}y=G%MnJMF7a+{X_aOvXNxD$SX$L-{Jq*5&!>8|GSV1o@6`78 zTle}f8*98>$(7yN*Si;>oeeha`=zp*EjHDgYqgeZv`Q|PZ3?kAS1U=dKe;-@8(RDZ IJL_usA9XyxjsO4v delta 2318 zcmcIlZERCj7=BOhy52kX)vg^|H+EYlSwBG7ijEGjcY|bXjtt8ThLDVjfdPrK`3r-# z1AoAP*yEW@XEcl-!d)O@hQ<;iXSUyLNzF3Va8L1Iz$9|ZTxLf<`70t4*jMC42(ClvrcI-zmwP9pA~ z%!45r$aLHUT}c7PJhs@9Rdc{H2zMIef;6_(prjY^%@H!a(EwdiUP$qS2-Ir>i)03i zJRtO~Km&IY#<9SvENMsEHk;iiPY8W`bXyEZ5p5#ctmtrBkUepMjNS8L$Y@<&v)#u7 zCWKj@GQ^~G%spRcX0$F($ZXX?1)c%~B@pD6ir2O+u=`v@^vKOC`@$$AcVs}j>=3^7 zR4UO6+QLQ`fhgZ-Z@!WijX-DIF>&kRCt8w&$>7?1T3S?5ml*R?1hWv0aQ!A+N zVA=`5$x!@9lrB7%UPa?k0}N%Vl(Z(0EHTshxthRJ@%!Ke!~VD+j_rJCi65wN=>M1a zfeLs0H;K8tna+u0TwXvKCrCI53fnB~F--W`w9rHd8!0)&KFRykI^n18fm)iV`%I0S z2xbOo6FX>JgUIn8jO$ILcD$+R3?ZA?Oz|SyL<5}#5Gh2bsTDxCH!r0JoUEl}9UEK0$A2+AlYz*ehn4IN7^qFNVK>|HCY0?k0ZLZsF_O zEOnfB{J_Wk+oTI4)11Qy-9E+Mvv_TzPI~=)K}{}%w1i%teEG!NrAOG$7TG=OrN?JB zUoOJcy(h$~_L=_6Cwe>-uTLbk1)_skkhhMSRA-nyTIMT``RS&O4YbD?4!#<1U@w$C zyE;}!okBR+qPS@0H*J_dH~ea0QA7sw!ol9JTho$JiMH9SKFJ7CK9}EsXp5!|XKnCi zPGp}VXDrkWd+(?#z`Mi#~x#k6R+v$D5_QE6mUrwT+BI6Py~t1M;m}e>&Qq!0_&yPC9riz|NMti(W^Jh#A{^2%@jCba{sj zUzc7xM7b*XaA$9qyN9ixT|+joowEZ)-G5?i;Ea&YaB+r7XAJmhycST)Jk(N9MGP-u zX(d8N_3ByN0vAxH=eY2lFGw!F8HeI+;~i%?_f_=%LN@3xz!C7?4{}&=@}t6?Pa6YvUIyNH@n2n zSJydf=j3MPx_`eu$J)%c3TB=YsCVgS`?qJE&j#jPLvSVAS+lPGUPMvfq*?dPM+N&G k-Fu0?Zem0Zh&(T?YuY-QBD=^AwpDJFV!DGEQ{}qf0UGO}SpWb4 From 2f903041495c942e4b32bf32656c4ddef1daca0c Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 12 Mar 2021 23:44:10 +0100 Subject: [PATCH 034/214] Add color and verbosity to Makefile --- 01_wait_forever/Cargo.toml | 1 - 01_wait_forever/Makefile | 37 ++++++++---- 01_wait_forever/build.rs | 1 + 02_runtime_init/Cargo.toml | 1 - 02_runtime_init/Makefile | 37 ++++++++---- 02_runtime_init/README.md | 2 +- 02_runtime_init/build.rs | 1 + 03_hacky_hello_world/Cargo.toml | 1 - 03_hacky_hello_world/Makefile | 37 ++++++++---- 03_hacky_hello_world/README.md | 8 +-- 03_hacky_hello_world/build.rs | 1 + 04_zero_overhead_abstraction/Cargo.toml | 1 - 04_zero_overhead_abstraction/Makefile | 37 ++++++++---- 04_zero_overhead_abstraction/README.md | 2 +- 04_zero_overhead_abstraction/build.rs | 1 + 05_safe_globals/Cargo.toml | 1 - 05_safe_globals/Makefile | 37 ++++++++---- 05_safe_globals/build.rs | 1 + 06_drivers_gpio_uart/Cargo.toml | 1 - 06_drivers_gpio_uart/Makefile | 37 ++++++++---- 06_drivers_gpio_uart/README.md | 18 +++--- 06_drivers_gpio_uart/build.rs | 1 + 07_uart_chainloader/Cargo.toml | 1 - 07_uart_chainloader/Makefile | 42 ++++++++----- 07_uart_chainloader/README.md | 39 +++++-------- 07_uart_chainloader/build.rs | 1 + 07_uart_chainloader/demo_payload_rpi3.img | Bin 6800 -> 6808 bytes 08_timestamps/Cargo.toml | 1 - 08_timestamps/Makefile | 37 ++++++++---- 08_timestamps/README.md | 39 +++++-------- 08_timestamps/build.rs | 1 + 09_hw_debug_JTAG/Cargo.toml | 1 - 09_hw_debug_JTAG/Makefile | 55 +++++++++++------- 09_hw_debug_JTAG/README.md | 38 ++++++------ 09_hw_debug_JTAG/build.rs | 1 + 10_privilege_level/Cargo.toml | 1 - 10_privilege_level/Makefile | 55 +++++++++++------- 10_privilege_level/build.rs | 1 + .../Cargo.toml | 1 - .../Makefile | 55 +++++++++++------- .../build.rs | 1 + 12_exceptions_part1_groundwork/Cargo.toml | 1 - 12_exceptions_part1_groundwork/Makefile | 55 +++++++++++------- 12_exceptions_part1_groundwork/build.rs | 1 + 13_integrated_testing/Makefile | 54 ++++++++++------- 13_integrated_testing/README.md | 55 +++++++----------- 13_integrated_testing/build.rs | 1 + 14_exceptions_part2_peripheral_IRQs/Makefile | 54 ++++++++++------- 14_exceptions_part2_peripheral_IRQs/build.rs | 1 + 15_virtual_mem_part2_mmio_remap/Makefile | 54 ++++++++++------- 15_virtual_mem_part2_mmio_remap/build.rs | 1 + X1_JTAG_boot/Cargo.toml | 1 - X1_JTAG_boot/Makefile | 37 ++++++++---- X1_JTAG_boot/build.rs | 1 + utils/color.mk.in | 5 ++ utils/minipush.rb | 1 + utils/miniterm.rb | 1 + 57 files changed, 570 insertions(+), 387 deletions(-) create mode 100644 utils/color.mk.in diff --git a/01_wait_forever/Cargo.toml b/01_wait_forever/Cargo.toml index 0595aa573..8a0187c43 100644 --- a/01_wait_forever/Cargo.toml +++ b/01_wait_forever/Cargo.toml @@ -4,7 +4,6 @@ version = "0.1.0" authors = ["Andre Richter "] edition = "2018" -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = [] diff --git a/01_wait_forever/Makefile b/01_wait_forever/Makefile index abecc406b..eb2cc7323 100644 --- a/01_wait_forever/Makefile +++ b/01_wait_forever/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -14,6 +16,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -d in_asm -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -24,6 +27,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -d in_asm -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -31,12 +35,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -53,8 +59,8 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) @@ -63,36 +69,45 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/01_wait_forever/build.rs b/01_wait_forever/build.rs index 40ee0284d..3f0a29110 100644 --- a/01_wait_forever/build.rs +++ b/01_wait_forever/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/02_runtime_init/Cargo.toml b/02_runtime_init/Cargo.toml index 0f798fcbe..a75809963 100644 --- a/02_runtime_init/Cargo.toml +++ b/02_runtime_init/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = [] diff --git a/02_runtime_init/Makefile b/02_runtime_init/Makefile index abecc406b..eb2cc7323 100644 --- a/02_runtime_init/Makefile +++ b/02_runtime_init/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -14,6 +16,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -d in_asm -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -24,6 +27,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -d in_asm -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -31,12 +35,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -53,8 +59,8 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) @@ -63,36 +69,45 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 50cce8798..9241a1950 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -33,9 +33,9 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml +[profile.release] +lto = true + - # The features section is used to select the target board. [features] default = [] + bsp_rpi3 = [] diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.S 02_runtime_init/src/_arch/aarch64/cpu/boot.S --- 01_wait_forever/src/_arch/aarch64/cpu/boot.S diff --git a/02_runtime_init/build.rs b/02_runtime_init/build.rs index 40ee0284d..3f0a29110 100644 --- a/02_runtime_init/build.rs +++ b/02_runtime_init/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/03_hacky_hello_world/Cargo.toml b/03_hacky_hello_world/Cargo.toml index 0f798fcbe..a75809963 100644 --- a/03_hacky_hello_world/Cargo.toml +++ b/03_hacky_hello_world/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = [] diff --git a/03_hacky_hello_world/Makefile b/03_hacky_hello_world/Makefile index 9d5a8f78b..75158b5b0 100644 --- a/03_hacky_hello_world/Makefile +++ b/03_hacky_hello_world/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -14,6 +16,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -24,6 +27,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -31,12 +35,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -53,8 +59,8 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) @@ -63,36 +69,45 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index c8dc66cc7..753aa0487 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -31,7 +31,7 @@ Kernel panic: Stopping here. diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile --- 02_runtime_init/Makefile +++ 03_hacky_hello_world/Makefile -@@ -11,7 +11,7 @@ +@@ -13,7 +13,7 @@ KERNEL_BIN = kernel8.img QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = raspi3 @@ -39,8 +39,8 @@ diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile + QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm - LINKER_FILE = src/bsp/raspberrypi/link.ld -@@ -21,7 +21,7 @@ + READELF_BINARY = aarch64-none-elf-readelf +@@ -24,7 +24,7 @@ KERNEL_BIN = kernel8.img QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = @@ -48,7 +48,7 @@ diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile + QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm - LINKER_FILE = src/bsp/raspberrypi/link.ld + READELF_BINARY = aarch64-none-elf-readelf diff -uNr 02_runtime_init/src/bsp/raspberrypi/console.rs 03_hacky_hello_world/src/bsp/raspberrypi/console.rs --- 02_runtime_init/src/bsp/raspberrypi/console.rs diff --git a/03_hacky_hello_world/build.rs b/03_hacky_hello_world/build.rs index 40ee0284d..3f0a29110 100644 --- a/03_hacky_hello_world/build.rs +++ b/03_hacky_hello_world/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/04_zero_overhead_abstraction/Cargo.toml b/04_zero_overhead_abstraction/Cargo.toml index 517455433..5deb70108 100644 --- a/04_zero_overhead_abstraction/Cargo.toml +++ b/04_zero_overhead_abstraction/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = [] diff --git a/04_zero_overhead_abstraction/Makefile b/04_zero_overhead_abstraction/Makefile index 9d5a8f78b..75158b5b0 100644 --- a/04_zero_overhead_abstraction/Makefile +++ b/04_zero_overhead_abstraction/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -14,6 +16,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -24,6 +27,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -31,12 +35,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -53,8 +59,8 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) @@ -63,36 +69,45 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index d20758d1c..aeb3bfb85 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -13,7 +13,7 @@ diff -uNr 03_hacky_hello_world/Cargo.toml 04_zero_overhead_abstraction/Cargo.toml --- 03_hacky_hello_world/Cargo.toml +++ 04_zero_overhead_abstraction/Cargo.toml -@@ -18,3 +18,8 @@ +@@ -17,3 +17,8 @@ ##-------------------------------------------------------------------------------------------------- [dependencies] diff --git a/04_zero_overhead_abstraction/build.rs b/04_zero_overhead_abstraction/build.rs index 40ee0284d..3f0a29110 100644 --- a/04_zero_overhead_abstraction/build.rs +++ b/04_zero_overhead_abstraction/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/05_safe_globals/Cargo.toml b/05_safe_globals/Cargo.toml index 517455433..5deb70108 100644 --- a/05_safe_globals/Cargo.toml +++ b/05_safe_globals/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = [] diff --git a/05_safe_globals/Makefile b/05_safe_globals/Makefile index 9d5a8f78b..75158b5b0 100644 --- a/05_safe_globals/Makefile +++ b/05_safe_globals/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -14,6 +16,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -24,6 +27,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -31,12 +35,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -53,8 +59,8 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) @@ -63,36 +69,45 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/05_safe_globals/build.rs b/05_safe_globals/build.rs index 40ee0284d..3f0a29110 100644 --- a/05_safe_globals/build.rs +++ b/05_safe_globals/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/06_drivers_gpio_uart/Cargo.toml b/06_drivers_gpio_uart/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/06_drivers_gpio_uart/Cargo.toml +++ b/06_drivers_gpio_uart/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/06_drivers_gpio_uart/Makefile b/06_drivers_gpio_uart/Makefile index ee7ebe389..07eb97b2e 100644 --- a/06_drivers_gpio_uart/Makefile +++ b/06_drivers_gpio_uart/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -30,6 +33,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -37,12 +41,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -61,8 +67,8 @@ DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -79,19 +85,22 @@ EXEC_MINITERM = ruby ../utils/miniterm.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -99,19 +108,25 @@ miniterm: @$(DOCKER_MINITERM) $(EXEC_MINITERM) $(DEV_SERIAL) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index a4176ca51..8b8e076d7 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -110,8 +110,8 @@ Miniterm 1.0 diff -uNr 05_safe_globals/Cargo.toml 06_drivers_gpio_uart/Cargo.toml --- 05_safe_globals/Cargo.toml +++ 06_drivers_gpio_uart/Cargo.toml -@@ -10,8 +10,8 @@ - # The features section is used to select the target board. +@@ -9,8 +9,8 @@ + [features] default = [] -bsp_rpi3 = [] @@ -121,7 +121,7 @@ diff -uNr 05_safe_globals/Cargo.toml 06_drivers_gpio_uart/Cargo.toml ##-------------------------------------------------------------------------------------------------- ## Dependencies -@@ -19,6 +19,9 @@ +@@ -18,6 +18,9 @@ [dependencies] @@ -135,7 +135,7 @@ diff -uNr 05_safe_globals/Cargo.toml 06_drivers_gpio_uart/Cargo.toml diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile --- 05_safe_globals/Makefile +++ 06_drivers_gpio_uart/Makefile -@@ -5,6 +5,12 @@ +@@ -7,6 +7,12 @@ # Default to the RPi3 BSP ?= rpi3 @@ -148,15 +148,15 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile # BSP-specific arguments ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat -@@ -52,13 +58,23 @@ +@@ -58,13 +64,23 @@ DOCKER_IMAGE = rustembedded/osdev-utils DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils +DOCKER_ARG_DEV = --privileged -v /dev:/dev - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) - DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +# Dockerize commands that require USB device passthrough only on Linux @@ -174,7 +174,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile all: $(KERNEL_BIN) -@@ -79,6 +95,9 @@ +@@ -88,6 +104,9 @@ @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -182,7 +182,7 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile + @$(DOCKER_MINITERM) $(EXEC_MINITERM) $(DEV_SERIAL) + clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs diff --git a/06_drivers_gpio_uart/build.rs b/06_drivers_gpio_uart/build.rs index 40ee0284d..3f0a29110 100644 --- a/06_drivers_gpio_uart/build.rs +++ b/06_drivers_gpio_uart/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/07_uart_chainloader/Cargo.toml b/07_uart_chainloader/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/07_uart_chainloader/Cargo.toml +++ b/07_uart_chainloader/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/07_uart_chainloader/Makefile b/07_uart_chainloader/Makefile index d32147d65..4fc73f554 100644 --- a/07_uart_chainloader/Makefile +++ b/07_uart_chainloader/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img @@ -31,6 +34,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img @@ -39,12 +43,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -63,8 +69,8 @@ DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -82,22 +88,26 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) -qemu qemuasm: - @echo "This board is not yet supported for QEMU." +qemu: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) qemuasm: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU with ASM output") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm endif @@ -105,22 +115,26 @@ chainboot: @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(CHAINBOOT_DEMO_PAYLOAD) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ - --section .text \ - --section .got \ + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 91242143d..e67c78109 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -110,9 +110,9 @@ Binary files 06_drivers_gpio_uart/demo_payload_rpi4.img and 07_uart_chainloader/ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile --- 06_drivers_gpio_uart/Makefile +++ 07_uart_chainloader/Makefile -@@ -21,7 +21,8 @@ - OBJDUMP_BINARY = aarch64-none-elf-objdump +@@ -24,7 +24,8 @@ NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic @@ -120,9 +120,9 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img -@@ -31,7 +32,8 @@ - OBJDUMP_BINARY = aarch64-none-elf-objdump +@@ -35,7 +36,8 @@ NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic @@ -130,7 +130,7 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile endif # Export for build.rs -@@ -68,13 +70,14 @@ +@@ -74,13 +76,14 @@ ifeq ($(UNAME_S),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) @@ -148,18 +148,13 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile all: $(KERNEL_BIN) -@@ -88,15 +91,18 @@ - $(DOC_CMD) --document-private-items --open - - ifeq ($(QEMU_MACHINE_TYPE),) --qemu: -+qemu qemuasm: - @echo "This board is not yet supported for QEMU." - else +@@ -102,10 +105,14 @@ qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +qemuasm: $(KERNEL_BIN) ++ $(call colorecho, "\nLaunching QEMU with ASM output") + @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm endif @@ -169,19 +164,15 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(CHAINBOOT_DEMO_PAYLOAD) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -108,7 +114,10 @@ - readelf --headers $(KERNEL_ELF) - - objdump: $(KERNEL_ELF) -- @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt -+ @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ -+ --section .text \ -+ --section .got \ -+ $(KERNEL_ELF) | rustfilt + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +@@ -122,6 +129,7 @@ + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ ++ --section .got \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs --- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs diff --git a/07_uart_chainloader/build.rs b/07_uart_chainloader/build.rs index 40ee0284d..3f0a29110 100644 --- a/07_uart_chainloader/build.rs +++ b/07_uart_chainloader/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 13cf8aa1ed21d4896c05b5c25caf8d17c69289fa..a46c0d4149a22954048d876138fda7825a398700 100755 GIT binary patch delta 2141 zcmZ8iZA?>F7=BN03%%v5EiIIa&=wF{@Uvp6i|Iuq6VQy5ZvNO5a9_&D3@%yr!|fJz zKU`hXlS8K2A4UJHjV4QwWn|fgnk-o|Gf6i8Ef%(HrNu3aS;BRbyZ78)jPoY9H}8F( z_j%v*ocG)&+b^~!Yvpf^Fy#TykMqLdJ+%uK%pfTdFifZwFn1at=7)InoC#)60yrYz znK(-D-KnC1!0YZECDwSf2k}-11Q!KMAQyyiJ_KRW2(xc+z9dcD()gtVf#VxAekta( zwiN~2FbDZ^_&5siWS+S49EAM_m_3BaA`%0q>;wLNrNQ3u+W#dlBbN)D({r`Ln7b0e z;v(F@nD!7c!Jh|AK&PQ2loLwRr9@I%Yr!5w{Yym5UpB}P>oZB=UJ>Z16YOnPuy01f zH5h;Kn`-02S9zbQ;y)$t5I`w-{(P87}y^?{rqrk~cj^;nH1 zZ0;a$z=50xI%kXgl;9O4ePnZJq`XiTu;^mv#qxn`o87?#DGA7(`z8Pjav=la^&a7; zk7}g3JL=X@g1{VaxWptWqqyo46DXH)3pr_7!rAbq{p1|9UpE4 z`*~OmEea*+5^QiAHYj65L)dVuQ3`v_K*!OgJ{#Bv(cvxw$gv2B`8X2V98$-51h1jJ zgJ>_epJVg*3?SBv*t&I#Xz#j+VNH#VqqV$X4qdCnqAR#U&RN&u_K%^n1x~5vYz{f2 zVt@HS0<9e(mcSrd8^>woq;+e@(Aw|O+7YytH)BC-d(lb@T06X|#<-Q-MnJc2l6Vw6 z?5J>tc-e14A6cB)VSIrQ@62h_^9F~viTa0KG~4`jDpO2S-0$U?pNrc}BsgO!yG6*C z>`wW%D(?7ja0a0UJB;Dl*EDl4lwl)&6f?32P4GH zjZ(@X0rlG?B_o*A9um2i6(F4v9O>+udLvs5skl#0nad;v5ED@=dxlW|WEIU}Lz&{D z6yJESXCuvE^=z{-wI9z|uGy4AZqh$pYp1GgNG&&z)G|)OGEPgLm05RMXd^}ElW!XW ztk3Eey>;~7=_;Ha_Lg;%Z*5Jz5Vt}**T5r^#2U#K)b&r-(OF*;r+seq*&iTns2tjH zn>Bc9iRG-VPM?)Nx(amaNJsK$HbT1@bHvVWg+Bv0DLL?jB1D?A^mG@dq=)UcZ*^ic%1;K)joc0seI(&7 z?P2fRLt#%7eKp=dhfVS5S!mR1nN{@y{~mMAgfSi!_jD@(lQcKtbOZw?m=hnxqvk#A z4|`XCd$-~bcPkNM&Wkv=I2cKs+a!2e=rCsPoo}HV|1GF#i|5=AV3qT~Q`H#gmDJm0 zVJ!&CpKP+yMHbj6m6vRI=v!*A7|QYe_upi_4j(bIamT0fQ<%1gi;XnX05n$&iYgm% zG=vmy-i!mEMZ=bE16_EGCM)Vr-lBlF5pUp&&omVlKeUN*-VrpGFP`%VKTgo1eQfZU zr5Ew$M~zK6t>UpYwHo+86}#@NXLp@0f4>2X;?)9$r%R9bb{+KeAK12iyXVEuD0-A- zwyJ8f!u(Y~^z5%GaJ5AInXE3v=%RLYpFDX2|My^2e_y4)f0j4II;$h{VOJsBsNr)})uZDMJ>I3qcx<%mS-wZd*Q>hpxJc*M>9O9e0q!&6db8$TccG1>4#)NQ3pVMh z7tcBi=oCv~^omf3#@GTzuT>Nx7gJmz@)z4)bLh#t45KyGw!8Y0D%M+Ob2TT(e&%jUA4ThamLztoYnba}pLRBSdQg#TW!3Yc@i00w5EHTzWAE zi5>v&AcV4A1Z}4sqq04)&FRXehY^o@A-0U1avTzEoTn(+i7KytRF>PSyfw(}+vtdK z?l}4&Urnr_L-|oiG@w)Xv}aUClC-w{Plw3v`q-*d2@{;ij-I6_v&1jxuXRY`yZw9f|~*$tlH z88JJA01uMG+38hw?nUgFFAxjPiP=dDo5fi|OhU|y0awIeR2NBMHHJ_zGkxwrj3+FK zYoblaDO;S?So06eOjLDyEs`_pL{bS!5xHS>&!#{uOr&TqxjUOf8F?T9@nA^&ChAv; zUz%IuVFE>7=(s~MBsd% zRn@;o>68|_gy6dZOjp_z!xTx?u`+T^E8(vAedq*XN*AR<{9jA_P=(e1U*d-oH(?2potmnkEq4>Awv0Dp8L!Vz%V34mDe%qZ$I*W0hN%65<)O=jyQ||G&mlp49J_~?6PfFvoVpjOybSk- zXFo!U;($1ZIaIAQ8LajTKf@(dd-Eu&m|iSNXVBTOBsCT4O@m# ze2^$bXDVgv#b$M;*yElpK3pBg2kJap)B(uuBU6F;6g%gMC!mKO9qXn??78%7dUrLK z6V>ByHIF%E%cbQ+%81&P;xV5$7PUiBzK2g|g5C2B9q3OPk%5#kNTd}RpAWpFd3-+d z0AYFrJ$GFS(;(`@qg6N0xv#@r&i}$RGlG@LyiOJ>F)(hkx0_nYW%gmydv1Iz!hTGK z;f7JG$YplO+eMu0l=q7I7pyAQ+|l8XM1X!S2j@28$)4vxtvFMyXn2vM+#f9u!tXTMRv*Ol8;r)7Qh!P$%?)p zJKx%#+%8~J9ASLw`M6JN^RA&MLkIS6?e7mAOr}wzAL?rMDhYph=G9SR{moyw760GS zQ_S7c(s{I{X2k}mj#Jhe)vk`iY){Leda$*|a#_|}R;<+wNelN{_-1R3?NuHBu xV8kCM%Pir4x_{O7t+g4qm7LECexF +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -30,6 +33,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -37,12 +41,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -61,8 +67,8 @@ DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -79,19 +85,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -99,19 +108,25 @@ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 43c165386..55c12a7f8 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -49,9 +49,9 @@ Binary files 07_uart_chainloader/demo_payload_rpi4.img and 08_timestamps/demo_pa diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile --- 07_uart_chainloader/Makefile +++ 08_timestamps/Makefile -@@ -21,8 +21,7 @@ - OBJDUMP_BINARY = aarch64-none-elf-objdump +@@ -24,8 +24,7 @@ NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img @@ -59,9 +59,9 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img -@@ -32,8 +31,7 @@ - OBJDUMP_BINARY = aarch64-none-elf-objdump +@@ -36,8 +35,7 @@ NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img @@ -69,7 +69,7 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile endif # Export for build.rs -@@ -76,8 +74,7 @@ +@@ -82,8 +80,7 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) EXEC_MINIPUSH = ruby ../utils/minipush.rb @@ -79,18 +79,13 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile all: $(KERNEL_BIN) -@@ -91,18 +88,15 @@ - $(DOC_CMD) --document-private-items --open - - ifeq ($(QEMU_MACHINE_TYPE),) --qemu qemuasm: -+qemu: - @echo "This board is not yet supported for QEMU." - else +@@ -105,14 +102,10 @@ qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) - -qemuasm: $(KERNEL_BIN) +- $(call colorecho, "\nLaunching QEMU with ASM output") - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm endif @@ -100,19 +95,15 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -114,10 +108,7 @@ - readelf --headers $(KERNEL_ELF) - - objdump: $(KERNEL_ELF) -- @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ -- --section .text \ -- --section .got \ -- $(KERNEL_ELF) | rustfilt -+ @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +@@ -129,7 +122,6 @@ + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ +- --section .got \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs 08_timestamps/src/_arch/aarch64/cpu/boot.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs diff --git a/08_timestamps/build.rs b/08_timestamps/build.rs index 40ee0284d..3f0a29110 100644 --- a/08_timestamps/build.rs +++ b/08_timestamps/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/09_hw_debug_JTAG/Cargo.toml b/09_hw_debug_JTAG/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/09_hw_debug_JTAG/Cargo.toml +++ b/09_hw_debug_JTAG/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/09_hw_debug_JTAG/Makefile b/09_hw_debug_JTAG/Makefile index 9105d297f..038244e7e 100644 --- a/09_hw_debug_JTAG/Makefile +++ b/09_hw_debug_JTAG/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -32,6 +35,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -41,12 +45,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -67,9 +73,9 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -91,19 +97,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -114,33 +123,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/09_hw_debug_JTAG/README.md b/09_hw_debug_JTAG/README.md index 918f72582..b13e1fa7f 100644 --- a/09_hw_debug_JTAG/README.md +++ b/09_hw_debug_JTAG/README.md @@ -306,25 +306,25 @@ Thanks to [@naotaco](https://github.com/naotaco) for laying the groundwork for t diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile --- 08_timestamps/Makefile +++ 09_hw_debug_JTAG/Makefile -@@ -20,6 +20,8 @@ - QEMU_RELEASE_ARGS = -serial stdio -display none +@@ -23,6 +23,8 @@ OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) -@@ -30,6 +32,8 @@ - QEMU_RELEASE_ARGS = -serial stdio -display none +@@ -34,6 +36,8 @@ OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -@@ -59,9 +63,12 @@ +@@ -65,9 +69,12 @@ DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils @@ -332,12 +332,12 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -+DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) - DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) ++DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux -@@ -69,12 +76,17 @@ +@@ -75,12 +82,17 @@ DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) @@ -356,7 +356,7 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile all: $(KERNEL_BIN) -@@ -98,6 +110,23 @@ +@@ -107,6 +119,19 @@ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) @@ -364,20 +364,16 @@ diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +openocd: ++ $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + -+define gen_gdb -+ RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) -+ @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -+endef -+ -+gdb: -+ $(call gen_gdb,-C debuginfo=2) -+ -+gdb-opt0: -+ $(call gen_gdb,-C debuginfo=2 -C opt-level=0) ++gdb: RUSTC_MISC_ARGS += -C debuginfo=2 ++gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 ++gdb gdb-opt0: $(KERNEL_ELF) ++ $(call colorecho, "\nLaunching GDB") ++ @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) ``` diff --git a/09_hw_debug_JTAG/build.rs b/09_hw_debug_JTAG/build.rs index 40ee0284d..3f0a29110 100644 --- a/09_hw_debug_JTAG/build.rs +++ b/09_hw_debug_JTAG/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/10_privilege_level/Cargo.toml b/10_privilege_level/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/10_privilege_level/Cargo.toml +++ b/10_privilege_level/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/10_privilege_level/Makefile b/10_privilege_level/Makefile index 9105d297f..038244e7e 100644 --- a/10_privilege_level/Makefile +++ b/10_privilege_level/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -32,6 +35,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -41,12 +45,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -67,9 +73,9 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -91,19 +97,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -114,33 +123,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/10_privilege_level/build.rs b/10_privilege_level/build.rs index 40ee0284d..3f0a29110 100644 --- a/10_privilege_level/build.rs +++ b/10_privilege_level/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/11_virtual_mem_part1_identity_mapping/Cargo.toml b/11_virtual_mem_part1_identity_mapping/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/11_virtual_mem_part1_identity_mapping/Cargo.toml +++ b/11_virtual_mem_part1_identity_mapping/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/11_virtual_mem_part1_identity_mapping/Makefile b/11_virtual_mem_part1_identity_mapping/Makefile index 9105d297f..038244e7e 100644 --- a/11_virtual_mem_part1_identity_mapping/Makefile +++ b/11_virtual_mem_part1_identity_mapping/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -32,6 +35,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -41,12 +45,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -67,9 +73,9 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -91,19 +97,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -114,33 +123,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/11_virtual_mem_part1_identity_mapping/build.rs b/11_virtual_mem_part1_identity_mapping/build.rs index 40ee0284d..3f0a29110 100644 --- a/11_virtual_mem_part1_identity_mapping/build.rs +++ b/11_virtual_mem_part1_identity_mapping/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/12_exceptions_part1_groundwork/Cargo.toml b/12_exceptions_part1_groundwork/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/12_exceptions_part1_groundwork/Cargo.toml +++ b/12_exceptions_part1_groundwork/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/12_exceptions_part1_groundwork/Makefile b/12_exceptions_part1_groundwork/Makefile index 9105d297f..038244e7e 100644 --- a/12_exceptions_part1_groundwork/Makefile +++ b/12_exceptions_part1_groundwork/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -32,6 +35,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -41,12 +45,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -67,9 +73,9 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -91,19 +97,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -114,33 +123,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/12_exceptions_part1_groundwork/build.rs b/12_exceptions_part1_groundwork/build.rs index 40ee0284d..3f0a29110 100644 --- a/12_exceptions_part1_groundwork/build.rs +++ b/12_exceptions_part1_groundwork/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/13_integrated_testing/Makefile b/13_integrated_testing/Makefile index 74e67da50..32d139243 100644 --- a/13_integrated_testing/Makefile +++ b/13_integrated_testing/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -21,6 +23,7 @@ ifeq ($(BSP),rpi3) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -34,6 +37,7 @@ else ifeq ($(BSP),rpi4) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -81,10 +85,10 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -106,19 +110,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu test: - @echo $(QEMU_MISSING_STRING) + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) define KERNEL_TEST_RUNNER @@ -134,10 +141,11 @@ endef export KERNEL_TEST_RUNNER test: FEATURES += --features test_build test: + $(call colorecho, "\nCompiling test(s) - $(BSP)") @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh @chmod +x target/kernel_test_runner.sh - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif chainboot: $(KERNEL_BIN) @@ -147,33 +155,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 3f62ef6c7..c0fc217b9 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -819,12 +819,7 @@ diff -uNr 12_exceptions_part1_groundwork/.cargo/config.toml 13_integrated_testin diff -uNr 12_exceptions_part1_groundwork/Cargo.toml 13_integrated_testing/Cargo.toml --- 12_exceptions_part1_groundwork/Cargo.toml +++ 13_integrated_testing/Cargo.toml -@@ -7,22 +7,49 @@ - [profile.release] - lto = true - --# The features section is used to select the target board. - [features] +@@ -11,17 +11,45 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] @@ -875,23 +870,23 @@ diff -uNr 12_exceptions_part1_groundwork/Cargo.toml 13_integrated_testing/Cargo. diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile --- 12_exceptions_part1_groundwork/Makefile +++ 13_integrated_testing/Makefile -@@ -18,6 +18,7 @@ +@@ -20,6 +20,7 @@ QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = raspi3 QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm - OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg -@@ -30,6 +31,7 @@ + READELF_BINARY = aarch64-none-elf-readelf +@@ -33,6 +34,7 @@ QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm - OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg -@@ -41,18 +43,30 @@ + READELF_BINARY = aarch64-none-elf-readelf +@@ -45,6 +47,15 @@ # Export for build.rs export LINKER_FILE @@ -904,19 +899,10 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile + endif +endif + -+QEMU_MISSING_STRING = "This board is not yet supported for QEMU." -+ - RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) - RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs + QEMU_MISSING_STRING = "This board is not yet supported for QEMU." --FEATURES = bsp_$(BSP) -+FEATURES = --features bsp_$(BSP) - COMPILER_ARGS = --target=$(TARGET) \ -- --features $(FEATURES) \ -+ $(FEATURES) \ - --release - - RUSTC_CMD = cargo rustc $(COMPILER_ARGS) + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) +@@ -59,6 +70,7 @@ DOC_CMD = cargo doc $(COMPILER_ARGS) CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) CHECK_CMD = cargo check $(COMPILER_ARGS) @@ -924,15 +910,15 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -@@ -69,6 +83,7 @@ +@@ -75,6 +87,7 @@ - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) - DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -+DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) - DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) + DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) ++DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux -@@ -85,8 +100,8 @@ +@@ -91,8 +104,8 @@ EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) EXEC_MINIPUSH = ruby ../utils/minipush.rb @@ -943,16 +929,16 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile all: $(KERNEL_BIN) -@@ -100,11 +115,29 @@ - $(DOC_CMD) --document-private-items --open +@@ -108,12 +121,31 @@ + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) -qemu: -- @echo "This board is not yet supported for QEMU." +qemu test: -+ @echo $(QEMU_MISSING_STRING) + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +define KERNEL_TEST_RUNNER @@ -968,10 +954,11 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile +export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build +test: ++ $(call colorecho, "\nCompiling test(s) - $(BSP)") + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh -+ RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) ++ @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif chainboot: $(KERNEL_BIN) diff --git a/13_integrated_testing/build.rs b/13_integrated_testing/build.rs index 40ee0284d..3f0a29110 100644 --- a/13_integrated_testing/build.rs +++ b/13_integrated_testing/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/14_exceptions_part2_peripheral_IRQs/Makefile b/14_exceptions_part2_peripheral_IRQs/Makefile index 74e67da50..32d139243 100644 --- a/14_exceptions_part2_peripheral_IRQs/Makefile +++ b/14_exceptions_part2_peripheral_IRQs/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -21,6 +23,7 @@ ifeq ($(BSP),rpi3) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -34,6 +37,7 @@ else ifeq ($(BSP),rpi4) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -81,10 +85,10 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -106,19 +110,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu test: - @echo $(QEMU_MISSING_STRING) + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) define KERNEL_TEST_RUNNER @@ -134,10 +141,11 @@ endef export KERNEL_TEST_RUNNER test: FEATURES += --features test_build test: + $(call colorecho, "\nCompiling test(s) - $(BSP)") @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh @chmod +x target/kernel_test_runner.sh - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif chainboot: $(KERNEL_BIN) @@ -147,33 +155,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/14_exceptions_part2_peripheral_IRQs/build.rs b/14_exceptions_part2_peripheral_IRQs/build.rs index 40ee0284d..3f0a29110 100644 --- a/14_exceptions_part2_peripheral_IRQs/build.rs +++ b/14_exceptions_part2_peripheral_IRQs/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 74e67da50..32d139243 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -21,6 +23,7 @@ ifeq ($(BSP),rpi3) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -34,6 +37,7 @@ else ifeq ($(BSP),rpi4) QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -81,10 +85,10 @@ DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_ARG_NET = --network host -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -106,19 +110,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu test: - @echo $(QEMU_MISSING_STRING) + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) define KERNEL_TEST_RUNNER @@ -134,10 +141,11 @@ endef export KERNEL_TEST_RUNNER test: FEATURES += --features test_build test: + $(call colorecho, "\nCompiling test(s) - $(BSP)") @mkdir -p target @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh @chmod +x target/kernel_test_runner.sh - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif chainboot: $(KERNEL_BIN) @@ -147,33 +155,35 @@ jtagboot: @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) openocd: + $(call colorecho, "\nLaunching OpenOCD") @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) -define gen_gdb - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC) $1" $(RUSTC_CMD) - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) -endef - -gdb: - $(call gen_gdb,-C debuginfo=2) - -gdb-opt0: - $(call gen_gdb,-C debuginfo=2 -C opt-level=0) +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/15_virtual_mem_part2_mmio_remap/build.rs b/15_virtual_mem_part2_mmio_remap/build.rs index 40ee0284d..3f0a29110 100644 --- a/15_virtual_mem_part2_mmio_remap/build.rs +++ b/15_virtual_mem_part2_mmio_remap/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/X1_JTAG_boot/Cargo.toml b/X1_JTAG_boot/Cargo.toml index b8906da1f..df5f7f974 100644 --- a/X1_JTAG_boot/Cargo.toml +++ b/X1_JTAG_boot/Cargo.toml @@ -7,7 +7,6 @@ edition = "2018" [profile.release] lto = true -# The features section is used to select the target board. [features] default = [] bsp_rpi3 = ["register"] diff --git a/X1_JTAG_boot/Makefile b/X1_JTAG_boot/Makefile index 27944c93d..4206c5282 100644 --- a/X1_JTAG_boot/Makefile +++ b/X1_JTAG_boot/Makefile @@ -2,6 +2,8 @@ ## ## Copyright (c) 2018-2021 Andre Richter +include ../utils/color.mk.in + # Default to the RPi3 BSP ?= rpi3 @@ -20,6 +22,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) @@ -30,6 +33,7 @@ else ifeq ($(BSP),rpi4) QEMU_RELEASE_ARGS = -serial stdio -display none OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -37,12 +41,14 @@ endif # Export for build.rs export LINKER_FILE +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs -FEATURES = bsp_$(BSP) +FEATURES = --features bsp_$(BSP) COMPILER_ARGS = --target=$(TARGET) \ - --features $(FEATURES) \ + $(FEATURES) \ --release RUSTC_CMD = cargo rustc $(COMPILER_ARGS) @@ -61,8 +67,8 @@ DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_ELFTOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) @@ -79,19 +85,22 @@ EXEC_MINIPUSH = ruby ../utils/minipush.rb all: $(KERNEL_BIN) $(KERNEL_ELF): - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) doc: - $(DOC_CMD) --document-private-items --open + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: - @echo "This board is not yet supported for QEMU." + $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif @@ -99,19 +108,25 @@ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) clippy: - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) clean: rm -rf target $(KERNEL_BIN) readelf: $(KERNEL_ELF) - readelf --headers $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) objdump: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(OBJDUMP_BINARY) --disassemble --demangle $(KERNEL_ELF) | rustfilt + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) - @$(DOCKER_ELFTOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt # For rust-analyzer check: diff --git a/X1_JTAG_boot/build.rs b/X1_JTAG_boot/build.rs index 40ee0284d..3f0a29110 100644 --- a/X1_JTAG_boot/build.rs +++ b/X1_JTAG_boot/build.rs @@ -4,4 +4,5 @@ fn main() { let linker_file = env::var("LINKER_FILE").unwrap(); println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); } diff --git a/utils/color.mk.in b/utils/color.mk.in new file mode 100644 index 000000000..1945a10c0 --- /dev/null +++ b/utils/color.mk.in @@ -0,0 +1,5 @@ +define colorecho + @tput setaf 6 + @echo $1 + @tput sgr0 +endef diff --git a/utils/minipush.rb b/utils/minipush.rb index bda97bb1f..14c1e6a5f 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -112,6 +112,7 @@ def run end end +puts puts 'Minipush 1.0'.cyan puts diff --git a/utils/miniterm.rb b/utils/miniterm.rb index 091566697..ff0c64fb6 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -126,6 +126,7 @@ def run end if __FILE__ == $PROGRAM_NAME + puts puts 'Miniterm 1.0'.cyan puts From e7b890c08481e508fa9a086f638f246458653b2f Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 13 Mar 2021 00:18:18 +0100 Subject: [PATCH 035/214] test fix for CI tput errors --- utils/color.mk.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/color.mk.in b/utils/color.mk.in index 1945a10c0..851bac875 100644 --- a/utils/color.mk.in +++ b/utils/color.mk.in @@ -1,5 +1,5 @@ define colorecho - @tput setaf 6 + @tput setaf 6 2> /dev/null || true @echo $1 - @tput sgr0 + @tput sgr0 2> /dev/null || true endef From eb2bee6bb1489cfff247d41f3bf4cbe80fbe6ca8 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 13 Mar 2021 00:25:14 +0100 Subject: [PATCH 036/214] newline --- 01_wait_forever/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/01_wait_forever/README.md b/01_wait_forever/README.md index 785e9817e..66dc29d29 100644 --- a/01_wait_forever/README.md +++ b/01_wait_forever/README.md @@ -33,6 +33,7 @@ ### Test it In the project folder, invoke QEMU and observe the CPU core spinning on `wfe`: + ```console $ make qemu [...] From d09374710dda2221320ca5fc599db1cc412720f1 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 15 Mar 2021 22:07:01 +0100 Subject: [PATCH 037/214] Some rework on virtual memory code - Mostly more spearation of concerns in 15. - Cleanups in other parts. --- .../README.md | 194 +++-- .../src/_arch/aarch64/memory/mmu.rs | 60 +- .../aarch64/memory/mmu/translation_table.rs | 54 +- .../src/bsp/raspberrypi/memory/mmu.rs | 4 +- .../src/main.rs | 5 +- .../src/memory/mmu.rs | 39 +- 12_exceptions_part1_groundwork/README.md | 14 +- .../src/_arch/aarch64/memory/mmu.rs | 60 +- .../aarch64/memory/mmu/translation_table.rs | 54 +- .../src/bsp/raspberrypi/memory/mmu.rs | 4 +- 12_exceptions_part1_groundwork/src/main.rs | 5 +- .../src/memory/mmu.rs | 39 +- 13_integrated_testing/README.md | 24 +- .../src/_arch/aarch64/memory/mmu.rs | 60 +- .../aarch64/memory/mmu/translation_table.rs | 54 +- .../src/bsp/raspberrypi/memory/mmu.rs | 4 +- 13_integrated_testing/src/lib.rs | 1 + 13_integrated_testing/src/main.rs | 4 +- 13_integrated_testing/src/memory/mmu.rs | 39 +- .../tests/02_exception_sync_page_fault.rs | 2 +- 14_exceptions_part2_peripheral_IRQs/README.md | 38 +- .../src/_arch/aarch64/memory/mmu.rs | 60 +- .../aarch64/memory/mmu/translation_table.rs | 54 +- .../src/bsp/raspberrypi/memory/mmu.rs | 4 +- .../src/main.rs | 4 +- .../src/memory/mmu.rs | 39 +- .../src/synchronization.rs | 18 + .../tests/02_exception_sync_page_fault.rs | 2 +- 15_virtual_mem_part2_mmio_remap/README.md | 805 +++++++++++------- .../src/_arch/aarch64/memory/mmu.rs | 94 +- .../aarch64/memory/mmu/translation_table.rs | 100 +-- .../src/bsp/raspberrypi/console.rs | 4 +- .../src/bsp/raspberrypi/memory/mmu.rs | 65 +- 15_virtual_mem_part2_mmio_remap/src/lib.rs | 1 + 15_virtual_mem_part2_mmio_remap/src/main.rs | 13 +- 15_virtual_mem_part2_mmio_remap/src/memory.rs | 53 +- .../src/memory/mmu.rs | 92 +- .../src/memory/mmu/mapping_record.rs | 18 +- .../src/memory/mmu/translation_table.rs | 15 +- .../src/memory/mmu/types.rs | 15 +- .../src/synchronization.rs | 18 + .../tests/02_exception_sync_page_fault.rs | 14 +- 42 files changed, 1383 insertions(+), 863 deletions(-) diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 7b822f68c..270c8422a 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -115,12 +115,12 @@ descriptors). In `translation_table.rs`, there is a definition of the actual translation table struct which is generic over the number of `LVL2` tables. The latter depends on the size of the target board's memory. Naturally, the `BSP` knows these details about the target board, and provides the size -through the constant `bsp::memory::mmu::KernelAddrSpaceSize::SIZE`. +through the constant `bsp::memory::mmu::KernelAddrSpace::SIZE`. This information is used by `translation_table.rs` to calculate the number of needed `LVL2` tables. Since one `LVL2` table in a `64 KiB` configuration covers `512 MiB`, all that needs to be done is to -divide `KernelAddrSpaceSize::SIZE` by `512 MiB` (there are several compile-time checks in place that -ensure that `KernelAddrSpaceSize` is a multiple of `512 MiB`). +divide `KernelAddrSpace::SIZE` by `512 MiB` (there are several compile-time checks in place that +ensure that `KernelAddrSpace::SIZE` is a multiple of `512 MiB`). The final table type is exported as `KernelTranslationTable`. Below is the respective excerpt from `translation_table.rs`: @@ -144,7 +144,7 @@ struct PageDescriptor { value: u64, } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -175,10 +175,6 @@ tables: //-------------------------------------------------------------------------------------------------- /// The kernel translation tables. -/// -/// # Safety -/// -/// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); ``` @@ -213,10 +209,10 @@ Afterwards, the [Translation Table Base Register 0 - EL1] is set up with the bas `lvl2` tables and the [Translation Control Register - EL1] is configured: ```rust - // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); +// Set the "Translation Table Base Register". +TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); - self.configure_translation_control(); +self.configure_translation_control(); ``` Finally, the `MMU` is turned on through the [System Control Register - EL1]. The last step also @@ -297,7 +293,7 @@ unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } ``` @@ -439,8 +435,8 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ -+ Block = 0, -+ Table = 1 ++ Reserved_Invalid = 0, ++ Page = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ @@ -468,19 +464,19 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + value: u64, +} + -+trait BaseAddr { -+ fn base_addr_u64(&self) -> u64; -+ fn base_addr_usize(&self) -> usize; ++trait StartAddr { ++ fn phys_start_addr_u64(&self) -> u64; ++ fn phys_start_addr_usize(&self) -> usize; +} + -+const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; ++const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -+/// aligned, hence the "reverse" order of appearance. ++/// aligned, so the lvl3 is put first. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { @@ -498,12 +494,13 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 +// Private Code +//-------------------------------------------------------------------------------------------------- + -+impl BaseAddr for [T; N] { -+ fn base_addr_u64(&self) -> u64 { ++// The binary is still identity mapped, so we don't need to convert here. ++impl StartAddr for [T; N] { ++ fn phys_start_addr_u64(&self) -> u64 { + self as *const T as u64 + } + -+ fn base_addr_usize(&self) -> usize { ++ fn phys_start_addr_usize(&self) -> usize { + self as *const _ as usize + } +} @@ -517,14 +514,14 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + } + + /// Create an instance pointing to the supplied address. -+ pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { ++ pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { + let val = InMemoryRegister::::new(0); + -+ let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; ++ let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; + val.write( -+ STAGE1_TABLE_DESCRIPTOR::VALID::True ++ STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table -+ + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), ++ + STAGE1_TABLE_DESCRIPTOR::VALID::True, + ); + + TableDescriptor { value: val.get() } @@ -577,16 +574,16 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + } + + /// Create an instance. -+ pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { ++ pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { + let val = InMemoryRegister::::new(0); + -+ let shifted = output_addr as u64 >> Granule64KiB::SHIFT; ++ let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; + val.write( -+ STAGE1_PAGE_DESCRIPTOR::VALID::True ++ STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + + STAGE1_PAGE_DESCRIPTOR::AF::True -+ + attribute_fields.into() -+ + STAGE1_PAGE_DESCRIPTOR::TYPE::Table -+ + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), ++ + STAGE1_PAGE_DESCRIPTOR::TYPE::Page ++ + STAGE1_PAGE_DESCRIPTOR::VALID::True ++ + attribute_fields.clone().into(), + ); + + Self { value: val.get() } @@ -599,10 +596,9 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + +impl FixedSizeTranslationTable { + /// Create an instance. -+ #[allow(clippy::assertions_on_constants)] + pub const fn new() -> Self { ++ // Can't have a zero-sized address space. + assert!(NUM_TABLES > 0); -+ assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -618,15 +614,15 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { + for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { + *l2_entry = -+ TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); ++ TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); + + for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { + let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); + -+ let (output_addr, attribute_fields) = ++ let (phys_output_addr, attribute_fields) = + bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; + -+ *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); ++ *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); + } + } + @@ -634,15 +630,15 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + } + + /// The translation table's base address to be used for programming the MMU. -+ pub fn base_address(&self) -> u64 { -+ self.lvl2.base_addr_u64() ++ pub fn phys_base_address(&self) -> u64 { ++ self.lvl2.phys_start_addr_u64() + } +} diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs --- 10_privilege_level/src/_arch/aarch64/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs -@@ -0,0 +1,146 @@ +@@ -0,0 +1,164 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -662,6 +658,7 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + bsp, memory, + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; ++use core::intrinsics::unlikely; +use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- @@ -678,15 +675,6 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; + -+/// The min supported address space size. -+pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB -+ -+/// The max supported address space size. -+pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -+ -+/// The supported address space size granule. -+pub type AddrSpaceSizeGranule = Granule512MiB; -+ +/// Constants for indexing the MAIR_EL1. +#[allow(dead_code)] +pub mod mair { @@ -711,6 +699,18 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part +// Private Code +//-------------------------------------------------------------------------------------------------- + ++impl memory::mmu::AddressSpace { ++ /// Checks for architectural restrictions. ++ pub const fn arch_address_space_size_sanity_checks() { ++ // Size must be at least one full 512 MiB table. ++ assert!((AS_SIZE modulo Granule512MiB::SIZE) == 0); ++ ++ // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 ++ // version. ++ assert!(AS_SIZE <= (1 << 48)); ++ } ++} ++ +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { @@ -727,19 +727,19 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { -+ let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -+ let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; ++ let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; + + TCR_EL1.write( -+ TCR_EL1::TBI0::Ignored -+ + TCR_EL1::IPS.val(ips) -+ + TCR_EL1::EPD1::DisableTTBR1Walks ++ TCR_EL1::TBI0::Used ++ + TCR_EL1::IPS::Bits_40 + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks -+ + TCR_EL1::T0SZ.val(t0sz), ++ + TCR_EL1::A1::TTBR0 ++ + TCR_EL1::T0SZ.val(t0sz) ++ + TCR_EL1::EPD1::DisableTTBR1Walks, + ); + } +} @@ -756,22 +756,31 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ ++use memory::mmu::MMUEnableError; + +impl memory::mmu::interface::MMU for MemoryManagementUnit { -+ unsafe fn init(&self) -> Result<(), &'static str> { -+ // Fail early if translation granule is not supported. Both RPis support it, though. -+ if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { -+ return Err("Translation granule not supported in HW"); ++ unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { ++ if unlikely(self.is_enabled()) { ++ return Err(MMUEnableError::AlreadyEnabled); ++ } ++ ++ // Fail early if translation granule is not supported. ++ if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { ++ return Err(MMUEnableError::Other( ++ "Translation granule not supported in HW", ++ )); + } + + // Prepare the memory attribute indirection register. + self.set_up_mair(); + + // Populate translation tables. -+ KERNEL_TABLES.populate_tt_entries()?; ++ KERNEL_TABLES ++ .populate_tt_entries() ++ .map_err(|e| MMUEnableError::Other(e))?; + + // Set the "Translation Table Base Register". -+ TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); ++ TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); + + self.configure_translation_control(); + @@ -788,6 +797,11 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + + Ok(()) + } ++ ++ #[inline(always)] ++ fn is_enabled(&self) -> bool { ++ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) ++ } +} diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -829,8 +843,8 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa +// Public Definitions +//-------------------------------------------------------------------------------------------------- + -+/// The address space size chosen by this BSP. -+pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; ++/// The kernel's address space defined by this BSP. ++pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; + +const NUM_MEM_RANGES: usize = 3; + @@ -1006,7 +1020,7 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs --- 10_privilege_level/src/main.rs +++ 11_virtual_mem_part1_identity_mapping/src/main.rs -@@ -108,7 +108,10 @@ +@@ -108,7 +108,11 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] @@ -1014,16 +1028,17 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s #![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] +#![feature(const_panic)] ++#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -132,9 +135,18 @@ +@@ -132,9 +136,18 @@ /// # Safety /// /// - Only a single core must be active and running this function. -/// - The init calls in this function must appear in the correct order. +/// - The init calls in this function must appear in the correct order: -+/// - Virtual memory must be activated before the device drivers. ++/// - Caching must be activated before the device drivers. +/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device +/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on +/// the RPi SoCs. @@ -1031,13 +1046,13 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s use driver::interface::DriverManager; + use memory::mmu::interface::MMU; + -+ if let Err(string) = memory::mmu::mmu().init() { ++ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { + panic!("MMU: {}", string); + } for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -158,6 +170,9 @@ +@@ -158,6 +171,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -1047,7 +1062,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -181,6 +196,13 @@ +@@ -181,6 +197,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -1084,7 +1099,7 @@ diff -uNr 10_privilege_level/src/memory/mmu/translation_table.rs 11_virtual_mem_ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs --- 10_privilege_level/src/memory/mmu.rs +++ 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs -@@ -0,0 +1,247 @@ +@@ -0,0 +1,264 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -1118,8 +1133,17 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map +// Public Definitions +//-------------------------------------------------------------------------------------------------- + ++/// MMU enable errors variants. ++#[allow(missing_docs)] ++#[derive(Debug)] ++pub enum MMUEnableError { ++ AlreadyEnabled, ++ Other(&'static str), ++} ++ +/// Memory Management interfaces. +pub mod interface { ++ use super::*; + + /// MMU functions. + pub trait MMU { @@ -1129,15 +1153,18 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map + /// # Safety + /// + /// - Changes the HW's global state. -+ unsafe fn init(&self) -> Result<(), &'static str>; ++ unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; ++ ++ /// Returns true if the MMU is enabled, false otherwise. ++ fn is_enabled(&self) -> bool; + } +} + +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + -+/// Describes the size of an address space. -+pub struct AddressSpaceSize; ++/// Describes properties of an address space. ++pub struct AddressSpace; + +/// Architecture agnostic translation types. +#[allow(missing_docs)] @@ -1195,6 +1222,15 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map +// Public Code +//-------------------------------------------------------------------------------------------------- + ++impl fmt::Display for MMUEnableError { ++ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { ++ match self { ++ MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), ++ MMUEnableError::Other(x) => write!(f, "{}", x), ++ } ++ } ++} ++ +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); @@ -1209,22 +1245,18 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map + } +} + -+impl AddressSpaceSize { ++impl AddressSpace { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). -+ pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; ++ pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); -+ assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); -+ assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); + -+ // Must adhere to architectural restrictions. -+ assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); -+ assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); -+ assert!((AS_SIZE modulo arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); ++ // Check for architectural restrictions as well. ++ Self::arch_address_space_size_sanity_checks(); + + AS_SIZE + } diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs index 3504d2571..2f7bf615a 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -17,6 +17,7 @@ use crate::{ bsp, memory, memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; +use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- @@ -33,15 +34,6 @@ struct MemoryManagementUnit; pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// The min supported address space size. -pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - -/// The max supported address space size. -pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB - -/// The supported address space size granule. -pub type AddrSpaceSizeGranule = Granule512MiB; - /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] pub mod mair { @@ -66,6 +58,18 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. fn set_up_mair(&self) { @@ -82,19 +86,19 @@ impl MemoryManagementUnit { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, ); } } @@ -111,22 +115,31 @@ pub fn mmu() -> &'static impl memory::mmu::interface::MMU { //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ +use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { - unsafe fn init(&self) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); } // Prepare the memory attribute indirection register. self.set_up_mair(); // Populate translation tables. - KERNEL_TABLES.populate_tt_entries()?; + KERNEL_TABLES + .populate_tt_entries() + .map_err(|e| MMUEnableError::Other(e))?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); self.configure_translation_control(); @@ -143,4 +156,9 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { Ok(()) } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } } diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs index cbe1d7832..f38d0895c 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -87,8 +87,8 @@ register_bitfields! {u64, AttrIndx OFFSET(2) NUMBITS(3) [], TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 + Reserved_Invalid = 0, + Page = 1 ], VALID OFFSET(0) NUMBITS(1) [ @@ -116,19 +116,19 @@ struct PageDescriptor { value: u64, } -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; +trait StartAddr { + fn phys_start_addr_u64(&self) -> u64; + fn phys_start_addr_usize(&self) -> usize; } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. +/// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] pub struct FixedSizeTranslationTable { @@ -146,12 +146,13 @@ pub type KernelTranslationTable = FixedSizeTranslationTable; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { +// The binary is still identity mapped, so we don't need to convert here. +impl StartAddr for [T; N] { + fn phys_start_addr_u64(&self) -> u64 { self as *const T as u64 } - fn base_addr_usize(&self) -> usize { + fn phys_start_addr_usize(&self) -> usize { self as *const _ as usize } } @@ -165,14 +166,14 @@ impl TableDescriptor { } /// Create an instance pointing to the supplied address. - pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { let val = InMemoryRegister::::new(0); - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + + STAGE1_TABLE_DESCRIPTOR::VALID::True, ); TableDescriptor { value: val.get() } @@ -225,16 +226,16 @@ impl PageDescriptor { } /// Create an instance. - pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), ); Self { value: val.get() } @@ -247,10 +248,9 @@ impl PageDescriptor { impl FixedSizeTranslationTable { /// Create an instance. - #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); - assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -266,15 +266,15 @@ impl FixedSizeTranslationTable { pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { *l2_entry = - TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); - let (output_addr, attribute_fields) = + let (phys_output_addr, attribute_fields) = bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); } } @@ -282,7 +282,7 @@ impl FixedSizeTranslationTable { } /// The translation table's base address to be used for programming the MMU. - pub fn base_address(&self) -> u64 { - self.lvl2.base_addr_u64() + pub fn phys_base_address(&self) -> u64 { + self.lvl2.phys_start_addr_u64() } } diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs index 911d10544..703bb3ba2 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -12,8 +12,8 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The address space size chosen by this BSP. -pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; +/// The kernel's address space defined by this BSP. +pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; const NUM_MEM_RANGES: usize = 3; diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 9f755c9ba..ccc842de4 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -112,6 +112,7 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] @@ -136,7 +137,7 @@ mod time; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Virtual memory must be activated before the device drivers. +/// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -144,7 +145,7 @@ unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs index efc9c447b..77cdcc061 100644 --- a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs @@ -31,8 +31,17 @@ pub use arch_mmu::mmu; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + /// Memory Management interfaces. pub mod interface { + use super::*; /// MMU functions. pub trait MMU { @@ -42,15 +51,18 @@ pub mod interface { /// # Safety /// /// - Changes the HW's global state. - unsafe fn init(&self) -> Result<(), &'static str>; + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; } } /// Describes the characteristics of a translation granule. pub struct TranslationGranule; -/// Describes the size of an address space. -pub struct AddressSpaceSize; +/// Describes properties of an address space. +pub struct AddressSpace; /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -108,6 +120,15 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + impl TranslationGranule { /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -122,22 +143,18 @@ impl TranslationGranule { } } -impl AddressSpaceSize { +impl AddressSpace { /// The address space size. pub const SIZE: usize = Self::size_checked(); /// The address space shift, aka log2(size). - pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; const fn size_checked() -> usize { assert!(AS_SIZE.is_power_of_two()); - assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); - assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); - // Must adhere to architectural restrictions. - assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); - assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); - assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); AS_SIZE } diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 53c953df0..0ce07116d 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -900,8 +900,8 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.r --- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -15,7 +15,7 @@ - /// The address space size chosen by this BSP. - pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; + /// The kernel's address space defined by this BSP. + pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; -const NUM_MEM_RANGES: usize = 3; +const NUM_MEM_RANGES: usize = 2; @@ -967,24 +967,24 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_p diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs --- 11_virtual_mem_part1_identity_mapping/src/main.rs +++ 12_exceptions_part1_groundwork/src/main.rs -@@ -113,6 +113,7 @@ - #![feature(const_generics)] +@@ -114,6 +114,7 @@ #![feature(const_panic)] + #![feature(core_intrinsics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] -@@ -144,6 +145,8 @@ +@@ -145,6 +146,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; + exception::handling_init(); + - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } -@@ -196,13 +199,28 @@ +@@ -197,13 +200,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs index 3504d2571..2f7bf615a 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs @@ -17,6 +17,7 @@ use crate::{ bsp, memory, memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; +use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- @@ -33,15 +34,6 @@ struct MemoryManagementUnit; pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// The min supported address space size. -pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - -/// The max supported address space size. -pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB - -/// The supported address space size granule. -pub type AddrSpaceSizeGranule = Granule512MiB; - /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] pub mod mair { @@ -66,6 +58,18 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. fn set_up_mair(&self) { @@ -82,19 +86,19 @@ impl MemoryManagementUnit { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, ); } } @@ -111,22 +115,31 @@ pub fn mmu() -> &'static impl memory::mmu::interface::MMU { //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ +use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { - unsafe fn init(&self) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); } // Prepare the memory attribute indirection register. self.set_up_mair(); // Populate translation tables. - KERNEL_TABLES.populate_tt_entries()?; + KERNEL_TABLES + .populate_tt_entries() + .map_err(|e| MMUEnableError::Other(e))?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); self.configure_translation_control(); @@ -143,4 +156,9 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { Ok(()) } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } } diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs index cbe1d7832..f38d0895c 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -87,8 +87,8 @@ register_bitfields! {u64, AttrIndx OFFSET(2) NUMBITS(3) [], TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 + Reserved_Invalid = 0, + Page = 1 ], VALID OFFSET(0) NUMBITS(1) [ @@ -116,19 +116,19 @@ struct PageDescriptor { value: u64, } -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; +trait StartAddr { + fn phys_start_addr_u64(&self) -> u64; + fn phys_start_addr_usize(&self) -> usize; } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. +/// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] pub struct FixedSizeTranslationTable { @@ -146,12 +146,13 @@ pub type KernelTranslationTable = FixedSizeTranslationTable; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { +// The binary is still identity mapped, so we don't need to convert here. +impl StartAddr for [T; N] { + fn phys_start_addr_u64(&self) -> u64 { self as *const T as u64 } - fn base_addr_usize(&self) -> usize { + fn phys_start_addr_usize(&self) -> usize { self as *const _ as usize } } @@ -165,14 +166,14 @@ impl TableDescriptor { } /// Create an instance pointing to the supplied address. - pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { let val = InMemoryRegister::::new(0); - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + + STAGE1_TABLE_DESCRIPTOR::VALID::True, ); TableDescriptor { value: val.get() } @@ -225,16 +226,16 @@ impl PageDescriptor { } /// Create an instance. - pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), ); Self { value: val.get() } @@ -247,10 +248,9 @@ impl PageDescriptor { impl FixedSizeTranslationTable { /// Create an instance. - #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); - assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -266,15 +266,15 @@ impl FixedSizeTranslationTable { pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { *l2_entry = - TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); - let (output_addr, attribute_fields) = + let (phys_output_addr, attribute_fields) = bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); } } @@ -282,7 +282,7 @@ impl FixedSizeTranslationTable { } /// The translation table's base address to be used for programming the MMU. - pub fn base_address(&self) -> u64 { - self.lvl2.base_addr_u64() + pub fn phys_base_address(&self) -> u64 { + self.lvl2.phys_start_addr_u64() } } diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs index fe98604d9..1775c07cd 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -12,8 +12,8 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The address space size chosen by this BSP. -pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; +/// The kernel's address space defined by this BSP. +pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; const NUM_MEM_RANGES: usize = 2; diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 9dee3a44c..8e8c089f1 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -112,6 +112,7 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(global_asm)] #![feature(panic_info_message)] @@ -137,7 +138,7 @@ mod time; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Virtual memory must be activated before the device drivers. +/// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -147,7 +148,7 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } diff --git a/12_exceptions_part1_groundwork/src/memory/mmu.rs b/12_exceptions_part1_groundwork/src/memory/mmu.rs index efc9c447b..77cdcc061 100644 --- a/12_exceptions_part1_groundwork/src/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/memory/mmu.rs @@ -31,8 +31,17 @@ pub use arch_mmu::mmu; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + /// Memory Management interfaces. pub mod interface { + use super::*; /// MMU functions. pub trait MMU { @@ -42,15 +51,18 @@ pub mod interface { /// # Safety /// /// - Changes the HW's global state. - unsafe fn init(&self) -> Result<(), &'static str>; + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; } } /// Describes the characteristics of a translation granule. pub struct TranslationGranule; -/// Describes the size of an address space. -pub struct AddressSpaceSize; +/// Describes properties of an address space. +pub struct AddressSpace; /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -108,6 +120,15 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + impl TranslationGranule { /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -122,22 +143,18 @@ impl TranslationGranule { } } -impl AddressSpaceSize { +impl AddressSpace { /// The address space size. pub const SIZE: usize = Self::size_checked(); /// The address space shift, aka log2(size). - pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; const fn size_checked() -> usize { assert!(AS_SIZE.is_power_of_two()); - assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); - assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); - // Must adhere to architectural restrictions. - assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); - assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); - assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); AS_SIZE } diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index c0fc217b9..452c864e5 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1026,7 +1026,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translatio --- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -286,3 +286,31 @@ - self.lvl2.base_addr_u64() + self.lvl2.phys_start_addr_u64() } } + @@ -1061,8 +1061,8 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translatio diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs -@@ -144,3 +144,22 @@ - Ok(()) +@@ -162,3 +162,22 @@ + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } + @@ -1194,7 +1194,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,171 @@ +@@ -0,0 +1,172 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1311,6 +1311,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +#![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] +#![feature(const_panic)] ++#![feature(core_intrinsics)] +#![feature(format_args_nl)] +#![feature(global_asm)] +#![feature(linkage)] @@ -1370,7 +1371,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,130 +6,12 @@ +@@ -6,131 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1480,6 +1481,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -#![feature(const_fn_fn_ptr_basics)] -#![feature(const_generics)] -#![feature(const_panic)] +-#![feature(core_intrinsics)] + #![feature(format_args_nl)] -#![feature(global_asm)] @@ -1503,7 +1505,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -141,6 +23,7 @@ +@@ -142,6 +23,7 @@ /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -1511,7 +1513,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -167,9 +50,7 @@ +@@ -168,9 +50,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1521,7 +1523,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -196,31 +77,6 @@ +@@ -197,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1557,7 +1559,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing/src/memory/mmu.rs --- 12_exceptions_part1_groundwork/src/memory/mmu.rs +++ 13_integrated_testing/src/memory/mmu.rs -@@ -54,7 +54,6 @@ +@@ -66,7 +66,6 @@ /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -1565,7 +1567,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing #[derive(Copy, Clone)] pub enum Translation { Identity, -@@ -244,4 +243,9 @@ +@@ -261,4 +260,9 @@ info!("{}", i); } } @@ -1910,7 +1912,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + -+ if let Err(string) = memory::mmu::mmu().init() { ++ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { + println!("MMU: {}", string); + cpu::qemu_exit_failure() + } diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs index 42ba0519b..29e8125d7 100644 --- a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -17,6 +17,7 @@ use crate::{ bsp, memory, memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; +use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- @@ -33,15 +34,6 @@ struct MemoryManagementUnit; pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// The min supported address space size. -pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - -/// The max supported address space size. -pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB - -/// The supported address space size granule. -pub type AddrSpaceSizeGranule = Granule512MiB; - /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] pub mod mair { @@ -66,6 +58,18 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. fn set_up_mair(&self) { @@ -82,19 +86,19 @@ impl MemoryManagementUnit { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, ); } } @@ -111,22 +115,31 @@ pub fn mmu() -> &'static impl memory::mmu::interface::MMU { //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ +use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { - unsafe fn init(&self) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); } // Prepare the memory attribute indirection register. self.set_up_mair(); // Populate translation tables. - KERNEL_TABLES.populate_tt_entries()?; + KERNEL_TABLES + .populate_tt_entries() + .map_err(|e| MMUEnableError::Other(e))?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); self.configure_translation_control(); @@ -143,6 +156,11 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { Ok(()) } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } } //-------------------------------------------------------------------------------------------------- diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs b/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs index 337f9aedf..73a93ff73 100644 --- a/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -87,8 +87,8 @@ register_bitfields! {u64, AttrIndx OFFSET(2) NUMBITS(3) [], TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 + Reserved_Invalid = 0, + Page = 1 ], VALID OFFSET(0) NUMBITS(1) [ @@ -116,19 +116,19 @@ struct PageDescriptor { value: u64, } -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; +trait StartAddr { + fn phys_start_addr_u64(&self) -> u64; + fn phys_start_addr_usize(&self) -> usize; } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. +/// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] pub struct FixedSizeTranslationTable { @@ -146,12 +146,13 @@ pub type KernelTranslationTable = FixedSizeTranslationTable; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { +// The binary is still identity mapped, so we don't need to convert here. +impl StartAddr for [T; N] { + fn phys_start_addr_u64(&self) -> u64 { self as *const T as u64 } - fn base_addr_usize(&self) -> usize { + fn phys_start_addr_usize(&self) -> usize { self as *const _ as usize } } @@ -165,14 +166,14 @@ impl TableDescriptor { } /// Create an instance pointing to the supplied address. - pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { let val = InMemoryRegister::::new(0); - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + + STAGE1_TABLE_DESCRIPTOR::VALID::True, ); TableDescriptor { value: val.get() } @@ -225,16 +226,16 @@ impl PageDescriptor { } /// Create an instance. - pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), ); Self { value: val.get() } @@ -247,10 +248,9 @@ impl PageDescriptor { impl FixedSizeTranslationTable { /// Create an instance. - #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); - assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -266,15 +266,15 @@ impl FixedSizeTranslationTable { pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { *l2_entry = - TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); - let (output_addr, attribute_fields) = + let (phys_output_addr, attribute_fields) = bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); } } @@ -282,8 +282,8 @@ impl FixedSizeTranslationTable { } /// The translation table's base address to be used for programming the MMU. - pub fn base_address(&self) -> u64 { - self.lvl2.base_addr_u64() + pub fn phys_base_address(&self) -> u64 { + self.lvl2.phys_start_addr_u64() } } diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs index 451d927b6..0ccfae002 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -12,8 +12,8 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The address space size chosen by this BSP. -pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; +/// The kernel's address space defined by this BSP. +pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; const NUM_MEM_RANGES: usize = 2; diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index 0fa216a98..b094daf3d 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -114,6 +114,7 @@ #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] #![feature(global_asm)] #![feature(linkage)] diff --git a/13_integrated_testing/src/main.rs b/13_integrated_testing/src/main.rs index 769f9e8f0..101124df8 100644 --- a/13_integrated_testing/src/main.rs +++ b/13_integrated_testing/src/main.rs @@ -19,7 +19,7 @@ use libkernel::{bsp, console, driver, exception, info, memory, time}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Virtual memory must be activated before the device drivers. +/// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. @@ -30,7 +30,7 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } diff --git a/13_integrated_testing/src/memory/mmu.rs b/13_integrated_testing/src/memory/mmu.rs index cca2951aa..ec4ca0748 100644 --- a/13_integrated_testing/src/memory/mmu.rs +++ b/13_integrated_testing/src/memory/mmu.rs @@ -31,8 +31,17 @@ pub use arch_mmu::mmu; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + /// Memory Management interfaces. pub mod interface { + use super::*; /// MMU functions. pub trait MMU { @@ -42,15 +51,18 @@ pub mod interface { /// # Safety /// /// - Changes the HW's global state. - unsafe fn init(&self) -> Result<(), &'static str>; + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; } } /// Describes the characteristics of a translation granule. pub struct TranslationGranule; -/// Describes the size of an address space. -pub struct AddressSpaceSize; +/// Describes properties of an address space. +pub struct AddressSpace; /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -107,6 +119,15 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + impl TranslationGranule { /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -121,22 +142,18 @@ impl TranslationGranule { } } -impl AddressSpaceSize { +impl AddressSpace { /// The address space size. pub const SIZE: usize = Self::size_checked(); /// The address space shift, aka log2(size). - pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; const fn size_checked() -> usize { assert!(AS_SIZE.is_power_of_two()); - assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); - assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); - // Must adhere to architectural restrictions. - assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); - assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); - assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); AS_SIZE } diff --git a/13_integrated_testing/tests/02_exception_sync_page_fault.rs b/13_integrated_testing/tests/02_exception_sync_page_fault.rs index 185089a3d..f1535d34c 100644 --- a/13_integrated_testing/tests/02_exception_sync_page_fault.rs +++ b/13_integrated_testing/tests/02_exception_sync_page_fault.rs @@ -29,7 +29,7 @@ unsafe fn kernel_init() -> ! { println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { println!("MMU: {}", string); cpu::qemu_exit_failure() } diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 1c3cb7f0c..5e1f45e32 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2296,7 +2296,7 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs --- 13_integrated_testing/src/lib.rs +++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -111,9 +111,11 @@ +@@ -111,6 +111,7 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -2304,11 +2304,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -+#![feature(core_intrinsics)] - #![feature(format_args_nl)] - #![feature(global_asm)] - #![feature(linkage)] -@@ -137,6 +139,7 @@ +@@ -138,6 +139,7 @@ pub mod exception; pub mod memory; pub mod print; @@ -2331,7 +2327,7 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ /// @@ -21,8 +21,8 @@ /// - The init calls in this function must appear in the correct order: - /// - Virtual memory must be activated before the device drivers. + /// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -/// the RPi SoCs. @@ -2590,7 +2586,7 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip type Data = T; fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { -@@ -72,6 +110,32 @@ +@@ -72,6 +110,50 @@ // mutable reference will ever only be given out once at a time. let data = unsafe { &mut *self.data.get() }; @@ -2614,14 +2610,32 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip + + let data = unsafe { &mut *self.data.get() }; + -+ f(data) -+ } + f(data) + } + + fn read(&self, f: impl FnOnce(&Self::Data) -> R) -> R { + let data = unsafe { &*self.data.get() }; + - f(data) - } ++ f(data) ++ } ++} ++ ++//-------------------------------------------------------------------------------------------------- ++// Testing ++//-------------------------------------------------------------------------------------------------- ++ ++#[cfg(test)] ++mod tests { ++ use super::*; ++ use test_macros::kernel_test; ++ ++ /// InitStateLock must be transparent. ++ #[kernel_test] ++ fn init_state_lock_is_transparent() { ++ use core::mem::size_of; ++ ++ assert_eq!(size_of::>(), size_of::()); ++ } } diff -uNr 13_integrated_testing/tests/03_exception_irq_sanity.rs 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 42ba0519b..29e8125d7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -17,6 +17,7 @@ use crate::{ bsp, memory, memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; +use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- @@ -33,15 +34,6 @@ struct MemoryManagementUnit; pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// The min supported address space size. -pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - -/// The max supported address space size. -pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB - -/// The supported address space size granule. -pub type AddrSpaceSizeGranule = Granule512MiB; - /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] pub mod mair { @@ -66,6 +58,18 @@ static MMU: MemoryManagementUnit = MemoryManagementUnit; // Private Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. fn set_up_mair(&self) { @@ -82,19 +86,19 @@ impl MemoryManagementUnit { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; + let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, ); } } @@ -111,22 +115,31 @@ pub fn mmu() -> &'static impl memory::mmu::interface::MMU { //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ +use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { - unsafe fn init(&self) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); } // Prepare the memory attribute indirection register. self.set_up_mair(); // Populate translation tables. - KERNEL_TABLES.populate_tt_entries()?; + KERNEL_TABLES + .populate_tt_entries() + .map_err(|e| MMUEnableError::Other(e))?; // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); + TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); self.configure_translation_control(); @@ -143,6 +156,11 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { Ok(()) } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } } //-------------------------------------------------------------------------------------------------- diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs index 337f9aedf..73a93ff73 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -87,8 +87,8 @@ register_bitfields! {u64, AttrIndx OFFSET(2) NUMBITS(3) [], TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 + Reserved_Invalid = 0, + Page = 1 ], VALID OFFSET(0) NUMBITS(1) [ @@ -116,19 +116,19 @@ struct PageDescriptor { value: u64, } -trait BaseAddr { - fn base_addr_u64(&self) -> u64; - fn base_addr_usize(&self) -> usize; +trait StartAddr { + fn phys_start_addr_u64(&self) -> u64; + fn phys_start_addr_usize(&self) -> usize; } -const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; +const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. +/// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] pub struct FixedSizeTranslationTable { @@ -146,12 +146,13 @@ pub type KernelTranslationTable = FixedSizeTranslationTable; // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn base_addr_u64(&self) -> u64 { +// The binary is still identity mapped, so we don't need to convert here. +impl StartAddr for [T; N] { + fn phys_start_addr_u64(&self) -> u64 { self as *const T as u64 } - fn base_addr_usize(&self) -> usize { + fn phys_start_addr_usize(&self) -> usize { self as *const _ as usize } } @@ -165,14 +166,14 @@ impl TableDescriptor { } /// Create an instance pointing to the supplied address. - pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { let val = InMemoryRegister::::new(0); - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + + STAGE1_TABLE_DESCRIPTOR::VALID::True, ); TableDescriptor { value: val.get() } @@ -225,16 +226,16 @@ impl PageDescriptor { } /// Create an instance. - pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { + pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), ); Self { value: val.get() } @@ -247,10 +248,9 @@ impl PageDescriptor { impl FixedSizeTranslationTable { /// Create an instance. - #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); - assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -266,15 +266,15 @@ impl FixedSizeTranslationTable { pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { *l2_entry = - TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); + TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); - let (output_addr, attribute_fields) = + let (phys_output_addr, attribute_fields) = bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; - *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); + *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); } } @@ -282,8 +282,8 @@ impl FixedSizeTranslationTable { } /// The translation table's base address to be used for programming the MMU. - pub fn base_address(&self) -> u64 { - self.lvl2.base_addr_u64() + pub fn phys_base_address(&self) -> u64 { + self.lvl2.phys_start_addr_u64() } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs index 451d927b6..0ccfae002 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs @@ -12,8 +12,8 @@ use core::ops::RangeInclusive; // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The address space size chosen by this BSP. -pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; +/// The kernel's address space defined by this BSP. +pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; const NUM_MEM_RANGES: usize = 2; diff --git a/14_exceptions_part2_peripheral_IRQs/src/main.rs b/14_exceptions_part2_peripheral_IRQs/src/main.rs index 6a21c8d41..0bd9f71dd 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/main.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/main.rs @@ -19,7 +19,7 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Virtual memory must be activated before the device drivers. +/// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ IRQSafeNullLocks instead of spinlocks), will fail to /// work on the RPi SoCs. @@ -30,7 +30,7 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs index cca2951aa..ec4ca0748 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs @@ -31,8 +31,17 @@ pub use arch_mmu::mmu; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + /// Memory Management interfaces. pub mod interface { + use super::*; /// MMU functions. pub trait MMU { @@ -42,15 +51,18 @@ pub mod interface { /// # Safety /// /// - Changes the HW's global state. - unsafe fn init(&self) -> Result<(), &'static str>; + unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; } } /// Describes the characteristics of a translation granule. pub struct TranslationGranule; -/// Describes the size of an address space. -pub struct AddressSpaceSize; +/// Describes properties of an address space. +pub struct AddressSpace; /// Architecture agnostic translation types. #[allow(missing_docs)] @@ -107,6 +119,15 @@ pub struct KernelVirtualLayout { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + impl TranslationGranule { /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -121,22 +142,18 @@ impl TranslationGranule { } } -impl AddressSpaceSize { +impl AddressSpace { /// The address space size. pub const SIZE: usize = Self::size_checked(); /// The address space shift, aka log2(size). - pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; const fn size_checked() -> usize { assert!(AS_SIZE.is_power_of_two()); - assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); - assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); - // Must adhere to architectural restrictions. - assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); - assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); - assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); AS_SIZE } diff --git a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs index fe9d454a8..94582732c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs @@ -139,3 +139,21 @@ impl interface::ReadWriteEx for InitStateLock { f(data) } } + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// InitStateLock must be transparent. + #[kernel_test] + fn init_state_lock_is_transparent() { + use core::mem::size_of; + + assert_eq!(size_of::>(), size_of::()); + } +} diff --git a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs index 185089a3d..f1535d34c 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs @@ -29,7 +29,7 @@ unsafe fn kernel_init() -> ! { println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { println!("MMU: {}", string); cpu::qemu_exit_failure() } diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 53eb8047f..ec117146f 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -16,7 +16,7 @@ - [Introduction](#introduction) - [Implementation](#implementation) - [A New Mapping API in `src/memory/mmu.rs`](#a-new-mapping-api-in-srcmemorymmutranslationtablers) - - [Using the new API in `bsp` code and drivers](#using-the-new-api-in-bsp-code-and-drivers) + - [The new APIs in action](#the-new-apis-in-action) - [Additional Changes](#additional-changes) - [Test it](#test-it) - [Diff to previous](#diff-to-previous) @@ -56,16 +56,16 @@ separation, this tutorial makes a start by changing the following things: 1. Instead of bulk-`identity mapping` the whole of the board's address space, only the particular parts that are needed will be mapped. -1. For now, the `kernel binary` stays identity mapped. This will be changed in the next tutorial as - it is a quite difficult and peculiar exercise to remap the kernel. +1. For now, the `kernel binary` stays identity mapped. This will be changed in the in the coming + tutorials as it is a quite difficult and peculiar exercise to remap the kernel. 1. Device `MMIO regions` are lazily remapped during the device driver's `init()`. 1. The remappings will populate the top of the virtual address space. In the `AArch64 MMU Driver`, we provide the top `256 MiB` for it. 1. It is possible to define the size of the virtual address space at compile time. We chose `8 GiB` for now, which means remapped MMIO virtual addresses will start at `7936 MiB` - (`0x1F0000000`). -1. We keep using `TTBR0` for the kernel page tables for now. This will be changed when we remap the - `kernel binary` in the next tutorial. + (`0x1_f000_0000`). +1. We keep using `TTBR0` for the kernel translation tables for now. This will be changed when we + remap the `kernel binary` in the coming tutorials. [ARM Cortex-A Series Programmer’s Guide for ARMv8-A]: https://developer.arm.com/documentation/den0024/latest/ [higher half kernel]: https://wiki.osdev.org/Higher_Half_Kernel @@ -80,10 +80,11 @@ kernel code** (`src/memory/**`). The way it worked was that the `architectural MMU code` would query the `bsp code` about the start and end of the physical address space, and any special regions in this space that need a mapping that _is not_ normal chacheable DRAM. It would then go ahead and map the whole address space at once -and never touch the page tables again during runtime. +and never touch the translation tables again during runtime. -Changing in this tutorial, **architecture** and **bsp** code will no longer talk to each other -directly. Instead, this is decoupled now through the kernel's **generic MMU subsystem code**. +Changing in this tutorial, **architecture** and **bsp** code will no longer autonomously create the +virtual memory mappings. Instead, this is now orchestrated by the kernel's **generic MMU subsystem +code**. ### A New Mapping API in `src/memory/mmu/translation_table.rs` @@ -93,7 +94,7 @@ First, we define an interface for operating on `translation tables`: /// Translation table operations. pub trait TranslationTable { /// Anything that needs to run before any of the other provided functions can be used. - unsafe fn init(&mut self); + fn init(&mut self); /// The translation table's base address to be used for programming the MMU. fn phys_base_address(&self) -> Address; @@ -107,12 +108,6 @@ pub trait TranslationTable { ) -> Result<(), &'static str>; /// Obtain a free virtual page slice in the MMIO region. - /// - /// The "MMIO region" is a distinct region of the implementor's choice, which allows - /// differentiating MMIO addresses from others. This can speed up debugging efforts. - /// Ideally, those MMIO addresses are also standing out visually so that a human eye can - /// identify them. For example, by allocating them from near the end of the virtual address - /// space. fn next_mmio_virt_page_slice( &mut self, num_pages: usize, @@ -123,11 +118,49 @@ pub trait TranslationTable { } ``` -The MMU driver (`src/_arch/_/memory/mmu.rs`) has one global instance for the kernel tables which -implements this interface, and which can be accessed by calling -`arch_mmu::kernel_translation_tables()` in the generic kernel code (`src/memory/mmu.rs`). From -there, we provide a couple of memory mapping functions that wrap around this interface , and which -are exported for the rest of the kernel to use: +In order to enable the generic kernel code to manipulate the kernel's translation tables, they must +first be made accessible. Until now, they were just a "hidden" struct in the `architectural` MMU +driver (`src/arch/.../memory/mmu.rs`). This made sense because the MMU driver code was the only code +that needed to be concerned with the table data structure, so having it accessible locally +simplified things. + +Since the tables need to be exposed to the rest of the kernel code now, it makes sense to move them +to `BSP` code. Because ultimately, it is the `BSP` that is defining the translation table's +properties, such as the size of the virtual address space that the tables need to cover. + +They are now defined in the global instances region of `src/bsp/.../memory/mmu.rs`. To control +access, they are guarded by an `InitStateLock`. + +```rust +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +/// The kernel translation tables. +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new()); +``` + +The struct `KernelTranslationTable` is a type alias defined in the same file, which in turn gets its +definition from an associated type of type `KernelVirtAddrSpace`, which itself is a type alias of +`memory::mmu::AddressSpace`. I know this sounds horribly complicated, but in the end this is just +some layers of `const generics` whose implementation is scattered between `generic` and `arch` code. +This is done to (1) ensure a sane compile-time definition of the translation table struct (by doing +various bounds checks), and (2) to separate concerns between generic `MMU` code and specializations +that come from the `architectural` part. + +In the end, these tables can be accessed by calling `bsp::memory::mmu::kernel_translation_tables()`: + +```rust +/// Return a reference to the kernel's translation tables. +pub fn kernel_translation_tables() -> &'static InitStateLock { + &KERNEL_TABLES +} +``` + +Finally, the generic kernel code (`src/memory/mmu.rs`) now provides a couple of memory mapping +functions that access and manipulate this instance. They are exported for the rest of the kernel to +use: ```rust /// Raw mapping of virtual to physical pages in the kernel translation tables. @@ -135,8 +168,8 @@ are exported for the rest of the kernel to use: /// Prevents mapping into the MMIO range of the tables. pub unsafe fn kernel_map_pages_at( name: &'static str, - phys_pages: &PageSliceDescriptor, virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str>; @@ -148,20 +181,39 @@ pub unsafe fn kernel_map_mmio( phys_mmio_descriptor: &MMIODescriptor, ) -> Result, &'static str>; -/// Map the kernel's binary and enable the MMU. -pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> ; +/// Map the kernel's binary. Returns the translation table's base address. +pub unsafe fn kernel_map_binary() -> Result, &'static str>; + +/// Enable the MMU and data + instruction caching. +pub unsafe fn enable_mmu_and_caching( + phys_tables_base_addr: Address, +) -> Result<(), MMUEnableError>; ``` -### Using the new API in `bsp` code and drivers +### The new APIs in action + +`kernel_map_binary()` and `enable_mmu_and_caching()` are used early in `kernel_init()` to set up +virtual memory: + +```rust +let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { + Err(string) => panic!("Error mapping kernel binary: {}", string), + Ok(addr) => addr, +}; + +if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { + panic!("Enabling MMU failed: {}", e); +} +``` -For now, there are two places where the new API is used. First, in `src/bsp/_/memory/mmu.rs`, which -provides a dedicated call to **map the kernel binary** (because it is the `BSP` that provides the -`linker script`, which in turn defines the final layout of the kernel in memory): +Both functions internally use `bsp` and `arch` specific code to achieve their goals. For example, +`memory::mmu::kernel_map_binary()` itself wraps around a `bsp` function of the same name +(`bsp::memory::mmu::kernel_map_binary()`): ```rust /// Map the kernel binary. pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel boot-core stack", &virt_stack_page_desc(), &phys_stack_page_desc(), @@ -172,12 +224,12 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { }, )?; - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel code and RO data", // omitted for brevity. )?; - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel data and bss", // omitted for brevity. )?; @@ -186,8 +238,8 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { } ``` -Second, in device drivers, which now expect an `MMIODescriptor` type instead of a raw address. The -following is an example for the `UART`: +Another user of the new APIs are device drivers, which now expect an `MMIODescriptor` type instead +of a raw address. The following is an example for the `UART`: ```rust impl PL011Uart { @@ -234,7 +286,7 @@ through them: ## Test it When you load the kernel, you can now see that the driver's MMIO virtual addresses start at -`0x1F0000000`: +`0x1_f000_0000`: Raspberry Pi 3: @@ -254,21 +306,21 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 67 KiB ========================================🦀 100% 33 KiB/s Time: 00:00:02 +[MP] ⏩ Pushing 67 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.064756] Booting on: Raspberry Pi 3 -[ 3.065839] MMU online: -[ 3.067010] ----------------------------------------------------------------------------------------------------------------- -[ 3.072868] Virtual Physical Size Attr Entity -[ 3.078725] ----------------------------------------------------------------------------------------------------------------- -[ 3.084585] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack -[ 3.089877] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data -[ 3.095213] 0x000090000..0x0001BFFFF --> 0x000090000..0x0001BFFFF | 1 MiB | C RW XN | Kernel data and bss -[ 3.100376] 0x1F0000000..0x1F000FFFF --> 0x03F200000..0x03F20FFFF | 64 KiB | Dev RW XN | BCM GPIO -[ 3.105060] | BCM PL011 UART -[ 3.110008] 0x1F0010000..0x1F001FFFF --> 0x03F000000..0x03F00FFFF | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 3.115863] ----------------------------------------------------------------------------------------------------------------- +[ 0.786819] Booting on: Raspberry Pi 3 +[ 0.787092] MMU online: +[ 0.787384] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.789128] Virtual Physical Size Attr Entity +[ 0.790873] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.792618] 0x0000_0000_0007_0000..0x0000_0000_0007_ffff --> 0x00_0007_0000..0x00_0007_ffff | 64 KiB | C RW XN | Kernel boot-core stack +[ 0.794221] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.795835] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.797406] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.798857] | BCM PL011 UART +[ 0.800374] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 0.802117] ------------------------------------------------------------------------------------------------------------------------------------------- ``` Raspberry Pi 4: @@ -289,22 +341,22 @@ Minipush 1.0 Raspberry Pi 4 [ML] Requesting binary -[MP] ⏩ Pushing 74 KiB ========================================🦀 100% 24 KiB/s Time: 00:00:03 +[MP] ⏩ Pushing 74 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.379342] Booting on: Raspberry Pi 4 -[ 3.379731] MMU online: -[ 3.380902] ----------------------------------------------------------------------------------------------------------------- -[ 3.386759] Virtual Physical Size Attr Entity -[ 3.392616] ----------------------------------------------------------------------------------------------------------------- -[ 3.398475] 0x000070000..0x00007FFFF --> 0x000070000..0x00007FFFF | 64 KiB | C RW XN | Kernel boot-core stack -[ 3.403768] 0x000080000..0x00008FFFF --> 0x000080000..0x00008FFFF | 64 KiB | C RO X | Kernel code and RO data -[ 3.409104] 0x000090000..0x0001BFFFF --> 0x000090000..0x0001BFFFF | 1 MiB | C RW XN | Kernel data and bss -[ 3.414267] 0x1F0000000..0x1F000FFFF --> 0x0FE200000..0x0FE20FFFF | 64 KiB | Dev RW XN | BCM GPIO -[ 3.418951] | BCM PL011 UART -[ 3.423898] 0x1F0010000..0x1F001FFFF --> 0x0FF840000..0x0FF84FFFF | 64 KiB | Dev RW XN | GICD -[ 3.428409] | GICC -[ 3.432921] ----------------------------------------------------------------------------------------------------------------- +[ 0.853908] Booting on: Raspberry Pi 4 +[ 0.854007] MMU online: +[ 0.854299] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.856043] Virtual Physical Size Attr Entity +[ 0.857788] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.859533] 0x0000_0000_0007_0000..0x0000_0000_0007_ffff --> 0x00_0007_0000..0x00_0007_ffff | 64 KiB | C RW XN | Kernel boot-core stack +[ 0.861137] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.862750] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.864321] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.865772] | BCM PL011 UART +[ 0.867289] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD +[ 0.868697] | GICC +[ 0.870105] ------------------------------------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous @@ -338,74 +390,82 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, + }, -+ Address, AddressType, Physical, Virtual, ++ Address, Physical, Virtual, }, }; use core::convert; -@@ -117,11 +120,11 @@ +@@ -117,12 +120,9 @@ } - trait BaseAddr { -- fn base_addr_u64(&self) -> u64; -- fn base_addr_usize(&self) -> usize; -+ fn phys_base_addr(&self) -> Address; + trait StartAddr { +- fn phys_start_addr_u64(&self) -> u64; +- fn phys_start_addr_usize(&self) -> usize; ++ fn phys_start_addr(&self) -> Address; } --const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; -+const NUM_LVL2_TABLES: usize = -+ bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; - +-const NUM_LVL2_TABLES: usize = bsp::memory::mmu::KernelAddrSpace::SIZE >> Granule512MiB::SHIFT; +- //-------------------------------------------------------------------------------------------------- // Public Definitions -@@ -137,6 +140,12 @@ + //-------------------------------------------------------------------------------------------------- +@@ -137,10 +137,13 @@ /// Table descriptors, covering 512 MiB windows. lvl2: [TableDescriptor; NUM_TABLES], -+ +-} + +-/// A translation table type for the kernel space. +-pub type KernelTranslationTable = FixedSizeTranslationTable; + /// Index of the next free MMIO page. + cur_l3_mmio_index: usize, + + /// Have the tables been initialized? + initialized: bool, - } ++} - /// A translation table type for the kernel space. -@@ -147,12 +156,9 @@ //-------------------------------------------------------------------------------------------------- + // Private Code +@@ -148,12 +151,8 @@ - impl BaseAddr for [T; N] { -- fn base_addr_u64(&self) -> u64 { + // The binary is still identity mapped, so we don't need to convert here. + impl StartAddr for [T; N] { +- fn phys_start_addr_u64(&self) -> u64 { - self as *const T as u64 - } - -- fn base_addr_usize(&self) -> usize { +- fn phys_start_addr_usize(&self) -> usize { - self as *const _ as usize -+ fn phys_base_addr(&self) -> Address { -+ // The binary is still identity mapped, so we don't need to convert here. ++ fn phys_start_addr(&self) -> Address { + Address::new(self as *const _ as usize) } } -@@ -225,20 +231,29 @@ +@@ -166,10 +165,10 @@ + } + + /// Create an instance pointing to the supplied address. +- pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: usize) -> Self { ++ pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: Address) -> Self { + let val = InMemoryRegister::::new(0); + +- let shifted = phys_next_lvl_table_addr >> Granule64KiB::SHIFT; ++ let shifted = phys_next_lvl_table_addr.into_usize() >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table +@@ -226,7 +225,10 @@ } /// Create an instance. -- pub fn from_output_addr(output_addr: usize, attribute_fields: AttributeFields) -> Self { +- pub fn from_output_addr(phys_output_addr: usize, attribute_fields: &AttributeFields) -> Self { + pub fn from_output_addr( -+ output_addr: *const Page, ++ phys_output_addr: *const Page, + attribute_fields: &AttributeFields, + ) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; - val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True - + STAGE1_PAGE_DESCRIPTOR::AF::True -- + attribute_fields.into() -+ + attribute_fields.clone().into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), - ); + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; +@@ -240,50 +242,193 @@ Self { value: val.get() } } @@ -418,21 +478,29 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans } //-------------------------------------------------------------------------------------------------- -@@ -246,44 +261,172 @@ + // Public Code //-------------------------------------------------------------------------------------------------- ++impl memory::mmu::AssociatedTranslationTable ++ for memory::mmu::AddressSpace ++where ++ [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, ++{ ++ type TableStartFromBottom = FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }>; ++} ++ impl FixedSizeTranslationTable { + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + /// Create an instance. - #[allow(clippy::assertions_on_constants)] ++ #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { + assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); ++ + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); -- assert!((bsp::memory::mmu::KernelAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); -+ assert!((bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE modulo Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -450,10 +518,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans - pub unsafe fn populate_tt_entries(&mut self) -> Result<(), &'static str> { - for (l2_nr, l2_entry) in self.lvl2.iter_mut().enumerate() { - *l2_entry = -- TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].base_addr_usize()); +- TableDescriptor::from_next_lvl_table_addr(self.lvl3[l2_nr].phys_start_addr_usize()); + /// The start address of the table's MMIO range. + #[inline(always)] -+ const fn mmio_start_addr(&self) -> Address { ++ fn mmio_start_addr(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), @@ -462,7 +530,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] -+ const fn mmio_end_addr_inclusive(&self) -> Address { ++ fn mmio_end_addr_inclusive(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) @@ -472,12 +540,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + + /// Helper to calculate the lvl2 and lvl3 indices from an address. + #[inline(always)] -+ fn lvl2_lvl3_index_from( ++ fn lvl2_lvl3_index_from( + &self, -+ addr: *const Page, ++ addr: *const Page, + ) -> Result<(usize, usize), &'static str> { -+ let lvl2_index = addr as usize >> Granule512MiB::SHIFT; -+ let lvl3_index = (addr as usize & Granule512MiB::MASK) >> Granule64KiB::SHIFT; ++ let addr = addr as usize; ++ let lvl2_index = addr >> Granule512MiB::SHIFT; ++ let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + + if lvl2_index > (NUM_TABLES - 1) { + return Err("Virtual page is out of bounds of translation table"); @@ -501,36 +570,32 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ -+ + +- for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { +- let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); +impl memory::mmu::translation_table::interface::TranslationTable + for FixedSizeTranslationTable +{ -+ unsafe fn init(&mut self) { ++ fn init(&mut self) { + if self.initialized { + return; + } - -- for (l3_nr, l3_entry) in self.lvl3[l2_nr].iter_mut().enumerate() { -- let virt_addr = (l2_nr << Granule512MiB::SHIFT) + (l3_nr << Granule64KiB::SHIFT); ++ + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { -+ let desc = TableDescriptor::from_next_lvl_table_addr( -+ self.lvl3[lvl2_nr].phys_base_addr().into_usize(), -+ ); ++ let desc = ++ TableDescriptor::from_next_lvl_table_addr(self.lvl3[lvl2_nr].phys_start_addr()); + *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; + } - -- let (output_addr, attribute_fields) = -- bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; ++ + fn phys_base_address(&self) -> Address { -+ self.lvl2.phys_base_addr() ++ self.lvl2.phys_start_addr() + } - -- *l3_entry = PageDescriptor::from_output_addr(output_addr, attribute_fields); ++ + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, @@ -542,15 +607,18 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + let p = phys_pages.as_slice(); + let v = virt_pages.as_slice(); + -+ if p.len() != v.len() { -+ return Err("Tried to map page slices with unequal sizes"); -+ } -+ + // No work to do for empty slices. -+ if p.is_empty() { ++ if v.is_empty() { + return Ok(()); + } -+ + +- let (phys_output_addr, attribute_fields) = +- bsp::memory::mmu::virt_mem_layout().virt_addr_properties(virt_addr)?; ++ if v.len() != p.len() { ++ return Err("Tried to map page slices with unequal sizes"); ++ } + +- *l3_entry = PageDescriptor::from_output_addr(phys_output_addr, &attribute_fields); + if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { + return Err("Tried to map outside of physical address space"); + } @@ -569,8 +637,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans } - /// The translation table's base address to be used for programming the MMU. -- pub fn base_address(&self) -> u64 { -- self.lvl2.base_addr_u64() +- pub fn phys_base_address(&self) -> u64 { +- self.lvl2.phys_start_addr_u64() + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, @@ -585,14 +653,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + return Err("Not enough MMIO space left"); + } + -+ let addr = (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) -+ | (self.cur_l3_mmio_index << Granule64KiB::SHIFT); ++ let addr = Address::new( ++ (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) ++ | (self.cur_l3_mmio_index << Granule64KiB::SHIFT), ++ ); + self.cur_l3_mmio_index += num_pages; + -+ Ok(PageSliceDescriptor::from_addr( -+ Address::new(addr), -+ num_pages, -+ )) ++ Ok(PageSliceDescriptor::from_addr(addr, num_pages)) + } + + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { @@ -609,11 +676,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans } } -@@ -292,6 +435,9 @@ +@@ -292,6 +437,9 @@ //-------------------------------------------------------------------------------------------------- #[cfg(test)] -+pub type MinSizeKernelTranslationTable = FixedSizeTranslationTable<1>; ++pub type MinSizeTranslationTable = FixedSizeTranslationTable<1>; + +#[cfg(test)] mod tests { @@ -623,93 +690,88 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs -@@ -15,7 +15,11 @@ +@@ -15,7 +15,7 @@ use crate::{ bsp, memory, - memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, -+ memory::{ -+ mmu::{translation_table::KernelTranslationTable, TranslationGranule}, -+ Address, Physical, -+ }, -+ synchronization::InitStateLock, ++ memory::{mmu::TranslationGranule, Address, Physical}, }; + use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; +@@ -45,13 +45,6 @@ + // Global instances + //-------------------------------------------------------------------------------------------------- -@@ -37,7 +41,7 @@ - pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - - /// The max supported address space size. --pub const MAX_ADDR_SPACE_SIZE: usize = 32 * 1024 * 1024 * 1024; // 32 GiB -+pub const MAX_ADDR_SPACE_SIZE: usize = 8 * 1024 * 1024 * 1024; // 8 GiB - - /// The supported address space size granule. - pub type AddrSpaceSizeGranule = Granule512MiB; -@@ -58,7 +62,8 @@ - /// # Safety - /// - /// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". +-/// The kernel translation tables. +-/// +-/// # Safety +-/// +-/// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static mut KERNEL_TABLES: KernelTranslationTable = KernelTranslationTable::new(); -+static KERNEL_TABLES: InitStateLock = -+ InitStateLock::new(KernelTranslationTable::new()); - +- static MMU: MemoryManagementUnit = MemoryManagementUnit; -@@ -83,7 +88,7 @@ + //-------------------------------------------------------------------------------------------------- +@@ -86,7 +79,7 @@ + /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -- let t0sz = (64 - bsp::memory::mmu::KernelAddrSpaceSize::SHIFT) as u64; -+ let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpaceSize::SHIFT) as u64; +- let t0sz = (64 - bsp::memory::mmu::KernelAddrSpace::SIZE_SHIFT) as u64; ++ let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored -@@ -103,6 +108,11 @@ - // Public Code - //-------------------------------------------------------------------------------------------------- - -+/// Return a guarded reference to the kernel's translation tables. -+pub fn kernel_translation_tables() -> &'static InitStateLock { -+ &KERNEL_TABLES -+} -+ - /// Return a reference to the MMU instance. - pub fn mmu() -> &'static impl memory::mmu::interface::MMU { - &MMU -@@ -113,7 +123,10 @@ - //------------------------------------------------------------------------------ + TCR_EL1::TBI0::Used +@@ -118,7 +111,10 @@ + use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { -- unsafe fn init(&self) -> Result<(), &'static str> { -+ unsafe fn enable( +- unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError> { ++ unsafe fn enable_mmu_and_caching( + &self, -+ kernel_table_phys_base_addr: Address, -+ ) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); -@@ -122,11 +135,8 @@ ++ phys_tables_base_addr: Address, ++ ) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } +@@ -133,13 +129,8 @@ // Prepare the memory attribute indirection register. self.set_up_mair(); - // Populate translation tables. -- KERNEL_TABLES.populate_tt_entries()?; +- KERNEL_TABLES +- .populate_tt_entries() +- .map_err(|e| MMUEnableError::Other(e))?; - // Set the "Translation Table Base Register". -- TTBR0_EL1.set_baddr(KERNEL_TABLES.base_address()); -+ TTBR0_EL1.set_baddr(kernel_table_phys_base_addr.into_usize() as u64); +- TTBR0_EL1.set_baddr(KERNEL_TABLES.phys_base_address()); ++ TTBR0_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); self.configure_translation_control(); -@@ -158,7 +168,7 @@ - #[kernel_test] - fn kernel_tables_in_bss() { - let bss_range = bsp::memory::bss_range_inclusive(); -- let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; -+ let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; - - assert!(bss_range.contains(&kernel_tables_addr)); +@@ -162,22 +153,3 @@ + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } + } +- +-//-------------------------------------------------------------------------------------------------- +-// Testing +-//-------------------------------------------------------------------------------------------------- +- +-#[cfg(test)] +-mod tests { +- use super::*; +- use test_macros::kernel_test; +- +- /// Check if KERNEL_TABLES is in .bss. +- #[kernel_test] +- fn kernel_tables_in_bss() { +- let bss_range = bsp::memory::bss_range_inclusive(); +- let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; +- +- assert!(bss_range.contains(&kernel_tables_addr)); +- } +-} diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -1305,7 +1367,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs 15_ use super::memory; -use crate::{bsp::device_driver, console}; -+use crate::{bsp::device_driver, console, cpu}; ++use crate::{bsp::device_driver, console, cpu, driver}; use core::fmt; //-------------------------------------------------------------------------------------------------- @@ -1315,7 +1377,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs 15_ pub unsafe fn panic_console_out() -> impl fmt::Write { - let mut panic_gpio = device_driver::PanicGPIO::new(memory::map::mmio::GPIO_START); - let mut panic_uart = device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START); -+ use crate::driver::interface::DeviceDriver; ++ use driver::interface::DeviceDriver; + let mut panic_gpio = device_driver::PanicGPIO::new(memory::map::mmio::GPIO_START.into_usize()); + let mut panic_uart = @@ -1379,7 +1441,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs -@@ -4,70 +4,131 @@ +@@ -4,70 +4,157 @@ //! BSP Memory Management Unit. @@ -1389,29 +1451,44 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +use crate::{ + common, + memory::{ -+ mmu as kernel_mmu, ++ mmu as generic_mmu, + mmu::{ -+ AccessPermissions, AddressSpaceSize, AttributeFields, MemAttributes, Page, -+ PageSliceDescriptor, TranslationGranule, ++ AccessPermissions, AddressSpace, AssociatedTranslationTable, AttributeFields, ++ MemAttributes, Page, PageSliceDescriptor, TranslationGranule, + }, + Physical, Virtual, + }, ++ synchronization::InitStateLock, +}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++type KernelTranslationTable = ++ ::TableStartFromBottom; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- --/// The address space size chosen by this BSP. --pub type KernelAddrSpaceSize = AddressSpaceSize<{ memory_map::END_INCLUSIVE + 1 }>; -- --const NUM_MEM_RANGES: usize = 2; +-/// The kernel's address space defined by this BSP. +-pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; +/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to +/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; +-const NUM_MEM_RANGES: usize = 2; ++/// The kernel's virtual address space defined by this BSP. ++pub type KernelVirtAddrSpace = AddressSpace<{ 8 * 1024 * 1024 * 1024 }>; + -/// The virtual memory layout. --/// ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++/// The kernel translation tables. + /// -/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. -/// It is agnostic of the paging granularity that the architecture's MMU will use. -pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::new( @@ -1439,8 +1516,12 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - }, - ], -); -+/// The address space size chosen by this BSP. -+pub type KernelVirtAddrSpaceSize = AddressSpaceSize<{ 8 * 1024 * 1024 * 1024 }>; ++/// It is mandatory that InitStateLock is transparent. ++/// ++/// That is, `size_of(InitStateLock) == size_of(KernelTranslationTable)`. ++/// There is a unit tests that checks this porperty. ++static KERNEL_TABLES: InitStateLock = ++ InitStateLock::new(KernelTranslationTable::new()); //-------------------------------------------------------------------------------------------------- // Private Code @@ -1477,10 +1558,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + let num_pages = size_to_num_pages(super::data_size()); + + PageSliceDescriptor::from_addr(super::virt_data_start(), num_pages) - } - --fn mmio_range_inclusive() -> RangeInclusive { -- RangeInclusive::new(memory_map::mmio::START, memory_map::mmio::END_INCLUSIVE) ++} ++ +// The binary is still identity mapped, so we don't need to convert in the following. + +/// The boot core's stack. @@ -1491,8 +1570,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// The Read-Only (RO) pages of the kernel binary. +fn phys_ro_page_desc() -> PageSliceDescriptor { + virt_ro_page_desc().into() -+} -+ + } + +-fn mmio_range_inclusive() -> RangeInclusive { +- RangeInclusive::new(memory_map::mmio::START, memory_map::mmio::END_INCLUSIVE) +/// The data pages of the kernel binary. +fn phys_data_page_desc() -> PageSliceDescriptor { + virt_data_page_desc().into() @@ -1505,6 +1586,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs -/// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout { - &LAYOUT ++/// Return a reference to the kernel's translation tables. ++pub fn kernel_translation_tables() -> &'static InitStateLock { ++ &KERNEL_TABLES ++} ++ +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( @@ -1519,7 +1605,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// +/// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. +pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { -+ kernel_mmu::kernel_map_pages_at( ++ generic_mmu::kernel_map_pages_at( + "Kernel boot-core stack", + &virt_stack_page_desc(), + &phys_stack_page_desc(), @@ -1530,7 +1616,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + }, + )?; + -+ kernel_mmu::kernel_map_pages_at( ++ generic_mmu::kernel_map_pages_at( + "Kernel code and RO data", + &virt_ro_page_desc(), + &phys_ro_page_desc(), @@ -1541,7 +1627,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + }, + )?; + -+ kernel_mmu::kernel_map_pages_at( ++ generic_mmu::kernel_map_pages_at( + "Kernel data and bss", + &virt_data_page_desc(), + &phys_data_page_desc(), @@ -1556,19 +1642,19 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs } //-------------------------------------------------------------------------------------------------- -@@ -82,14 +143,12 @@ +@@ -82,14 +169,12 @@ /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { - const SIXTYFOUR_KIB: usize = 65536; +- +- for i in LAYOUT.inner().iter() { +- let start: usize = *(i.virtual_range)().start(); +- let end: usize = *(i.virtual_range)().end() + 1; + for i in [virt_stack_page_desc, virt_ro_page_desc, virt_data_page_desc].iter() { + let start: usize = i().start_addr().into_usize(); + let end: usize = i().end_addr().into_usize(); -- for i in LAYOUT.inner().iter() { -- let start: usize = *(i.virtual_range)().start(); -- let end: usize = *(i.virtual_range)().end() + 1; -- - assert_eq!(start modulo SIXTYFOUR_KIB, 0); - assert_eq!(end modulo SIXTYFOUR_KIB, 0); + assert_eq!(start modulo KernelGranule::SIZE, 0); @@ -1576,7 +1662,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs assert!(end >= start); } } -@@ -97,17 +156,18 @@ +@@ -97,18 +182,28 @@ /// Ensure the kernel's virtual memory layout is free of overlaps. #[kernel_test] fn virt_mem_layout_has_no_overlaps() { @@ -1592,20 +1678,30 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - assert!(!second_range().contains(first_range().start())); - assert!(!second_range().contains(first_range().end())); + let layout = [ -+ virt_stack_page_desc().into_usize_range_inclusive(), -+ virt_ro_page_desc().into_usize_range_inclusive(), -+ virt_data_page_desc().into_usize_range_inclusive(), ++ virt_stack_page_desc(), ++ virt_ro_page_desc(), ++ virt_data_page_desc(), + ]; + + for (i, first_range) in layout.iter().enumerate() { + for second_range in layout.iter().skip(i + 1) { -+ assert!(!first_range.contains(second_range.start())); -+ assert!(!first_range.contains(second_range.end())); -+ assert!(!second_range.contains(first_range.start())); -+ assert!(!second_range.contains(first_range.end())); ++ assert!(!first_range.contains(second_range.start_addr())); ++ assert!(!first_range.contains(second_range.end_addr_inclusive())); ++ assert!(!second_range.contains(first_range.start_addr())); ++ assert!(!second_range.contains(first_range.end_addr_inclusive())); } } } ++ ++ /// Check if KERNEL_TABLES is in .bss. ++ #[kernel_test] ++ fn kernel_tables_in_bss() { ++ let bss_range = super::super::bss_range_inclusive(); ++ let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; ++ ++ assert!(bss_range.contains(&kernel_tables_addr)); ++ } + } diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -1939,15 +2035,16 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs --- 14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 15_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -112,6 +112,7 @@ +@@ -112,6 +112,8 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] ++#![feature(const_evaluatable_checked)] +#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -133,6 +134,7 @@ +@@ -133,6 +135,7 @@ mod synchronization; pub mod bsp; @@ -1959,7 +2056,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_mmio_remap/src/main.rs --- 14_exceptions_part2_peripheral_IRQs/src/main.rs +++ 15_virtual_mem_part2_mmio_remap/src/main.rs -@@ -26,21 +26,34 @@ +@@ -26,21 +26,39 @@ #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; @@ -1967,14 +2064,18 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_m exception::handling_init(); -- if let Err(string) = memory::mmu::mmu().init() { +- if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { - panic!("MMU: {}", string); -+ if let Err(string) = memory::mmu::kernel_map_binary_and_enable_mmu() { -+ panic!("Enabling MMU failed: {}", string); - } -+ // Printing will silently fail fail from here on, because the driver's MMIO is not remapped yet. - -- for i in bsp::driver::driver_manager().all_device_drivers().iter() { ++ let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { ++ Err(string) => panic!("Error mapping kernel binary: {}", string), ++ Ok(addr) => addr, ++ }; ++ ++ if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { ++ panic!("Enabling MMU failed: {}", e); ++ } ++ // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. ++ + // Bring up the drivers needed for printing first. + for i in bsp::driver::driver_manager() + .early_print_device_drivers() @@ -1982,10 +2083,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_m + { + // Any encountered errors cannot be printed yet, obviously, so just safely park the CPU. + i.init().unwrap_or_else(|_| cpu::wait_forever()); -+ } + } + bsp::driver::driver_manager().post_early_print_device_driver_init(); + // Printing available again from here on. -+ + +- for i in bsp::driver::driver_manager().all_device_drivers().iter() { + // Now bring up the remaining drivers. + for i in bsp::driver::driver_manager() + .non_early_print_device_drivers() @@ -2000,7 +2102,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_m // Let device drivers register and enable their handlers with the interrupt controller. for i in bsp::driver::driver_manager().all_device_drivers() { -@@ -66,8 +79,8 @@ +@@ -66,8 +84,8 @@ info!("Booting on: {}", bsp::board_name()); @@ -2129,12 +2231,12 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + const KIB_RSHIFT: u32 = 10; // log2(1024). + const MIB_RSHIFT: u32 = 20; // log2(1024 * 1024). + -+ info!(" -----------------------------------------------------------------------------------------------------------------"); ++ info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + info!( -+ " {:^24} {:^24} {:^7} {:^9} {:^35}", ++ " {:^44} {:^30} {:^7} {:^9} {:^35}", + "Virtual", "Physical", "Size", "Attr", "Entity" + ); -+ info!(" -----------------------------------------------------------------------------------------------------------------"); ++ info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + + for i in self + .inner @@ -2142,10 +2244,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + .filter(|x| x.is_some()) + .map(|x| x.unwrap()) + { -+ let virt_start = i.virt_start_addr.into_usize(); ++ let virt_start = i.virt_start_addr; + let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; -+ let phys_start = i.phys_pages.start_addr().into_usize(); -+ let phys_end_inclusive = i.phys_pages.end_addr_inclusive().into_usize(); ++ let phys_start = i.phys_pages.start_addr(); ++ let phys_end_inclusive = i.phys_pages.end_addr_inclusive(); + let size = i.phys_pages.size(); + + let (size, unit) = if (size >> MIB_RSHIFT) > 0 { @@ -2173,7 +2275,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + }; + + info!( -+ " {:#011X}..{:#011X} --> {:#011X}..{:#011X} | \ ++ " {}..{} --> {}..{} | \ + {: >3} {} | {: <3} {} {: <2} | {}", + virt_start, + virt_end_inclusive, @@ -2190,14 +2292,14 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + for k in i.users[1..].iter() { + if let Some(additional_user) = *k { + info!( -+ " | {}", ++ " | {}", + additional_user + ); + } + } + } + -+ info!(" -----------------------------------------------------------------------------------------------------------------"); ++ info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + } +} + @@ -2241,7 +2343,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs --- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs -@@ -8,7 +8,104 @@ +@@ -8,7 +8,105 @@ #[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] mod arch_translation_table; @@ -2253,7 +2355,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.r //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- - pub use arch_translation_table::KernelTranslationTable; +-pub use arch_translation_table::KernelTranslationTable; ++#[cfg(target_arch = "aarch64")] ++pub use arch_translation_table::FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -2271,7 +2375,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.r + /// + /// - Implementor must ensure that this function can run only once or is harmless if invoked + /// multiple times. -+ unsafe fn init(&mut self); ++ fn init(&mut self); + + /// The translation table's base address to be used for programming the MMU. + fn phys_base_address(&self) -> Address; @@ -2317,17 +2421,17 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.r +mod tests { + use super::*; + use crate::bsp; -+ use arch_translation_table::MinSizeKernelTranslationTable; ++ use arch_translation_table::MinSizeTranslationTable; + use interface::TranslationTable; + use test_macros::kernel_test; + -+ /// Sanity checks for the kernel TranslationTable implementation. ++ /// Sanity checks for the TranslationTable implementation. + #[kernel_test] + fn translationtable_implementation_sanity() { -+ // Need to take care that `tables` fits into the stack. -+ let mut tables = MinSizeKernelTranslationTable::new(); ++ // This will occupy a lot of space on the stack. ++ let mut tables = MinSizeTranslationTable::new(); + -+ unsafe { tables.init() }; ++ tables.init(); + + let x = tables.next_mmio_virt_page_slice(0); + assert!(x.is_err()); @@ -2350,7 +2454,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.r diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs --- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs -@@ -0,0 +1,213 @@ +@@ -0,0 +1,210 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -2361,7 +2465,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + bsp, common, + memory::{Address, AddressType, Physical, Virtual}, +}; -+use core::{convert::From, marker::PhantomData, ops::RangeInclusive}; ++use core::{convert::From, marker::PhantomData}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -2474,6 +2578,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + self.start + (self.size() - 1) + } + ++ /// Check if an address is contained within this descriptor. ++ pub fn contains(&self, addr: Address) -> bool { ++ (addr >= self.start_addr()) && (addr <= self.end_addr_inclusive()) ++ } ++ + /// Return a non-mutable slice of Pages. + /// + /// # Safety @@ -2482,14 +2591,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + pub unsafe fn as_slice(&self) -> &[Page] { + core::slice::from_raw_parts(self.first_page_ptr(), self.num_pages) + } -+ -+ /// Return the inclusive address range of the slice. -+ pub fn into_usize_range_inclusive(self) -> RangeInclusive { -+ RangeInclusive::new( -+ self.start_addr().into_usize(), -+ self.end_addr_inclusive().into_usize(), -+ ) -+ } +} + +impl From> for PageSliceDescriptor { @@ -2568,7 +2669,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs -@@ -3,29 +3,22 @@ +@@ -3,29 +3,23 @@ // Copyright (c) 2020-2021 Andre Richter //! Memory Management Unit. @@ -2597,6 +2698,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + memory::{Address, Physical, Virtual}, + synchronization, warn, +}; ++use core::fmt; -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports @@ -2606,33 +2708,28 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p //-------------------------------------------------------------------------------------------------- // Public Definitions -@@ -33,16 +26,20 @@ - - /// Memory Management interfaces. - pub mod interface { -+ use super::*; +@@ -45,13 +39,15 @@ /// MMU functions. pub trait MMU { - /// Called by the kernel during early init. Supposed to take the translation tables from the - /// `BSP`-supplied `virt_mem_layout()` and install/activate them for the respective MMU. -+ /// Turns on the MMU. ++ /// Turns on the MMU for the first time and enables data and instruction caching. /// /// # Safety /// -+ /// - Must only be called after the kernel translation tables have been init()'ed. /// - Changes the HW's global state. -- unsafe fn init(&self) -> Result<(), &'static str>; -+ unsafe fn enable( +- unsafe fn enable_mmu_and_caching(&self) -> Result<(), MMUEnableError>; ++ unsafe fn enable_mmu_and_caching( + &self, -+ kernel_table_phys_base_addr: Address, -+ ) -> Result<(), &'static str>; - } - } ++ phys_tables_base_addr: Address, ++ ) -> Result<(), MMUEnableError>; -@@ -52,55 +49,35 @@ - /// Describes the size of an address space. - pub struct AddressSpaceSize; + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; +@@ -64,55 +60,43 @@ + /// Describes properties of an address space. + pub struct AddressSpace; -/// Architecture agnostic translation types. -#[allow(missing_docs)] @@ -2665,8 +2762,14 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - pub mem_attributes: MemAttributes, - pub acc_perms: AccessPermissions, - pub execute_never: bool, --} -- ++/// Intended to be implemented for [`AddressSpace`]. ++pub trait AssociatedTranslationTable { ++ /// A translation table whose address range is: ++ /// ++ /// [0, AS_SIZE - 1] ++ type TableStartFromBottom; + } + -/// Architecture agnostic descriptor for a memory range. -#[allow(missing_docs)] -pub struct TranslationDescriptor { @@ -2675,11 +2778,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p - pub physical_range_translation: Translation, - pub attribute_fields: AttributeFields, -} -- --/// Type for expressing the kernel's virtual memory layout. --pub struct KernelVirtualLayout { -- /// The last (inclusive) address of the address space. -- max_virt_addr_inclusive: usize, +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- @@ -2701,9 +2799,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { -+ arch_mmu::kernel_translation_tables() ++ bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; -+ + +-/// Type for expressing the kernel's virtual memory layout. +-pub struct KernelVirtualLayout { +- /// The last (inclusive) address of the address space. +- max_virt_addr_inclusive: usize, + if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { + warn!("{}", x); + } @@ -2714,7 +2816,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p } //-------------------------------------------------------------------------------------------------- -@@ -111,6 +88,9 @@ +@@ -132,6 +116,9 @@ /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -2724,7 +2826,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p /// The granule's shift, aka log2(size). pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; -@@ -142,110 +122,89 @@ +@@ -159,110 +146,103 @@ } } @@ -2848,7 +2950,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { -+ let is_mmio = arch_mmu::kernel_translation_tables() ++ let is_mmio = bsp::memory::mmu::kernel_translation_tables() + .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); + if is_mmio { + return Err("Attempt to manually map into MMIO region"); @@ -2881,8 +2983,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + addr + // Otherwise, allocate a new virtual page slice and map it. + } else { -+ let virt_pages: PageSliceDescriptor = arch_mmu::kernel_translation_tables() -+ .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; ++ let virt_pages: PageSliceDescriptor = ++ bsp::memory::mmu::kernel_translation_tables() ++ .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; + + kernel_map_pages_at_unchecked( + name, @@ -2901,19 +3004,32 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + Ok(virt_addr + offset_into_start_page) +} + -+/// Map the kernel's binary and enable the MMU. ++/// Map the kernel's binary. Returns the translation table's base address. +/// +/// # Safety +/// -+/// - Crucial function during kernel init. Changes the the complete memory view of the processor. -+pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> { -+ let phys_base_addr = arch_mmu::kernel_translation_tables().write(|tables| { -+ tables.init(); -+ tables.phys_base_address() -+ }); ++/// - See [`bsp::memory::mmu::kernel_map_binary()`]. ++pub unsafe fn kernel_map_binary() -> Result, &'static str> { ++ let phys_kernel_tables_base_addr = ++ bsp::memory::mmu::kernel_translation_tables().write(|tables| { ++ tables.init(); ++ tables.phys_base_address() ++ }); + + bsp::memory::mmu::kernel_map_binary()?; -+ arch_mmu::mmu().enable(phys_base_addr) ++ ++ Ok(phys_kernel_tables_base_addr) ++} ++ ++/// Enable the MMU and data + instruction caching. ++/// ++/// # Safety ++/// ++/// - Crucial function during kernel init. Changes the the complete memory view of the processor. ++pub unsafe fn enable_mmu_and_caching( ++ phys_tables_base_addr: Address, ++) -> Result<(), MMUEnableError> { ++ arch_mmu::mmu().enable_mmu_and_caching(phys_tables_base_addr) +} + +/// Human-readable print of all recorded kernel mappings. @@ -2924,13 +3040,17 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2_mmio_remap/src/memory.rs --- 14_exceptions_part2_peripheral_IRQs/src/memory.rs +++ 15_virtual_mem_part2_mmio_remap/src/memory.rs -@@ -6,12 +6,85 @@ +@@ -6,12 +6,136 @@ pub mod mmu; -use core::ops::RangeInclusive; +use crate::common; -+use core::{marker::PhantomData, ops::RangeInclusive}; ++use core::{ ++ fmt, ++ marker::PhantomData, ++ ops::{AddAssign, RangeInclusive, SubAssign}, ++}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -2997,6 +3117,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2 + } +} + ++impl AddAssign for Address { ++ fn add_assign(&mut self, other: Self) { ++ *self = Self { ++ value: self.value + other.into_usize(), ++ _address_type: PhantomData, ++ }; ++ } ++} ++ +impl core::ops::Sub for Address { + type Output = Self; + @@ -3007,6 +3136,44 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2 + } + } +} ++ ++impl SubAssign for Address { ++ fn sub_assign(&mut self, other: Self) { ++ *self = Self { ++ value: self.value - other.into_usize(), ++ _address_type: PhantomData, ++ }; ++ } ++} ++ ++impl fmt::Display for Address { ++ // Don't expect to see physical addresses greater than 40 bit. ++ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { ++ let q3: u8 = ((self.value >> 32) & 0xff) as u8; ++ let q2: u16 = ((self.value >> 16) & 0xffff) as u16; ++ let q1: u16 = (self.value & 0xffff) as u16; ++ ++ write!(f, "0x")?; ++ write!(f, "{:02x}_", q3)?; ++ write!(f, "{:04x}_", q2)?; ++ write!(f, "{:04x}", q1) ++ } ++} ++ ++impl fmt::Display for Address { ++ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { ++ let q4: u16 = ((self.value >> 48) & 0xffff) as u16; ++ let q3: u16 = ((self.value >> 32) & 0xffff) as u16; ++ let q2: u16 = ((self.value >> 16) & 0xffff) as u16; ++ let q1: u16 = (self.value & 0xffff) as u16; ++ ++ write!(f, "0x")?; ++ write!(f, "{:04x}_", q4)?; ++ write!(f, "{:04x}_", q3)?; ++ write!(f, "{:04x}_", q2)?; ++ write!(f, "{:04x}", q1) ++ } ++} + /// Zero out an inclusive memory range. /// @@ -3024,17 +3191,25 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault exception::handling_init(); bsp::console::qemu_bring_up_console(); -@@ -29,10 +29,22 @@ +@@ -29,10 +29,30 @@ println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); -- if let Err(string) = memory::mmu::mmu().init() { +- if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { - println!("MMU: {}", string); -+ if let Err(string) = memory::mmu::kernel_map_binary_and_enable_mmu() { -+ println!("Enabling MMU failed: {}", string); ++ let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { ++ Err(string) => { ++ println!("Error mapping kernel binary: {}", string); ++ cpu::qemu_exit_failure() ++ } ++ Ok(addr) => addr, ++ }; ++ ++ if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { ++ println!("Enabling MMU failed: {}", e); cpu::qemu_exit_failure() } -+ // Printing will silently fail fail from here on, because the driver's MMIO is not remapped yet. ++ // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. + + // Bring up the drivers needed for printing first. + for i in bsp::driver::driver_manager() diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs index ebf454ce9..223115168 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -15,12 +15,9 @@ use crate::{ bsp, memory, - memory::{ - mmu::{translation_table::KernelTranslationTable, TranslationGranule}, - Address, Physical, - }, - synchronization::InitStateLock, + memory::{mmu::TranslationGranule, Address, Physical}, }; +use core::intrinsics::unlikely; use cortex_a::{barrier, regs::*}; //-------------------------------------------------------------------------------------------------- @@ -37,15 +34,6 @@ struct MemoryManagementUnit; pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; -/// The min supported address space size. -pub const MIN_ADDR_SPACE_SIZE: usize = 1024 * 1024 * 1024; // 1 GiB - -/// The max supported address space size. -pub const MAX_ADDR_SPACE_SIZE: usize = 8 * 1024 * 1024 * 1024; // 8 GiB - -/// The supported address space size granule. -pub type AddrSpaceSizeGranule = Granule512MiB; - /// Constants for indexing the MAIR_EL1. #[allow(dead_code)] pub mod mair { @@ -57,20 +45,24 @@ pub mod mair { // Global instances //-------------------------------------------------------------------------------------------------- -/// The kernel translation tables. -/// -/// # Safety -/// -/// - Supposed to land in `.bss`. Therefore, ensure that all initial member values boil down to "0". -static KERNEL_TABLES: InitStateLock = - InitStateLock::new(KernelTranslationTable::new()); - static MMU: MemoryManagementUnit = MemoryManagementUnit; //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. fn set_up_mair(&self) { @@ -87,19 +79,19 @@ impl MemoryManagementUnit { /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { - let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); - let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpaceSize::SHIFT) as u64; + let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; TCR_EL1.write( - TCR_EL1::TBI0::Ignored - + TCR_EL1::IPS.val(ips) - + TCR_EL1::EPD1::DisableTTBR1Walks + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + TCR_EL1::TG0::KiB_64 + TCR_EL1::SH0::Inner + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + TCR_EL1::EPD0::EnableTTBR0Walks - + TCR_EL1::T0SZ.val(t0sz), + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, ); } } @@ -108,11 +100,6 @@ impl MemoryManagementUnit { // Public Code //-------------------------------------------------------------------------------------------------- -/// Return a guarded reference to the kernel's translation tables. -pub fn kernel_translation_tables() -> &'static InitStateLock { - &KERNEL_TABLES -} - /// Return a reference to the MMU instance. pub fn mmu() -> &'static impl memory::mmu::interface::MMU { &MMU @@ -121,22 +108,29 @@ pub fn mmu() -> &'static impl memory::mmu::interface::MMU { //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ +use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { - unsafe fn enable( + unsafe fn enable_mmu_and_caching( &self, - kernel_table_phys_base_addr: Address, - ) -> Result<(), &'static str> { - // Fail early if translation granule is not supported. Both RPis support it, though. - if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { - return Err("Translation granule not supported in HW"); + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); } // Prepare the memory attribute indirection register. self.set_up_mair(); // Set the "Translation Table Base Register". - TTBR0_EL1.set_baddr(kernel_table_phys_base_addr.into_usize() as u64); + TTBR0_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); self.configure_translation_control(); @@ -153,23 +147,9 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { Ok(()) } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check if KERNEL_TABLES is in .bss. - #[kernel_test] - fn kernel_tables_in_bss() { - let bss_range = bsp::memory::bss_range_inclusive(); - let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; - assert!(bss_range.contains(&kernel_tables_addr)); + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs index f682d6a49..15784a1b6 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -20,7 +20,7 @@ use crate::{ arch_mmu::{Granule512MiB, Granule64KiB}, AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, }, - Address, AddressType, Physical, Virtual, + Address, Physical, Virtual, }, }; use core::convert; @@ -90,8 +90,8 @@ register_bitfields! {u64, AttrIndx OFFSET(2) NUMBITS(3) [], TYPE OFFSET(1) NUMBITS(1) [ - Block = 0, - Table = 1 + Reserved_Invalid = 0, + Page = 1 ], VALID OFFSET(0) NUMBITS(1) [ @@ -119,19 +119,16 @@ struct PageDescriptor { value: u64, } -trait BaseAddr { - fn phys_base_addr(&self) -> Address; +trait StartAddr { + fn phys_start_addr(&self) -> Address; } -const NUM_LVL2_TABLES: usize = - bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE >> Granule512MiB::SHIFT; - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB -/// aligned, hence the "reverse" order of appearance. +/// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] pub struct FixedSizeTranslationTable { @@ -148,16 +145,13 @@ pub struct FixedSizeTranslationTable { initialized: bool, } -/// A translation table type for the kernel space. -pub type KernelTranslationTable = FixedSizeTranslationTable; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BaseAddr for [T; N] { - fn phys_base_addr(&self) -> Address { - // The binary is still identity mapped, so we don't need to convert here. +// The binary is still identity mapped, so we don't need to convert here. +impl StartAddr for [T; N] { + fn phys_start_addr(&self) -> Address { Address::new(self as *const _ as usize) } } @@ -171,14 +165,14 @@ impl TableDescriptor { } /// Create an instance pointing to the supplied address. - pub fn from_next_lvl_table_addr(next_lvl_table_addr: usize) -> Self { + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: Address) -> Self { let val = InMemoryRegister::::new(0); - let shifted = next_lvl_table_addr >> Granule64KiB::SHIFT; + let shifted = phys_next_lvl_table_addr.into_usize() >> Granule64KiB::SHIFT; val.write( - STAGE1_TABLE_DESCRIPTOR::VALID::True + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table - + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64), + + STAGE1_TABLE_DESCRIPTOR::VALID::True, ); TableDescriptor { value: val.get() } @@ -232,18 +226,18 @@ impl PageDescriptor { /// Create an instance. pub fn from_output_addr( - output_addr: *const Page, + phys_output_addr: *const Page, attribute_fields: &AttributeFields, ) -> Self { let val = InMemoryRegister::::new(0); - let shifted = output_addr as u64 >> Granule64KiB::SHIFT; + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; val.write( - STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + STAGE1_PAGE_DESCRIPTOR::AF::True - + attribute_fields.clone().into() - + STAGE1_PAGE_DESCRIPTOR::TYPE::Table - + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted), + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), ); Self { value: val.get() } @@ -260,6 +254,14 @@ impl PageDescriptor { // Public Code //-------------------------------------------------------------------------------------------------- +impl memory::mmu::AssociatedTranslationTable + for memory::mmu::AddressSpace +where + [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, +{ + type TableStartFromBottom = FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }>; +} + impl FixedSizeTranslationTable { // Reserve the last 256 MiB of the address space for MMIO mappings. const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; @@ -269,8 +271,9 @@ impl FixedSizeTranslationTable { #[allow(clippy::assertions_on_constants)] pub const fn new() -> Self { assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); + + // Can't have a zero-sized address space. assert!(NUM_TABLES > 0); - assert!((bsp::memory::mmu::KernelVirtAddrSpaceSize::SIZE % Granule512MiB::SIZE) == 0); Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], @@ -282,7 +285,7 @@ impl FixedSizeTranslationTable { /// The start address of the table's MMIO range. #[inline(always)] - const fn mmio_start_addr(&self) -> Address { + fn mmio_start_addr(&self) -> Address { Address::new( (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), @@ -291,7 +294,7 @@ impl FixedSizeTranslationTable { /// The inclusive end address of the table's MMIO range. #[inline(always)] - const fn mmio_end_addr_inclusive(&self) -> Address { + fn mmio_end_addr_inclusive(&self) -> Address { Address::new( (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) | (8191 << Granule64KiB::SHIFT) @@ -301,12 +304,13 @@ impl FixedSizeTranslationTable { /// Helper to calculate the lvl2 and lvl3 indices from an address. #[inline(always)] - fn lvl2_lvl3_index_from( + fn lvl2_lvl3_index_from( &self, - addr: *const Page, + addr: *const Page, ) -> Result<(usize, usize), &'static str> { - let lvl2_index = addr as usize >> Granule512MiB::SHIFT; - let lvl3_index = (addr as usize & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + let addr = addr as usize; + let lvl2_index = addr >> Granule512MiB::SHIFT; + let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; if lvl2_index > (NUM_TABLES - 1) { return Err("Virtual page is out of bounds of translation table"); @@ -334,16 +338,15 @@ impl FixedSizeTranslationTable { impl memory::mmu::translation_table::interface::TranslationTable for FixedSizeTranslationTable { - unsafe fn init(&mut self) { + fn init(&mut self) { if self.initialized { return; } // Populate the l2 entries. for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { - let desc = TableDescriptor::from_next_lvl_table_addr( - self.lvl3[lvl2_nr].phys_base_addr().into_usize(), - ); + let desc = + TableDescriptor::from_next_lvl_table_addr(self.lvl3[lvl2_nr].phys_start_addr()); *lvl2_entry = desc; } @@ -352,7 +355,7 @@ impl memory::mmu::translation_table::interface::Transla } fn phys_base_address(&self) -> Address { - self.lvl2.phys_base_addr() + self.lvl2.phys_start_addr() } unsafe fn map_pages_at( @@ -366,15 +369,15 @@ impl memory::mmu::translation_table::interface::Transla let p = phys_pages.as_slice(); let v = virt_pages.as_slice(); - if p.len() != v.len() { - return Err("Tried to map page slices with unequal sizes"); - } - // No work to do for empty slices. - if p.is_empty() { + if v.is_empty() { return Ok(()); } + if v.len() != p.len() { + return Err("Tried to map page slices with unequal sizes"); + } + if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { return Err("Tried to map outside of physical address space"); } @@ -406,14 +409,13 @@ impl memory::mmu::translation_table::interface::Transla return Err("Not enough MMIO space left"); } - let addr = (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) - | (self.cur_l3_mmio_index << Granule64KiB::SHIFT); + let addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (self.cur_l3_mmio_index << Granule64KiB::SHIFT), + ); self.cur_l3_mmio_index += num_pages; - Ok(PageSliceDescriptor::from_addr( - Address::new(addr), - num_pages, - )) + Ok(PageSliceDescriptor::from_addr(addr, num_pages)) } fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { @@ -435,7 +437,7 @@ impl memory::mmu::translation_table::interface::Transla //-------------------------------------------------------------------------------------------------- #[cfg(test)] -pub type MinSizeKernelTranslationTable = FixedSizeTranslationTable<1>; +pub type MinSizeTranslationTable = FixedSizeTranslationTable<1>; #[cfg(test)] mod tests { diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs index f63a9e9bb..abf8f89cc 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs @@ -5,7 +5,7 @@ //! BSP console facilities. use super::memory; -use crate::{bsp::device_driver, console, cpu}; +use crate::{bsp::device_driver, console, cpu, driver}; use core::fmt; //-------------------------------------------------------------------------------------------------- @@ -23,7 +23,7 @@ use core::fmt; /// /// - Use only for printing during a panic. pub unsafe fn panic_console_out() -> impl fmt::Write { - use crate::driver::interface::DeviceDriver; + use driver::interface::DeviceDriver; let mut panic_gpio = device_driver::PanicGPIO::new(memory::map::mmio::GPIO_START.into_usize()); let mut panic_uart = diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs index 76b93e47a..842384dea 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -7,15 +7,23 @@ use crate::{ common, memory::{ - mmu as kernel_mmu, + mmu as generic_mmu, mmu::{ - AccessPermissions, AddressSpaceSize, AttributeFields, MemAttributes, Page, - PageSliceDescriptor, TranslationGranule, + AccessPermissions, AddressSpace, AssociatedTranslationTable, AttributeFields, + MemAttributes, Page, PageSliceDescriptor, TranslationGranule, }, Physical, Virtual, }, + synchronization::InitStateLock, }; +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +type KernelTranslationTable = + ::TableStartFromBottom; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,8 +32,21 @@ use crate::{ /// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; -/// The address space size chosen by this BSP. -pub type KernelVirtAddrSpaceSize = AddressSpaceSize<{ 8 * 1024 * 1024 * 1024 }>; +/// The kernel's virtual address space defined by this BSP. +pub type KernelVirtAddrSpace = AddressSpace<{ 8 * 1024 * 1024 * 1024 }>; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +/// The kernel translation tables. +/// +/// It is mandatory that InitStateLock is transparent. +/// +/// That is, `size_of(InitStateLock) == size_of(KernelTranslationTable)`. +/// There is a unit tests that checks this porperty. +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new()); //-------------------------------------------------------------------------------------------------- // Private Code @@ -81,6 +102,11 @@ fn phys_data_page_desc() -> PageSliceDescriptor { // Public Code //-------------------------------------------------------------------------------------------------- +/// Return a reference to the kernel's translation tables. +pub fn kernel_translation_tables() -> &'static InitStateLock { + &KERNEL_TABLES +} + /// Pointer to the last page of the physical address space. pub fn phys_addr_space_end_page() -> *const Page { common::align_down( @@ -95,7 +121,7 @@ pub fn phys_addr_space_end_page() -> *const Page { /// /// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel boot-core stack", &virt_stack_page_desc(), &phys_stack_page_desc(), @@ -106,7 +132,7 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { }, )?; - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel code and RO data", &virt_ro_page_desc(), &phys_ro_page_desc(), @@ -117,7 +143,7 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { }, )?; - kernel_mmu::kernel_map_pages_at( + generic_mmu::kernel_map_pages_at( "Kernel data and bss", &virt_data_page_desc(), &phys_data_page_desc(), @@ -157,18 +183,27 @@ mod tests { #[kernel_test] fn virt_mem_layout_has_no_overlaps() { let layout = [ - virt_stack_page_desc().into_usize_range_inclusive(), - virt_ro_page_desc().into_usize_range_inclusive(), - virt_data_page_desc().into_usize_range_inclusive(), + virt_stack_page_desc(), + virt_ro_page_desc(), + virt_data_page_desc(), ]; for (i, first_range) in layout.iter().enumerate() { for second_range in layout.iter().skip(i + 1) { - assert!(!first_range.contains(second_range.start())); - assert!(!first_range.contains(second_range.end())); - assert!(!second_range.contains(first_range.start())); - assert!(!second_range.contains(first_range.end())); + assert!(!first_range.contains(second_range.start_addr())); + assert!(!first_range.contains(second_range.end_addr_inclusive())); + assert!(!second_range.contains(first_range.start_addr())); + assert!(!second_range.contains(first_range.end_addr_inclusive())); } } } + + /// Check if KERNEL_TABLES is in .bss. + #[kernel_test] + fn kernel_tables_in_bss() { + let bss_range = super::super::bss_range_inclusive(); + let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; + + assert!(bss_range.contains(&kernel_tables_addr)); + } } diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index dad5e903c..30f684af4 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -112,6 +112,7 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] +#![feature(const_evaluatable_checked)] #![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/15_virtual_mem_part2_mmio_remap/src/main.rs b/15_virtual_mem_part2_mmio_remap/src/main.rs index 11bb6902d..40d3ed410 100644 --- a/15_virtual_mem_part2_mmio_remap/src/main.rs +++ b/15_virtual_mem_part2_mmio_remap/src/main.rs @@ -19,7 +19,7 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Virtual memory must be activated before the device drivers. +/// - Caching must be activated before the device drivers. /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device /// drivers (which currently employ IRQSafeNullLocks instead of spinlocks), will fail to /// work on the RPi SoCs. @@ -29,10 +29,15 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); - if let Err(string) = memory::mmu::kernel_map_binary_and_enable_mmu() { - panic!("Enabling MMU failed: {}", string); + let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { + Err(string) => panic!("Error mapping kernel binary: {}", string), + Ok(addr) => addr, + }; + + if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { + panic!("Enabling MMU failed: {}", e); } - // Printing will silently fail fail from here on, because the driver's MMIO is not remapped yet. + // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. // Bring up the drivers needed for printing first. for i in bsp::driver::driver_manager() diff --git a/15_virtual_mem_part2_mmio_remap/src/memory.rs b/15_virtual_mem_part2_mmio_remap/src/memory.rs index 1493b1a94..515731eb7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory.rs @@ -7,7 +7,11 @@ pub mod mmu; use crate::common; -use core::{marker::PhantomData, ops::RangeInclusive}; +use core::{ + fmt, + marker::PhantomData, + ops::{AddAssign, RangeInclusive, SubAssign}, +}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -74,6 +78,15 @@ impl core::ops::Add for Address { } } +impl AddAssign for Address { + fn add_assign(&mut self, other: Self) { + *self = Self { + value: self.value + other.into_usize(), + _address_type: PhantomData, + }; + } +} + impl core::ops::Sub for Address { type Output = Self; @@ -85,6 +98,44 @@ impl core::ops::Sub for Address { } } +impl SubAssign for Address { + fn sub_assign(&mut self, other: Self) { + *self = Self { + value: self.value - other.into_usize(), + _address_type: PhantomData, + }; + } +} + +impl fmt::Display for Address { + // Don't expect to see physical addresses greater than 40 bit. + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q3: u8 = ((self.value >> 32) & 0xff) as u8; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:02x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + +impl fmt::Display for Address { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q4: u16 = ((self.value >> 48) & 0xffff) as u16; + let q3: u16 = ((self.value >> 32) & 0xffff) as u16; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:04x}_", q4)?; + write!(f, "{:04x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + /// Zero out an inclusive memory range. /// /// # Safety diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index b02055937..7a0115af0 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -17,6 +17,7 @@ use crate::{ memory::{Address, Physical, Virtual}, synchronization, warn, }; +use core::fmt; pub use types::*; @@ -24,30 +25,48 @@ pub use types::*; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + /// Memory Management interfaces. pub mod interface { use super::*; /// MMU functions. pub trait MMU { - /// Turns on the MMU. + /// Turns on the MMU for the first time and enables data and instruction caching. /// /// # Safety /// - /// - Must only be called after the kernel translation tables have been init()'ed. /// - Changes the HW's global state. - unsafe fn enable( + unsafe fn enable_mmu_and_caching( &self, - kernel_table_phys_base_addr: Address, - ) -> Result<(), &'static str>; + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; } } /// Describes the characteristics of a translation granule. pub struct TranslationGranule; -/// Describes the size of an address space. -pub struct AddressSpaceSize; +/// Describes properties of an address space. +pub struct AddressSpace; + +/// Intended to be implemented for [`AddressSpace`]. +pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// + /// [0, AS_SIZE - 1] + type TableStartFromBottom; +} //-------------------------------------------------------------------------------------------------- // Private Code @@ -70,7 +89,7 @@ unsafe fn kernel_map_pages_at_unchecked( phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - arch_mmu::kernel_translation_tables() + bsp::memory::mmu::kernel_translation_tables() .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { @@ -84,6 +103,15 @@ unsafe fn kernel_map_pages_at_unchecked( // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + impl TranslationGranule { /// The granule's size. pub const SIZE: usize = Self::size_checked(); @@ -101,22 +129,18 @@ impl TranslationGranule { } } -impl AddressSpaceSize { +impl AddressSpace { /// The address space size. pub const SIZE: usize = Self::size_checked(); /// The address space shift, aka log2(size). - pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; const fn size_checked() -> usize { assert!(AS_SIZE.is_power_of_two()); - assert!(arch_mmu::MIN_ADDR_SPACE_SIZE.is_power_of_two()); - assert!(arch_mmu::MAX_ADDR_SPACE_SIZE.is_power_of_two()); - // Must adhere to architectural restrictions. - assert!(AS_SIZE >= arch_mmu::MIN_ADDR_SPACE_SIZE); - assert!(AS_SIZE <= arch_mmu::MAX_ADDR_SPACE_SIZE); - assert!((AS_SIZE % arch_mmu::AddrSpaceSizeGranule::SIZE) == 0); + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); AS_SIZE } @@ -136,7 +160,7 @@ pub unsafe fn kernel_map_pages_at( phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - let is_mmio = arch_mmu::kernel_translation_tables() + let is_mmio = bsp::memory::mmu::kernel_translation_tables() .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); if is_mmio { return Err("Attempt to manually map into MMIO region"); @@ -169,8 +193,9 @@ pub unsafe fn kernel_map_mmio( addr // Otherwise, allocate a new virtual page slice and map it. } else { - let virt_pages: PageSliceDescriptor = arch_mmu::kernel_translation_tables() - .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; + let virt_pages: PageSliceDescriptor = + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; kernel_map_pages_at_unchecked( name, @@ -189,19 +214,32 @@ pub unsafe fn kernel_map_mmio( Ok(virt_addr + offset_into_start_page) } -/// Map the kernel's binary and enable the MMU. +/// Map the kernel's binary. Returns the translation table's base address. /// /// # Safety /// -/// - Crucial function during kernel init. Changes the the complete memory view of the processor. -pub unsafe fn kernel_map_binary_and_enable_mmu() -> Result<(), &'static str> { - let phys_base_addr = arch_mmu::kernel_translation_tables().write(|tables| { - tables.init(); - tables.phys_base_address() - }); +/// - See [`bsp::memory::mmu::kernel_map_binary()`]. +pub unsafe fn kernel_map_binary() -> Result, &'static str> { + let phys_kernel_tables_base_addr = + bsp::memory::mmu::kernel_translation_tables().write(|tables| { + tables.init(); + tables.phys_base_address() + }); bsp::memory::mmu::kernel_map_binary()?; - arch_mmu::mmu().enable(phys_base_addr) + + Ok(phys_kernel_tables_base_addr) +} + +/// Enable the MMU and data + instruction caching. +/// +/// # Safety +/// +/// - Crucial function during kernel init. Changes the the complete memory view of the processor. +pub unsafe fn enable_mmu_and_caching( + phys_tables_base_addr: Address, +) -> Result<(), MMUEnableError> { + arch_mmu::mmu().enable_mmu_and_caching(phys_tables_base_addr) } /// Human-readable print of all recorded kernel mappings. diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs index cd46403fd..791ec4910 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -111,12 +111,12 @@ impl MappingRecord { const KIB_RSHIFT: u32 = 10; // log2(1024). const MIB_RSHIFT: u32 = 20; // log2(1024 * 1024). - info!(" -----------------------------------------------------------------------------------------------------------------"); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); info!( - " {:^24} {:^24} {:^7} {:^9} {:^35}", + " {:^44} {:^30} {:^7} {:^9} {:^35}", "Virtual", "Physical", "Size", "Attr", "Entity" ); - info!(" -----------------------------------------------------------------------------------------------------------------"); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); for i in self .inner @@ -124,10 +124,10 @@ impl MappingRecord { .filter(|x| x.is_some()) .map(|x| x.unwrap()) { - let virt_start = i.virt_start_addr.into_usize(); + let virt_start = i.virt_start_addr; let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; - let phys_start = i.phys_pages.start_addr().into_usize(); - let phys_end_inclusive = i.phys_pages.end_addr_inclusive().into_usize(); + let phys_start = i.phys_pages.start_addr(); + let phys_end_inclusive = i.phys_pages.end_addr_inclusive(); let size = i.phys_pages.size(); let (size, unit) = if (size >> MIB_RSHIFT) > 0 { @@ -155,7 +155,7 @@ impl MappingRecord { }; info!( - " {:#011X}..{:#011X} --> {:#011X}..{:#011X} | \ + " {}..{} --> {}..{} | \ {: >3} {} | {: <3} {} {: <2} | {}", virt_start, virt_end_inclusive, @@ -172,14 +172,14 @@ impl MappingRecord { for k in i.users[1..].iter() { if let Some(additional_user) = *k { info!( - " | {}", + " | {}", additional_user ); } } } - info!(" -----------------------------------------------------------------------------------------------------------------"); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); } } diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs index 7e16e6060..65af83cb1 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs @@ -16,7 +16,8 @@ use crate::memory::{ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_translation_table::KernelTranslationTable; +#[cfg(target_arch = "aarch64")] +pub use arch_translation_table::FixedSizeTranslationTable; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -34,7 +35,7 @@ pub mod interface { /// /// - Implementor must ensure that this function can run only once or is harmless if invoked /// multiple times. - unsafe fn init(&mut self); + fn init(&mut self); /// The translation table's base address to be used for programming the MMU. fn phys_base_address(&self) -> Address; @@ -80,17 +81,17 @@ pub mod interface { mod tests { use super::*; use crate::bsp; - use arch_translation_table::MinSizeKernelTranslationTable; + use arch_translation_table::MinSizeTranslationTable; use interface::TranslationTable; use test_macros::kernel_test; - /// Sanity checks for the kernel TranslationTable implementation. + /// Sanity checks for the TranslationTable implementation. #[kernel_test] fn translationtable_implementation_sanity() { - // Need to take care that `tables` fits into the stack. - let mut tables = MinSizeKernelTranslationTable::new(); + // This will occupy a lot of space on the stack. + let mut tables = MinSizeTranslationTable::new(); - unsafe { tables.init() }; + tables.init(); let x = tables.next_mmio_virt_page_slice(0); assert!(x.is_err()); diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs index 594315882..f83ebb890 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs @@ -8,7 +8,7 @@ use crate::{ bsp, common, memory::{Address, AddressType, Physical, Virtual}, }; -use core::{convert::From, marker::PhantomData, ops::RangeInclusive}; +use core::{convert::From, marker::PhantomData}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -121,6 +121,11 @@ impl PageSliceDescriptor { self.start + (self.size() - 1) } + /// Check if an address is contained within this descriptor. + pub fn contains(&self, addr: Address) -> bool { + (addr >= self.start_addr()) && (addr <= self.end_addr_inclusive()) + } + /// Return a non-mutable slice of Pages. /// /// # Safety @@ -129,14 +134,6 @@ impl PageSliceDescriptor { pub unsafe fn as_slice(&self) -> &[Page] { core::slice::from_raw_parts(self.first_page_ptr(), self.num_pages) } - - /// Return the inclusive address range of the slice. - pub fn into_usize_range_inclusive(self) -> RangeInclusive { - RangeInclusive::new( - self.start_addr().into_usize(), - self.end_addr_inclusive().into_usize(), - ) - } } impl From> for PageSliceDescriptor { diff --git a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs index fe9d454a8..94582732c 100644 --- a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs +++ b/15_virtual_mem_part2_mmio_remap/src/synchronization.rs @@ -139,3 +139,21 @@ impl interface::ReadWriteEx for InitStateLock { f(data) } } + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// InitStateLock must be transparent. + #[kernel_test] + fn init_state_lock_is_transparent() { + use core::mem::size_of; + + assert_eq!(size_of::>(), size_of::()); + } +} diff --git a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs index 45f12a1f7..940866a03 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs @@ -29,11 +29,19 @@ unsafe fn kernel_init() -> ! { println!("Testing synchronous exception handling by causing a page fault"); println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::kernel_map_binary_and_enable_mmu() { - println!("Enabling MMU failed: {}", string); + let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { + Err(string) => { + println!("Error mapping kernel binary: {}", string); + cpu::qemu_exit_failure() + } + Ok(addr) => addr, + }; + + if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { + println!("Enabling MMU failed: {}", e); cpu::qemu_exit_failure() } - // Printing will silently fail fail from here on, because the driver's MMIO is not remapped yet. + // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. // Bring up the drivers needed for printing first. for i in bsp::driver::driver_manager() From fa4a2de3317959124be724b229ad3e28ccdc9f87 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 15 Mar 2021 22:20:26 +0100 Subject: [PATCH 038/214] MMIODescriptors are always physical --- 15_virtual_mem_part2_mmio_remap/README.md | 138 ++++++++---------- .../src/bsp/device_driver/arm/gicv2.rs | 25 ++-- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 17 +-- .../bcm/bcm2xxx_interrupt_controller.rs | 8 +- .../peripheral_ic.rs | 15 +- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 13 +- .../src/memory/mmu.rs | 8 +- .../src/memory/mmu/mapping_record.rs | 4 +- .../src/memory/mmu/types.rs | 16 +- 9 files changed, 111 insertions(+), 133 deletions(-) diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index ec117146f..9a6c6df64 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -941,25 +941,22 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs -@@ -79,7 +79,11 @@ +@@ -79,7 +79,8 @@ mod gicc; mod gicd; -use crate::{bsp, cpu, driver, exception, synchronization, synchronization::InitStateLock}; -+use crate::{ -+ bsp, cpu, driver, exception, memory, memory::Physical, synchronization, -+ synchronization::InitStateLock, -+}; ++use crate::{bsp, cpu, driver, exception, memory, synchronization, synchronization::InitStateLock}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- // Private Definitions -@@ -96,12 +100,18 @@ +@@ -96,12 +97,18 @@ /// Representation of the GIC. pub struct GICv2 { -+ gicd_phys_mmio_descriptor: memory::mmu::MMIODescriptor, -+ gicc_phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ gicd_mmio_descriptor: memory::mmu::MMIODescriptor, ++ gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + /// The Distributor. gicd: gicd::GICD, @@ -973,7 +970,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, } -@@ -118,11 +128,17 @@ +@@ -118,11 +125,17 @@ /// /// # Safety /// @@ -981,21 +978,21 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs - pub const unsafe fn new(gicd_mmio_start_addr: usize, gicc_mmio_start_addr: usize) -> Self { + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( -+ gicd_phys_mmio_descriptor: memory::mmu::MMIODescriptor, -+ gicc_phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ gicd_mmio_descriptor: memory::mmu::MMIODescriptor, ++ gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { Self { - gicd: gicd::GICD::new(gicd_mmio_start_addr), - gicc: gicc::GICC::new(gicc_mmio_start_addr), -+ gicd_phys_mmio_descriptor, -+ gicc_phys_mmio_descriptor, -+ gicd: gicd::GICD::new(gicd_phys_mmio_descriptor.start_addr().into_usize()), -+ gicc: gicc::GICC::new(gicc_phys_mmio_descriptor.start_addr().into_usize()), ++ gicd_mmio_descriptor, ++ gicc_mmio_descriptor, ++ gicd: gicd::GICD::new(gicd_mmio_descriptor.start_addr().into_usize()), ++ gicc: gicc::GICC::new(gicc_mmio_descriptor.start_addr().into_usize()), + is_mmio_remapped: AtomicBool::new(false), handler_table: InitStateLock::new([None; Self::NUM_IRQS]), } } -@@ -139,6 +155,22 @@ +@@ -139,6 +152,22 @@ } unsafe fn init(&self) -> Result<(), &'static str> { @@ -1004,11 +1001,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs + let mut virt_addr; + + // GICD -+ virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_phys_mmio_descriptor)?; ++ virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_mmio_descriptor)?; + self.gicd.set_mmio(virt_addr.into_usize()); + + // GICC -+ virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_phys_mmio_descriptor)?; ++ virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_mmio_descriptor)?; + self.gicc.set_mmio(virt_addr.into_usize()); + + // Conclude remapping. @@ -1027,9 +1024,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, -- synchronization::IRQSafeNullLock, -+ bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::Physical, -+ synchronization, synchronization::IRQSafeNullLock, ++ bsp::device_driver::common::MMIODerefWrapper, driver, memory, synchronization, + synchronization::IRQSafeNullLock, }; +use core::sync::atomic::{AtomicUsize, Ordering}; use register::{mmio::*, register_bitfields, register_structs}; @@ -1039,7 +1035,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Representation of the GPIO HW. pub struct GPIO { -+ phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, inner: IRQSafeNullLock, } @@ -1064,32 +1060,29 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Disable pull-up/down on pins 14 and 15. #[cfg(feature = "bsp_rpi3")] fn disable_pud_14_15_bcm2837(&mut self) { -@@ -190,10 +206,14 @@ +@@ -190,10 +206,12 @@ /// /// # Safety /// - /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: usize) -> Self { + /// - The user must ensure to provide correct MMIO descriptors. -+ pub const unsafe fn new(phys_mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { ++ pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { Self { - inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), -+ phys_mmio_descriptor, ++ mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), -+ inner: IRQSafeNullLock::new(GPIOInner::new( -+ phys_mmio_descriptor.start_addr().into_usize(), -+ )), ++ inner: IRQSafeNullLock::new(GPIOInner::new(mmio_descriptor.start_addr().into_usize())), } } -@@ -212,4 +232,27 @@ +@@ -212,4 +230,26 @@ fn compatible(&self) -> &'static str { "BCM GPIO" } + + unsafe fn init(&self) -> Result<(), &'static str> { -+ let virt_addr = -+ memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)?; ++ let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; @@ -1114,7 +1107,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs -@@ -2,12 +2,14 @@ +@@ -2,12 +2,12 @@ // // Copyright (c) 2020-2021 Andre Richter @@ -1125,17 +1118,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ use crate::{ bsp::device_driver::common::MMIODerefWrapper, - exception, synchronization, -+ driver, exception, memory, -+ memory::Physical, -+ synchronization, ++ driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; use register::{mmio::*, register_structs}; -@@ -51,11 +53,13 @@ +@@ -51,11 +51,13 @@ /// Representation of the peripheral interrupt controller. pub struct PeripheralIC { -+ phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ mmio_descriptor: memory::mmu::MMIODescriptor, + /// Access to write registers is guarded with a lock. wo_registers: IRQSafeNullLock, @@ -1146,20 +1137,20 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, -@@ -70,21 +74,26 @@ +@@ -70,21 +72,26 @@ /// /// # Safety /// - /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: usize) -> Self { + /// - The user must ensure to provide correct MMIO descriptors. -+ pub const unsafe fn new(phys_mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { -+ let addr = phys_mmio_descriptor.start_addr().into_usize(); ++ pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { ++ let addr = mmio_descriptor.start_addr().into_usize(); + Self { - wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), - ro_registers: ReadOnlyRegisters::new(mmio_start_addr), -+ phys_mmio_descriptor, ++ mmio_descriptor, + wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(addr)), + ro_registers: InitStateLock::new(ReadOnlyRegisters::new(addr)), handler_table: InitStateLock::new([None; InterruptController::NUM_PERIPHERAL_IRQS]), @@ -1180,7 +1171,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } } -@@ -93,6 +102,25 @@ +@@ -93,6 +100,24 @@ //------------------------------------------------------------------------------ use synchronization::interface::{Mutex, ReadWriteEx}; @@ -1191,8 +1182,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = -+ memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)? -+ .into_usize(); ++ memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?.into_usize(); + + self.wo_registers + .lock(|regs| *regs = WriteOnlyRegisters::new(virt_addr)); @@ -1215,7 +1205,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ mod peripheral_ic; -use crate::{driver, exception}; -+use crate::{driver, exception, memory, memory::Physical}; ++use crate::{driver, exception, memory}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -1227,12 +1217,12 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ - pub const unsafe fn new(_local_mmio_start_addr: usize, periph_mmio_start_addr: usize) -> Self { + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( -+ _phys_local_mmio_descriptor: memory::mmu::MMIODescriptor, -+ phys_periph_mmio_descriptor: memory::mmu::MMIODescriptor, ++ _local_mmio_descriptor: memory::mmu::MMIODescriptor, ++ periph_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { Self { - periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), -+ periph: peripheral_ic::PeripheralIC::new(phys_periph_mmio_descriptor), ++ periph: peripheral_ic::PeripheralIC::new(periph_mmio_descriptor), } } } @@ -1256,15 +1246,14 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, -- synchronization, synchronization::IRQSafeNullLock, + bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, -+ memory::Physical, synchronization, synchronization::IRQSafeNullLock, -+}; + synchronization, synchronization::IRQSafeNullLock, + }; +-use core::fmt; +use core::{ + fmt, + sync::atomic::{AtomicUsize, Ordering}, - }; --use core::fmt; ++}; use register::{mmio::*, register_bitfields, register_structs}; //-------------------------------------------------------------------------------------------------- @@ -1272,7 +1261,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Representation of the UART. pub struct PL011Uart { -+ phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, inner: IRQSafeNullLock, irq_number: bsp::device_driver::IRQNumber, @@ -1312,26 +1301,25 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + /// - The user must ensure to provide correct IRQ numbers. pub const unsafe fn new( - mmio_start_addr: usize, -+ phys_mmio_descriptor: memory::mmu::MMIODescriptor, ++ mmio_descriptor: memory::mmu::MMIODescriptor, irq_number: bsp::device_driver::IRQNumber, ) -> Self { Self { - inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), -+ phys_mmio_descriptor, ++ mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), + inner: IRQSafeNullLock::new(PL011UartInner::new( -+ phys_mmio_descriptor.start_addr().into_usize(), ++ mmio_descriptor.start_addr().into_usize(), + )), irq_number, } } -@@ -413,7 +433,14 @@ +@@ -413,7 +433,13 @@ } unsafe fn init(&self) -> Result<(), &'static str> { - self.inner.lock(|inner| inner.init()); -+ let virt_addr = -+ memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)?; ++ let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; @@ -1341,7 +1329,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ Ok(()) } -@@ -432,6 +459,16 @@ +@@ -432,6 +458,16 @@ Ok(()) } @@ -2319,10 +2307,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 +} + +pub fn kernel_find_and_insert_mmio_duplicate( -+ phys_mmio_descriptor: &MMIODescriptor, ++ mmio_descriptor: &MMIODescriptor, + new_user: &'static str, +) -> Option> { -+ let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); ++ let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + + KERNEL_MAPPING_RECORD.write(|mr| { + let dup = mr.find_duplicate(&phys_pages)?; @@ -2512,8 +2500,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + +/// An MMIO descriptor for use in device drivers. +#[derive(Copy, Clone)] -+pub struct MMIODescriptor { -+ start_addr: Address, ++pub struct MMIODescriptor { ++ start_addr: Address, + size: usize, +} + @@ -2602,8 +2590,8 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + } +} + -+impl From> for PageSliceDescriptor { -+ fn from(desc: MMIODescriptor) -> Self { ++impl From for PageSliceDescriptor { ++ fn from(desc: MMIODescriptor) -> Self { + let start_page_addr = desc + .start_addr + .align_down(bsp::memory::mmu::KernelGranule::SIZE); @@ -2623,21 +2611,21 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual +// MMIODescriptor +//------------------------------------------------------------------------------ + -+impl MMIODescriptor { ++impl MMIODescriptor { + /// Create an instance. -+ pub const fn new(start_addr: Address, size: usize) -> Self { ++ pub const fn new(start_addr: Address, size: usize) -> Self { + assert!(size > 0); + + Self { start_addr, size } + } + + /// Return the start address. -+ pub const fn start_addr(&self) -> Address { ++ pub const fn start_addr(&self) -> Address { + self.start_addr + } + + /// Return the inclusive end address. -+ pub fn end_addr_inclusive(&self) -> Address { ++ pub fn end_addr_inclusive(&self) -> Address { + self.start_addr + (self.size - 1) + } + @@ -2970,15 +2958,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p +/// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. +pub unsafe fn kernel_map_mmio( + name: &'static str, -+ phys_mmio_descriptor: &MMIODescriptor, ++ mmio_descriptor: &MMIODescriptor, +) -> Result, &'static str> { -+ let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); ++ let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let offset_into_start_page = -+ phys_mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; ++ mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + + // Check if an identical page slice has been mapped for another driver. If so, reuse it. + let virt_addr = if let Some(addr) = -+ mapping_record::kernel_find_and_insert_mmio_duplicate(phys_mmio_descriptor, name) ++ mapping_record::kernel_find_and_insert_mmio_duplicate(mmio_descriptor, name) + { + addr + // Otherwise, allocate a new virtual page slice and map it. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs index 4c0e11249..c317abaea 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs @@ -79,10 +79,7 @@ mod gicc; mod gicd; -use crate::{ - bsp, cpu, driver, exception, memory, memory::Physical, synchronization, - synchronization::InitStateLock, -}; +use crate::{bsp, cpu, driver, exception, memory, synchronization, synchronization::InitStateLock}; use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- @@ -100,8 +97,8 @@ pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER /// Representation of the GIC. pub struct GICv2 { - gicd_phys_mmio_descriptor: memory::mmu::MMIODescriptor, - gicc_phys_mmio_descriptor: memory::mmu::MMIODescriptor, + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, /// The Distributor. gicd: gicd::GICD, @@ -130,14 +127,14 @@ impl GICv2 { /// /// - The user must ensure to provide correct MMIO descriptors. pub const unsafe fn new( - gicd_phys_mmio_descriptor: memory::mmu::MMIODescriptor, - gicc_phys_mmio_descriptor: memory::mmu::MMIODescriptor, + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, ) -> Self { Self { - gicd_phys_mmio_descriptor, - gicc_phys_mmio_descriptor, - gicd: gicd::GICD::new(gicd_phys_mmio_descriptor.start_addr().into_usize()), - gicc: gicc::GICC::new(gicc_phys_mmio_descriptor.start_addr().into_usize()), + gicd_mmio_descriptor, + gicc_mmio_descriptor, + gicd: gicd::GICD::new(gicd_mmio_descriptor.start_addr().into_usize()), + gicc: gicc::GICC::new(gicc_mmio_descriptor.start_addr().into_usize()), is_mmio_remapped: AtomicBool::new(false), handler_table: InitStateLock::new([None; Self::NUM_IRQS]), } @@ -160,11 +157,11 @@ impl driver::interface::DeviceDriver for GICv2 { let mut virt_addr; // GICD - virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_phys_mmio_descriptor)?; + virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_mmio_descriptor)?; self.gicd.set_mmio(virt_addr.into_usize()); // GICC - virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_phys_mmio_descriptor)?; + virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_mmio_descriptor)?; self.gicc.set_mmio(virt_addr.into_usize()); // Conclude remapping. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 45dbe75c6..e444812ba 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -5,8 +5,8 @@ //! GPIO Driver. use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, memory, memory::Physical, - synchronization, synchronization::IRQSafeNullLock, + bsp::device_driver::common::MMIODerefWrapper, driver, memory, synchronization, + synchronization::IRQSafeNullLock, }; use core::sync::atomic::{AtomicUsize, Ordering}; use register::{mmio::*, register_bitfields, register_structs}; @@ -118,7 +118,7 @@ pub use GPIOInner as PanicGPIO; /// Representation of the GPIO HW. pub struct GPIO { - phys_mmio_descriptor: memory::mmu::MMIODescriptor, + mmio_descriptor: memory::mmu::MMIODescriptor, virt_mmio_start_addr: AtomicUsize, inner: IRQSafeNullLock, } @@ -207,13 +207,11 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide correct MMIO descriptors. - pub const unsafe fn new(phys_mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { Self { - phys_mmio_descriptor, + mmio_descriptor, virt_mmio_start_addr: AtomicUsize::new(0), - inner: IRQSafeNullLock::new(GPIOInner::new( - phys_mmio_descriptor.start_addr().into_usize(), - )), + inner: IRQSafeNullLock::new(GPIOInner::new(mmio_descriptor.start_addr().into_usize())), } } @@ -234,8 +232,7 @@ impl driver::interface::DeviceDriver for GPIO { } unsafe fn init(&self) -> Result<(), &'static str> { - let virt_addr = - memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)?; + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; self.inner .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index 69c001adf..0bd84e3cf 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -6,7 +6,7 @@ mod peripheral_ic; -use crate::{driver, exception, memory, memory::Physical}; +use crate::{driver, exception, memory}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -80,11 +80,11 @@ impl InterruptController { /// /// - The user must ensure to provide correct MMIO descriptors. pub const unsafe fn new( - _phys_local_mmio_descriptor: memory::mmu::MMIODescriptor, - phys_periph_mmio_descriptor: memory::mmu::MMIODescriptor, + _local_mmio_descriptor: memory::mmu::MMIODescriptor, + periph_mmio_descriptor: memory::mmu::MMIODescriptor, ) -> Self { Self { - periph: peripheral_ic::PeripheralIC::new(phys_periph_mmio_descriptor), + periph: peripheral_ic::PeripheralIC::new(periph_mmio_descriptor), } } } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 654ac3c70..abc3af711 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -7,9 +7,7 @@ use super::{InterruptController, PendingIRQs, PeripheralIRQ}; use crate::{ bsp::device_driver::common::MMIODerefWrapper, - driver, exception, memory, - memory::Physical, - synchronization, + driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; use register::{mmio::*, register_structs}; @@ -53,7 +51,7 @@ type HandlerTable = /// Representation of the peripheral interrupt controller. pub struct PeripheralIC { - phys_mmio_descriptor: memory::mmu::MMIODescriptor, + mmio_descriptor: memory::mmu::MMIODescriptor, /// Access to write registers is guarded with a lock. wo_registers: IRQSafeNullLock, @@ -75,11 +73,11 @@ impl PeripheralIC { /// # Safety /// /// - The user must ensure to provide correct MMIO descriptors. - pub const unsafe fn new(phys_mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { - let addr = phys_mmio_descriptor.start_addr().into_usize(); + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + let addr = mmio_descriptor.start_addr().into_usize(); Self { - phys_mmio_descriptor, + mmio_descriptor, wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(addr)), ro_registers: InitStateLock::new(ReadOnlyRegisters::new(addr)), handler_table: InitStateLock::new([None; InterruptController::NUM_PERIPHERAL_IRQS]), @@ -109,8 +107,7 @@ impl driver::interface::DeviceDriver for PeripheralIC { unsafe fn init(&self) -> Result<(), &'static str> { let virt_addr = - memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)? - .into_usize(); + memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?.into_usize(); self.wo_registers .lock(|regs| *regs = WriteOnlyRegisters::new(virt_addr)); diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 4b1e9c937..3adb9137d 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -11,7 +11,7 @@ use crate::{ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, - memory::Physical, synchronization, synchronization::IRQSafeNullLock, + synchronization, synchronization::IRQSafeNullLock, }; use core::{ fmt, @@ -235,7 +235,7 @@ pub use PL011UartInner as PanicUart; /// Representation of the UART. pub struct PL011Uart { - phys_mmio_descriptor: memory::mmu::MMIODescriptor, + mmio_descriptor: memory::mmu::MMIODescriptor, virt_mmio_start_addr: AtomicUsize, inner: IRQSafeNullLock, irq_number: bsp::device_driver::IRQNumber, @@ -408,14 +408,14 @@ impl PL011Uart { /// - The user must ensure to provide correct MMIO descriptors. /// - The user must ensure to provide correct IRQ numbers. pub const unsafe fn new( - phys_mmio_descriptor: memory::mmu::MMIODescriptor, + mmio_descriptor: memory::mmu::MMIODescriptor, irq_number: bsp::device_driver::IRQNumber, ) -> Self { Self { - phys_mmio_descriptor, + mmio_descriptor, virt_mmio_start_addr: AtomicUsize::new(0), inner: IRQSafeNullLock::new(PL011UartInner::new( - phys_mmio_descriptor.start_addr().into_usize(), + mmio_descriptor.start_addr().into_usize(), )), irq_number, } @@ -433,8 +433,7 @@ impl driver::interface::DeviceDriver for PL011Uart { } unsafe fn init(&self) -> Result<(), &'static str> { - let virt_addr = - memory::mmu::kernel_map_mmio(self.compatible(), &self.phys_mmio_descriptor)?; + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; self.inner .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index 7a0115af0..e942f7f3e 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -180,15 +180,15 @@ pub unsafe fn kernel_map_pages_at( /// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. pub unsafe fn kernel_map_mmio( name: &'static str, - phys_mmio_descriptor: &MMIODescriptor, + mmio_descriptor: &MMIODescriptor, ) -> Result, &'static str> { - let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); let offset_into_start_page = - phys_mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; // Check if an identical page slice has been mapped for another driver. If so, reuse it. let virt_addr = if let Some(addr) = - mapping_record::kernel_find_and_insert_mmio_duplicate(phys_mmio_descriptor, name) + mapping_record::kernel_find_and_insert_mmio_duplicate(mmio_descriptor, name) { addr // Otherwise, allocate a new virtual page slice and map it. diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs index 791ec4910..eab62fb39 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -199,10 +199,10 @@ pub fn kernel_add( } pub fn kernel_find_and_insert_mmio_duplicate( - phys_mmio_descriptor: &MMIODescriptor, + mmio_descriptor: &MMIODescriptor, new_user: &'static str, ) -> Option> { - let phys_pages: PageSliceDescriptor = phys_mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); KERNEL_MAPPING_RECORD.write(|mr| { let dup = mr.find_duplicate(&phys_pages)?; diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs index f83ebb890..2e52e8f3d 100644 --- a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +++ b/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs @@ -55,8 +55,8 @@ pub struct AttributeFields { /// An MMIO descriptor for use in device drivers. #[derive(Copy, Clone)] -pub struct MMIODescriptor { - start_addr: Address, +pub struct MMIODescriptor { + start_addr: Address, size: usize, } @@ -145,8 +145,8 @@ impl From> for PageSliceDescriptor { } } -impl From> for PageSliceDescriptor { - fn from(desc: MMIODescriptor) -> Self { +impl From for PageSliceDescriptor { + fn from(desc: MMIODescriptor) -> Self { let start_page_addr = desc .start_addr .align_down(bsp::memory::mmu::KernelGranule::SIZE); @@ -166,21 +166,21 @@ impl From> for PageSliceDescriptor MMIODescriptor { +impl MMIODescriptor { /// Create an instance. - pub const fn new(start_addr: Address, size: usize) -> Self { + pub const fn new(start_addr: Address, size: usize) -> Self { assert!(size > 0); Self { start_addr, size } } /// Return the start address. - pub const fn start_addr(&self) -> Address { + pub const fn start_addr(&self) -> Address { self.start_addr } /// Return the inclusive end address. - pub fn end_addr_inclusive(&self) -> Address { + pub fn end_addr_inclusive(&self) -> Address { self.start_addr + (self.size - 1) } From 223989adb9cce379d650b9cd67581c0a26df2f6f Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 16 Mar 2021 22:36:06 +0100 Subject: [PATCH 039/214] Overhaul linker script --- 01_wait_forever/Makefile | 1 - 01_wait_forever/src/bsp/raspberrypi/link.ld | 23 +- 02_runtime_init/Makefile | 1 + 02_runtime_init/README.md | 62 ++- 02_runtime_init/src/bsp/raspberrypi/link.ld | 45 +- 03_hacky_hello_world/Makefile | 1 + .../src/bsp/raspberrypi/link.ld | 45 +- 04_zero_overhead_abstraction/Makefile | 1 + 04_zero_overhead_abstraction/README.md | 37 +- .../src/bsp/raspberrypi/link.ld | 46 ++- .../src/bsp/raspberrypi/memory.rs | 18 +- 05_safe_globals/Makefile | 1 + 05_safe_globals/src/bsp/raspberrypi/link.ld | 46 ++- 05_safe_globals/src/bsp/raspberrypi/memory.rs | 18 +- 06_drivers_gpio_uart/Makefile | 1 + 06_drivers_gpio_uart/README.md | 21 +- .../src/bsp/raspberrypi/link.ld | 46 ++- .../src/bsp/raspberrypi/memory.rs | 21 +- 07_uart_chainloader/README.md | 119 +++--- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6808 -> 6824 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6656 -> 6672 bytes .../src/bsp/raspberrypi/link.ld | 49 +-- .../src/bsp/raspberrypi/memory.rs | 7 +- 07_uart_chainloader/src/relocate.rs | 16 +- 08_timestamps/Makefile | 1 + 08_timestamps/README.md | 117 +++--- 08_timestamps/src/bsp/raspberrypi/link.ld | 46 ++- 08_timestamps/src/bsp/raspberrypi/memory.rs | 21 +- 09_hw_debug_JTAG/Makefile | 1 + 09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld | 46 ++- .../src/bsp/raspberrypi/memory.rs | 21 +- 10_privilege_level/Makefile | 1 + 10_privilege_level/README.md | 2 +- .../src/bsp/raspberrypi/link.ld | 46 ++- .../src/bsp/raspberrypi/memory.rs | 21 +- .../Makefile | 1 + .../README.md | 94 ++--- .../src/bsp/raspberrypi/link.ld | 52 ++- .../src/bsp/raspberrypi/memory.rs | 23 +- .../src/bsp/raspberrypi/memory/mmu.rs | 6 +- 12_exceptions_part1_groundwork/Makefile | 1 + 12_exceptions_part1_groundwork/README.md | 52 +-- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 24 +- .../src/bsp/raspberrypi/link.ld | 55 +-- .../src/bsp/raspberrypi/memory.rs | 23 +- .../src/bsp/raspberrypi/memory/mmu.rs | 6 +- 13_integrated_testing/Makefile | 1 + .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 24 +- .../src/bsp/raspberrypi/link.ld | 55 +-- .../src/bsp/raspberrypi/memory.rs | 23 +- .../src/bsp/raspberrypi/memory/mmu.rs | 6 +- 14_exceptions_part2_peripheral_IRQs/Makefile | 1 + 14_exceptions_part2_peripheral_IRQs/README.md | 4 +- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 24 +- .../src/bsp/raspberrypi/link.ld | 55 +-- .../src/bsp/raspberrypi/memory.rs | 23 +- .../src/bsp/raspberrypi/memory/mmu.rs | 6 +- 15_virtual_mem_part2_mmio_remap/Makefile | 1 + 15_virtual_mem_part2_mmio_remap/README.md | 384 +++++++++++------- .../src/_arch/aarch64/exception.S | 2 +- .../src/_arch/aarch64/exception.rs | 42 +- .../src/bsp/raspberrypi/link.ld | 74 ++-- .../src/bsp/raspberrypi/memory.rs | 94 +++-- .../src/bsp/raspberrypi/memory/mmu.rs | 89 ++-- X1_JTAG_boot/Makefile | 1 + X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8144 -> 8152 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6832 -> 6840 bytes X1_JTAG_boot/src/bsp/raspberrypi/link.ld | 46 ++- X1_JTAG_boot/src/bsp/raspberrypi/memory.rs | 21 +- 72 files changed, 1339 insertions(+), 928 deletions(-) diff --git a/01_wait_forever/Makefile b/01_wait_forever/Makefile index eb2cc7323..2e44e7ac5 100644 --- a/01_wait_forever/Makefile +++ b/01_wait_forever/Makefile @@ -102,7 +102,6 @@ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ - --section .rodata \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/01_wait_forever/src/bsp/raspberrypi/link.ld b/01_wait_forever/src/bsp/raspberrypi/link.ld index 81f01c11d..7d649f884 100644 --- a/01_wait_forever/src/bsp/raspberrypi/link.ld +++ b/01_wait_forever/src/bsp/raspberrypi/link.ld @@ -3,15 +3,26 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + ***********************************************************************************************/ .text : { - *(.text._start) *(.text*) - } - - /DISCARD/ : { *(.comment*) } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx } diff --git a/02_runtime_init/Makefile b/02_runtime_init/Makefile index eb2cc7323..a20b283b4 100644 --- a/02_runtime_init/Makefile +++ b/02_runtime_init/Makefile @@ -103,6 +103,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 9241a1950..5f5245d57 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -9,8 +9,7 @@ ## Notable additions - More sections in linker script: - - `.rodata`, `.data` - - `.bss` + - `.rodata`, `.got`, `.data`, `.bss` - `_start()`: - Halt core if core != `core0`. - `core0` jumps to the `runtime_init()` Rust function. @@ -37,6 +36,19 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml default = [] bsp_rpi3 = [] +diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile +--- 01_wait_forever/Makefile ++++ 02_runtime_init/Makefile +@@ -102,6 +102,8 @@ + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ ++ --section .rodata \ ++ --section .got \ + $(KERNEL_ELF) | rustfilt + + nm: $(KERNEL_ELF) + diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.S 02_runtime_init/src/_arch/aarch64/cpu/boot.S --- 01_wait_forever/src/_arch/aarch64/cpu/boot.S +++ 02_runtime_init/src/_arch/aarch64/cpu/boot.S @@ -97,33 +109,45 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aar diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld -@@ -13,5 +13,27 @@ - *(.text._start) *(.text*) - } +@@ -11,6 +11,7 @@ + PHDRS + { + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ ++ segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ + } -+ .rodata : -+ { -+ *(.rodata*) -+ } + SECTIONS +@@ -18,11 +19,30 @@ + . = __rpi_load_addr; + + /*********************************************************************************************** +- * Code ++ * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + .text : + { + KEEP(*(.text._start)) + *(.text*) + } :segment_rx + -+ .data : -+ { -+ *(.data*) -+ } ++ .rodata : ALIGN(8) { *(.rodata*) } :segment_rx ++ .got : ALIGN(8) { *(.got) } :segment_rx ++ ++ /*********************************************************************************************** ++ * Data + BSS ++ ***********************************************************************************************/ ++ .data : { *(.data*) } :segment_rw + + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -+ .bss ALIGN(8): ++ .bss : ALIGN(8) + { + __bss_start = .; + *(.bss*); + . = ALIGN(8); + -+ /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ -+ . += 8; ++ . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ + __bss_end_inclusive = . - 8; -+ } -+ - /DISCARD/ : { *(.comment*) } ++ } :NONE } diff -uNr 01_wait_forever/src/bsp/raspberrypi/memory.rs 02_runtime_init/src/bsp/raspberrypi/memory.rs diff --git a/02_runtime_init/src/bsp/raspberrypi/link.ld b/02_runtime_init/src/bsp/raspberrypi/link.ld index 573abc5f0..acb7fe28b 100644 --- a/02_runtime_init/src/bsp/raspberrypi/link.ld +++ b/02_runtime_init/src/bsp/raspberrypi/link.ld @@ -3,37 +3,46 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/03_hacky_hello_world/Makefile b/03_hacky_hello_world/Makefile index 75158b5b0..2ed82a5f1 100644 --- a/03_hacky_hello_world/Makefile +++ b/03_hacky_hello_world/Makefile @@ -103,6 +103,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld index 573abc5f0..acb7fe28b 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld +++ b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld @@ -3,37 +3,46 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/04_zero_overhead_abstraction/Makefile b/04_zero_overhead_abstraction/Makefile index 75158b5b0..2ed82a5f1 100644 --- a/04_zero_overhead_abstraction/Makefile +++ b/04_zero_overhead_abstraction/Makefile @@ -103,6 +103,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index aeb3bfb85..358d01e71 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -167,20 +167,43 @@ diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs 04_zero_overhead_abstr +/// Used by `arch` code to find the early boot core. +pub const BOOT_CORE_ID: usize = 0; +diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/link.ld 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld +--- 03_hacky_hello_world/src/bsp/raspberrypi/link.ld ++++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld +@@ -21,6 +21,7 @@ + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ ++ __rx_start = .; + .text : + { + KEEP(*(.text._start)) + diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs 04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs --- 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs +++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs -@@ -17,9 +17,25 @@ +@@ -12,14 +12,36 @@ + + // Symbols from the linker script. + extern "Rust" { ++ static __rx_start: UnsafeCell<()>; ++ + static __bss_start: UnsafeCell; + static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- -+// Public Definitions ++// Private Code +//-------------------------------------------------------------------------------------------------- + -+/// The board's memory map. -+#[rustfmt::skip] -+pub(super) mod map { -+ pub const BOOT_CORE_STACK_END: usize = 0x8_0000; ++/// Start address of the Read+Execute (RX) range. ++/// ++/// # Safety ++/// ++/// - Value is provided by the linker script and must be trusted as-is. ++#[inline(always)] ++fn rx_start() -> usize { ++ unsafe { __rx_start.get() as usize } +} + +//-------------------------------------------------------------------------------------------------- @@ -190,7 +213,7 @@ diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs 04_zero_overhead_ab +/// Exclusive end address of the boot core's stack. +#[inline(always)] +pub fn boot_core_stack_end() -> usize { -+ map::BOOT_CORE_STACK_END ++ rx_start() +} + /// Return the inclusive range spanning the .bss section. diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld b/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs index 77fdafd7a..bffe8eab0 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs @@ -12,18 +12,24 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- -// Public Definitions +// Private Code //-------------------------------------------------------------------------------------------------- -/// The board's memory map. -#[rustfmt::skip] -pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -33,7 +39,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/05_safe_globals/Makefile b/05_safe_globals/Makefile index 75158b5b0..2ed82a5f1 100644 --- a/05_safe_globals/Makefile +++ b/05_safe_globals/Makefile @@ -103,6 +103,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/05_safe_globals/src/bsp/raspberrypi/link.ld b/05_safe_globals/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/05_safe_globals/src/bsp/raspberrypi/link.ld +++ b/05_safe_globals/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/05_safe_globals/src/bsp/raspberrypi/memory.rs b/05_safe_globals/src/bsp/raspberrypi/memory.rs index 77fdafd7a..bffe8eab0 100644 --- a/05_safe_globals/src/bsp/raspberrypi/memory.rs +++ b/05_safe_globals/src/bsp/raspberrypi/memory.rs @@ -12,18 +12,24 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- -// Public Definitions +// Private Code //-------------------------------------------------------------------------------------------------- -/// The board's memory map. -#[rustfmt::skip] -pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -33,7 +39,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/06_drivers_gpio_uart/Makefile b/06_drivers_gpio_uart/Makefile index 07eb97b2e..1dd6cc2b9 100644 --- a/06_drivers_gpio_uart/Makefile +++ b/06_drivers_gpio_uart/Makefile @@ -122,6 +122,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 8b8e076d7..7dea308f3 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -1110,10 +1110,16 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/driver.rs 06_drivers_gpio_uart/src diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs --- 05_safe_globals/src/bsp/raspberrypi/memory.rs +++ 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs -@@ -24,6 +24,29 @@ - #[rustfmt::skip] - pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; +@@ -19,6 +19,38 @@ + } + + //-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++/// The board's physical memory map. ++#[rustfmt::skip] ++pub(super) mod map { + + pub const GPIO_OFFSET: usize = 0x0020_0000; + pub const UART_OFFSET: usize = 0x0020_1000; @@ -1137,10 +1143,13 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src + pub const GPIO_START: usize = START + GPIO_OFFSET; + pub const PL011_UART_START: usize = START + UART_OFFSET; + } - } - ++} ++ ++//-------------------------------------------------------------------------------------------------- + // Private Code //-------------------------------------------------------------------------------------------------- + diff -uNr 05_safe_globals/src/bsp/raspberrypi.rs 06_drivers_gpio_uart/src/bsp/raspberrypi.rs --- 05_safe_globals/src/bsp/raspberrypi.rs +++ 06_drivers_gpio_uart/src/bsp/raspberrypi.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs index fb47b1418..56a3306e5 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -12,6 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -20,10 +22,9 @@ extern "Rust" { // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -49,6 +50,20 @@ pub(super) mod map { } } +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -56,7 +71,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index e67c78109..923615b56 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -165,14 +165,6 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -122,6 +129,7 @@ - @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ - --section .text \ - --section .rodata \ -+ --section .got \ - $(KERNEL_ELF) | rustfilt - - nm: $(KERNEL_ELF) diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs --- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs @@ -276,77 +268,82 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 07_uart_chainloader/src/bsp/raspberrypi/link.ld --- 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ 07_uart_chainloader/src/bsp/raspberrypi/link.ld -@@ -5,12 +5,15 @@ +@@ -16,12 +16,13 @@ SECTIONS { -- /* Set current address to the value from which the RPi starts execution */ -- . = 0x80000; +- . = __rpi_load_addr; + /* Set the link address to 32 MiB */ + . = 0x2000000; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ +- __rx_start = .; + __binary_start = .; .text : { -- *(.text._start) *(.text*) -+ *(.text._start) -+ KEEP(*(.text.runtime_init)) -+ *(.text*); - } - - .rodata : -@@ -35,5 +38,16 @@ + KEEP(*(.text._start)) +@@ -46,4 +47,10 @@ + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - -+ .got : -+ { -+ *(.got*) -+ } + } :NONE + + /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ + . = ALIGN(8); + __binary_end_inclusive = . - 8; + + __runtime_init_reloc = runtime_init; -+ - /DISCARD/ : { *(.comment*) } } diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader/src/bsp/raspberrypi/memory.rs --- 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ 07_uart_chainloader/src/bsp/raspberrypi/memory.rs -@@ -12,6 +12,9 @@ +@@ -12,10 +12,12 @@ // Symbols from the linker script. extern "Rust" { +- static __rx_start: UnsafeCell<()>; +- + static __binary_start: UnsafeCell; -+ static __binary_end_inclusive: UnsafeCell; -+ static __runtime_init_reloc: UnsafeCell; static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; ++ static __binary_end_inclusive: UnsafeCell; ++ ++ static __runtime_init_reloc: UnsafeCell; } -@@ -23,10 +26,12 @@ - /// The board's memory map. + + //-------------------------------------------------------------------------------------------------- +@@ -25,9 +27,12 @@ + /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { -- pub const BOOT_CORE_STACK_END: usize = 0x8_0000; + pub const BOOT_CORE_STACK_END: usize = 0x8_0000; ++ ++ pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; - pub const UART_OFFSET: usize = 0x0020_1000; -+ pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; -+ + pub const GPIO_OFFSET: usize = 0x0020_0000; + pub const UART_OFFSET: usize = 0x0020_1000; /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -59,13 +64,35 @@ - map::BOOT_CORE_STACK_END +@@ -51,36 +56,44 @@ } --/// Return the inclusive range spanning the .bss section. + //-------------------------------------------------------------------------------------------------- +-// Private Code ++// Public Code + //-------------------------------------------------------------------------------------------------- + +-/// Start address of the Read+Execute (RX) range. ++/// Exclusive end address of the boot core's stack. ++#[inline(always)] ++pub fn boot_core_stack_end() -> usize { ++ map::BOOT_CORE_STACK_END ++} ++ +/// The address on which the Raspberry firmware loads every binary by default. +#[inline(always)] +pub fn board_default_load_addr() -> *const u64 { @@ -354,21 +351,33 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader +} + +/// Return the inclusive range spanning the relocated kernel binary. -+/// -+/// # Safety -+/// + /// + /// # Safety + /// +-/// - Value is provided by the linker script and must be trusted as-is. +-#[inline(always)] +-fn rx_start() -> usize { +- unsafe { __rx_start.get() as usize } +/// - Values are provided by the linker script and must be trusted as-is. +/// - The linker-provided addresses must be u64 aligned. +pub fn relocated_binary_range_inclusive() -> RangeInclusive<*mut u64> { + unsafe { RangeInclusive::new(__binary_start.get(), __binary_end_inclusive.get()) } -+} -+ + } + +-//-------------------------------------------------------------------------------------------------- +-// Public Code +-//-------------------------------------------------------------------------------------------------- +- +-/// Exclusive end address of the boot core's stack. +/// The relocated address of function `runtime_init()`. -+#[inline(always)] + #[inline(always)] +-pub fn boot_core_stack_end() -> usize { +- rx_start() +pub fn relocated_runtime_init_addr() -> *const u64 { + unsafe { __runtime_init_reloc.get() as _ } -+} -+ + } + +-/// Return the inclusive range spanning the .bss section. +/// Return the inclusive range spanning the relocated .bss section. /// /// # Safety @@ -498,7 +507,7 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs diff -uNr 06_drivers_gpio_uart/src/relocate.rs 07_uart_chainloader/src/relocate.rs --- 06_drivers_gpio_uart/src/relocate.rs +++ 07_uart_chainloader/src/relocate.rs -@@ -0,0 +1,51 @@ +@@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -537,16 +546,14 @@ diff -uNr 06_drivers_gpio_uart/src/relocate.rs 07_uart_chainloader/src/relocate. + current_binary_start_addr = current_binary_start_addr.offset(1); + } + -+ // The following function calls form a hack to achieve an "absolute jump" to -+ // `runtime_init::runtime_init()` by forcing an indirection through the global offset table -+ // (GOT), so that execution continues from the relocated binary. -+ // -+ // Without this, the address of `runtime_init()` would be calculated as a relative offset from -+ // the current program counter, since we are compiling as `position independent code`. This -+ // would cause us to keep executing from the address to which the firmware loaded us, instead of -+ // the relocated position. ++ // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by ++ // forcing an indirection through the global offset table (GOT), so that execution continues ++ // from the relocated binary. + // -+ // There likely is a more elegant way to do this. ++ // Without the indirection through the assembly, the address of `runtime_init()` would be ++ // calculated as a relative offset from the current program counter, since we are compiling as ++ // `position independent code`. This would cause us to keep executing from the address to which ++ // the firmware loaded us, instead of the relocated position. + let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; + cpu::branch_to_raw_addr(relocated_runtime_init_addr) +} diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index a46c0d4149a22954048d876138fda7825a398700..79eef6b4ca54773cbe15dd88f8b48e0ea9a8541a 100755 GIT binary patch delta 1022 zcmZ8fL1@!p6#oCTS=+hUrdyh(*tKjQeDD1)S?*r#E{Pt5 zo_l+Ab1j_UuuWTV_KO|;hSH=>(CGw zJi2y>v`~kc{tOAxO!bmgyPZO1AALTJjAB}ABOe}yNWy?yfxK_MjlAejT4*EC#U0GR z&p<}(43M3th&S0yvu45AD55wliS*z!6d++q7~`k`jU&j( z2LOj8`K!l4wnT4qwRU`mZ0M~pD*vl4fFMbU8OdSkSGHq;VY&i4CSXX)&t3^TRRP9s zF|WN21*W$${UpQmAF|{BdWGqGUE>Tlu=);$*%Tbw0$=!AVm{)GvCQNW!!H@Om(w#Z zvMQDpwxlpKcT|Kfxtyn)EGw`XmV6U_hm8Z4bJ&C)Igf26yQgfk!wbyM$|}7+GTzVl dEm)Ut2%~JqC}S^x7D>@poGzJKDk)Dr*z delta 1028 zcmZ8fUuaTM96smj-Mni$hV6WMgNEh&Ogp%esLFtV4Pn%3x{ zjqvOY+FQ9^vZq3VeA@_u=%EI&*Xf>?MG*DkirKl|GZ;H?@A3P7-|zeVxxNgn1`2`( zK2N>rl^K{WpJPr!DB&5+Lnz~)nsKIPDTlQa5X$Ap>?p&^F67OWx7x!j^p}lYzo8(X zLB{|nbOP|Y9s_8$IEa=Yw(76Qxn8seaw$z$w2=mmt!9kb+z*4spGjQ_5H(oG%lb)g z-h}EJfU50nB}iCVvY{&R*>!76VH{BFuaM+!=?9 z*3JZeVhn~P7rK$_M;SJkS^!S8qk%}*Y(ABM^i}a()aFv$P`u$|qm>z<$M0i7h1<)e zbVGRG7#j%08{Sa75rMkz90Zrhx(W<=faF0Lk~y;OLAHOYTWdKXob3+6P5i3+8J{*I zk4W5zURj`b15fvS@*ZM63R`ocYCk~rcF<_fYfInyH6^7los1_0hfC?c>QcK;pFlhB z0lL1g)>C?cFqZ-VFAGWN#5KX}onlBbIUbeBe4IWm9EqGlVYYMZ)~kJ~y48t&JBw+z_EyycdB&#Jajpy3?F?p?6*Ft@5F57r-~AO%S(mb5Cglu{$w_3i9?E!&n= zs}Xu?|It+$^I-vN`Ff!f<03sN-o_<5BlhB4K4$B~xRk#k?G>a_kVFIdiRzsaZp}N_ zE@CwhBWn=K)_}}hCn6Vf?PTZ$t#uv433|a5!Z+x!tFz_3o0P*gy(}W)1ZGdF9Y~zB(NycJTHR z9gsu2G?f&;ATkX)*f2&iGTOcy`CbqwPvB{BJ2?oTE=zWI|~;6lh}VW{3Z)LPgNA zS<%B4V`S8Zwwu*B{&cabn*E8w9Ma+)I8ks)-IX2*rXxu-UfXWMA%Haiuhniz7lTBE zlcy5_VgrA3Jit^_QYN_udexBjkttWYD$P=&aMzYeUQj))%@QiuA?84jla8;bFfN{& zG(R?z1v8Ga(WS0R2eyB6U~~ilM#T+80KX@*k)%Lp({E$MX(q79H0Ex_3O(yiy6M|8^LS*R=jN+HkW+mf|jfSkoH2T@I z6_3+(YD@wGiqPGvH@ZUrQLxR(@`&R-od>o41KV0!w4=L|P<@$RSH1W?A@9bry!+m~{^hULJ(i6pzE}Vz!$1j*iuJq~Cb0(=uoE`7I31F)Q$W566!!(54Tlwu%}Q6%Sek6$M2+c}l8ciwfrV z7raTW2iab12_ABhL{ao4Sn=Yeh(Z!k6e*fwU1xXy3OcYm`+YMz-^~1fzAe|5_sckv znP@}_p|DKqMlA7T@(`AJP436RLc47@V!3ct>6MgnjJ06l^}H<##tVxUen-E$j?D+i zF8~;?bpt!YHnSV|c))c4BmAr@j>q^dS9h}PWtD_YtH{6_D6_9~F=0j94!KM%j+&m8 zw5p7(8kaSsc(lar?o>-M#$IABQ%QjNXMl9xKA}qQB+X0T7e*7zLEcg3C){xae$Blk z+S$ljsH&g%M^9p=s-9J}>8#zToR0#3?oK2xCD@eg8?z(ao*I(0AG-eahsVukE(4o6 z0$Tk(u=-)thGW2d^w;e!zD!+C3ft6vLq|<)9c*NMq*HkyYc<`b>641qY~U2sEge6b zqAKrcs)xVzJ83(lzROTI`$hgm4YmVi&_0ZeQOX#x!3HDhobEBRTEppL4I-%yOcxtw zd+L1(?QYh_w|F)i@CLyJ1hbyiYAMEK`tx`s%53DXjPS>v;J~+@daIHyUbp2kr~SV5 ztqK$!#5rLsT?@t7S|4Ic6>pL^;Z==kJB*aw86w1P?GyN<4815{_#Fa1sg zB5$l*-tJwG$WME-IKXROV?PbPn>z;Sju@Lr$eMLJF{Wqf?CBK#ZYEB1KLs{DOLNcY zZI+gpA{V!&0RG;$21mHZzYd!?hJu}K5)~wC2Fw3K%Xf9}itO3BHkpj<@61r+ALd!S znM^N|HLb$$`CqByp?{-QJ{Sl_Cj%Bsm&~-Pz%;&TFL*J~k$o1lvfP604O@6lVM4^| z(wKHvV7jiR{hvQ9;&ZAwf$bvys=%TP33~a(;K}Heko_sb6@epSG;>{>9Db|0knl}n zZ6Q(IiUJ&zt*=wUH}Sopp!2B9Vsrd_D20c(7V4m5ph$8qvR$aEqMqT?^lTKoOYprs Y89t|u3wEhsckqdDvtzMnf>* */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { /* Set the link address to 32 MiB */ . = 0x2000000; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ __binary_start = .; .text : { - *(.text._start) - KEEP(*(.text.runtime_init)) - *(.text*); - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - .got : - { - *(.got*) - } + } :NONE /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ . = ALIGN(8); __binary_end_inclusive = . - 8; __runtime_init_reloc = runtime_init; - - /DISCARD/ : { *(.comment*) } } diff --git a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs index 4b2642e36..5abfb9ef0 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs @@ -13,17 +13,18 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { static __binary_start: UnsafeCell; - static __binary_end_inclusive: UnsafeCell; - static __runtime_init_reloc: UnsafeCell; static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; + static __binary_end_inclusive: UnsafeCell; + + static __runtime_init_reloc: UnsafeCell; } //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { pub const BOOT_CORE_STACK_END: usize = 0x8_0000; diff --git a/07_uart_chainloader/src/relocate.rs b/07_uart_chainloader/src/relocate.rs index 85ddab672..6ac6bccea 100644 --- a/07_uart_chainloader/src/relocate.rs +++ b/07_uart_chainloader/src/relocate.rs @@ -36,16 +36,14 @@ pub unsafe fn relocate_self() -> ! { current_binary_start_addr = current_binary_start_addr.offset(1); } - // The following function calls form a hack to achieve an "absolute jump" to - // `runtime_init::runtime_init()` by forcing an indirection through the global offset table - // (GOT), so that execution continues from the relocated binary. + // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by + // forcing an indirection through the global offset table (GOT), so that execution continues + // from the relocated binary. // - // Without this, the address of `runtime_init()` would be calculated as a relative offset from - // the current program counter, since we are compiling as `position independent code`. This - // would cause us to keep executing from the address to which the firmware loaded us, instead of - // the relocated position. - // - // There likely is a more elegant way to do this. + // Without the indirection through the assembly, the address of `runtime_init()` would be + // calculated as a relative offset from the current program counter, since we are compiling as + // `position independent code`. This would cause us to keep executing from the address to which + // the firmware loaded us, instead of the relocated position. let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; cpu::branch_to_raw_addr(relocated_runtime_init_addr) } diff --git a/08_timestamps/Makefile b/08_timestamps/Makefile index 4206c5282..b5a56d075 100644 --- a/08_timestamps/Makefile +++ b/08_timestamps/Makefile @@ -122,6 +122,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/08_timestamps/README.md b/08_timestamps/README.md index 55c12a7f8..eebdd20cd 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -96,14 +96,6 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -@@ -129,7 +122,6 @@ - @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ - --section .text \ - --section .rodata \ -- --section .got \ - $(KERNEL_ELF) | rustfilt - - nm: $(KERNEL_ELF) diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs 08_timestamps/src/_arch/aarch64/cpu/boot.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs @@ -369,65 +361,60 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08 diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/link.ld 08_timestamps/src/bsp/raspberrypi/link.ld --- 07_uart_chainloader/src/bsp/raspberrypi/link.ld +++ 08_timestamps/src/bsp/raspberrypi/link.ld -@@ -5,15 +5,12 @@ +@@ -16,13 +16,12 @@ SECTIONS { - /* Set the link address to 32 MiB */ - . = 0x2000000; -+ /* Set current address to the value from which the RPi starts execution */ -+ . = 0x80000; ++ . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ - __binary_start = .; ++ __rx_start = .; .text : { -- *(.text._start) -- KEEP(*(.text.runtime_init)) -- *(.text*); -+ *(.text._start) *(.text*) - } - - .rodata : -@@ -38,16 +35,5 @@ + KEEP(*(.text._start)) +@@ -47,10 +46,4 @@ + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - -- .got : -- { -- *(.got*) -- } + } :NONE - - /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ - . = ALIGN(8); - __binary_end_inclusive = . - 8; - - __runtime_init_reloc = runtime_init; -- - /DISCARD/ : { *(.comment*) } } diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bsp/raspberrypi/memory.rs --- 07_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ 08_timestamps/src/bsp/raspberrypi/memory.rs -@@ -12,9 +12,6 @@ +@@ -12,12 +12,10 @@ // Symbols from the linker script. extern "Rust" { - static __binary_start: UnsafeCell; -- static __binary_end_inclusive: UnsafeCell; -- static __runtime_init_reloc: UnsafeCell; ++ static __rx_start: UnsafeCell<()>; ++ static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; +- static __binary_end_inclusive: UnsafeCell; +- +- static __runtime_init_reloc: UnsafeCell; } -@@ -26,12 +23,10 @@ - /// The board's memory map. + + //-------------------------------------------------------------------------------------------------- +@@ -27,12 +25,9 @@ + /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; -+ pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - -- pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; - +- pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; + - pub const GPIO_OFFSET: usize = 0x0020_0000; - pub const UART_OFFSET: usize = 0x0020_1000; + pub const GPIO_OFFSET: usize = 0x0020_0000; @@ -435,10 +422,20 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -64,35 +59,13 @@ - map::BOOT_CORE_STACK_END +@@ -56,44 +51,36 @@ } + //-------------------------------------------------------------------------------------------------- +-// Public Code ++// Private Code + //-------------------------------------------------------------------------------------------------- + +-/// Exclusive end address of the boot core's stack. +-#[inline(always)] +-pub fn boot_core_stack_end() -> usize { +- map::BOOT_CORE_STACK_END +-} +- -/// The address on which the Raspberry firmware loads every binary by default. -#[inline(always)] -pub fn board_default_load_addr() -> *const u64 { @@ -446,21 +443,33 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs -} - -/// Return the inclusive range spanning the relocated kernel binary. --/// --/// # Safety --/// ++/// Start address of the Read+Execute (RX) range. + /// + /// # Safety + /// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn relocated_binary_range_inclusive() -> RangeInclusive<*mut u64> { - unsafe { RangeInclusive::new(__binary_start.get(), __binary_end_inclusive.get()) } --} -- ++/// - Value is provided by the linker script and must be trusted as-is. ++#[inline(always)] ++fn rx_start() -> usize { ++ unsafe { __rx_start.get() as usize } + } + -/// The relocated address of function `runtime_init()`. --#[inline(always)] ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Exclusive end address of the boot core's stack. + #[inline(always)] -pub fn relocated_runtime_init_addr() -> *const u64 { - unsafe { __runtime_init_reloc.get() as _ } --} -- ++pub fn boot_core_stack_end() -> usize { ++ rx_start() + } + -/// Return the inclusive range spanning the relocated .bss section. +/// Return the inclusive range spanning the .bss section. /// @@ -674,7 +683,7 @@ diff -uNr 07_uart_chainloader/src/print.rs 08_timestamps/src/print.rs diff -uNr 07_uart_chainloader/src/relocate.rs 08_timestamps/src/relocate.rs --- 07_uart_chainloader/src/relocate.rs +++ 08_timestamps/src/relocate.rs -@@ -1,51 +0,0 @@ +@@ -1,49 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter @@ -713,16 +722,14 @@ diff -uNr 07_uart_chainloader/src/relocate.rs 08_timestamps/src/relocate.rs - current_binary_start_addr = current_binary_start_addr.offset(1); - } - -- // The following function calls form a hack to achieve an "absolute jump" to -- // `runtime_init::runtime_init()` by forcing an indirection through the global offset table -- // (GOT), so that execution continues from the relocated binary. -- // -- // Without this, the address of `runtime_init()` would be calculated as a relative offset from -- // the current program counter, since we are compiling as `position independent code`. This -- // would cause us to keep executing from the address to which the firmware loaded us, instead of -- // the relocated position. +- // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by +- // forcing an indirection through the global offset table (GOT), so that execution continues +- // from the relocated binary. - // -- // There likely is a more elegant way to do this. +- // Without the indirection through the assembly, the address of `runtime_init()` would be +- // calculated as a relative offset from the current program counter, since we are compiling as +- // `position independent code`. This would cause us to keep executing from the address to which +- // the firmware loaded us, instead of the relocated position. - let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; - cpu::branch_to_raw_addr(relocated_runtime_init_addr) -} diff --git a/08_timestamps/src/bsp/raspberrypi/link.ld b/08_timestamps/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/08_timestamps/src/bsp/raspberrypi/link.ld +++ b/08_timestamps/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/08_timestamps/src/bsp/raspberrypi/memory.rs b/08_timestamps/src/bsp/raspberrypi/memory.rs index fb47b1418..56a3306e5 100644 --- a/08_timestamps/src/bsp/raspberrypi/memory.rs +++ b/08_timestamps/src/bsp/raspberrypi/memory.rs @@ -12,6 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -20,10 +22,9 @@ extern "Rust" { // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -49,6 +50,20 @@ pub(super) mod map { } } +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -56,7 +71,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/09_hw_debug_JTAG/Makefile b/09_hw_debug_JTAG/Makefile index 038244e7e..e3dcaae86 100644 --- a/09_hw_debug_JTAG/Makefile +++ b/09_hw_debug_JTAG/Makefile @@ -147,6 +147,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs index fb47b1418..56a3306e5 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs @@ -12,6 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -20,10 +22,9 @@ extern "Rust" { // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -49,6 +50,20 @@ pub(super) mod map { } } +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -56,7 +71,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/10_privilege_level/Makefile b/10_privilege_level/Makefile index 038244e7e..e3dcaae86 100644 --- a/10_privilege_level/Makefile +++ b/10_privilege_level/Makefile @@ -147,6 +147,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 0d58e281a..2c4355487 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -136,7 +136,7 @@ Finally, we set the stack pointer for `SP_EL1` and call `ERET`: ```rust // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. -SP_EL1.set(bsp::cpu::BOOT_CORE_STACK_START); +SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. asm::eret() diff --git a/10_privilege_level/src/bsp/raspberrypi/link.ld b/10_privilege_level/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/10_privilege_level/src/bsp/raspberrypi/link.ld +++ b/10_privilege_level/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/10_privilege_level/src/bsp/raspberrypi/memory.rs b/10_privilege_level/src/bsp/raspberrypi/memory.rs index fb47b1418..56a3306e5 100644 --- a/10_privilege_level/src/bsp/raspberrypi/memory.rs +++ b/10_privilege_level/src/bsp/raspberrypi/memory.rs @@ -12,6 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -20,10 +22,9 @@ extern "Rust" { // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -49,6 +50,20 @@ pub(super) mod map { } } +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -56,7 +71,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/11_virtual_mem_part1_identity_mapping/Makefile b/11_virtual_mem_part1_identity_mapping/Makefile index 038244e7e..e3dcaae86 100644 --- a/11_virtual_mem_part1_identity_mapping/Makefile +++ b/11_virtual_mem_part1_identity_mapping/Makefile @@ -147,6 +147,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 270c8422a..bebf518e9 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -224,10 +224,16 @@ enables caching for data and instructions. ### `link.ld` -We need to align the `ro` section to `64 KiB` so that it doesn't overlap with the next section that -needs read/write attributes. This blows up the binary in size, but is a small price to pay -considering that it reduces the amount of static paging entries significantly, when compared to the -classical `4 KiB` granule. +We need to align the `rx` segment to `64 KiB` so that it doesn't overlap with the next section that +needs read/write attributes instead of read/execute attributes: + +```ld.s +. = ALIGN(64K); /* Align to page boundary */ +__rx_end_exclusive = .; +``` + +This blows up the binary in size, but is a small price to pay considering that it reduces the amount +of static paging entries significantly, when compared to the classical `4 KiB` granule. ## Address translation examples @@ -807,23 +813,16 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld --- 10_privilege_level/src/bsp/raspberrypi/link.ld +++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld -@@ -8,6 +8,7 @@ - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; - -+ __ro_start = .; - .text : - { - *(.text._start) *(.text*) -@@ -17,6 +18,8 @@ - { - *(.rodata*) - } -+ . = ALIGN(65536); /* Fill up to 64 KiB */ -+ __ro_end = .; +@@ -31,6 +31,9 @@ + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { ++ . = ALIGN(64K); /* Align to page boundary */ ++ __rx_end_exclusive = .; ++ + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs --- 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs @@ -857,7 +856,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa + [ + TranslationDescriptor { + name: "Kernel code and RO data", -+ virtual_range: ro_range_inclusive, ++ virtual_range: rx_range_inclusive, + physical_range_translation: Translation::Identity, + attribute_fields: AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, @@ -892,10 +891,10 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa +// Private Code +//-------------------------------------------------------------------------------------------------- + -+fn ro_range_inclusive() -> RangeInclusive { ++fn rx_range_inclusive() -> RangeInclusive { + // Notice the subtraction to turn the exclusive end into an inclusive end. + #[allow(clippy::range_minus_one)] -+ RangeInclusive::new(super::ro_start(), super::ro_end() - 1) ++ RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) +} + +fn remapped_mmio_range_inclusive() -> RangeInclusive { @@ -928,17 +927,16 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- -@@ -14,6 +16,8 @@ +@@ -13,6 +15,7 @@ + // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; ++ static __rx_end_exclusive: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; -+ static __ro_start: UnsafeCell<()>; -+ static __ro_end: UnsafeCell<()>; - } - - //-------------------------------------------------------------------------------------------------- -@@ -23,6 +27,21 @@ - /// The board's memory map. +@@ -25,6 +28,20 @@ + /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { + /// The inclusive end address of the memory map. @@ -955,11 +953,10 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ + /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on + /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. + pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; -+ - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; -@@ -36,6 +55,7 @@ + pub const UART_OFFSET: usize = 0x0020_1000; +@@ -37,6 +54,7 @@ pub const START: usize = 0x3F00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -967,7 +964,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } /// Physical devices. -@@ -46,10 +66,35 @@ +@@ -47,6 +65,7 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -975,35 +972,24 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } } - //-------------------------------------------------------------------------------------------------- -+// Private Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Start address of the Read-Only (RO) range. -+/// -+/// # Safety -+/// -+/// - Value is provided by the linker script and must be trusted as-is. -+#[inline(always)] -+fn ro_start() -> usize { -+ unsafe { __ro_start.get() as usize } -+} -+ -+/// Size of the Read-Only (RO) range of the kernel binary. +@@ -64,6 +83,16 @@ + unsafe { __rx_start.get() as usize } + } + ++/// Exclusive end address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] -+fn ro_end() -> usize { -+ unsafe { __ro_end.get() as usize } ++fn rx_end_exclusive() -> usize { ++ unsafe { __rx_end_exclusive.get() as usize } +} + -+//-------------------------------------------------------------------------------------------------- + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- - diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/src/bsp.rs --- 10_privilege_level/src/bsp.rs +++ 11_virtual_mem_part1_identity_mapping/src/bsp.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld index aec8e79b7..6c63ba10f 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -3,40 +3,50 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; - __ro_start = .; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } - . = ALIGN(65536); /* Fill up to 64 KiB */ - __ro_end = .; + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; + + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs index d4bcf0a3b..bb18dd201 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -14,17 +14,18 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; - static __ro_end: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { /// The inclusive end address of the memory map. @@ -42,8 +43,6 @@ pub(super) mod map { /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -74,24 +73,24 @@ pub(super) mod map { // Private Code //-------------------------------------------------------------------------------------------------- -/// Start address of the Read-Only (RO) range. +/// Start address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_start() -> usize { - unsafe { __ro_start.get() as usize } +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } -/// Size of the Read-Only (RO) range of the kernel binary. +/// Exclusive end address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_end() -> usize { - unsafe { __ro_end.get() as usize } +fn rx_end_exclusive() -> usize { + unsafe { __rx_end_exclusive.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -101,7 +100,7 @@ fn ro_end() -> usize { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs index 703bb3ba2..cf9cfcae2 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -26,7 +26,7 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne [ TranslationDescriptor { name: "Kernel code and RO data", - virtual_range: ro_range_inclusive, + virtual_range: rx_range_inclusive, physical_range_translation: Translation::Identity, attribute_fields: AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, @@ -61,10 +61,10 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne // Private Code //-------------------------------------------------------------------------------------------------- -fn ro_range_inclusive() -> RangeInclusive { +fn rx_range_inclusive() -> RangeInclusive { // Notice the subtraction to turn the exclusive end into an inclusive end. #[allow(clippy::range_minus_one)] - RangeInclusive::new(super::ro_start(), super::ro_end() - 1) + RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) } fn remapped_mmio_range_inclusive() -> RangeInclusive { diff --git a/12_exceptions_part1_groundwork/Makefile b/12_exceptions_part1_groundwork/Makefile index 038244e7e..e3dcaae86 100644 --- a/12_exceptions_part1_groundwork/Makefile +++ b/12_exceptions_part1_groundwork/Makefile @@ -147,6 +147,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 0ce07116d..b495742d6 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -238,7 +238,7 @@ because `Rust` has no stable convention ([yet](https://github.com/rust-lang/rfcs Next, we craft the exception vector table: ```asm -.section .exception_vectors, "ax", @progbits +.section .text // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 @@ -324,7 +324,7 @@ The actual handlers referenced from the assembly can now branch to it for the ti ```rust #[no_mangle] -unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { +unsafe extern "C" fn current_elx_irq(e: &mut ExceptionContext) { default_exception_handler(e); } ``` @@ -480,7 +480,7 @@ General purpose register: diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs --- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs -@@ -11,7 +11,230 @@ +@@ -11,7 +11,224 @@ //! //! crate::exception::arch_exception @@ -541,18 +541,18 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 +//------------------------------------------------------------------------------ + +#[no_mangle] -+unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { -+ default_exception_handler(e); ++unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { ++ panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] -+unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) { -+ default_exception_handler(e); ++unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { ++ panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] -+unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) { -+ default_exception_handler(e); ++unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { ++ panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +//------------------------------------------------------------------------------ @@ -646,9 +646,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 + writeln!(f, " - {}", ec_translation)?; + + // Raw print of instruction specific syndrome. -+ write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; -+ -+ Ok(()) ++ write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS)) + } +} + @@ -681,9 +679,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 + + write!(f, " Illegal Execution State (IL): {}", + to_flag_str(self.0.is_set(SPSR_EL1::IL)) -+ )?; -+ -+ Ok(()) ++ ) + } +} + @@ -704,15 +700,13 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 + for (i, reg) in self.gpr.iter().enumerate() { + write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; + } -+ write!(f, " lr : {:#018x}", self.lr)?; -+ -+ Ok(()) ++ write!(f, " lr : {:#018x}", self.lr) + } +} //-------------------------------------------------------------------------------------------------- // Public Code -@@ -28,3 +251,23 @@ +@@ -28,3 +245,23 @@ _ => (PrivilegeLevel::Unknown, "Unknown"), } } @@ -793,7 +787,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S 12 +//-------------------------------------------------------------------------------------------------- +// The exception vector table. +//-------------------------------------------------------------------------------------------------- -+.section .exception_vectors, "ax", @progbits ++.section .text + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 @@ -880,22 +874,6 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S 12 + + eret -diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld 12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld ---- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld -+++ 12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld -@@ -14,6 +14,11 @@ - *(.text._start) *(.text*) - } - -+ .exception_vectors : -+ { -+ *(.exception_vectors*) -+ } -+ - .rodata : - { - *(.rodata*) - diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs --- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +++ 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -926,7 +904,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.r virtual_range: mmio_range_inclusive, physical_range_translation: Translation::Identity, @@ -67,11 +57,6 @@ - RangeInclusive::new(super::ro_start(), super::ro_end() - 1) + RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) } -fn remapped_mmio_range_inclusive() -> RangeInclusive { diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S index 4d125334c..fd7b1f930 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S @@ -50,7 +50,7 @@ //-------------------------------------------------------------------------------------------------- // The exception vector table. //-------------------------------------------------------------------------------------------------- -.section .exception_vectors, "ax", @progbits +.section .text // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs index 64f1fd23e..f08d0ce4a 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -67,18 +67,18 @@ fn default_exception_handler(e: &ExceptionContext) { //------------------------------------------------------------------------------ #[no_mangle] -unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } //------------------------------------------------------------------------------ @@ -172,9 +172,7 @@ impl fmt::Display for EsrEL1 { writeln!(f, " - {}", ec_translation)?; // Raw print of instruction specific syndrome. - write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; - - Ok(()) + write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS)) } } @@ -207,9 +205,7 @@ impl fmt::Display for SpsrEL1 { write!(f, " Illegal Execution State (IL): {}", to_flag_str(self.0.is_set(SPSR_EL1::IL)) - )?; - - Ok(()) + ) } } @@ -230,9 +226,7 @@ impl fmt::Display for ExceptionContext { for (i, reg) in self.gpr.iter().enumerate() { write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; } - write!(f, " lr : {:#018x}", self.lr)?; - - Ok(()) + write!(f, " lr : {:#018x}", self.lr) } } diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld index 4dfe60c7f..6c63ba10f 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld @@ -3,45 +3,50 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; - __ro_start = .; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .exception_vectors : - { - *(.exception_vectors*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .rodata : - { - *(.rodata*) - } - . = ALIGN(65536); /* Fill up to 64 KiB */ - __ro_end = .; + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs index d4bcf0a3b..bb18dd201 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs @@ -14,17 +14,18 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; - static __ro_end: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { /// The inclusive end address of the memory map. @@ -42,8 +43,6 @@ pub(super) mod map { /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -74,24 +73,24 @@ pub(super) mod map { // Private Code //-------------------------------------------------------------------------------------------------- -/// Start address of the Read-Only (RO) range. +/// Start address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_start() -> usize { - unsafe { __ro_start.get() as usize } +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } -/// Size of the Read-Only (RO) range of the kernel binary. +/// Exclusive end address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_end() -> usize { - unsafe { __ro_end.get() as usize } +fn rx_end_exclusive() -> usize { + unsafe { __rx_end_exclusive.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -101,7 +100,7 @@ fn ro_end() -> usize { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs index 1775c07cd..ef4bf32cc 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -26,7 +26,7 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne [ TranslationDescriptor { name: "Kernel code and RO data", - virtual_range: ro_range_inclusive, + virtual_range: rx_range_inclusive, physical_range_translation: Translation::Identity, attribute_fields: AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, @@ -51,10 +51,10 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne // Private Code //-------------------------------------------------------------------------------------------------- -fn ro_range_inclusive() -> RangeInclusive { +fn rx_range_inclusive() -> RangeInclusive { // Notice the subtraction to turn the exclusive end into an inclusive end. #[allow(clippy::range_minus_one)] - RangeInclusive::new(super::ro_start(), super::ro_end() - 1) + RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) } fn mmio_range_inclusive() -> RangeInclusive { diff --git a/13_integrated_testing/Makefile b/13_integrated_testing/Makefile index 32d139243..4c2fb069a 100644 --- a/13_integrated_testing/Makefile +++ b/13_integrated_testing/Makefile @@ -179,6 +179,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/13_integrated_testing/src/_arch/aarch64/exception.S b/13_integrated_testing/src/_arch/aarch64/exception.S index 4d125334c..fd7b1f930 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.S +++ b/13_integrated_testing/src/_arch/aarch64/exception.S @@ -50,7 +50,7 @@ //-------------------------------------------------------------------------------------------------- // The exception vector table. //-------------------------------------------------------------------------------------------------- -.section .exception_vectors, "ax", @progbits +.section .text // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 diff --git a/13_integrated_testing/src/_arch/aarch64/exception.rs b/13_integrated_testing/src/_arch/aarch64/exception.rs index 0e640ec69..e67841f9f 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception.rs @@ -67,18 +67,18 @@ fn default_exception_handler(e: &ExceptionContext) { //------------------------------------------------------------------------------ #[no_mangle] -unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } //------------------------------------------------------------------------------ @@ -162,9 +162,7 @@ impl fmt::Display for EsrEL1 { writeln!(f, " - {}", ec_translation)?; // Raw print of instruction specific syndrome. - write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; - - Ok(()) + write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS)) } } @@ -197,9 +195,7 @@ impl fmt::Display for SpsrEL1 { write!(f, " Illegal Execution State (IL): {}", to_flag_str(self.0.is_set(SPSR_EL1::IL)) - )?; - - Ok(()) + ) } } @@ -220,9 +216,7 @@ impl fmt::Display for ExceptionContext { for (i, reg) in self.gpr.iter().enumerate() { write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; } - write!(f, " lr : {:#018x}", self.lr)?; - - Ok(()) + write!(f, " lr : {:#018x}", self.lr) } } diff --git a/13_integrated_testing/src/bsp/raspberrypi/link.ld b/13_integrated_testing/src/bsp/raspberrypi/link.ld index 4dfe60c7f..6c63ba10f 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/link.ld +++ b/13_integrated_testing/src/bsp/raspberrypi/link.ld @@ -3,45 +3,50 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; - __ro_start = .; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .exception_vectors : - { - *(.exception_vectors*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .rodata : - { - *(.rodata*) - } - . = ALIGN(65536); /* Fill up to 64 KiB */ - __ro_end = .; + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory.rs b/13_integrated_testing/src/bsp/raspberrypi/memory.rs index d4bcf0a3b..bb18dd201 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory.rs @@ -14,17 +14,18 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; - static __ro_end: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { /// The inclusive end address of the memory map. @@ -42,8 +43,6 @@ pub(super) mod map { /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -74,24 +73,24 @@ pub(super) mod map { // Private Code //-------------------------------------------------------------------------------------------------- -/// Start address of the Read-Only (RO) range. +/// Start address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_start() -> usize { - unsafe { __ro_start.get() as usize } +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } -/// Size of the Read-Only (RO) range of the kernel binary. +/// Exclusive end address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_end() -> usize { - unsafe { __ro_end.get() as usize } +fn rx_end_exclusive() -> usize { + unsafe { __rx_end_exclusive.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -101,7 +100,7 @@ fn ro_end() -> usize { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs index 0ccfae002..f4769ad39 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -26,7 +26,7 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne [ TranslationDescriptor { name: "Kernel code and RO data", - virtual_range: ro_range_inclusive, + virtual_range: rx_range_inclusive, physical_range_translation: Translation::Identity, attribute_fields: AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, @@ -51,10 +51,10 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne // Private Code //-------------------------------------------------------------------------------------------------- -fn ro_range_inclusive() -> RangeInclusive { +fn rx_range_inclusive() -> RangeInclusive { // Notice the subtraction to turn the exclusive end into an inclusive end. #[allow(clippy::range_minus_one)] - RangeInclusive::new(super::ro_start(), super::ro_end() - 1) + RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) } fn mmio_range_inclusive() -> RangeInclusive { diff --git a/14_exceptions_part2_peripheral_IRQs/Makefile b/14_exceptions_part2_peripheral_IRQs/Makefile index 32d139243..4c2fb069a 100644 --- a/14_exceptions_part2_peripheral_IRQs/Makefile +++ b/14_exceptions_part2_peripheral_IRQs/Makefile @@ -179,6 +179,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 5e1f45e32..9ae718d77 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -2056,7 +2056,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception.rs 14_exceptions_p diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs --- 13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs -@@ -52,10 +52,12 @@ +@@ -51,10 +51,12 @@ pub mod mmio { use super::*; @@ -2073,7 +2073,7 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part } /// Physical devices. -@@ -66,6 +68,8 @@ +@@ -65,6 +67,8 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S index 4d125334c..fd7b1f930 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S @@ -50,7 +50,7 @@ //-------------------------------------------------------------------------------------------------- // The exception vector table. //-------------------------------------------------------------------------------------------------- -.section .exception_vectors, "ax", @progbits +.section .text // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs index 5cf9eb5ca..de4f1f185 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -68,18 +68,18 @@ fn default_exception_handler(e: &ExceptionContext) { //------------------------------------------------------------------------------ #[no_mangle] -unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } //------------------------------------------------------------------------------ @@ -166,9 +166,7 @@ impl fmt::Display for EsrEL1 { writeln!(f, " - {}", ec_translation)?; // Raw print of instruction specific syndrome. - write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; - - Ok(()) + write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS)) } } @@ -201,9 +199,7 @@ impl fmt::Display for SpsrEL1 { write!(f, " Illegal Execution State (IL): {}", to_flag_str(self.0.is_set(SPSR_EL1::IL)) - )?; - - Ok(()) + ) } } @@ -224,9 +220,7 @@ impl fmt::Display for ExceptionContext { for (i, reg) in self.gpr.iter().enumerate() { write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; } - write!(f, " lr : {:#018x}", self.lr)?; - - Ok(()) + write!(f, " lr : {:#018x}", self.lr) } } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld index 4dfe60c7f..6c63ba10f 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld @@ -3,45 +3,50 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; - __ro_start = .; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .exception_vectors : - { - *(.exception_vectors*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .rodata : - { - *(.rodata*) - } - . = ALIGN(65536); /* Fill up to 64 KiB */ - __ro_end = .; + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs index 4ac3b57b9..d07f2dc34 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -14,17 +14,18 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; - static __ro_end: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { /// The inclusive end address of the memory map. @@ -42,8 +43,6 @@ pub(super) mod map { /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -78,24 +77,24 @@ pub(super) mod map { // Private Code //-------------------------------------------------------------------------------------------------- -/// Start address of the Read-Only (RO) range. +/// Start address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_start() -> usize { - unsafe { __ro_start.get() as usize } +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } } -/// Size of the Read-Only (RO) range of the kernel binary. +/// Exclusive end address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_end() -> usize { - unsafe { __ro_end.get() as usize } +fn rx_end_exclusive() -> usize { + unsafe { __rx_end_exclusive.get() as usize } } //-------------------------------------------------------------------------------------------------- @@ -105,7 +104,7 @@ fn ro_end() -> usize { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs index 0ccfae002..f4769ad39 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs @@ -26,7 +26,7 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne [ TranslationDescriptor { name: "Kernel code and RO data", - virtual_range: ro_range_inclusive, + virtual_range: rx_range_inclusive, physical_range_translation: Translation::Identity, attribute_fields: AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, @@ -51,10 +51,10 @@ pub static LAYOUT: KernelVirtualLayout = KernelVirtualLayout::ne // Private Code //-------------------------------------------------------------------------------------------------- -fn ro_range_inclusive() -> RangeInclusive { +fn rx_range_inclusive() -> RangeInclusive { // Notice the subtraction to turn the exclusive end into an inclusive end. #[allow(clippy::range_minus_one)] - RangeInclusive::new(super::ro_start(), super::ro_end() - 1) + RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) } fn mmio_range_inclusive() -> RangeInclusive { diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/15_virtual_mem_part2_mmio_remap/Makefile index 32d139243..4c2fb069a 100644 --- a/15_virtual_mem_part2_mmio_remap/Makefile +++ b/15_virtual_mem_part2_mmio_remap/Makefile @@ -179,6 +179,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 9a6c6df64..18c72b74c 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -276,6 +276,13 @@ unsafe fn init(&self) -> Result<(), &'static str> { There's a couple of changes not covered in this tutorial text, but the reader should ideally skim through them: +- [`src/bsp/raspberrypi/memory.rs`](src/bsp/raspberrypi/memory.rs) and + [`src/bsp/raspberrypi/link.ld`](src/bsp/raspberrypi/link.ld) changed the location of the boot + core's stack. It is now located after the data segment, and separated by an unmapped `guard page`. + There is also supporting code in + [`src/_arch/aarch64/exception.rs`](src/_arch/aarch64/exception.rs) that runs on data aborts and + checks if the fault address lies within the `stack guard page`. This can be an indication that a + kernel stack overflow happened. - [`src/memory/mmu/types.rs`](src/memory/mmu/types.rs) introduces a couple of supporting types, like `Page`. - [`src/memory/mmu/mapping_record.rs`](src/memory/mmu/mapping_record.rs) provides the generic kernel @@ -309,18 +316,18 @@ Minipush 1.0 [MP] ⏩ Pushing 67 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.786819] Booting on: Raspberry Pi 3 -[ 0.787092] MMU online: -[ 0.787384] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.789128] Virtual Physical Size Attr Entity -[ 0.790873] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.792618] 0x0000_0000_0007_0000..0x0000_0000_0007_ffff --> 0x00_0007_0000..0x00_0007_ffff | 64 KiB | C RW XN | Kernel boot-core stack -[ 0.794221] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 0.795835] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss -[ 0.797406] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 0.798857] | BCM PL011 UART -[ 0.800374] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 0.802117] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.789721] Booting on: Raspberry Pi 3 +[ 0.789994] MMU online: +[ 0.790286] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.792030] Virtual Physical Size Attr Entity +[ 0.793774] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.795520] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.797134] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.798704] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 0.800307] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.801759] | BCM PL011 UART +[ 0.803276] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 0.805020] ------------------------------------------------------------------------------------------------------------------------------------------- ``` Raspberry Pi 4: @@ -344,19 +351,19 @@ Minipush 1.0 [MP] ⏩ Pushing 74 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.853908] Booting on: Raspberry Pi 4 -[ 0.854007] MMU online: -[ 0.854299] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.856043] Virtual Physical Size Attr Entity -[ 0.857788] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.859533] 0x0000_0000_0007_0000..0x0000_0000_0007_ffff --> 0x00_0007_0000..0x00_0007_ffff | 64 KiB | C RW XN | Kernel boot-core stack -[ 0.861137] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 0.862750] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss -[ 0.864321] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 0.865772] | BCM PL011 UART -[ 0.867289] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD -[ 0.868697] | GICC -[ 0.870105] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.870371] Booting on: Raspberry Pi 4 +[ 0.870470] MMU online: +[ 0.870763] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.872507] Virtual Physical Size Attr Entity +[ 0.874251] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.875996] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.877611] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.879181] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 0.880784] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.882235] | BCM PL011 UART +[ 0.883752] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD +[ 0.885160] | GICC +[ 0.886569] ------------------------------------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous @@ -375,6 +382,55 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs 15_v // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. asm::eret() +diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +--- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs ++++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +@@ -11,7 +11,11 @@ + //! + //! crate::exception::arch_exception + +-use crate::{bsp, exception}; ++use crate::{ ++ bsp::{self}, ++ exception, ++ memory::Address, ++}; + use core::{cell::UnsafeCell, fmt}; + use cortex_a::{barrier, regs::*}; + use register::InMemoryRegister; +@@ -50,6 +54,20 @@ + // Private Code + //-------------------------------------------------------------------------------------------------- + ++/// Check if additional context can be derived from a data abort. ++fn inspect_data_abort(f: &mut fmt::Formatter) -> fmt::Result { ++ let fault_addr = Address::new(FAR_EL1.get() as usize); ++ ++ if bsp::memory::mmu::virt_boot_core_stack_guard_page_desc().contains(fault_addr) { ++ writeln!( ++ f, ++ "\n\n >> Attempted to access the guard page of the kernel's boot core stack <<" ++ )?; ++ } ++ ++ Ok(()) ++} ++ + /// Prints verbose information about the exception and then panics. + fn default_exception_handler(e: &ExceptionContext) { + panic!( +@@ -166,7 +184,9 @@ + writeln!(f, " - {}", ec_translation)?; + + // Raw print of instruction specific syndrome. +- write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS)) ++ write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; ++ ++ inspect_data_abort(f) + } + } + + diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs --- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -1413,23 +1469,41 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs 15_v diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld -@@ -42,6 +42,11 @@ - . += 8; +@@ -37,6 +37,7 @@ + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ ++ __rw_start = .; + .data : { *(.data*) } :segment_rw + + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ +@@ -49,4 +50,21 @@ + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } -+ . = ALIGN(65536); -+ __data_end = .; -+ -+ __ro_size = __ro_end - __ro_start; -+ __data_size = __data_end - __ro_end; - - /DISCARD/ : { *(.comment*) } + } :NONE ++ ++ . = ALIGN(64K); /* Align to page boundary */ ++ __rw_end_exclusive = .; ++ ++ /*********************************************************************************************** ++ * Guard Page between boot core stack and data ++ ***********************************************************************************************/ ++ __boot_core_stack_guard_page_start = .; ++ . += 64K; ++ __boot_core_stack_guard_page_end_exclusive = .; ++ ++ /*********************************************************************************************** ++ * Boot Core Stack ++ ***********************************************************************************************/ ++ __boot_core_stack_start = .; ++ . += 512K; ++ __boot_core_stack_end_exclusive = .; } diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs -@@ -4,70 +4,157 @@ +@@ -4,70 +4,164 @@ //! BSP Memory Management Unit. @@ -1465,16 +1539,16 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to +/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; - --const NUM_MEM_RANGES: usize = 2; ++ +/// The kernel's virtual address space defined by this BSP. +pub type KernelVirtAddrSpace = AddressSpace<{ 8 * 1024 * 1024 * 1024 }>; --/// The virtual memory layout. +-const NUM_MEM_RANGES: usize = 2; +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- -+ + +-/// The virtual memory layout. +/// The kernel translation tables. /// -/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. @@ -1484,7 +1558,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - [ - TranslationDescriptor { - name: "Kernel code and RO data", -- virtual_range: ro_range_inclusive, +- virtual_range: rx_range_inclusive, - physical_range_translation: Translation::Identity, - attribute_fields: AttributeFields { - mem_attributes: MemAttributes::CacheableDRAM, @@ -1515,10 +1589,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs // Private Code //-------------------------------------------------------------------------------------------------- --fn ro_range_inclusive() -> RangeInclusive { +-fn rx_range_inclusive() -> RangeInclusive { - // Notice the subtraction to turn the exclusive end into an inclusive end. - #[allow(clippy::range_minus_one)] -- RangeInclusive::new(super::ro_start(), super::ro_end() - 1) +- RangeInclusive::new(super::rx_start(), super::rx_end_exclusive() - 1) +/// Helper function for calculating the number of pages the given parameter spans. +const fn size_to_num_pages(size: usize) -> usize { + assert!(size > 0); @@ -1527,44 +1601,44 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + size >> KernelGranule::SHIFT +} + -+/// The boot core's stack. -+fn virt_stack_page_desc() -> PageSliceDescriptor { -+ let num_pages = size_to_num_pages(super::boot_core_stack_size()); ++/// The Read+Execute (RX) pages of the kernel binary. ++fn virt_rx_page_desc() -> PageSliceDescriptor { ++ let num_pages = size_to_num_pages(super::rx_size()); + -+ PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) ++ PageSliceDescriptor::from_addr(super::virt_rx_start(), num_pages) +} + -+/// The Read-Only (RO) pages of the kernel binary. -+fn virt_ro_page_desc() -> PageSliceDescriptor { -+ let num_pages = size_to_num_pages(super::ro_size()); ++/// The Read+Write (RW) pages of the kernel binary. ++fn virt_rw_page_desc() -> PageSliceDescriptor { ++ let num_pages = size_to_num_pages(super::rw_size()); + -+ PageSliceDescriptor::from_addr(super::virt_ro_start(), num_pages) ++ PageSliceDescriptor::from_addr(super::virt_rw_start(), num_pages) +} + -+/// The data pages of the kernel binary. -+fn virt_data_page_desc() -> PageSliceDescriptor { -+ let num_pages = size_to_num_pages(super::data_size()); ++/// The boot core's stack. ++fn virt_boot_core_stack_page_desc() -> PageSliceDescriptor { ++ let num_pages = size_to_num_pages(super::boot_core_stack_size()); + -+ PageSliceDescriptor::from_addr(super::virt_data_start(), num_pages) ++ PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) +} + +// The binary is still identity mapped, so we don't need to convert in the following. + -+/// The boot core's stack. -+fn phys_stack_page_desc() -> PageSliceDescriptor { -+ virt_stack_page_desc().into() -+} -+ -+/// The Read-Only (RO) pages of the kernel binary. -+fn phys_ro_page_desc() -> PageSliceDescriptor { -+ virt_ro_page_desc().into() ++/// The Read+Execute (RX) pages of the kernel binary. ++fn phys_rx_page_desc() -> PageSliceDescriptor { ++ virt_rx_page_desc().into() } -fn mmio_range_inclusive() -> RangeInclusive { - RangeInclusive::new(memory_map::mmio::START, memory_map::mmio::END_INCLUSIVE) -+/// The data pages of the kernel binary. -+fn phys_data_page_desc() -> PageSliceDescriptor { -+ virt_data_page_desc().into() ++/// The Read+Write (RW) pages of the kernel binary. ++fn phys_rw_page_desc() -> PageSliceDescriptor { ++ virt_rw_page_desc().into() ++} ++ ++/// The boot core's stack. ++fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { ++ virt_boot_core_stack_page_desc().into() } //-------------------------------------------------------------------------------------------------- @@ -1579,6 +1653,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + &KERNEL_TABLES +} + ++/// The boot core's stack guard page. ++pub fn virt_boot_core_stack_guard_page_desc() -> PageSliceDescriptor { ++ let num_pages = size_to_num_pages(super::boot_core_stack_guard_page_size()); ++ ++ PageSliceDescriptor::from_addr(super::virt_boot_core_stack_guard_page_start(), num_pages) ++} ++ +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( @@ -1594,31 +1675,31 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. +pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { + generic_mmu::kernel_map_pages_at( -+ "Kernel boot-core stack", -+ &virt_stack_page_desc(), -+ &phys_stack_page_desc(), ++ "Kernel code and RO data", ++ &virt_rx_page_desc(), ++ &phys_rx_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, -+ acc_perms: AccessPermissions::ReadWrite, -+ execute_never: true, ++ acc_perms: AccessPermissions::ReadOnly, ++ execute_never: false, + }, + )?; + + generic_mmu::kernel_map_pages_at( -+ "Kernel code and RO data", -+ &virt_ro_page_desc(), -+ &phys_ro_page_desc(), ++ "Kernel data and bss", ++ &virt_rw_page_desc(), ++ &phys_rw_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, -+ acc_perms: AccessPermissions::ReadOnly, -+ execute_never: false, ++ acc_perms: AccessPermissions::ReadWrite, ++ execute_never: true, + }, + )?; + + generic_mmu::kernel_map_pages_at( -+ "Kernel data and bss", -+ &virt_data_page_desc(), -+ &phys_data_page_desc(), ++ "Kernel boot-core stack", ++ &virt_boot_core_stack_page_desc(), ++ &phys_boot_core_stack_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, @@ -1630,7 +1711,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs } //-------------------------------------------------------------------------------------------------- -@@ -82,14 +169,12 @@ +@@ -82,14 +176,18 @@ /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { @@ -1639,7 +1720,13 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - for i in LAYOUT.inner().iter() { - let start: usize = *(i.virtual_range)().start(); - let end: usize = *(i.virtual_range)().end() + 1; -+ for i in [virt_stack_page_desc, virt_ro_page_desc, virt_data_page_desc].iter() { ++ for i in [ ++ virt_rx_page_desc, ++ virt_rw_page_desc, ++ virt_boot_core_stack_page_desc, ++ ] ++ .iter() ++ { + let start: usize = i().start_addr().into_usize(); + let end: usize = i().end_addr().into_usize(); @@ -1650,7 +1737,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs assert!(end >= start); } } -@@ -97,18 +182,28 @@ +@@ -97,18 +195,28 @@ /// Ensure the kernel's virtual memory layout is free of overlaps. #[kernel_test] fn virt_mem_layout_has_no_overlaps() { @@ -1666,9 +1753,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs - assert!(!second_range().contains(first_range().start())); - assert!(!second_range().contains(first_range().end())); + let layout = [ -+ virt_stack_page_desc(), -+ virt_ro_page_desc(), -+ virt_data_page_desc(), ++ virt_rx_page_desc(), ++ virt_rw_page_desc(), ++ virt_boot_core_stack_page_desc(), + ]; + + for (i, first_range) in layout.iter().enumerate() { @@ -1694,7 +1781,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs -@@ -3,9 +3,41 @@ +@@ -3,9 +3,40 @@ // Copyright (c) 2018-2021 Andre Richter //! BSP Memory Management. @@ -1703,31 +1790,30 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v +//! copies the binary to 0x8_0000: +//! +//! +---------------------------------------------+ -+//! | | 0x0 -+//! | Unmapped | -+//! | | 0x6_FFFF -+//! +---------------------------------------------+ -+//! | BOOT_CORE_STACK_START | 0x7_0000 -+//! | | ^ -+//! | ... | | Stack growth direction -+//! | | | -+//! | BOOT_CORE_STACK_END_INCLUSIVE | 0x7_FFFF -+//! +---------------------------------------------+ -+//! | RO_START == BOOT_CORE_STACK_END | 0x8_0000 +//! | | ++//! | Unmapped | +//! | | ++//! +---------------------------------------------+ ++//! | | rx_start @ 0x8_0000 +//! | .text | -+//! | .exception_vectors | +//! | .rodata | -+//! | | -+//! | RO_END_INCLUSIVE | 0x8_0000 + __ro_size - 1 ++//! | .got | ++//! | | rx_end_inclusive +//! +---------------------------------------------+ -+//! | RO_END == DATA_START | 0x8_0000 + __ro_size -+//! | | ++//! | | rw_start == ro_end +//! | .data | +//! | .bss | ++//! | | rw_end_inclusive ++//! +---------------------------------------------+ ++//! | | rw_end ++//! | Unmapped Boot-core Stack Guard Page | +//! | | -+//! | DATA_END_INCLUSIVE | 0x8_0000 + __ro_size + __data_size - 1 ++//! +---------------------------------------------+ ++//! | | boot_core_stack_start ^ ++//! | | | stack ++//! | Boot-core Stack | | growth ++//! | | | direction ++//! | | boot_core_stack_end_inclusive | +//! +---------------------------------------------+ pub mod mmu; @@ -1736,21 +1822,25 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- -@@ -17,47 +49,39 @@ +@@ -17,8 +48,16 @@ + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + ++ static __rw_start: UnsafeCell<()>; static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; -- static __ro_end: UnsafeCell<()>; -+ static __ro_size: UnsafeCell<()>; -+ static __data_size: UnsafeCell<()>; ++ static __rw_end_exclusive: UnsafeCell<()>; ++ ++ static __boot_core_stack_start: UnsafeCell<()>; ++ static __boot_core_stack_end_exclusive: UnsafeCell<()>; ++ ++ static __boot_core_stack_guard_page_start: UnsafeCell<()>; ++ static __boot_core_stack_guard_page_end_exclusive: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- - // Public Definitions - //-------------------------------------------------------------------------------------------------- - --/// The board's memory map. -+/// The board's physical memory map. +@@ -28,35 +67,26 @@ + /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - /// The inclusive end address of the memory map. @@ -1767,13 +1857,10 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v - /// physical address that is not backed by any DRAM (e.g. accessing an address close to 4 GiB on - /// an RPi3 that comes with 1 GiB of RAM). This would result in a crash or other kind of error. - pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; -+ use super::*; - -- pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - - pub const GPIO_OFFSET: usize = 0x0020_0000; - pub const UART_OFFSET: usize = 0x0020_1000; -+ pub const BOOT_CORE_STACK_SIZE: usize = 0x1_0000; ++ use super::*; /// Physical devices. #[cfg(feature = "bsp_rpi3")] @@ -1802,7 +1889,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } /// Physical devices. -@@ -65,13 +89,22 @@ +@@ -64,13 +94,22 @@ pub mod mmio { use super::*; @@ -1831,53 +1918,72 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } //-------------------------------------------------------------------------------------------------- -@@ -84,8 +117,8 @@ +@@ -83,18 +122,69 @@ /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] --fn ro_start() -> usize { -- unsafe { __ro_start.get() as usize } -+fn virt_ro_start() -> Address { -+ Address::new(unsafe { __ro_start.get() as usize }) +-fn rx_start() -> usize { +- unsafe { __rx_start.get() as usize } ++fn virt_rx_start() -> Address { ++ Address::new(unsafe { __rx_start.get() as usize }) } - /// Size of the Read-Only (RO) range of the kernel binary. -@@ -94,8 +127,42 @@ +-/// Exclusive end address of the Read+Execute (RX) range. ++/// Size of the Read+Execute (RX) range. + /// + /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] --fn ro_end() -> usize { -- unsafe { __ro_end.get() as usize } -+fn ro_size() -> usize { -+ unsafe { __ro_size.get() as usize } +-fn rx_end_exclusive() -> usize { +- unsafe { __rx_end_exclusive.get() as usize } ++fn rx_size() -> usize { ++ unsafe { (__rx_end_exclusive.get() as usize) - (__rx_start.get() as usize) } +} + -+/// Start address of the data range. ++/// Start address of the Read+Write (RW) range. +#[inline(always)] -+fn virt_data_start() -> Address { -+ virt_ro_start() + ro_size() ++fn virt_rw_start() -> Address { ++ Address::new(unsafe { __rw_start.get() as usize }) +} + -+/// Size of the data range. ++/// Size of the Read+Write (RW) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] -+fn data_size() -> usize { -+ unsafe { __data_size.get() as usize } ++fn rw_size() -> usize { ++ unsafe { (__rw_end_exclusive.get() as usize) - (__rw_start.get() as usize) } +} + +/// Start address of the boot core's stack. +#[inline(always)] +fn virt_boot_core_stack_start() -> Address { -+ virt_ro_start() - map::BOOT_CORE_STACK_SIZE ++ Address::new(unsafe { __boot_core_stack_start.get() as usize }) +} + +/// Size of the boot core's stack. +#[inline(always)] +fn boot_core_stack_size() -> usize { -+ map::BOOT_CORE_STACK_SIZE ++ unsafe { ++ (__boot_core_stack_end_exclusive.get() as usize) - (__boot_core_stack_start.get() as usize) ++ } ++} ++ ++/// Start address of the boot core's stack guard page. ++#[inline(always)] ++fn virt_boot_core_stack_guard_page_start() -> Address { ++ Address::new(unsafe { __boot_core_stack_guard_page_start.get() as usize }) ++} ++ ++/// Size of the boot core's stack guard page. ++#[inline(always)] ++fn boot_core_stack_guard_page_size() -> usize { ++ unsafe { ++ (__boot_core_stack_guard_page_end_exclusive.get() as usize) ++ - (__boot_core_stack_guard_page_start.get() as usize) ++ } +} + +/// Exclusive end address of the physical address space. @@ -1887,12 +1993,12 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } //-------------------------------------------------------------------------------------------------- -@@ -104,8 +171,10 @@ +@@ -103,8 +193,10 @@ /// Exclusive end address of the boot core's stack. #[inline(always)] -pub fn boot_core_stack_end() -> usize { -- map::BOOT_CORE_STACK_END +- rx_start() +pub fn phys_boot_core_stack_end() -> Address { + // The binary is still identity mapped, so we don't need to convert here. + let end = virt_boot_core_stack_start().into_usize() + boot_core_stack_size(); diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S index 4d125334c..fd7b1f930 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S @@ -50,7 +50,7 @@ //-------------------------------------------------------------------------------------------------- // The exception vector table. //-------------------------------------------------------------------------------------------------- -.section .exception_vectors, "ax", @progbits +.section .text // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs index 5cf9eb5ca..b93387344 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -11,7 +11,11 @@ //! //! crate::exception::arch_exception -use crate::{bsp, exception}; +use crate::{ + bsp::{self}, + exception, + memory::Address, +}; use core::{cell::UnsafeCell, fmt}; use cortex_a::{barrier, regs::*}; use register::InMemoryRegister; @@ -50,6 +54,20 @@ struct EsrEL1; // Private Code //-------------------------------------------------------------------------------------------------- +/// Check if additional context can be derived from a data abort. +fn inspect_data_abort(f: &mut fmt::Formatter) -> fmt::Result { + let fault_addr = Address::new(FAR_EL1.get() as usize); + + if bsp::memory::mmu::virt_boot_core_stack_guard_page_desc().contains(fault_addr) { + writeln!( + f, + "\n\n >> Attempted to access the guard page of the kernel's boot core stack <<" + )?; + } + + Ok(()) +} + /// Prints verbose information about the exception and then panics. fn default_exception_handler(e: &ExceptionContext) { panic!( @@ -68,18 +86,18 @@ fn default_exception_handler(e: &ExceptionContext) { //------------------------------------------------------------------------------ #[no_mangle] -unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } #[no_mangle] -unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) { - default_exception_handler(e); +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") } //------------------------------------------------------------------------------ @@ -168,7 +186,7 @@ impl fmt::Display for EsrEL1 { // Raw print of instruction specific syndrome. write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; - Ok(()) + inspect_data_abort(f) } } @@ -201,9 +219,7 @@ impl fmt::Display for SpsrEL1 { write!(f, " Illegal Execution State (IL): {}", to_flag_str(self.0.is_set(SPSR_EL1::IL)) - )?; - - Ok(()) + ) } } @@ -224,9 +240,7 @@ impl fmt::Display for ExceptionContext { for (i, reg) in self.gpr.iter().enumerate() { write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; } - write!(f, " lr : {:#018x}", self.lr)?; - - Ok(()) + write!(f, " lr : {:#018x}", self.lr) } } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld index e5975256f..f2b5d05d7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -3,50 +3,68 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; - __ro_start = .; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .exception_vectors : - { - *(.exception_vectors*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .rodata : - { - *(.rodata*) - } - . = ALIGN(65536); /* Fill up to 64 KiB */ - __ro_end = .; + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + __rw_start = .; + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - . = ALIGN(65536); - __data_end = .; + } :NONE + + . = ALIGN(64K); /* Align to page boundary */ + __rw_end_exclusive = .; - __ro_size = __ro_end - __ro_start; - __data_size = __data_end - __ro_end; + /*********************************************************************************************** + * Guard Page between boot core stack and data + ***********************************************************************************************/ + __boot_core_stack_guard_page_start = .; + . += 64K; + __boot_core_stack_guard_page_end_exclusive = .; - /DISCARD/ : { *(.comment*) } + /*********************************************************************************************** + * Boot Core Stack + ***********************************************************************************************/ + __boot_core_stack_start = .; + . += 512K; + __boot_core_stack_end_exclusive = .; } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs index 7b48d7b51..19545c003 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -8,31 +8,30 @@ //! copies the binary to 0x8_0000: //! //! +---------------------------------------------+ -//! | | 0x0 -//! | Unmapped | -//! | | 0x6_FFFF -//! +---------------------------------------------+ -//! | BOOT_CORE_STACK_START | 0x7_0000 -//! | | ^ -//! | ... | | Stack growth direction -//! | | | -//! | BOOT_CORE_STACK_END_INCLUSIVE | 0x7_FFFF -//! +---------------------------------------------+ -//! | RO_START == BOOT_CORE_STACK_END | 0x8_0000 //! | | +//! | Unmapped | //! | | +//! +---------------------------------------------+ +//! | | rx_start @ 0x8_0000 //! | .text | -//! | .exception_vectors | //! | .rodata | -//! | | -//! | RO_END_INCLUSIVE | 0x8_0000 + __ro_size - 1 +//! | .got | +//! | | rx_end_inclusive //! +---------------------------------------------+ -//! | RO_END == DATA_START | 0x8_0000 + __ro_size -//! | | +//! | | rw_start == rx_end //! | .data | //! | .bss | +//! | | rw_end_inclusive +//! +---------------------------------------------+ +//! | | rw_end +//! | Unmapped Boot-core Stack Guard Page | //! | | -//! | DATA_END_INCLUSIVE | 0x8_0000 + __ro_size + __data_size - 1 +//! +---------------------------------------------+ +//! | | boot_core_stack_start ^ +//! | | | stack +//! | Boot-core Stack | | growth +//! | | | direction +//! | | boot_core_stack_end_inclusive | //! +---------------------------------------------+ pub mod mmu; @@ -46,11 +45,19 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + + static __rw_start: UnsafeCell<()>; static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __ro_start: UnsafeCell<()>; - static __ro_size: UnsafeCell<()>; - static __data_size: UnsafeCell<()>; + static __rw_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_start: UnsafeCell<()>; + static __boot_core_stack_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_guard_page_start: UnsafeCell<()>; + static __boot_core_stack_guard_page_end_exclusive: UnsafeCell<()>; } //-------------------------------------------------------------------------------------------------- @@ -62,8 +69,6 @@ extern "Rust" { pub(super) mod map { use super::*; - pub const BOOT_CORE_STACK_SIZE: usize = 0x1_0000; - /// Physical devices. #[cfg(feature = "bsp_rpi3")] pub mod mmio { @@ -111,52 +116,69 @@ pub(super) mod map { // Private Code //-------------------------------------------------------------------------------------------------- -/// Start address of the Read-Only (RO) range. +/// Start address of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn virt_ro_start() -> Address { - Address::new(unsafe { __ro_start.get() as usize }) +fn virt_rx_start() -> Address { + Address::new(unsafe { __rx_start.get() as usize }) } -/// Size of the Read-Only (RO) range of the kernel binary. +/// Size of the Read+Execute (RX) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn ro_size() -> usize { - unsafe { __ro_size.get() as usize } +fn rx_size() -> usize { + unsafe { (__rx_end_exclusive.get() as usize) - (__rx_start.get() as usize) } } -/// Start address of the data range. +/// Start address of the Read+Write (RW) range. #[inline(always)] -fn virt_data_start() -> Address { - virt_ro_start() + ro_size() +fn virt_rw_start() -> Address { + Address::new(unsafe { __rw_start.get() as usize }) } -/// Size of the data range. +/// Size of the Read+Write (RW) range. /// /// # Safety /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] -fn data_size() -> usize { - unsafe { __data_size.get() as usize } +fn rw_size() -> usize { + unsafe { (__rw_end_exclusive.get() as usize) - (__rw_start.get() as usize) } } /// Start address of the boot core's stack. #[inline(always)] fn virt_boot_core_stack_start() -> Address { - virt_ro_start() - map::BOOT_CORE_STACK_SIZE + Address::new(unsafe { __boot_core_stack_start.get() as usize }) } /// Size of the boot core's stack. #[inline(always)] fn boot_core_stack_size() -> usize { - map::BOOT_CORE_STACK_SIZE + unsafe { + (__boot_core_stack_end_exclusive.get() as usize) - (__boot_core_stack_start.get() as usize) + } +} + +/// Start address of the boot core's stack guard page. +#[inline(always)] +fn virt_boot_core_stack_guard_page_start() -> Address { + Address::new(unsafe { __boot_core_stack_guard_page_start.get() as usize }) +} + +/// Size of the boot core's stack guard page. +#[inline(always)] +fn boot_core_stack_guard_page_size() -> usize { + unsafe { + (__boot_core_stack_guard_page_end_exclusive.get() as usize) + - (__boot_core_stack_guard_page_start.get() as usize) + } } /// Exclusive end address of the physical address space. diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs index 842384dea..88fc80bd5 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -60,42 +60,42 @@ const fn size_to_num_pages(size: usize) -> usize { size >> KernelGranule::SHIFT } -/// The boot core's stack. -fn virt_stack_page_desc() -> PageSliceDescriptor { - let num_pages = size_to_num_pages(super::boot_core_stack_size()); +/// The Read+Execute (RX) pages of the kernel binary. +fn virt_rx_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rx_size()); - PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) + PageSliceDescriptor::from_addr(super::virt_rx_start(), num_pages) } -/// The Read-Only (RO) pages of the kernel binary. -fn virt_ro_page_desc() -> PageSliceDescriptor { - let num_pages = size_to_num_pages(super::ro_size()); +/// The Read+Write (RW) pages of the kernel binary. +fn virt_rw_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rw_size()); - PageSliceDescriptor::from_addr(super::virt_ro_start(), num_pages) + PageSliceDescriptor::from_addr(super::virt_rw_start(), num_pages) } -/// The data pages of the kernel binary. -fn virt_data_page_desc() -> PageSliceDescriptor { - let num_pages = size_to_num_pages(super::data_size()); +/// The boot core's stack. +fn virt_boot_core_stack_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_size()); - PageSliceDescriptor::from_addr(super::virt_data_start(), num_pages) + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) } // The binary is still identity mapped, so we don't need to convert in the following. -/// The boot core's stack. -fn phys_stack_page_desc() -> PageSliceDescriptor { - virt_stack_page_desc().into() +/// The Read+Execute (RX) pages of the kernel binary. +fn phys_rx_page_desc() -> PageSliceDescriptor { + virt_rx_page_desc().into() } -/// The Read-Only (RO) pages of the kernel binary. -fn phys_ro_page_desc() -> PageSliceDescriptor { - virt_ro_page_desc().into() +/// The Read+Write (RW) pages of the kernel binary. +fn phys_rw_page_desc() -> PageSliceDescriptor { + virt_rw_page_desc().into() } -/// The data pages of the kernel binary. -fn phys_data_page_desc() -> PageSliceDescriptor { - virt_data_page_desc().into() +/// The boot core's stack. +fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { + virt_boot_core_stack_page_desc().into() } //-------------------------------------------------------------------------------------------------- @@ -107,6 +107,13 @@ pub fn kernel_translation_tables() -> &'static InitStateLock PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_guard_page_size()); + + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_guard_page_start(), num_pages) +} + /// Pointer to the last page of the physical address space. pub fn phys_addr_space_end_page() -> *const Page { common::align_down( @@ -122,31 +129,31 @@ pub fn phys_addr_space_end_page() -> *const Page { /// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { generic_mmu::kernel_map_pages_at( - "Kernel boot-core stack", - &virt_stack_page_desc(), - &phys_stack_page_desc(), + "Kernel code and RO data", + &virt_rx_page_desc(), + &phys_rx_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, - acc_perms: AccessPermissions::ReadWrite, - execute_never: true, + acc_perms: AccessPermissions::ReadOnly, + execute_never: false, }, )?; generic_mmu::kernel_map_pages_at( - "Kernel code and RO data", - &virt_ro_page_desc(), - &phys_ro_page_desc(), + "Kernel data and bss", + &virt_rw_page_desc(), + &phys_rw_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, - acc_perms: AccessPermissions::ReadOnly, - execute_never: false, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, }, )?; generic_mmu::kernel_map_pages_at( - "Kernel data and bss", - &virt_data_page_desc(), - &phys_data_page_desc(), + "Kernel boot-core stack", + &virt_boot_core_stack_page_desc(), + &phys_boot_core_stack_page_desc(), &AttributeFields { mem_attributes: MemAttributes::CacheableDRAM, acc_perms: AccessPermissions::ReadWrite, @@ -169,7 +176,13 @@ mod tests { /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { - for i in [virt_stack_page_desc, virt_ro_page_desc, virt_data_page_desc].iter() { + for i in [ + virt_rx_page_desc, + virt_rw_page_desc, + virt_boot_core_stack_page_desc, + ] + .iter() + { let start: usize = i().start_addr().into_usize(); let end: usize = i().end_addr().into_usize(); @@ -183,9 +196,9 @@ mod tests { #[kernel_test] fn virt_mem_layout_has_no_overlaps() { let layout = [ - virt_stack_page_desc(), - virt_ro_page_desc(), - virt_data_page_desc(), + virt_rx_page_desc(), + virt_rw_page_desc(), + virt_boot_core_stack_page_desc(), ]; for (i, first_range) in layout.iter().enumerate() { diff --git a/X1_JTAG_boot/Makefile b/X1_JTAG_boot/Makefile index 4206c5282..b5a56d075 100644 --- a/X1_JTAG_boot/Makefile +++ b/X1_JTAG_boot/Makefile @@ -122,6 +122,7 @@ objdump: $(KERNEL_ELF) @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ --section .rodata \ + --section .got \ $(KERNEL_ELF) | rustfilt nm: $(KERNEL_ELF) diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index de6648804151541fd045c79f2f5dcf824f6094db..d45cc1ff0028c6d32ecd176a828fc440da668d97 100755 GIT binary patch delta 478 zcmca$f5U!)C_4kg29AsFKQ~G!@-jMaw&MNFDEQE5A~yrW1RkK&D`$p@*@>I&__dkD z3;2pI@{0LpKG^1((6Hi$uF zCqQY9$&E7NnhDGh(E=!~0ks(H3LwDa Lj2n|z%BuqaNydb6 delta 496 zcmca%f5Co&C^O@Npp7z$yo|=1?RY;k3SKmt$j!hofro)%!z*WoiP?dho%pqx#1sW4 zdUBqe#RZh|dT2NC{r~?T-hZ^++$<=>2$EaVv!f`B^VF=@Ao0nJ6SF_sPF^jfC3aD9 z;=A?CD_`m}{Nw~`^{!nzTi}u1=GQ`8jDfe@e{!-q?DP~BG3I1t*qH(3%THz4>Gc2q zbP$`9VMY+=Oop9yP(Dab9?1Xae{jVC=J;zM%mFkLWG=@=_n(?qC%cJgGhUrsBvQ_J zaq=sXel@7WG&oPp0;$pjI{Y2bfp4`LR%Y$$4D#YxRCEsLiTR>RjEst#4~e~FWMu#X zmB||=#5Dt$fTjb11_OkafbyN3eH8+H3=9nwLLGxb7#OBZ{wN{NIAb%Xq&~Ca26>3$ z2pI^y1IllJ^1((6)`&r5d!V$+>yOnjx!urg~`XOI`qqN3N28798@|NleJN85?7jTu*R*RGoV_A|r8x5i8> zv;U@D1{yM*Ma4BOn1JelK!*WBGeG%H&b|r( zJ_d${3ZafcAq)&tCT|oKXPmM5q-Z&l;tVN>;sgl@y#UJZfbzjcTUv-hWIdp?2Gj}( zu!RgT3mIaVA?k9Vw8rF(65@#S delta 363 zcmdmCy1{gUC^O@Npp7!Jyo}15t$05(3LZ3?$j!hofro)%!z*WoiP?^u?fA8s7z-z7 z2`*+7oXjUAF8k1S;%j5ZmE5(fX21Q+F!8N1)5`3>X_tXoCiEmub`sie>goDZi`8Lg zH%miMmZ*ra7AwO}4xkt(!>S<8rwls}Fvnj5VGf`U4o8qU!vsy&$!5aZjINWDgpCBc;?iH$W}hFnOYsIsk1kQ^^1T diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld index 573abc5f0..87e6a976f 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld +++ b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld @@ -3,37 +3,47 @@ * Copyright (c) 2018-2021 Andre Richter */ +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + SECTIONS { - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; + . = __rpi_load_addr; + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; .text : { - *(.text._start) *(.text*) - } + KEEP(*(.text._start)) + *(.text*) + } :segment_rx - .rodata : - { - *(.rodata*) - } + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx - .data : - { - *(.data*) - } + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss ALIGN(8): + .bss : ALIGN(8) { __bss_start = .; *(.bss*); . = ALIGN(8); - /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - . += 8; + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; - } - - /DISCARD/ : { *(.comment*) } + } :NONE } diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs index fb47b1418..56a3306e5 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs @@ -12,6 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -20,10 +22,9 @@ extern "Rust" { // Public Definitions //-------------------------------------------------------------------------------------------------- -/// The board's memory map. +/// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; @@ -49,6 +50,20 @@ pub(super) mod map { } } +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_start() -> usize { + unsafe { __rx_start.get() as usize } +} + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -56,7 +71,7 @@ pub(super) mod map { /// Exclusive end address of the boot core's stack. #[inline(always)] pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END + rx_start() } /// Return the inclusive range spanning the .bss section. From 5887503f8a00dfcd76b3c1ce76c396e6b2d98f4d Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 16 Mar 2021 23:05:50 +0100 Subject: [PATCH 040/214] Fix explanation --- .../README.md | 13 ++++----- .../src/main.rs | 7 ++--- 12_exceptions_part1_groundwork/README.md | 4 +-- 12_exceptions_part1_groundwork/src/main.rs | 7 ++--- 13_integrated_testing/README.md | 12 ++++---- 13_integrated_testing/src/main.rs | 7 ++--- 14_exceptions_part2_peripheral_IRQs/README.md | 29 +++++++++++++------ .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/main.rs | 7 ++--- 15_virtual_mem_part2_mmio_remap/README.md | 15 ++++++++-- 15_virtual_mem_part2_mmio_remap/src/main.rs | 7 ++--- 11 files changed, 62 insertions(+), 48 deletions(-) diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index bebf518e9..8534ad881 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -1018,16 +1018,15 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s #![feature(format_args_nl)] #![feature(panic_info_message)] #![feature(trait_alias)] -@@ -132,9 +136,18 @@ +@@ -132,9 +136,17 @@ /// # Safety /// /// - Only a single core must be active and running this function. -/// - The init calls in this function must appear in the correct order. +/// - The init calls in this function must appear in the correct order: -+/// - Caching must be activated before the device drivers. -+/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -+/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -+/// the RPi SoCs. ++/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, ++/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ ++/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; + use memory::mmu::interface::MMU; @@ -1038,7 +1037,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -158,6 +171,9 @@ +@@ -158,6 +170,9 @@ info!("Booting on: {}", bsp::board_name()); @@ -1048,7 +1047,7 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -181,6 +197,13 @@ +@@ -181,6 +196,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index ccc842de4..9037541d1 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -137,10 +137,9 @@ mod time; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Caching must be activated before the device drivers. -/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -/// the RPi SoCs. +/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index b495742d6..99838f3eb 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -953,7 +953,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] -@@ -145,6 +146,8 @@ +@@ -144,6 +145,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -962,7 +962,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } -@@ -197,13 +200,28 @@ +@@ -196,13 +199,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index 8e8c089f1..aed3404a0 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -138,10 +138,9 @@ mod time; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Caching must be activated before the device drivers. -/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -/// the RPi SoCs. +/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 452c864e5..65423fc4c 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1505,15 +1505,15 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -142,6 +23,7 @@ - /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device - /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on - /// the RPi SoCs. +@@ -141,6 +22,7 @@ + /// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, + /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ + /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. +#[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -168,9 +50,7 @@ +@@ -167,9 +49,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1523,7 +1523,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -197,31 +77,6 @@ +@@ -196,31 +76,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/13_integrated_testing/src/main.rs b/13_integrated_testing/src/main.rs index 101124df8..77c9d494b 100644 --- a/13_integrated_testing/src/main.rs +++ b/13_integrated_testing/src/main.rs @@ -19,10 +19,9 @@ use libkernel::{bsp, console, driver, exception, info, memory, time}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Caching must be activated before the device drivers. -/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -/// the RPi SoCs. +/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 9ae718d77..9b4436b8c 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -857,6 +857,19 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception.rs 14_exceptions_par #[no_mangle] +diff -uNr 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +--- 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs ++++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +@@ -149,7 +149,7 @@ + barrier::isb(barrier::SY); + + // Enable the MMU and turn on data and instruction caching. +- SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); ++ SCTLR_EL1.modify(SCTLR_EL1::M::Enable); + + // Force MMU init to complete before next instruction. + barrier::isb(barrier::SY); + diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs --- 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -2325,18 +2338,16 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ /// Early init code. /// -@@ -21,8 +21,8 @@ +@@ -21,7 +21,7 @@ /// - The init calls in this function must appear in the correct order: - /// - Caching must be activated before the device drivers. - /// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device --/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on --/// the RPi SoCs. -+/// drivers (which currently employ IRQSafeNullLocks instead of spinlocks), will fail to -+/// work on the RPi SoCs. + /// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, + /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +-/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. ++/// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; -@@ -42,15 +42,27 @@ +@@ -41,15 +41,27 @@ bsp::driver::driver_manager().post_device_driver_init(); // println! is usable from here on. @@ -2366,7 +2377,7 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ info!("Booting on: {}", bsp::board_name()); -@@ -77,12 +89,9 @@ +@@ -76,12 +88,9 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 29e8125d7..cbf04a98e 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -149,7 +149,7 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { barrier::isb(barrier::SY); // Enable the MMU and turn on data and instruction caching. - SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); + SCTLR_EL1.modify(SCTLR_EL1::M::Enable); // Force MMU init to complete before next instruction. barrier::isb(barrier::SY); diff --git a/14_exceptions_part2_peripheral_IRQs/src/main.rs b/14_exceptions_part2_peripheral_IRQs/src/main.rs index 0bd9f71dd..6ec5a670c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/main.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/main.rs @@ -19,10 +19,9 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Caching must be activated before the device drivers. -/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ IRQSafeNullLocks instead of spinlocks), will fail to -/// work on the RPi SoCs. +/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 18c72b74c..5576de8fd 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -805,6 +805,15 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 self.configure_translation_control(); +@@ -149,7 +140,7 @@ + barrier::isb(barrier::SY); + + // Enable the MMU and turn on data and instruction caching. +- SCTLR_EL1.modify(SCTLR_EL1::M::Enable); ++ SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); + + // Force MMU init to complete before next instruction. + barrier::isb(barrier::SY); @@ -162,22 +153,3 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } @@ -1800,7 +1809,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v +//! | .got | +//! | | rx_end_inclusive +//! +---------------------------------------------+ -+//! | | rw_start == ro_end ++//! | | rw_start == rx_end +//! | .data | +//! | .bss | +//! | | rw_end_inclusive @@ -2150,7 +2159,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_mmio_remap/src/main.rs --- 14_exceptions_part2_peripheral_IRQs/src/main.rs +++ 15_virtual_mem_part2_mmio_remap/src/main.rs -@@ -26,21 +26,39 @@ +@@ -25,21 +25,39 @@ #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; @@ -2196,7 +2205,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_m // Let device drivers register and enable their handlers with the interrupt controller. for i in bsp::driver::driver_manager().all_device_drivers() { -@@ -66,8 +84,8 @@ +@@ -65,8 +83,8 @@ info!("Booting on: {}", bsp::board_name()); diff --git a/15_virtual_mem_part2_mmio_remap/src/main.rs b/15_virtual_mem_part2_mmio_remap/src/main.rs index 40d3ed410..0dd1c28d2 100644 --- a/15_virtual_mem_part2_mmio_remap/src/main.rs +++ b/15_virtual_mem_part2_mmio_remap/src/main.rs @@ -19,10 +19,9 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order: -/// - Caching must be activated before the device drivers. -/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -/// drivers (which currently employ IRQSafeNullLocks instead of spinlocks), will fail to -/// work on the RPi SoCs. +/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; From 6a9af3c2020d3092802ae35b51b22cc93e2aa1a9 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 16 Mar 2021 23:24:59 +0100 Subject: [PATCH 041/214] change static type --- 04_zero_overhead_abstraction/README.md | 2 +- 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs | 2 +- 05_safe_globals/src/bsp/raspberrypi/cpu.rs | 2 +- 06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs | 2 +- 07_uart_chainloader/src/bsp/raspberrypi/cpu.rs | 2 +- 08_timestamps/src/bsp/raspberrypi/cpu.rs | 2 +- 09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs | 2 +- 10_privilege_level/src/bsp/raspberrypi/cpu.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 2 +- 12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs | 2 +- 13_integrated_testing/src/bsp/raspberrypi/cpu.rs | 2 +- 14_exceptions_part2_peripheral_IRQs/README.md | 2 +- .../src/bsp/device_driver/arm/gicv2.rs | 2 +- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs | 2 +- 15_virtual_mem_part2_mmio_remap/README.md | 2 +- .../src/bsp/device_driver/arm/gicv2.rs | 2 +- 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs | 2 +- X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md index 358d01e71..69db5c72b 100644 --- a/04_zero_overhead_abstraction/README.md +++ b/04_zero_overhead_abstraction/README.md @@ -165,7 +165,7 @@ diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs 04_zero_overhead_abstr +//-------------------------------------------------------------------------------------------------- + +/// Used by `arch` code to find the early boot core. -+pub const BOOT_CORE_ID: usize = 0; ++pub const BOOT_CORE_ID: u64 = 0; diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/link.ld 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld --- 03_hacky_hello_world/src/bsp/raspberrypi/link.ld diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs +++ b/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/05_safe_globals/src/bsp/raspberrypi/cpu.rs b/05_safe_globals/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/05_safe_globals/src/bsp/raspberrypi/cpu.rs +++ b/05_safe_globals/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/08_timestamps/src/bsp/raspberrypi/cpu.rs b/08_timestamps/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/08_timestamps/src/bsp/raspberrypi/cpu.rs +++ b/08_timestamps/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/10_privilege_level/src/bsp/raspberrypi/cpu.rs b/10_privilege_level/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/10_privilege_level/src/bsp/raspberrypi/cpu.rs +++ b/10_privilege_level/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 9b4436b8c..54e33071c 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -1357,7 +1357,7 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2.rs 14_exceptions + } + + unsafe fn init(&self) -> Result<(), &'static str> { -+ if cpu::smp::core_id::() == bsp::cpu::BOOT_CORE_ID { ++ if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + self.gicd.boot_core_init(); + } + diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs index edbc41660..33713459c 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs @@ -139,7 +139,7 @@ impl driver::interface::DeviceDriver for GICv2 { } unsafe fn init(&self) -> Result<(), &'static str> { - if cpu::smp::core_id::() == bsp::cpu::BOOT_CORE_ID { + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { self.gicd.boot_core_init(); } diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index 5576de8fd..c512c51c6 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -1077,7 +1077,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs + self.is_mmio_remapped.store(true, Ordering::Relaxed); + } + - if cpu::smp::core_id::() == bsp::cpu::BOOT_CORE_ID { + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { self.gicd.boot_core_init(); } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs index c317abaea..51c321868 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs @@ -168,7 +168,7 @@ impl driver::interface::DeviceDriver for GICv2 { self.is_mmio_remapped.store(true, Ordering::Relaxed); } - if cpu::smp::core_id::() == bsp::cpu::BOOT_CORE_ID { + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { self.gicd.boot_core_init(); } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs index 3951618d7..16ab927d7 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,4 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: usize = 0; +pub const BOOT_CORE_ID: u64 = 0; From 2432c0d283883ea87b5f2e70eed36971d74d44df Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 20 Mar 2021 09:41:43 +0100 Subject: [PATCH 042/214] Revert to assembly boot code It is too risky to rely on the compiler to not insert any operations using the stack. Having a stack-setting call in Rust using the cortex-a crate as the first action in a Rust-only _start() function does not work if you're subsequently using the stack, because the compiler often inserts the operations to make room on the stack to prepare a function call BEFORE the call to set the stack, which crashes the boot process. Hence, keep on using a small piece of assembly boot code throughout. --- .editorconfig | 5 +- 01_wait_forever/Cargo.toml | 3 + 01_wait_forever/README.md | 2 +- 01_wait_forever/src/_arch/aarch64/cpu/boot.S | 11 - 01_wait_forever/src/_arch/aarch64/cpu/boot.rs | 4 +- 01_wait_forever/src/_arch/aarch64/cpu/boot.s | 20 + 01_wait_forever/src/bsp/raspberrypi/link.ld | 1 - 01_wait_forever/src/main.rs | 6 +- 02_runtime_init/Cargo.lock | 28 ++ 02_runtime_init/Cargo.toml | 4 + 02_runtime_init/README.md | 193 +++++++--- 02_runtime_init/src/_arch/aarch64/cpu.rs | 12 +- 02_runtime_init/src/_arch/aarch64/cpu/boot.S | 21 - 02_runtime_init/src/_arch/aarch64/cpu/boot.rs | 22 +- 02_runtime_init/src/_arch/aarch64/cpu/boot.s | 42 ++ 02_runtime_init/src/bsp/raspberrypi.rs | 1 + .../src/bsp/raspberrypi/cpu.rs | 4 +- 02_runtime_init/src/bsp/raspberrypi/link.ld | 16 +- 02_runtime_init/src/main.rs | 6 +- 02_runtime_init/src/runtime_init.rs | 1 - 03_hacky_hello_world/Cargo.lock | 28 ++ 03_hacky_hello_world/Cargo.toml | 5 + 03_hacky_hello_world/README.md | 29 +- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs | 12 +- .../src/_arch/aarch64/cpu/boot.S | 21 - .../src/_arch/aarch64/cpu/boot.rs | 22 +- .../src/_arch/aarch64/cpu/boot.s | 42 ++ 03_hacky_hello_world/src/bsp/raspberrypi.rs | 1 + .../src/bsp/raspberrypi/cpu.rs | 14 +- .../src/bsp/raspberrypi/link.ld | 16 +- 03_hacky_hello_world/src/main.rs | 6 +- 03_hacky_hello_world/src/runtime_init.rs | 1 - .../.vscode/settings.json | 7 - 04_zero_overhead_abstraction/Cargo.lock | 34 -- 04_zero_overhead_abstraction/Cargo.toml | 24 -- 04_zero_overhead_abstraction/Makefile | 115 ------ 04_zero_overhead_abstraction/README.md | 301 --------------- 04_zero_overhead_abstraction/build.rs | 8 - .../src/_arch/aarch64/cpu.rs | 26 -- .../src/_arch/aarch64/cpu/boot.rs | 41 -- .../src/_arch/aarch64/cpu/smp.rs | 29 -- 04_zero_overhead_abstraction/src/bsp.rs | 11 - .../src/bsp/raspberrypi.rs | 9 - .../src/bsp/raspberrypi/console.rs | 47 --- .../src/bsp/raspberrypi/link.ld | 49 --- .../src/bsp/raspberrypi/memory.rs | 59 --- 04_zero_overhead_abstraction/src/console.rs | 19 - 04_zero_overhead_abstraction/src/cpu.rs | 18 - 04_zero_overhead_abstraction/src/cpu/boot.rs | 9 - 04_zero_overhead_abstraction/src/main.rs | 133 ------- 04_zero_overhead_abstraction/src/memory.rs | 30 -- .../src/panic_wait.rs | 19 - 04_zero_overhead_abstraction/src/print.rs | 38 -- .../src/runtime_init.rs | 37 -- 05_safe_globals/src/_arch/aarch64/cpu/boot.rs | 26 +- 05_safe_globals/src/_arch/aarch64/cpu/boot.s | 42 ++ 05_safe_globals/src/_arch/aarch64/cpu/smp.rs | 29 -- 05_safe_globals/src/bsp/raspberrypi/cpu.rs | 4 +- 05_safe_globals/src/bsp/raspberrypi/link.ld | 17 +- 05_safe_globals/src/bsp/raspberrypi/memory.rs | 22 -- 05_safe_globals/src/cpu.rs | 2 - 05_safe_globals/src/cpu/smp.rs | 14 - 05_safe_globals/src/main.rs | 6 +- 06_drivers_gpio_uart/README.md | 15 +- .../src/_arch/aarch64/cpu/boot.rs | 26 +- .../src/_arch/aarch64/cpu/boot.s | 42 ++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 17 +- .../src/bsp/raspberrypi/memory.rs | 22 -- 06_drivers_gpio_uart/src/cpu.rs | 2 - 06_drivers_gpio_uart/src/cpu/smp.rs | 14 - 06_drivers_gpio_uart/src/main.rs | 6 +- 07_uart_chainloader/Makefile | 4 +- 07_uart_chainloader/README.md | 358 +++++------------- 07_uart_chainloader/demo_payload_rpi3.img | Bin 6824 -> 6840 bytes 07_uart_chainloader/demo_payload_rpi4.img | Bin 6672 -> 6680 bytes 07_uart_chainloader/src/_arch/aarch64/cpu.rs | 16 - .../src/_arch/aarch64/cpu/boot.rs | 26 +- .../src/_arch/aarch64/cpu/boot.s | 53 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 30 +- .../src/bsp/raspberrypi/memory.rs | 30 +- 07_uart_chainloader/src/cpu.rs | 4 +- 07_uart_chainloader/src/cpu/smp.rs | 14 - 07_uart_chainloader/src/main.rs | 12 +- 07_uart_chainloader/src/relocate.rs | 49 --- 07_uart_chainloader/src/runtime_init.rs | 3 +- 08_timestamps/README.md | 290 ++++---------- 08_timestamps/src/_arch/aarch64/cpu/boot.rs | 26 +- 08_timestamps/src/_arch/aarch64/cpu/boot.s | 42 ++ 08_timestamps/src/_arch/aarch64/cpu/smp.rs | 29 -- 08_timestamps/src/bsp/raspberrypi/cpu.rs | 4 +- 08_timestamps/src/bsp/raspberrypi/link.ld | 17 +- 08_timestamps/src/bsp/raspberrypi/memory.rs | 22 -- 08_timestamps/src/cpu.rs | 2 - 08_timestamps/src/cpu/smp.rs | 14 - 08_timestamps/src/main.rs | 6 +- 09_hw_debug_JTAG/README.md | 16 +- .../src/_arch/aarch64/cpu/boot.rs | 26 +- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s | 42 ++ 09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs | 29 -- 09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs | 4 +- 09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld | 17 +- .../src/bsp/raspberrypi/memory.rs | 22 -- 09_hw_debug_JTAG/src/cpu.rs | 2 - 09_hw_debug_JTAG/src/cpu/smp.rs | 14 - 09_hw_debug_JTAG/src/main.rs | 6 +- 10_privilege_level/README.md | 210 +++++----- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- 10_privilege_level/src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 17 +- .../src/bsp/raspberrypi/memory.rs | 22 -- 10_privilege_level/src/cpu.rs | 2 - 10_privilege_level/src/cpu/smp.rs | 14 - 10_privilege_level/src/main.rs | 6 +- .../README.md | 49 ++- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 16 +- .../src/bsp/raspberrypi/memory.rs | 6 - .../src/cpu.rs | 2 - .../src/cpu/smp.rs | 14 - .../src/main.rs | 6 +- 12_exceptions_part1_groundwork/README.md | 291 +++++++------- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- .../src/_arch/aarch64/exception.S | 138 ------- .../src/_arch/aarch64/exception.rs | 2 +- .../src/_arch/aarch64/exception.s | 148 ++++++++ .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 16 +- .../src/bsp/raspberrypi/memory.rs | 6 - 12_exceptions_part1_groundwork/src/cpu.rs | 2 - 12_exceptions_part1_groundwork/src/cpu/smp.rs | 14 - 12_exceptions_part1_groundwork/src/main.rs | 5 +- 13_integrated_testing/README.md | 26 +- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 -- .../src/_arch/aarch64/exception.S | 138 ------- .../src/_arch/aarch64/exception.rs | 2 +- .../src/_arch/aarch64/exception.s | 148 ++++++++ .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 16 +- .../src/bsp/raspberrypi/memory.rs | 6 - 13_integrated_testing/src/cpu.rs | 2 - 13_integrated_testing/src/cpu/smp.rs | 14 - 13_integrated_testing/src/lib.rs | 5 +- .../tests/00_console_sanity.rs | 4 +- 14_exceptions_part2_peripheral_IRQs/README.md | 83 +++- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/exception.S | 138 ------- .../src/_arch/aarch64/exception.rs | 2 +- .../src/_arch/aarch64/exception.s | 148 ++++++++ .../src/_arch/aarch64/memory/mmu.rs | 2 +- .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 16 +- .../src/bsp/raspberrypi/memory.rs | 6 - .../src/lib.rs | 5 +- .../tests/00_console_sanity.rs | 4 +- 15_virtual_mem_part2_mmio_remap/README.md | 63 +-- .../src/_arch/aarch64/cpu/boot.rs | 48 +-- .../src/_arch/aarch64/cpu/boot.s | 48 +++ .../src/_arch/aarch64/exception.S | 138 ------- .../src/_arch/aarch64/exception.rs | 2 +- .../src/_arch/aarch64/exception.s | 148 ++++++++ .../src/bsp/raspberrypi/cpu.rs | 4 +- .../src/bsp/raspberrypi/link.ld | 19 +- .../src/bsp/raspberrypi/memory.rs | 8 - 15_virtual_mem_part2_mmio_remap/src/lib.rs | 5 +- .../tests/00_console_sanity.rs | 4 +- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8152 -> 8176 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6840 -> 6848 bytes X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs | 26 +- X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s | 42 ++ X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs | 29 -- X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs | 4 +- X1_JTAG_boot/src/bsp/raspberrypi/link.ld | 17 +- X1_JTAG_boot/src/bsp/raspberrypi/memory.rs | 22 -- X1_JTAG_boot/src/cpu.rs | 2 - X1_JTAG_boot/src/cpu/smp.rs | 14 - X1_JTAG_boot/src/main.rs | 6 +- rust-toolchain | 2 +- 191 files changed, 2586 insertions(+), 3695 deletions(-) delete mode 100644 01_wait_forever/src/_arch/aarch64/cpu/boot.S create mode 100644 01_wait_forever/src/_arch/aarch64/cpu/boot.s delete mode 100644 02_runtime_init/src/_arch/aarch64/cpu/boot.S create mode 100644 02_runtime_init/src/_arch/aarch64/cpu/boot.s rename {04_zero_overhead_abstraction => 02_runtime_init}/src/bsp/raspberrypi/cpu.rs (82%) delete mode 100644 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S create mode 100644 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s rename 04_zero_overhead_abstraction/src/cpu/smp.rs => 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs (63%) delete mode 100644 04_zero_overhead_abstraction/.vscode/settings.json delete mode 100644 04_zero_overhead_abstraction/Cargo.lock delete mode 100644 04_zero_overhead_abstraction/Cargo.toml delete mode 100644 04_zero_overhead_abstraction/Makefile delete mode 100644 04_zero_overhead_abstraction/README.md delete mode 100644 04_zero_overhead_abstraction/build.rs delete mode 100644 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs delete mode 100644 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs delete mode 100644 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs delete mode 100644 04_zero_overhead_abstraction/src/bsp.rs delete mode 100644 04_zero_overhead_abstraction/src/bsp/raspberrypi.rs delete mode 100644 04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs delete mode 100644 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld delete mode 100644 04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs delete mode 100644 04_zero_overhead_abstraction/src/console.rs delete mode 100644 04_zero_overhead_abstraction/src/cpu.rs delete mode 100644 04_zero_overhead_abstraction/src/cpu/boot.rs delete mode 100644 04_zero_overhead_abstraction/src/main.rs delete mode 100644 04_zero_overhead_abstraction/src/memory.rs delete mode 100644 04_zero_overhead_abstraction/src/panic_wait.rs delete mode 100644 04_zero_overhead_abstraction/src/print.rs delete mode 100644 04_zero_overhead_abstraction/src/runtime_init.rs create mode 100644 05_safe_globals/src/_arch/aarch64/cpu/boot.s delete mode 100644 05_safe_globals/src/_arch/aarch64/cpu/smp.rs delete mode 100644 05_safe_globals/src/cpu/smp.rs create mode 100644 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s delete mode 100644 06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs delete mode 100644 06_drivers_gpio_uart/src/cpu/smp.rs create mode 100644 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s delete mode 100644 07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs delete mode 100644 07_uart_chainloader/src/cpu/smp.rs delete mode 100644 07_uart_chainloader/src/relocate.rs create mode 100644 08_timestamps/src/_arch/aarch64/cpu/boot.s delete mode 100644 08_timestamps/src/_arch/aarch64/cpu/smp.rs delete mode 100644 08_timestamps/src/cpu/smp.rs create mode 100644 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s delete mode 100644 09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs delete mode 100644 09_hw_debug_JTAG/src/cpu/smp.rs create mode 100644 10_privilege_level/src/_arch/aarch64/cpu/boot.s delete mode 100644 10_privilege_level/src/_arch/aarch64/cpu/smp.rs delete mode 100644 10_privilege_level/src/cpu/smp.rs create mode 100644 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s delete mode 100644 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs delete mode 100644 11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs create mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s delete mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs delete mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S create mode 100644 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s delete mode 100644 12_exceptions_part1_groundwork/src/cpu/smp.rs create mode 100644 13_integrated_testing/src/_arch/aarch64/cpu/boot.s delete mode 100644 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs delete mode 100644 13_integrated_testing/src/_arch/aarch64/exception.S create mode 100644 13_integrated_testing/src/_arch/aarch64/exception.s delete mode 100644 13_integrated_testing/src/cpu/smp.rs create mode 100644 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s delete mode 100644 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S create mode 100644 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s create mode 100644 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s delete mode 100644 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S create mode 100644 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s create mode 100644 X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s delete mode 100644 X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs delete mode 100644 X1_JTAG_boot/src/cpu/smp.rs diff --git a/.editorconfig b/.editorconfig index 38ed8df1c..dd8892ca7 100644 --- a/.editorconfig +++ b/.editorconfig @@ -23,8 +23,9 @@ indent_size = 4 [*.rs] indent_size = 4 -[*.S] -indent_size = 4 +[*.s] +indent_style = tab +indent_size = 8 [*.sh] indent_size = 4 diff --git a/01_wait_forever/Cargo.toml b/01_wait_forever/Cargo.toml index 8a0187c43..a75809963 100644 --- a/01_wait_forever/Cargo.toml +++ b/01_wait_forever/Cargo.toml @@ -4,6 +4,9 @@ version = "0.1.0" authors = ["Andre Richter "] edition = "2018" +[profile.release] +lto = true + [features] default = [] bsp_rpi3 = [] diff --git a/01_wait_forever/README.md b/01_wait_forever/README.md index 66dc29d29..bf214b68f 100644 --- a/01_wait_forever/README.md +++ b/01_wait_forever/README.md @@ -23,7 +23,7 @@ - Only `.text` section. - `main.rs`: Important [inner attributes]: - `#![no_std]`, `#![no_main]` -- `boot.S`: Assembly `_start()` function that executes `wfe` (Wait For Event), halting all cores +- `boot.s`: Assembly `_start()` function that executes `wfe` (Wait For Event), halting all cores that are executing `_start()`. - We (have to) define a `#[panic_handler]` function to make the compiler happy. - Make it `unimplemented!()` because it will be stripped out since it is not used. diff --git a/01_wait_forever/src/_arch/aarch64/cpu/boot.S b/01_wait_forever/src/_arch/aarch64/cpu/boot.S deleted file mode 100644 index 2de0c61c5..000000000 --- a/01_wait_forever/src/_arch/aarch64/cpu/boot.S +++ /dev/null @@ -1,11 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -.section ".text._start" - -.global _start - -_start: -1: wfe // Wait for event - b 1b // In case an event happened, jump back to 1 diff --git a/01_wait_forever/src/_arch/aarch64/cpu/boot.rs b/01_wait_forever/src/_arch/aarch64/cpu/boot.rs index c3c325d3a..4bc6ed6da 100644 --- a/01_wait_forever/src/_arch/aarch64/cpu/boot.rs +++ b/01_wait_forever/src/_arch/aarch64/cpu/boot.rs @@ -11,5 +11,5 @@ //! //! crate::cpu::boot::arch_boot -// Assembly counterpart to this file. Includes function _start(). -global_asm!(include_str!("boot.S")); +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); diff --git a/01_wait_forever/src/_arch/aarch64/cpu/boot.s b/01_wait_forever/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..fb05382cd --- /dev/null +++ b/01_wait_forever/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/01_wait_forever/src/bsp/raspberrypi/link.ld b/01_wait_forever/src/bsp/raspberrypi/link.ld index 7d649f884..60c393d6b 100644 --- a/01_wait_forever/src/bsp/raspberrypi/link.ld +++ b/01_wait_forever/src/bsp/raspberrypi/link.ld @@ -23,6 +23,5 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) } :segment_rx } diff --git a/01_wait_forever/src/main.rs b/01_wait_forever/src/main.rs index 8cd4b1441..b01a84cb0 100644 --- a/01_wait_forever/src/main.rs +++ b/01_wait_forever/src/main.rs @@ -100,10 +100,8 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. -//! -//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. #![feature(asm)] #![feature(global_asm)] diff --git a/02_runtime_init/Cargo.lock b/02_runtime_init/Cargo.lock index 3e9c31110..3b926a347 100644 --- a/02_runtime_init/Cargo.lock +++ b/02_runtime_init/Cargo.lock @@ -1,6 +1,34 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + +[[package]] +name = "cortex-a" +version = "5.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +dependencies = [ + "register", +] + [[package]] name = "kernel" version = "0.1.0" +dependencies = [ + "cortex-a", +] +[[package]] +name = "register" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" +dependencies = [ + "tock-registers", +] + +[[package]] +name = "tock-registers" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" diff --git a/02_runtime_init/Cargo.toml b/02_runtime_init/Cargo.toml index a75809963..4a03b9623 100644 --- a/02_runtime_init/Cargo.toml +++ b/02_runtime_init/Cargo.toml @@ -17,3 +17,7 @@ bsp_rpi4 = [] ##-------------------------------------------------------------------------------------------------- [dependencies] + +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 5f5245d57..64e36440e 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -2,22 +2,28 @@ ## tl;dr -- We extend `boot.S` to call into Rust code for the first time. There, we zero the [bss] section +- We extend `boot.s` to call into Rust code for the first time. There, we zero the [bss] section before execution is halted with a call to `panic()`. - Check out `make qemu` again to see the additional code run. ## Notable additions -- More sections in linker script: - - `.rodata`, `.got`, `.data`, `.bss` -- `_start()`: - - Halt core if core != `core0`. - - `core0` jumps to the `runtime_init()` Rust function. -- `runtime_init()` in `runtime_init.rs` +- More additions to the linker script: + - New sections: `.rodata`, `.got`, `.data`, `.bss`. + - A dedicated place for linking boot-time arguments that need to be read by `_start()`. +- `_start()` in `_arch/__arch_name__/cpu/boot.s`: + 1. Halt core if core != core0. + 1. Set up the `stack pointer`. + 1. Jump to the `_start_rust()` function, defined in `arch/__arch_name__/cpu/boot.rs`. +- `runtime_init()` in `runtime_init.rs`: - Zeros the `.bss` section. - - Calls `kernel_init()`, which calls `panic!()`, which eventually halts `core0` as well. + - Calls `kernel_init()`, which calls `panic!()`, which eventually halts core0 as well. +- The library now uses the [cortex-a] crate, which provides zero-overhead abstractions and wraps + `unsafe` parts when dealing with the CPU's resources. + - See it in action in `_arch/__arch_name__/cpu.rs`. [bss]: https://en.wikipedia.org/wiki/.bss +[cortex-a]: https://github.com/rust-embedded/cortex-a ## Diff to previous ```diff @@ -25,16 +31,14 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml --- 01_wait_forever/Cargo.toml +++ 02_runtime_init/Cargo.toml -@@ -4,6 +4,9 @@ - authors = ["Andre Richter "] - edition = "2018" +@@ -17,3 +17,7 @@ + ##-------------------------------------------------------------------------------------------------- -+[profile.release] -+lto = true + [dependencies] + - [features] - default = [] - bsp_rpi3 = [] ++# Platform specific dependencies ++[target.'cfg(target_arch = "aarch64")'.dependencies] ++cortex-a = { version = "5.x.x" } diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile --- 01_wait_forever/Makefile @@ -49,32 +53,78 @@ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile nm: $(KERNEL_ELF) -diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.S 02_runtime_init/src/_arch/aarch64/cpu/boot.S ---- 01_wait_forever/src/_arch/aarch64/cpu/boot.S -+++ 02_runtime_init/src/_arch/aarch64/cpu/boot.S -@@ -7,5 +7,15 @@ - .global _start +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.rs 02_runtime_init/src/_arch/aarch64/cpu/boot.rs +--- 01_wait_forever/src/_arch/aarch64/cpu/boot.rs ++++ 02_runtime_init/src/_arch/aarch64/cpu/boot.rs +@@ -11,5 +11,23 @@ + //! + //! crate::cpu::boot::arch_boot + ++use crate::runtime_init; ++ + // Assembly counterpart to this file. + global_asm!(include_str!("boot.s")); ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// The Rust entry of the `kernel` binary. ++/// ++/// The function is called from the assembly `_start` function. ++/// ++/// # Safety ++/// ++/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. ++#[no_mangle] ++pub unsafe fn _start_rust() -> ! { ++ runtime_init::runtime_init() ++} +diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch/aarch64/cpu/boot.s +--- 01_wait_forever/src/_arch/aarch64/cpu/boot.s ++++ 02_runtime_init/src/_arch/aarch64/cpu/boot.s +@@ -3,6 +3,12 @@ + // Copyright (c) 2021 Andre Richter + + //-------------------------------------------------------------------------------------------------- ++// Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++.equ _core_id_mask, 0b11 ++ ++//-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- + .section .text._start +@@ -11,6 +17,22 @@ + // fn _start() + //------------------------------------------------------------------------------ _start: --1: wfe // Wait for event -- b 1b // In case an event happened, jump back to 1 -+ mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register -+ and x1, x1, #3 // Clear all bits except [1:0], which hold core id -+ cbz x1, 2f // Jump to label 2 if we are core 0 -+1: wfe // Wait for event -+ b 1b // In case an event happened, jump back to 1 -+2: // If we are here, we are core0 -+ ldr x1, =_start // Load address of function "_start()" -+ mov sp, x1 // Set start of stack to before our code, aka first -+ // address before "_start()" -+ bl runtime_init // Jump to the "runtime_init()" kernel function -+ b 1b // We should never reach here. But just in case, -+ // park this core aswell ++ // Only proceed on the boot core. Park it otherwise. ++ mrs x1, MPIDR_EL1 ++ and x1, x1, _core_id_mask ++ ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs ++ cmp x1, x2 ++ b.ne 1f ++ ++ // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. ++ ++ // Set the stack pointer. ++ ldr x0, =__boot_core_stack_end_exclusive ++ mov sp, x0 ++ ++ // Jump to Rust code. ++ b _start_rust ++ + // Infinitely wait for events (aka "park the core"). + 1: wfe + b 1b diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs --- 01_wait_forever/src/_arch/aarch64/cpu.rs +++ 02_runtime_init/src/_arch/aarch64/cpu.rs -@@ -0,0 +1,30 @@ +@@ -0,0 +1,26 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -88,6 +138,8 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aar +//! +//! crate::cpu::arch_cpu + ++use cortex_a::asm; ++ +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- @@ -95,21 +147,34 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aar +/// Pause execution on the core. +#[inline(always)] +pub fn wait_forever() -> ! { -+ unsafe { -+ loop { -+ #[rustfmt::skip] -+ asm!( -+ "wfe", -+ options(nomem, nostack, preserves_flags) -+ ); -+ } ++ loop { ++ asm::wfe() + } +} +diff -uNr 01_wait_forever/src/bsp/raspberrypi/cpu.rs 02_runtime_init/src/bsp/raspberrypi/cpu.rs +--- 01_wait_forever/src/bsp/raspberrypi/cpu.rs ++++ 02_runtime_init/src/bsp/raspberrypi/cpu.rs +@@ -0,0 +1,14 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! BSP Processor code. ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++/// Used by `arch` code to find the early boot core. ++#[no_mangle] ++#[link_section = ".text._start_arguments"] ++pub static BOOT_CORE_ID: u64 = 0; + diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld -@@ -11,6 +11,7 @@ +@@ -11,17 +11,52 @@ PHDRS { segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ @@ -117,8 +182,13 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/ra } SECTIONS -@@ -18,11 +19,30 @@ + { . = __rpi_load_addr; ++ /* ^ */ ++ /* | stack */ ++ /* | growth */ ++ /* | direction */ ++ __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** - * Code @@ -127,7 +197,16 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/ra .text : { KEEP(*(.text._start)) - *(.text*) ++ ++ /* Special constants (or statics in Rust speak) needed by _start(). ++ * ++ * They are placed in close proximity to _start() from where they will be read. This ensures ++ * that position-independent, PC-relative loads can be emitted. ++ */ ++ *(.text._start_arguments) ++ ++ *(.text._start_rust) /* The Rust entry point */ ++ *(.text*) /* Everything else */ } :segment_rx + + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx @@ -195,11 +274,12 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/memory.rs 02_runtime_init/src/bsp/ diff -uNr 01_wait_forever/src/bsp/raspberrypi.rs 02_runtime_init/src/bsp/raspberrypi.rs --- 01_wait_forever/src/bsp/raspberrypi.rs +++ 02_runtime_init/src/bsp/raspberrypi.rs -@@ -4,4 +4,4 @@ +@@ -4,4 +4,5 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -// Coming soon. ++pub mod cpu; +pub mod memory; diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs @@ -223,18 +303,18 @@ diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs --- 01_wait_forever/src/main.rs +++ 02_runtime_init/src/main.rs -@@ -102,8 +102,10 @@ +@@ -102,14 +102,25 @@ //! - //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. - //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. + //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. - //! - //! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html ++//! +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - #![feature(asm)] +-#![feature(asm)] #![feature(global_asm)] -@@ -112,6 +114,15 @@ + #![no_main] + #![no_std] mod bsp; mod cpu; @@ -306,7 +386,7 @@ diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.rs --- 01_wait_forever/src/runtime_init.rs +++ 02_runtime_init/src/runtime_init.rs -@@ -0,0 +1,38 @@ +@@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -339,7 +419,6 @@ diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.r +/// # Safety +/// +/// - Only a single core must be active and running this function. -+#[no_mangle] +pub unsafe fn runtime_init() -> ! { + zero_bss(); + diff --git a/02_runtime_init/src/_arch/aarch64/cpu.rs b/02_runtime_init/src/_arch/aarch64/cpu.rs index 4fd7e3132..5f97ea410 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu.rs +++ b/02_runtime_init/src/_arch/aarch64/cpu.rs @@ -11,6 +11,8 @@ //! //! crate::cpu::arch_cpu +use cortex_a::asm; + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -18,13 +20,7 @@ /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { - unsafe { - loop { - #[rustfmt::skip] - asm!( - "wfe", - options(nomem, nostack, preserves_flags) - ); - } + loop { + asm::wfe() } } diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.S b/02_runtime_init/src/_arch/aarch64/cpu/boot.S deleted file mode 100644 index 449cf5027..000000000 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.S +++ /dev/null @@ -1,21 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -.section ".text._start" - -.global _start - -_start: - mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register - and x1, x1, #3 // Clear all bits except [1:0], which hold core id - cbz x1, 2f // Jump to label 2 if we are core 0 -1: wfe // Wait for event - b 1b // In case an event happened, jump back to 1 -2: // If we are here, we are core0 - ldr x1, =_start // Load address of function "_start()" - mov sp, x1 // Set start of stack to before our code, aka first - // address before "_start()" - bl runtime_init // Jump to the "runtime_init()" kernel function - b 1b // We should never reach here. But just in case, - // park this core aswell diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.rs b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs index c3c325d3a..c85bb94b8 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.rs +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs @@ -11,5 +11,23 @@ //! //! crate::cpu::boot::arch_boot -// Assembly counterpart to this file. Includes function _start(). -global_asm!(include_str!("boot.S")); +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The Rust entry of the `kernel` binary. +/// +/// The function is called from the assembly `_start` function. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +#[no_mangle] +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() +} diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.s b/02_runtime_init/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/02_runtime_init/src/bsp/raspberrypi.rs b/02_runtime_init/src/bsp/raspberrypi.rs index f9bd5b888..10535d7b8 100644 --- a/02_runtime_init/src/bsp/raspberrypi.rs +++ b/02_runtime_init/src/bsp/raspberrypi.rs @@ -4,4 +4,5 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. +pub mod cpu; pub mod memory; diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs b/02_runtime_init/src/bsp/raspberrypi/cpu.rs similarity index 82% rename from 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs rename to 02_runtime_init/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs +++ b/02_runtime_init/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/02_runtime_init/src/bsp/raspberrypi/link.ld b/02_runtime_init/src/bsp/raspberrypi/link.ld index acb7fe28b..3b73b3c7e 100644 --- a/02_runtime_init/src/bsp/raspberrypi/link.ld +++ b/02_runtime_init/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -24,7 +29,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/02_runtime_init/src/main.rs b/02_runtime_init/src/main.rs index fd9c36bc3..faad6b091 100644 --- a/02_runtime_init/src/main.rs +++ b/02_runtime_init/src/main.rs @@ -100,14 +100,12 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![feature(asm)] #![feature(global_asm)] #![no_main] #![no_std] diff --git a/02_runtime_init/src/runtime_init.rs b/02_runtime_init/src/runtime_init.rs index 068dde843..ee0946863 100644 --- a/02_runtime_init/src/runtime_init.rs +++ b/02_runtime_init/src/runtime_init.rs @@ -30,7 +30,6 @@ unsafe fn zero_bss() { /// # Safety /// /// - Only a single core must be active and running this function. -#[no_mangle] pub unsafe fn runtime_init() -> ! { zero_bss(); diff --git a/03_hacky_hello_world/Cargo.lock b/03_hacky_hello_world/Cargo.lock index 3e9c31110..3b926a347 100644 --- a/03_hacky_hello_world/Cargo.lock +++ b/03_hacky_hello_world/Cargo.lock @@ -1,6 +1,34 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + +[[package]] +name = "cortex-a" +version = "5.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +dependencies = [ + "register", +] + [[package]] name = "kernel" version = "0.1.0" +dependencies = [ + "cortex-a", +] +[[package]] +name = "register" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" +dependencies = [ + "tock-registers", +] + +[[package]] +name = "tock-registers" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" diff --git a/03_hacky_hello_world/Cargo.toml b/03_hacky_hello_world/Cargo.toml index a75809963..5deb70108 100644 --- a/03_hacky_hello_world/Cargo.toml +++ b/03_hacky_hello_world/Cargo.toml @@ -17,3 +17,8 @@ bsp_rpi4 = [] ##-------------------------------------------------------------------------------------------------- [dependencies] + +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } + diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 753aa0487..a002a6198 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -4,8 +4,8 @@ - Introducing global `print!()` macros to enable "printf debugging" at the earliest. - To keep tutorial length reasonable, printing functions for now "abuse" a QEMU property that lets - us use the RPi's `UART` without setting it up properly. -- Using the real hardware `UART` is enabled step-by-step in following tutorials. + us use the Raspberry's `UART` without setting it up properly. +- Using the real hardware `UART` is enabled step-by-step in following tutorials. ## Notable additions @@ -28,6 +28,15 @@ Kernel panic: Stopping here. ## Diff to previous ```diff +diff -uNr 02_runtime_init/Cargo.toml 03_hacky_hello_world/Cargo.toml +--- 02_runtime_init/Cargo.toml ++++ 03_hacky_hello_world/Cargo.toml +@@ -21,3 +21,4 @@ + # Platform specific dependencies + [target.'cfg(target_arch = "aarch64")'.dependencies] + cortex-a = { version = "5.x.x" } ++ + diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile --- 02_runtime_init/Makefile +++ 03_hacky_hello_world/Makefile @@ -105,11 +114,12 @@ diff -uNr 02_runtime_init/src/bsp/raspberrypi/console.rs 03_hacky_hello_world/sr diff -uNr 02_runtime_init/src/bsp/raspberrypi.rs 03_hacky_hello_world/src/bsp/raspberrypi.rs --- 02_runtime_init/src/bsp/raspberrypi.rs +++ 03_hacky_hello_world/src/bsp/raspberrypi.rs -@@ -4,4 +4,5 @@ +@@ -4,5 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. +pub mod console; + pub mod cpu; pub mod memory; diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs @@ -139,19 +149,10 @@ diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs --- 02_runtime_init/src/main.rs +++ 03_hacky_hello_world/src/main.rs -@@ -101,21 +101,25 @@ - //! # Boot flow - //! - //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. --//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.S`. -+//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. - //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +@@ -106,14 +106,18 @@ //! --//! [`cpu::boot::arch_boot::_start()`]: ../src/kernel/cpu/up/_arch/aarch64/cpu/boot.rs.html -+//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - #![feature(asm)] +#![feature(format_args_nl)] #![feature(global_asm)] +#![feature(panic_info_message)] @@ -167,7 +168,7 @@ diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs mod runtime_init; /// Early init code. -@@ -124,5 +128,7 @@ +@@ -122,5 +126,7 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs index 4fd7e3132..5f97ea410 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu.rs +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu.rs @@ -11,6 +11,8 @@ //! //! crate::cpu::arch_cpu +use cortex_a::asm; + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -18,13 +20,7 @@ /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { - unsafe { - loop { - #[rustfmt::skip] - asm!( - "wfe", - options(nomem, nostack, preserves_flags) - ); - } + loop { + asm::wfe() } } diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S deleted file mode 100644 index 449cf5027..000000000 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S +++ /dev/null @@ -1,21 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -.section ".text._start" - -.global _start - -_start: - mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register - and x1, x1, #3 // Clear all bits except [1:0], which hold core id - cbz x1, 2f // Jump to label 2 if we are core 0 -1: wfe // Wait for event - b 1b // In case an event happened, jump back to 1 -2: // If we are here, we are core0 - ldr x1, =_start // Load address of function "_start()" - mov sp, x1 // Set start of stack to before our code, aka first - // address before "_start()" - bl runtime_init // Jump to the "runtime_init()" kernel function - b 1b // We should never reach here. But just in case, - // park this core aswell diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs index c3c325d3a..c85bb94b8 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs @@ -11,5 +11,23 @@ //! //! crate::cpu::boot::arch_boot -// Assembly counterpart to this file. Includes function _start(). -global_asm!(include_str!("boot.S")); +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The Rust entry of the `kernel` binary. +/// +/// The function is called from the assembly `_start` function. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +#[no_mangle] +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() +} diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/03_hacky_hello_world/src/bsp/raspberrypi.rs b/03_hacky_hello_world/src/bsp/raspberrypi.rs index ac9d53e95..fdb7551a1 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi.rs @@ -5,4 +5,5 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. pub mod console; +pub mod cpu; pub mod memory; diff --git a/04_zero_overhead_abstraction/src/cpu/smp.rs b/03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs similarity index 63% rename from 04_zero_overhead_abstraction/src/cpu/smp.rs rename to 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs index 38230ce1e..d09ff9568 100644 --- a/04_zero_overhead_abstraction/src/cpu/smp.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs @@ -2,13 +2,13 @@ // // Copyright (c) 2018-2021 Andre Richter -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; +//! BSP Processor code. //-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports +// Public Definitions //-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; + +/// Used by `arch` code to find the early boot core. +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld index acb7fe28b..3b73b3c7e 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld +++ b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -24,7 +29,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/03_hacky_hello_world/src/main.rs b/03_hacky_hello_world/src/main.rs index 0815d5e21..bc290ddde 100644 --- a/03_hacky_hello_world/src/main.rs +++ b/03_hacky_hello_world/src/main.rs @@ -100,14 +100,12 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![feature(asm)] #![feature(format_args_nl)] #![feature(global_asm)] #![feature(panic_info_message)] diff --git a/03_hacky_hello_world/src/runtime_init.rs b/03_hacky_hello_world/src/runtime_init.rs index 068dde843..ee0946863 100644 --- a/03_hacky_hello_world/src/runtime_init.rs +++ b/03_hacky_hello_world/src/runtime_init.rs @@ -30,7 +30,6 @@ unsafe fn zero_bss() { /// # Safety /// /// - Only a single core must be active and running this function. -#[no_mangle] pub unsafe fn runtime_init() -> ! { zero_bss(); diff --git a/04_zero_overhead_abstraction/.vscode/settings.json b/04_zero_overhead_abstraction/.vscode/settings.json deleted file mode 100644 index a0d6a9202..000000000 --- a/04_zero_overhead_abstraction/.vscode/settings.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "editor.formatOnSave": true, - "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], - "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] -} diff --git a/04_zero_overhead_abstraction/Cargo.lock b/04_zero_overhead_abstraction/Cargo.lock deleted file mode 100644 index 3b926a347..000000000 --- a/04_zero_overhead_abstraction/Cargo.lock +++ /dev/null @@ -1,34 +0,0 @@ -# This file is automatically @generated by Cargo. -# It is not intended for manual editing. -version = 3 - -[[package]] -name = "cortex-a" -version = "5.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" -dependencies = [ - "register", -] - -[[package]] -name = "kernel" -version = "0.1.0" -dependencies = [ - "cortex-a", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - -[[package]] -name = "tock-registers" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" diff --git a/04_zero_overhead_abstraction/Cargo.toml b/04_zero_overhead_abstraction/Cargo.toml deleted file mode 100644 index 5deb70108..000000000 --- a/04_zero_overhead_abstraction/Cargo.toml +++ /dev/null @@ -1,24 +0,0 @@ -[package] -name = "kernel" -version = "0.1.0" -authors = ["Andre Richter "] -edition = "2018" - -[profile.release] -lto = true - -[features] -default = [] -bsp_rpi3 = [] -bsp_rpi4 = [] - -##-------------------------------------------------------------------------------------------------- -## Dependencies -##-------------------------------------------------------------------------------------------------- - -[dependencies] - -# Platform specific dependencies -[target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } - diff --git a/04_zero_overhead_abstraction/Makefile b/04_zero_overhead_abstraction/Makefile deleted file mode 100644 index 2ed82a5f1..000000000 --- a/04_zero_overhead_abstraction/Makefile +++ /dev/null @@ -1,115 +0,0 @@ -## SPDX-License-Identifier: MIT OR Apache-2.0 -## -## Copyright (c) 2018-2021 Andre Richter - -include ../utils/color.mk.in - -# Default to the RPi3 -BSP ?= rpi3 - -# BSP-specific arguments -ifeq ($(BSP),rpi3) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img - QEMU_BINARY = qemu-system-aarch64 - QEMU_MACHINE_TYPE = raspi3 - QEMU_RELEASE_ARGS = -serial stdio -display none - OBJDUMP_BINARY = aarch64-none-elf-objdump - NM_BINARY = aarch64-none-elf-nm - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -else ifeq ($(BSP),rpi4) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img - QEMU_BINARY = qemu-system-aarch64 - QEMU_MACHINE_TYPE = - QEMU_RELEASE_ARGS = -serial stdio -display none - OBJDUMP_BINARY = aarch64-none-elf-objdump - NM_BINARY = aarch64-none-elf-nm - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -endif - -# Export for build.rs -export LINKER_FILE - -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." - -RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) -RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs - -FEATURES = --features bsp_$(BSP) -COMPILER_ARGS = --target=$(TARGET) \ - $(FEATURES) \ - --release - -RUSTC_CMD = cargo rustc $(COMPILER_ARGS) -DOC_CMD = cargo doc $(COMPILER_ARGS) -CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) -CHECK_CMD = cargo check $(COMPILER_ARGS) -OBJCOPY_CMD = rust-objcopy \ - --strip-all \ - -O binary - -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t - -DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) - -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) - -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check - -all: $(KERNEL_BIN) - -$(KERNEL_ELF): - $(call colorecho, "\nCompiling kernel - $(BSP)") - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) - -$(KERNEL_BIN): $(KERNEL_ELF) - @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) - -doc: - $(call colorecho, "\nGenerating docs") - @$(DOC_CMD) --document-private-items --open - -ifeq ($(QEMU_MACHINE_TYPE),) -qemu: - $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else -qemu: $(KERNEL_BIN) - $(call colorecho, "\nLaunching QEMU") - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -endif - -clippy: - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) - -clean: - rm -rf target $(KERNEL_BIN) - -readelf: $(KERNEL_ELF) - $(call colorecho, "\nLaunching readelf") - @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) - -objdump: $(KERNEL_ELF) - $(call colorecho, "\nLaunching objdump") - @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ - --section .text \ - --section .rodata \ - --section .got \ - $(KERNEL_ELF) | rustfilt - -nm: $(KERNEL_ELF) - $(call colorecho, "\nLaunching nm") - @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt - -# For rust-analyzer -check: - @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json diff --git a/04_zero_overhead_abstraction/README.md b/04_zero_overhead_abstraction/README.md deleted file mode 100644 index 69db5c72b..000000000 --- a/04_zero_overhead_abstraction/README.md +++ /dev/null @@ -1,301 +0,0 @@ -# Tutorial 04 - Zero Overhead Abstraction - -## tl;dr - -- All hand-written assembly is replaced by Rust code from the [cortex-a] crate, which provides - zero-overhead abstractions and wraps the `unsafe` parts. - -[cortex-a]: https://github.com/rust-embedded/cortex-a - -## Diff to previous -```diff - -diff -uNr 03_hacky_hello_world/Cargo.toml 04_zero_overhead_abstraction/Cargo.toml ---- 03_hacky_hello_world/Cargo.toml -+++ 04_zero_overhead_abstraction/Cargo.toml -@@ -17,3 +17,8 @@ - ##-------------------------------------------------------------------------------------------------- - - [dependencies] -+ -+# Platform specific dependencies -+[target.'cfg(target_arch = "aarch64")'.dependencies] -+cortex-a = { version = "5.x.x" } -+ - -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs ---- 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs -@@ -11,5 +11,31 @@ - //! - //! crate::cpu::boot::arch_boot - --// Assembly counterpart to this file. Includes function _start(). --global_asm!(include_str!("boot.S")); -+use crate::{bsp, cpu}; -+use cortex_a::regs::*; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// The entry of the `kernel` binary. -+/// -+/// The function must be named `_start`, because the linker is looking for this exact name. -+/// -+/// # Safety -+/// -+/// - Linker script must ensure to place this function where it is expected by the target machine. -+/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -+/// actually set (`SP.set()`). -+#[no_mangle] -+pub unsafe fn _start() -> ! { -+ use crate::runtime_init; -+ -+ if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { -+ SP.set(bsp::memory::boot_core_stack_end() as u64); -+ runtime_init::runtime_init() -+ } else { -+ // If not core0, infinitely wait for events. -+ cpu::wait_forever() -+ } -+} - -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.S ---- 03_hacky_hello_world/src/_arch/aarch64/cpu/boot.S -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.S -@@ -1,21 +0,0 @@ --// SPDX-License-Identifier: MIT OR Apache-2.0 --// --// Copyright (c) 2018-2021 Andre Richter -- --.section ".text._start" -- --.global _start -- --_start: -- mrs x1, mpidr_el1 // Read Multiprocessor Affinity Register -- and x1, x1, #3 // Clear all bits except [1:0], which hold core id -- cbz x1, 2f // Jump to label 2 if we are core 0 --1: wfe // Wait for event -- b 1b // In case an event happened, jump back to 1 --2: // If we are here, we are core0 -- ldr x1, =_start // Load address of function "_start()" -- mov sp, x1 // Set start of stack to before our code, aka first -- // address before "_start()" -- bl runtime_init // Jump to the "runtime_init()" kernel function -- b 1b // We should never reach here. But just in case, -- // park this core aswell - -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs ---- 03_hacky_hello_world/src/_arch/aarch64/cpu/smp.rs -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs -@@ -0,0 +1,29 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Architectural symmetric multiprocessing. -+//! -+//! # Orientation -+//! -+//! Since arch modules are imported into generic modules using the path attribute, the path of this -+//! file is: -+//! -+//! crate::cpu::smp::arch_smp -+ -+use cortex_a::regs::*; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Return the executing core's id. -+#[inline(always)] -+pub fn core_id() -> T -+where -+ T: From, -+{ -+ const CORE_MASK: u64 = 0b11; -+ -+ T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -+} - -diff -uNr 03_hacky_hello_world/src/_arch/aarch64/cpu.rs 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs ---- 03_hacky_hello_world/src/_arch/aarch64/cpu.rs -+++ 04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs -@@ -11,6 +11,8 @@ - //! - //! crate::cpu::arch_cpu - -+use cortex_a::asm; -+ - //-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- -@@ -18,13 +20,7 @@ - /// Pause execution on the core. - #[inline(always)] - pub fn wait_forever() -> ! { -- unsafe { -- loop { -- #[rustfmt::skip] -- asm!( -- "wfe", -- options(nomem, nostack, preserves_flags) -- ); -- } -+ loop { -+ asm::wfe() - } - } - -diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs ---- 03_hacky_hello_world/src/bsp/raspberrypi/cpu.rs -+++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/cpu.rs -@@ -0,0 +1,12 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! BSP Processor code. -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Definitions -+//-------------------------------------------------------------------------------------------------- -+ -+/// Used by `arch` code to find the early boot core. -+pub const BOOT_CORE_ID: u64 = 0; - -diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/link.ld 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld ---- 03_hacky_hello_world/src/bsp/raspberrypi/link.ld -+++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld -@@ -21,6 +21,7 @@ - /*********************************************************************************************** - * Code + RO Data + Global Offset Table - ***********************************************************************************************/ -+ __rx_start = .; - .text : - { - KEEP(*(.text._start)) - -diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs 04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs ---- 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs -+++ 04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs -@@ -12,14 +12,36 @@ - - // Symbols from the linker script. - extern "Rust" { -+ static __rx_start: UnsafeCell<()>; -+ - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; - } - - //-------------------------------------------------------------------------------------------------- -+// Private Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Start address of the Read+Execute (RX) range. -+/// -+/// # Safety -+/// -+/// - Value is provided by the linker script and must be trusted as-is. -+#[inline(always)] -+fn rx_start() -> usize { -+ unsafe { __rx_start.get() as usize } -+} -+ -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- - -+/// Exclusive end address of the boot core's stack. -+#[inline(always)] -+pub fn boot_core_stack_end() -> usize { -+ rx_start() -+} -+ - /// Return the inclusive range spanning the .bss section. - /// - /// # Safety - -diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi.rs 04_zero_overhead_abstraction/src/bsp/raspberrypi.rs ---- 03_hacky_hello_world/src/bsp/raspberrypi.rs -+++ 04_zero_overhead_abstraction/src/bsp/raspberrypi.rs -@@ -5,4 +5,5 @@ - //! Top-level BSP file for the Raspberry Pi 3 and 4. - - pub mod console; -+pub mod cpu; - pub mod memory; - -diff -uNr 03_hacky_hello_world/src/cpu/smp.rs 04_zero_overhead_abstraction/src/cpu/smp.rs ---- 03_hacky_hello_world/src/cpu/smp.rs -+++ 04_zero_overhead_abstraction/src/cpu/smp.rs -@@ -0,0 +1,14 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Symmetric multiprocessing. -+ -+#[cfg(target_arch = "aarch64")] -+#[path = "../_arch/aarch64/cpu/smp.rs"] -+mod arch_smp; -+ -+//-------------------------------------------------------------------------------------------------- -+// Architectural Public Reexports -+//-------------------------------------------------------------------------------------------------- -+pub use arch_smp::core_id; - -diff -uNr 03_hacky_hello_world/src/cpu.rs 04_zero_overhead_abstraction/src/cpu.rs ---- 03_hacky_hello_world/src/cpu.rs -+++ 04_zero_overhead_abstraction/src/cpu.rs -@@ -10,6 +10,8 @@ - - mod boot; - -+pub mod smp; -+ - //-------------------------------------------------------------------------------------------------- - // Architectural Public Reexports - //-------------------------------------------------------------------------------------------------- - -diff -uNr 03_hacky_hello_world/src/main.rs 04_zero_overhead_abstraction/src/main.rs ---- 03_hacky_hello_world/src/main.rs -+++ 04_zero_overhead_abstraction/src/main.rs -@@ -107,9 +107,7 @@ - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - --#![feature(asm)] - #![feature(format_args_nl)] --#![feature(global_asm)] - #![feature(panic_info_message)] - #![no_main] - #![no_std] -@@ -128,7 +126,8 @@ - /// - /// - Only a single core must be active and running this function. - unsafe fn kernel_init() -> ! { -- println!("[0] Hello from Rust!"); -+ println!("[0] Hello from pure Rust!"); - -- panic!("Stopping here.") -+ println!("[1] Stopping here."); -+ cpu::wait_forever() - } - -diff -uNr 03_hacky_hello_world/src/runtime_init.rs 04_zero_overhead_abstraction/src/runtime_init.rs ---- 03_hacky_hello_world/src/runtime_init.rs -+++ 04_zero_overhead_abstraction/src/runtime_init.rs -@@ -30,7 +30,6 @@ - /// # Safety - /// - /// - Only a single core must be active and running this function. --#[no_mangle] - pub unsafe fn runtime_init() -> ! { - zero_bss(); - -``` diff --git a/04_zero_overhead_abstraction/build.rs b/04_zero_overhead_abstraction/build.rs deleted file mode 100644 index 3f0a29110..000000000 --- a/04_zero_overhead_abstraction/build.rs +++ /dev/null @@ -1,8 +0,0 @@ -use std::env; - -fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); - - println!("cargo:rerun-if-changed={}", linker_file); - println!("cargo:rerun-if-changed=build.rs"); -} diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs deleted file mode 100644 index 5f97ea410..000000000 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu.rs +++ /dev/null @@ -1,26 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural processor code. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::arch_cpu - -use cortex_a::asm; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Pause execution on the core. -#[inline(always)] -pub fn wait_forever() -> ! { - loop { - asm::wfe() - } -} diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs deleted file mode 100644 index 549b5927a..000000000 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/boot.rs +++ /dev/null @@ -1,41 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2021 Andre Richter - -//! Architectural boot code. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::boot::arch_boot - -use crate::{bsp, cpu}; -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). -#[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } -} diff --git a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs b/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/04_zero_overhead_abstraction/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/04_zero_overhead_abstraction/src/bsp.rs b/04_zero_overhead_abstraction/src/bsp.rs deleted file mode 100644 index 2b92251f5..000000000 --- a/04_zero_overhead_abstraction/src/bsp.rs +++ /dev/null @@ -1,11 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Conditional reexporting of Board Support Packages. - -#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -mod raspberrypi; - -#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -pub use raspberrypi::*; diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs deleted file mode 100644 index fdb7551a1..000000000 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi.rs +++ /dev/null @@ -1,9 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Top-level BSP file for the Raspberry Pi 3 and 4. - -pub mod console; -pub mod cpu; -pub mod memory; diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs deleted file mode 100644 index fca7042e6..000000000 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,47 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! BSP console facilities. - -use crate::console; -use core::fmt; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// A mystical, magical device for generating QEMU output out of the void. -struct QEMUOutput; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are -/// used to implement the `kernel`'s `print!` and `println!` macros. By implementing `write_str()`, -/// we get `write_fmt()` automatically. -/// -/// See [`src/print.rs`]. -/// -/// [`src/print.rs`]: ../../print/index.html -impl fmt::Write for QEMUOutput { - fn write_str(&mut self, s: &str) -> fmt::Result { - for c in s.chars() { - unsafe { - core::ptr::write_volatile(0x3F20_1000 as *mut u8, c as u8); - } - } - - Ok(()) - } -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> impl console::interface::Write { - QEMUOutput {} -} diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld b/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld deleted file mode 100644 index 87e6a976f..000000000 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/link.ld +++ /dev/null @@ -1,49 +0,0 @@ -/* SPDX-License-Identifier: MIT OR Apache-2.0 - * - * Copyright (c) 2018-2021 Andre Richter - */ - -/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ -__rpi_load_addr = 0x80000; - -ENTRY(__rpi_load_addr) - -PHDRS -{ - segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ - segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ -} - -SECTIONS -{ - . = __rpi_load_addr; - - /*********************************************************************************************** - * Code + RO Data + Global Offset Table - ***********************************************************************************************/ - __rx_start = .; - .text : - { - KEEP(*(.text._start)) - *(.text*) - } :segment_rx - - .rodata : ALIGN(8) { *(.rodata*) } :segment_rx - .got : ALIGN(8) { *(.got) } :segment_rx - - /*********************************************************************************************** - * Data + BSS - ***********************************************************************************************/ - .data : { *(.data*) } :segment_rw - - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) - { - __bss_start = .; - *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; - } :NONE -} diff --git a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs b/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs deleted file mode 100644 index bffe8eab0..000000000 --- a/04_zero_overhead_abstraction/src/bsp/raspberrypi/memory.rs +++ /dev/null @@ -1,59 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! BSP Memory Management. - -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __rx_start: UnsafeCell<()>; - - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/04_zero_overhead_abstraction/src/console.rs b/04_zero_overhead_abstraction/src/console.rs deleted file mode 100644 index 27b79f7dd..000000000 --- a/04_zero_overhead_abstraction/src/console.rs +++ /dev/null @@ -1,19 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! System console. - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// Console interfaces. -pub mod interface { - /// Console write functions. - /// - /// `core::fmt::Write` is exactly what we need for now. Re-export it here because - /// implementing `console::Write` gives a better hint to the reader about the - /// intention. - pub use core::fmt::Write; -} diff --git a/04_zero_overhead_abstraction/src/cpu.rs b/04_zero_overhead_abstraction/src/cpu.rs deleted file mode 100644 index 6f326b325..000000000 --- a/04_zero_overhead_abstraction/src/cpu.rs +++ /dev/null @@ -1,18 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2020-2021 Andre Richter - -//! Processor code. - -#[cfg(target_arch = "aarch64")] -#[path = "_arch/aarch64/cpu.rs"] -mod arch_cpu; - -mod boot; - -pub mod smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_cpu::wait_forever; diff --git a/04_zero_overhead_abstraction/src/cpu/boot.rs b/04_zero_overhead_abstraction/src/cpu/boot.rs deleted file mode 100644 index 1dc5c1808..000000000 --- a/04_zero_overhead_abstraction/src/cpu/boot.rs +++ /dev/null @@ -1,9 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2021 Andre Richter - -//! Boot code. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/boot.rs"] -mod arch_boot; diff --git a/04_zero_overhead_abstraction/src/main.rs b/04_zero_overhead_abstraction/src/main.rs deleted file mode 100644 index 9dfc7d007..000000000 --- a/04_zero_overhead_abstraction/src/main.rs +++ /dev/null @@ -1,133 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -// Rust embedded logo for `make doc`. -#![doc(html_logo_url = "https://git.io/JeGIp")] - -//! The `kernel` binary. -//! -//! # Code organization and architecture -//! -//! The code is divided into different *modules*, each representing a typical **subsystem** of the -//! `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example, -//! `src/memory.rs` contains code that is concerned with all things memory management. -//! -//! ## Visibility of processor architecture code -//! -//! Some of the `kernel`'s subsystems depend on low-level code that is specific to the target -//! processor architecture. For each supported processor architecture, there exists a subfolder in -//! `src/_arch`, for example, `src/_arch/aarch64`. -//! -//! The architecture folders mirror the subsystem modules laid out in `src`. For example, -//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go -//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in -//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic -//! module's name prefixed with `arch_`. -//! -//! For example, this is the top of `src/memory/mmu.rs`: -//! -//! ``` -//! #[cfg(target_arch = "aarch64")] -//! #[path = "../_arch/aarch64/memory/mmu.rs"] -//! mod arch_mmu; -//! ``` -//! -//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. -//! This way, each architecture specific module can provide its implementation of an item, while the -//! caller must not be concerned which architecture has been conditionally compiled. -//! -//! ## BSP code -//! -//! `BSP` stands for Board Support Package. `BSP` code is organized under `src/bsp.rs` and contains -//! target board specific definitions and functions. These are things such as the board's memory map -//! or instances of drivers for devices that are featured on the respective board. -//! -//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the -//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is -//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. -//! -//! ## Kernel interfaces -//! -//! Both `arch` and `bsp` contain code that is conditionally compiled depending on the actual target -//! and board for which the kernel is compiled. For example, the `interrupt controller` hardware of -//! the `Raspberry Pi 3` and the `Raspberry Pi 4` is different, but we want the rest of the `kernel` -//! code to play nicely with any of the two without much hassle. -//! -//! In order to provide a clean abstraction between `arch`, `bsp` and `generic kernel code`, -//! `interface` traits are provided *whenever possible* and *where it makes sense*. They are defined -//! in the respective subsystem module and help to enforce the idiom of *program to an interface, -//! not an implementation*. For example, there will be a common IRQ handling interface which the two -//! different interrupt controller `drivers` of both Raspberrys will implement, and only export the -//! interface to the rest of the `kernel`. -//! -//! ``` -//! +-------------------+ -//! | Interface (Trait) | -//! | | -//! +--+-------------+--+ -//! ^ ^ -//! | | -//! | | -//! +----------+--+ +--+----------+ -//! | kernel code | | bsp code | -//! | | | arch code | -//! +-------------+ +-------------+ -//! ``` -//! -//! # Summary -//! -//! For a logical `kernel` subsystem, corresponding code can be distributed over several physical -//! locations. Here is an example for the **memory** subsystem: -//! -//! - `src/memory.rs` and `src/memory/**/*` -//! - Common code that is agnostic of target processor architecture and `BSP` characteristics. -//! - Example: A function to zero a chunk of memory. -//! - Interfaces for the memory subsystem that are implemented by `arch` or `BSP` code. -//! - Example: An `MMU` interface that defines `MMU` function prototypes. -//! - `src/bsp/__board_name__/memory.rs` and `src/bsp/__board_name__/memory/**/*` -//! - `BSP` specific code. -//! - Example: The board's memory map (physical addresses of DRAM and MMIO devices). -//! - `src/_arch/__arch_name__/memory.rs` and `src/_arch/__arch_name__/memory/**/*` -//! - Processor architecture specific code. -//! - Example: Implementation of the `MMU` interface for the `__arch_name__` processor -//! architecture. -//! -//! From a namespace perspective, **memory** subsystem code lives in: -//! -//! - `crate::memory::*` -//! - `crate::bsp::memory::*` -//! -//! # Boot flow -//! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - -#![feature(format_args_nl)] -#![feature(panic_info_message)] -#![no_main] -#![no_std] - -mod bsp; -mod console; -mod cpu; -mod memory; -mod panic_wait; -mod print; -mod runtime_init; - -/// Early init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -unsafe fn kernel_init() -> ! { - println!("[0] Hello from pure Rust!"); - - println!("[1] Stopping here."); - cpu::wait_forever() -} diff --git a/04_zero_overhead_abstraction/src/memory.rs b/04_zero_overhead_abstraction/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/04_zero_overhead_abstraction/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/04_zero_overhead_abstraction/src/panic_wait.rs b/04_zero_overhead_abstraction/src/panic_wait.rs deleted file mode 100644 index 255eedeeb..000000000 --- a/04_zero_overhead_abstraction/src/panic_wait.rs +++ /dev/null @@ -1,19 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! A panic handler that infinitely waits. - -use crate::{cpu, println}; -use core::panic::PanicInfo; - -#[panic_handler] -fn panic(info: &PanicInfo) -> ! { - if let Some(args) = info.message() { - println!("\nKernel panic: {}", args); - } else { - println!("\nKernel panic!"); - } - - cpu::wait_forever() -} diff --git a/04_zero_overhead_abstraction/src/print.rs b/04_zero_overhead_abstraction/src/print.rs deleted file mode 100644 index 8ea5db240..000000000 --- a/04_zero_overhead_abstraction/src/print.rs +++ /dev/null @@ -1,38 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Printing. - -use crate::{bsp, console}; -use core::fmt; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -#[doc(hidden)] -pub fn _print(args: fmt::Arguments) { - use console::interface::Write; - - bsp::console::console().write_fmt(args).unwrap(); -} - -/// Prints without a newline. -/// -/// Carbon copy from -#[macro_export] -macro_rules! print { - ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); -} - -/// Prints with a newline. -/// -/// Carbon copy from -#[macro_export] -macro_rules! println { - () => ($crate::print!("\n")); - ($($arg:tt)*) => ({ - $crate::print::_print(format_args_nl!($($arg)*)); - }) -} diff --git a/04_zero_overhead_abstraction/src/runtime_init.rs b/04_zero_overhead_abstraction/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/04_zero_overhead_abstraction/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/05_safe_globals/src/_arch/aarch64/cpu/boot.rs b/05_safe_globals/src/_arch/aarch64/cpu/boot.rs index 549b5927a..c85bb94b8 100644 --- a/05_safe_globals/src/_arch/aarch64/cpu/boot.rs +++ b/05_safe_globals/src/_arch/aarch64/cpu/boot.rs @@ -11,31 +11,23 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; -use cortex_a::regs::*; +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/05_safe_globals/src/_arch/aarch64/cpu/boot.s b/05_safe_globals/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/05_safe_globals/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs b/05_safe_globals/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/05_safe_globals/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/05_safe_globals/src/bsp/raspberrypi/cpu.rs b/05_safe_globals/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/05_safe_globals/src/bsp/raspberrypi/cpu.rs +++ b/05_safe_globals/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/05_safe_globals/src/bsp/raspberrypi/link.ld b/05_safe_globals/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/05_safe_globals/src/bsp/raspberrypi/link.ld +++ b/05_safe_globals/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/05_safe_globals/src/bsp/raspberrypi/memory.rs b/05_safe_globals/src/bsp/raspberrypi/memory.rs index bffe8eab0..5c6b1ea5b 100644 --- a/05_safe_globals/src/bsp/raspberrypi/memory.rs +++ b/05_safe_globals/src/bsp/raspberrypi/memory.rs @@ -12,36 +12,14 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/05_safe_globals/src/cpu.rs b/05_safe_globals/src/cpu.rs index 6f326b325..d8f780823 100644 --- a/05_safe_globals/src/cpu.rs +++ b/05_safe_globals/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/05_safe_globals/src/cpu/smp.rs b/05_safe_globals/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/05_safe_globals/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/05_safe_globals/src/main.rs b/05_safe_globals/src/main.rs index 839558158..579f90ddc 100644 --- a/05_safe_globals/src/main.rs +++ b/05_safe_globals/src/main.rs @@ -100,14 +100,14 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/06_drivers_gpio_uart/README.md b/06_drivers_gpio_uart/README.md index 7dea308f3..bd7c02089 100644 --- a/06_drivers_gpio_uart/README.md +++ b/06_drivers_gpio_uart/README.md @@ -18,7 +18,8 @@ - A `driver::interface::DeviceDriver` trait is added for abstracting `BSP` driver implementations from kernel code. - Drivers are stored in `src/bsp/device_driver`, and can be reused between `BSP`s. - - We introduce the `GPIO` driver, which pinmuxes the RPi's PL011 UART. + - We introduce the `GPIO` driver, which pinmuxes (that is, routing signals from inside the `SoC` + to actual HW pins) the RPi's PL011 UART. - Note how this driver differentiates between **RPi 3** and **RPi4**. Their HW is different, so we have to account for it in SW. - Most importantly, the `PL011Uart` driver: It implements the `console::interface::*` traits and @@ -1110,7 +1111,7 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/driver.rs 06_drivers_gpio_uart/src diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs --- 05_safe_globals/src/bsp/raspberrypi/memory.rs +++ 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs -@@ -19,6 +19,38 @@ +@@ -17,6 +17,38 @@ } //-------------------------------------------------------------------------------------------------- @@ -1146,7 +1147,7 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src +} + +//-------------------------------------------------------------------------------------------------- - // Private Code + // Public Code //-------------------------------------------------------------------------------------------------- @@ -1249,7 +1250,7 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs diff -uNr 05_safe_globals/src/cpu.rs 06_drivers_gpio_uart/src/cpu.rs --- 05_safe_globals/src/cpu.rs +++ 06_drivers_gpio_uart/src/cpu.rs -@@ -15,4 +15,7 @@ +@@ -13,4 +13,7 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- @@ -1311,15 +1312,15 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs --- 05_safe_globals/src/main.rs +++ 06_drivers_gpio_uart/src/main.rs -@@ -107,6 +107,8 @@ - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +@@ -106,6 +106,8 @@ + //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +#![allow(clippy::clippy::upper_case_acronyms)] +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] + #![feature(global_asm)] #![feature(panic_info_message)] - #![feature(trait_alias)] @@ -116,6 +118,7 @@ mod bsp; mod console; diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs index 549b5927a..c85bb94b8 100644 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs @@ -11,31 +11,23 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; -use cortex_a::regs::*; +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs b/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs index 56a3306e5..c6d65e36b 100644 --- a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ b/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -12,8 +12,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -50,30 +48,10 @@ pub(super) mod map { } } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/06_drivers_gpio_uart/src/cpu.rs b/06_drivers_gpio_uart/src/cpu.rs index f0df4791f..c5e956ea6 100644 --- a/06_drivers_gpio_uart/src/cpu.rs +++ b/06_drivers_gpio_uart/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/06_drivers_gpio_uart/src/cpu/smp.rs b/06_drivers_gpio_uart/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/06_drivers_gpio_uart/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/06_drivers_gpio_uart/src/main.rs b/06_drivers_gpio_uart/src/main.rs index 627b5b59b..0f3c493df 100644 --- a/06_drivers_gpio_uart/src/main.rs +++ b/06_drivers_gpio_uart/src/main.rs @@ -100,16 +100,16 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/07_uart_chainloader/Makefile b/07_uart_chainloader/Makefile index 4fc73f554..ce9d5a640 100644 --- a/07_uart_chainloader/Makefile +++ b/07_uart_chainloader/Makefile @@ -24,7 +24,7 @@ ifeq ($(BSP),rpi3) NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat @@ -36,7 +36,7 @@ else ifeq ($(BSP),rpi4) NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img endif diff --git a/07_uart_chainloader/README.md b/07_uart_chainloader/README.md index 923615b56..a60fe9051 100644 --- a/07_uart_chainloader/README.md +++ b/07_uart_chainloader/README.md @@ -3,22 +3,34 @@ ## tl;dr - Running from an SD card was a nice experience, but it would be extremely tedious to do it for - every new binary. Let's write a [chainloader] using [position independent code]. + every new binary. So let's write a [chainloader]. - This will be the last binary you need to put on the SD card. Each following tutorial will provide a `chainboot` target in the `Makefile` that lets you conveniently load the kernel over `UART`. [chainloader]: https://en.wikipedia.org/wiki/Chain_loading -[position independent code]: https://en.wikipedia.org/wiki/Position-independent_code + ## Note -Please note that there is a lot of stuff going on in this tutorial that is very hard to grasp by -only looking at the source code changes. +Please note that there is stuff going on in this tutorial that is very hard to grasp by only looking +at the source code changes. + +The gist of it is that in `boot.s`, we are writing a piece of [position independent code] which +automatically determines where the firmware has loaded the binary (`0x8_0000`), and where it was +linked to (`0x200_0000`, see `link.ld`). The binary then copies itself from loaded to linked address +(aka "relocating" itself), and then jumps to the relocated version of `_start_rust()`. + +Since the chainloader has put itself "out of the way" now, it can now receive another kernel binary +from the `UART` and copy it to the standard load address of the RPi firmware at `0x8_0000`. Finally, +it jumps to `0x8_0000` and the newly loaded binary transparently executes as if it had been loaded +from SD card all along. Please bear with me until I find the time to write it all down here elaborately. For the time being, please see this tutorial as an enabler for a convenience feature that allows booting the following tutorials in a quick manner. +[position independent code]: https://en.wikipedia.org/wiki/Position-independent_code + ## Install and test it Our chainloader is called `MiniLoad` and is inspired by [raspbootin]. @@ -71,34 +83,36 @@ Minipush 1.0 [3] Echoing input now ``` -In this tutorial, a version of the kernel from the previous tutorial is loaded -for demo purposes. In subsequent tuts, it will be the working directory's -kernel. +In this tutorial, a version of the kernel from the previous tutorial is loaded for demo purposes. In +subsequent tutorials, it will be the working directory's kernel. ## Test it -The `Makefile` in this tutorial has an additional target, `qemuasm`, that lets -you nicely observe the jump from the loaded address (`0x80_XXX`) to the -relocated code at (`0x0200_0XXX`): +The `Makefile` in this tutorial has an additional target, `qemuasm`, that lets you nicely observe +how the kernel, after relocating itself, jumps the load address region (`0x80_XXX`) to the relocated +code at (`0x0200_0XXX`): ```console $ make qemuasm [...] +N: +0x00080030: 58000140 ldr x0, #0x80058 +0x00080034: 9100001f mov sp, x0 +0x00080038: 58000141 ldr x1, #0x80060 +0x0008003c: d61f0020 br x1 + +---------------- IN: -0x0008098c: b0000008 adrp x8, #0x81000 -0x00080990: b0000000 adrp x0, #0x81000 -0x00080994: 912a8000 add x0, x0, #0xaa0 -0x00080998: f9471908 ldr x8, [x8, #0xe30] -0x0008099c: d63f0100 blr x8 +0x02000070: 9400044c bl #0x20011a0 ---------------- IN: -0x02000b1c: b0000008 adrp x8, #0x2001000 -0x02000b20: b0000009 adrp x9, #0x2001000 -0x02000b24: f9475d08 ldr x8, [x8, #0xeb8] -0x02000b28: f9476129 ldr x9, [x9, #0xec0] -0x02000b2c: eb08013f cmp x9, x8 -0x02000b30: 540000c2 b.hs #0x2000b48 +0x020011a0: 90000008 adrp x8, #0x2001000 +0x020011a4: 90000009 adrp x9, #0x2001000 +0x020011a8: f9446508 ldr x8, [x8, #0x8c8] +0x020011ac: f9446929 ldr x9, [x9, #0x8d0] +0x020011b0: eb08013f cmp x9, x8 +0x020011b4: 54000109 b.ls #0x20011d4 [...] ``` @@ -110,22 +124,18 @@ Binary files 06_drivers_gpio_uart/demo_payload_rpi4.img and 07_uart_chainloader/ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile --- 06_drivers_gpio_uart/Makefile +++ 07_uart_chainloader/Makefile -@@ -24,7 +24,8 @@ - NM_BINARY = aarch64-none-elf-nm +@@ -25,6 +25,7 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld -- RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -+ RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 + CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img -@@ -35,7 +36,8 @@ - NM_BINARY = aarch64-none-elf-nm +@@ -36,6 +37,7 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld -- RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -+ RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 + CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img endif @@ -166,47 +176,47 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs ---- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs -+++ 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs -@@ -29,11 +29,11 @@ - /// actually set (`SP.set()`). - #[no_mangle] - pub unsafe fn _start() -> ! { -- use crate::runtime_init; -+ use crate::relocate; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); -- runtime_init::runtime_init() -+ relocate::relocate_self() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - -diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs 07_uart_chainloader/src/_arch/aarch64/cpu.rs ---- 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs -+++ 07_uart_chainloader/src/_arch/aarch64/cpu.rs -@@ -35,3 +35,19 @@ - asm::wfe() - } - } +diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s +--- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s ++++ 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s +@@ -22,20 +22,31 @@ + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 +- b.ne 1f ++ b.ne 2f + +- // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. ++ // If execution reaches here, it is the boot core. + -+/// Branch to a raw integer value. -+/// -+/// # Safety -+/// -+/// - This is highly unsafe. Use with care. -+#[inline(always)] -+pub unsafe fn branch_to_raw_addr(addr: usize) -> ! { -+ asm!( -+ "blr {destination:x}", -+ destination = in(reg) addr, -+ options(nomem, nostack) -+ ); ++ // Next, relocate the binary. ++ adr x0, __binary_nonzero_start // The address the binary got loaded to. ++ ldr x1, =__binary_nonzero_start // The address the binary was linked to. ++ ldr x2, =__binary_nonzero_end_exclusive + -+ core::intrinsics::unreachable() -+} ++1: ldr x3, [x0], #8 ++ str x3, [x1], #8 ++ cmp x1, x2 ++ b.lo 1b + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + +- // Jump to Rust code. +- b _start_rust ++ // Jump to the relocated Rust code. ++ ldr x1, =_start_rust ++ br x1 + + // Infinitely wait for events (aka "park the core"). +-1: wfe +- b 1b ++2: wfe ++ b 2b + + .size _start, . - _start + .type _start, function diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -268,58 +278,46 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 07_uart_chainloader/src/bsp/raspberrypi/link.ld --- 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ 07_uart_chainloader/src/bsp/raspberrypi/link.ld -@@ -16,12 +16,13 @@ +@@ -16,7 +16,8 @@ SECTIONS { - . = __rpi_load_addr; + /* Set the link address to 32 MiB */ + . = 0x2000000; - + /* ^ */ + /* | stack */ + /* | growth */ +@@ -26,6 +27,7 @@ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ -- __rx_start = .; -+ __binary_start = .; ++ __binary_nonzero_start = .; .text : { KEEP(*(.text._start)) -@@ -46,4 +47,10 @@ - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; - } :NONE -+ +@@ -49,8 +51,12 @@ + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw + + /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ + . = ALIGN(8); -+ __binary_end_inclusive = . - 8; ++ __binary_nonzero_end_exclusive = .; + -+ __runtime_init_reloc = runtime_init; - } + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ +- .bss : ALIGN(8) ++ .bss : + { + __bss_start = .; + *(.bss*); diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader/src/bsp/raspberrypi/memory.rs --- 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ 07_uart_chainloader/src/bsp/raspberrypi/memory.rs -@@ -12,10 +12,12 @@ - - // Symbols from the linker script. - extern "Rust" { -- static __rx_start: UnsafeCell<()>; -- -+ static __binary_start: UnsafeCell; - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -+ static __binary_end_inclusive: UnsafeCell; -+ -+ static __runtime_init_reloc: UnsafeCell; - } - - //-------------------------------------------------------------------------------------------------- -@@ -25,9 +27,12 @@ +@@ -23,9 +23,10 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { -+ pub const BOOT_CORE_STACK_END: usize = 0x8_0000; -+ + pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; @@ -329,111 +327,34 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -51,36 +56,44 @@ - } - - //-------------------------------------------------------------------------------------------------- --// Private Code -+// Public Code +@@ -52,7 +53,13 @@ + // Public Code //-------------------------------------------------------------------------------------------------- --/// Start address of the Read+Execute (RX) range. -+/// Exclusive end address of the boot core's stack. -+#[inline(always)] -+pub fn boot_core_stack_end() -> usize { -+ map::BOOT_CORE_STACK_END -+} -+ +-/// Return the inclusive range spanning the .bss section. +/// The address on which the Raspberry firmware loads every binary by default. +#[inline(always)] +pub fn board_default_load_addr() -> *const u64 { + map::BOARD_DEFAULT_LOAD_ADDRESS as _ +} + -+/// Return the inclusive range spanning the relocated kernel binary. - /// - /// # Safety - /// --/// - Value is provided by the linker script and must be trusted as-is. --#[inline(always)] --fn rx_start() -> usize { -- unsafe { __rx_start.get() as usize } -+/// - Values are provided by the linker script and must be trusted as-is. -+/// - The linker-provided addresses must be u64 aligned. -+pub fn relocated_binary_range_inclusive() -> RangeInclusive<*mut u64> { -+ unsafe { RangeInclusive::new(__binary_start.get(), __binary_end_inclusive.get()) } - } - --//-------------------------------------------------------------------------------------------------- --// Public Code --//-------------------------------------------------------------------------------------------------- -- --/// Exclusive end address of the boot core's stack. -+/// The relocated address of function `runtime_init()`. - #[inline(always)] --pub fn boot_core_stack_end() -> usize { -- rx_start() -+pub fn relocated_runtime_init_addr() -> *const u64 { -+ unsafe { __runtime_init_reloc.get() as _ } - } - --/// Return the inclusive range spanning the .bss section. +/// Return the inclusive range spanning the relocated .bss section. /// /// # Safety /// - /// - Values are provided by the linker script and must be trusted as-is. - /// - The linker-provided addresses must be u64 aligned. --pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { -+pub fn relocated_bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - -diff -uNr 06_drivers_gpio_uart/src/cpu.rs 07_uart_chainloader/src/cpu.rs ---- 06_drivers_gpio_uart/src/cpu.rs -+++ 07_uart_chainloader/src/cpu.rs -@@ -15,7 +15,7 @@ - //-------------------------------------------------------------------------------------------------- - // Architectural Public Reexports - //-------------------------------------------------------------------------------------------------- --pub use arch_cpu::{nop, wait_forever}; -+pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; - - #[cfg(feature = "bsp_rpi3")] - pub use arch_cpu::spin_for_cycles; diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs --- 06_drivers_gpio_uart/src/main.rs +++ 07_uart_chainloader/src/main.rs -@@ -102,13 +102,17 @@ - //! - //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. - //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. --//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -+//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. -+//! 3. Finally, [`runtime_init::runtime_init()`] is called. - //! - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -+//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html +@@ -107,6 +107,7 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] -+#![feature(core_intrinsics)] #![feature(format_args_nl)] - #![feature(panic_info_message)] - #![feature(trait_alias)] -@@ -122,6 +126,7 @@ - mod memory; - mod panic_wait; - mod print; -+mod relocate; - mod runtime_init; - mod synchronization; - -@@ -150,29 +155,49 @@ + #![feature(global_asm)] +@@ -150,29 +151,49 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -504,81 +425,6 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + kernel() } -diff -uNr 06_drivers_gpio_uart/src/relocate.rs 07_uart_chainloader/src/relocate.rs ---- 06_drivers_gpio_uart/src/relocate.rs -+++ 07_uart_chainloader/src/relocate.rs -@@ -0,0 +1,49 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Relocation code. -+ -+use crate::{bsp, cpu}; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Relocates the own binary from `bsp::memory::board_default_load_addr()` to the `__binary_start` -+/// address from the linker script. -+/// -+/// # Safety -+/// -+/// - Only a single core must be active and running this function. -+/// - Function must not use the `bss` section. -+#[inline(never)] -+pub unsafe fn relocate_self() -> ! { -+ let range = bsp::memory::relocated_binary_range_inclusive(); -+ let mut relocated_binary_start_addr = *range.start(); -+ let relocated_binary_end_addr_inclusive = *range.end(); -+ -+ // The address of where the previous firmware loaded us. -+ let mut current_binary_start_addr = bsp::memory::board_default_load_addr(); -+ -+ // Copy the whole binary. -+ while relocated_binary_start_addr <= relocated_binary_end_addr_inclusive { -+ core::ptr::write_volatile( -+ relocated_binary_start_addr, -+ core::ptr::read_volatile(current_binary_start_addr), -+ ); -+ relocated_binary_start_addr = relocated_binary_start_addr.offset(1); -+ current_binary_start_addr = current_binary_start_addr.offset(1); -+ } -+ -+ // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by -+ // forcing an indirection through the global offset table (GOT), so that execution continues -+ // from the relocated binary. -+ // -+ // Without the indirection through the assembly, the address of `runtime_init()` would be -+ // calculated as a relative offset from the current program counter, since we are compiling as -+ // `position independent code`. This would cause us to keep executing from the address to which -+ // the firmware loaded us, instead of the relocated position. -+ let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; -+ cpu::branch_to_raw_addr(relocated_runtime_init_addr) -+} - -diff -uNr 06_drivers_gpio_uart/src/runtime_init.rs 07_uart_chainloader/src/runtime_init.rs ---- 06_drivers_gpio_uart/src/runtime_init.rs -+++ 07_uart_chainloader/src/runtime_init.rs -@@ -17,7 +17,7 @@ - /// - Must only be called pre `kernel_init()`. - #[inline(always)] - unsafe fn zero_bss() { -- memory::zero_volatile(bsp::memory::bss_range_inclusive()); -+ memory::zero_volatile(bsp::memory::relocated_bss_range_inclusive()); - } - - //-------------------------------------------------------------------------------------------------- -@@ -30,6 +30,7 @@ - /// # Safety - /// - /// - Only a single core must be active and running this function. -+#[no_mangle] - pub unsafe fn runtime_init() -> ! { - zero_bss(); - - diff -uNr 06_drivers_gpio_uart/update.sh 07_uart_chainloader/update.sh --- 06_drivers_gpio_uart/update.sh +++ 07_uart_chainloader/update.sh diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/07_uart_chainloader/demo_payload_rpi3.img index 79eef6b4ca54773cbe15dd88f8b48e0ea9a8541a..91602b45b02de681e2e3f0706864f16396592fa9 100755 GIT binary patch delta 1220 zcmZ8fUrZE782@JXj@wlTw+G8|N3M5^oX25eO9|m$!Yv191;jrWv|Y6+qpNu;gr8B1G3Uch_C$WY)znW76}2+UDtsvRy3quEg97`Yzk=*q?L z7tn;WUOh9TH{lH)VZpX~eSk|ouA*tQj+Ni*o17foKEHgWMw@~a#{8Y#+3>Ko1-+5tAaoeZb? zN214%eAL<+J=vF`Zx#gCZ)d>iKb2RdK)UOm44w=A-<(%mOdC~R>B-59lk_&r_&UbF z@28U!N^&<-jIV{a5O( zkSk+}nR5(3XSm!!>8|X0{mf2{JgQfuE4<6ju~Q?{K}DRYcG*rM20J(@L{hUQ=|J8>p?Z+ZSrUEqI8aVhZ&n9gQ2*%!q^ou9g4pLP_ZV^ delta 1147 zcmZ8fOK1~O6uoaIV34YE?wVh0Bf~ zL{YTQC-_;2C`!|fh9H=Y)DH?ST!<~YOS|bu5vgvbNaLH#E2tOV+;`5o@80|Fyys1$ zP3kCYo(Kv0!T>-|f0&=h=ku}%@KkI`m>~rCEp90j&rvZ7MlwBMB4lLxxO#+ia*;oe z;-tvYiBP83u?L}e=CQDYTOY5%(X#(-e4*RL60V7uMYy?s+y5C7J5{*~l z$s2yG8sx4g=6YF)pS9IkXxh_=))2o`@7w6Xiw^W z8E%48o#;w%RW}*)HW%wP_`v-tJSZ5+3t{rzEBg`&vrf>nM}4J@bpm8vi$+pRkYGuB z!`6fujVH_)62H0-^Z>2N(FMAO2OUPTEvMm%-}zhCx6sp0wD!}tjF!RHX)La#xvhF6v$x&aZ%jy-JvU=Kj*3{^eDA;9&>A(s$ zM2^U@;e!D?%EQoeosjGrM>$_c59HCe&fePl2gp-1d8qf8woI2ic@ewhXm diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/07_uart_chainloader/demo_payload_rpi4.img index 4ec2899812281f5c492516fa530527f937f5446b..b1060665b4af3eadced31cec72de42265985d110 100755 GIT binary patch delta 2148 zcmZ8iZ){Ul6hF7G-TKx-|In^oH(dW1O1mL{3Uq~NA8eQ~G|E^Ai66QJ3}I|y=NC@!(U^2In$eTfn7;3ln()P4g>6J1fbObu9RQ6p(u(6zzmPCF(aJt zga3yKA$3z}h3jUJDvAvX$dMjFj;RuXTs%9!gm#+8zg_yA-K7U5zk26QVC2l@Vj2`H)(Z)yN% zNwDUvM5@G){9_`PhYirV%_Jq-MWFlL;B2;oa|PnY4+-f3^rdvni*BjqQ}p+_eKF1> zUl!75b$^j_pr6OPWh5>OjnR@I<{u}!@=)malm5 zh?L+eklEz=tJwP@CRbY79P!DQu^GDTdo&>R+ z!M)ifwG5%}oZBbIQm73ioK}f*QjC-s#t>}wxeJ*C?YSc zdsgY>qH#6w(VPc=E!!=;X9$E?S!gFyqw9??5j>3^F+Fd1CJ>^L?pit%zLYCglUxxc z{Rt&qcwN#LneV!$FWf1*f4nn1es}cWYNv?|j4obug^-i%`r=i!={3{>khNl_QY#MT z-y7%<@iC8WC&{qaY$0-m9k;bQ94^dZ%*`M_-tv6=8TPxa(crIV)%NvK8l;mX(=ihU z6;O+OH=P*pvmuAFW#2F?r}zbsH3u5lUVD&qv7`3-z?lZh-?$qcW5JNhuE?g`J-I4} zLy`@2v9Ijm)ze{G6kK@|ffL0ehTMHc`Rjhaoz4vaol8Nkau#Ui6_R^D2o$@RJu$+b zax`Lw=N#L39!L1usRr7Pd1gKlawWx>Q@%Ilmh*EQDq0>KMr4`B0sL%UIVDv={H_2M zwcu{{r^B<3k1O7rBSsK0Y=m{(E0QfLDvHf4fj^!kQlUoNqBT=w8=Mq8A7oz;Gc&m)||3dt5UywF{E9->uD9!ZOU{EU=_S#6gY1-N;t&k{KGjHH?K~v zc>!J*v`-m*m)Lf%M|{(!Sx>XHH%`(l=f!#Fp4AJUF1-qe&c}BN|6bPZHr<|LtLqLp XuIbjLx^*+VP!|@xdgWert1kQ>Zelic delta 2218 zcmcIlZ){Ul6hHU9wtF38{j+xKI+p&Kvt5UR6zK+ud&n>_6DwU~Of-S2iTu&v<~Jp2 zw_zNDrq{cOL}QpnLi-KG&x!n(LV&wafk0TaG)lh^ys zpWiv>{?56tAFj`?PoIRYOFpxDV;4Yjr(eFL>w1#{a7y{Jx9lT8zcIm>AN;=-<(-%H z5typPNv;9ZvTlPy4@d*Uh?|Z=cNj8>b^uoo_y>XnlHmiEe#PuvS3;X&&m_Vwh?Zi| zy$$_ZxSy&H3=>EvxzUPG z#5CEO-y^BzwY)k)=66Yuw3^gzP=I#2!D)4X(>Ej!Y(YH_lEJ;!rFQk9ZLixK^-anH zBf?h1T|t|IHW$X5mwgdUCi&$UQbkX8I=s0W}v$*Q07Ia z7WAZsF)H!RIcz6Q8-10@l!862w8=S51}a%HiSLjqf^!h?yQ%g&&>7T!EWuc%1$!?O z%5jzWnOq4^#pfXa(qT??1)&WH^04DV{vZSpsAcG&h|jnFNdPfaS}mI*wgb zXFtDDeD(Z|;>Ld_Y>LxNq7%WDX9@X$U9Df+7+6PF0A%Yh|J)X!`^GoYBa_Xn)AkHe z*|4pR46_rqEsm)M%wfVt0>ueC)t6$6HlL(m}HVO{38a48h-&q@C zkO_|Y6fwt5@F@4UF7+BXez)epAL}7%q1L>$+=q9o*36!AZnC0u1f@DNd*8Vxtp%x! zxEIv`HQ|1+7%*a{%$dXrT|dgZXv8h|mLq1haM+Yd>~*`M5i=AN)L9{)uwI1%ADHUU zAj#Lc9v#@T%-Hc06UVoe~^H7Iv>CVdn5OIl3e)0M}~#L>r3@xtqMD8X;A6fJGVWj~e!+ z`={-vJ$IU~H&$h2&{$gs2lo%++ZCb5kvG=eem}DdjXmif`JW{&yYUMUZ?4MANejY` z6Kt##LdMHuth@cQ!oHk!RLBH^AtwQ@On`{mtq?*O<6+64$9QqG7&09_Y diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/07_uart_chainloader/src/_arch/aarch64/cpu.rs index 893c7580b..ade53a3ca 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -35,19 +35,3 @@ pub fn wait_forever() -> ! { asm::wfe() } } - -/// Branch to a raw integer value. -/// -/// # Safety -/// -/// - This is highly unsafe. Use with care. -#[inline(always)] -pub unsafe fn branch_to_raw_addr(addr: usize) -> ! { - asm!( - "blr {destination:x}", - destination = in(reg) addr, - options(nomem, nostack) - ); - - core::intrinsics::unreachable() -} diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs index b343792f3..c85bb94b8 100644 --- a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs +++ b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs @@ -11,31 +11,23 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; -use cortex_a::regs::*; +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] -pub unsafe fn _start() -> ! { - use crate::relocate; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - relocate::relocate_self() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.s b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..4f45ebd12 --- /dev/null +++ b/07_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 2f + + // If execution reaches here, it is the boot core. + + // Next, relocate the binary. + adr x0, __binary_nonzero_start // The address the binary got loaded to. + ldr x1, =__binary_nonzero_start // The address the binary was linked to. + ldr x2, =__binary_nonzero_end_exclusive + +1: ldr x3, [x0], #8 + str x3, [x1], #8 + cmp x1, x2 + b.lo 1b + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to the relocated Rust code. + ldr x1, =_start_rust + br x1 + + // Infinitely wait for events (aka "park the core"). +2: wfe + b 2b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs b/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/07_uart_chainloader/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/07_uart_chainloader/src/bsp/raspberrypi/link.ld b/07_uart_chainloader/src/bsp/raspberrypi/link.ld index 5642ed4e9..f75239061 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/link.ld +++ b/07_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -18,15 +18,29 @@ SECTIONS { /* Set the link address to 32 MiB */ . = 0x2000000; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __binary_start = .; + __binary_nonzero_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx @@ -37,8 +51,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw + /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ + . = ALIGN(8); + __binary_nonzero_end_exclusive = .; + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + .bss : { __bss_start = .; *(.bss*); @@ -47,10 +65,4 @@ SECTIONS . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; } :NONE - - /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ - . = ALIGN(8); - __binary_end_inclusive = . - 8; - - __runtime_init_reloc = runtime_init; } diff --git a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs index 5abfb9ef0..9a66cc8ac 100644 --- a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ b/07_uart_chainloader/src/bsp/raspberrypi/memory.rs @@ -12,12 +12,8 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __binary_start: UnsafeCell; static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; - static __binary_end_inclusive: UnsafeCell; - - static __runtime_init_reloc: UnsafeCell; } //-------------------------------------------------------------------------------------------------- @@ -27,8 +23,6 @@ extern "Rust" { /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { - pub const BOOT_CORE_STACK_END: usize = 0x8_0000; - pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; pub const GPIO_OFFSET: usize = 0x0020_0000; @@ -59,41 +53,19 @@ pub(super) mod map { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - map::BOOT_CORE_STACK_END -} - /// The address on which the Raspberry firmware loads every binary by default. #[inline(always)] pub fn board_default_load_addr() -> *const u64 { map::BOARD_DEFAULT_LOAD_ADDRESS as _ } -/// Return the inclusive range spanning the relocated kernel binary. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn relocated_binary_range_inclusive() -> RangeInclusive<*mut u64> { - unsafe { RangeInclusive::new(__binary_start.get(), __binary_end_inclusive.get()) } -} - -/// The relocated address of function `runtime_init()`. -#[inline(always)] -pub fn relocated_runtime_init_addr() -> *const u64 { - unsafe { __runtime_init_reloc.get() as _ } -} - /// Return the inclusive range spanning the relocated .bss section. /// /// # Safety /// /// - Values are provided by the linker script and must be trusted as-is. /// - The linker-provided addresses must be u64 aligned. -pub fn relocated_bss_range_inclusive() -> RangeInclusive<*mut u64> { +pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { let range; unsafe { range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); diff --git a/07_uart_chainloader/src/cpu.rs b/07_uart_chainloader/src/cpu.rs index 7de5c29c1..c5e956ea6 100644 --- a/07_uart_chainloader/src/cpu.rs +++ b/07_uart_chainloader/src/cpu.rs @@ -10,12 +10,10 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; +pub use arch_cpu::{nop, wait_forever}; #[cfg(feature = "bsp_rpi3")] pub use arch_cpu::spin_for_cycles; diff --git a/07_uart_chainloader/src/cpu/smp.rs b/07_uart_chainloader/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/07_uart_chainloader/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/07_uart_chainloader/src/main.rs b/07_uart_chainloader/src/main.rs index 4fc6449af..1ac8140cc 100644 --- a/07_uart_chainloader/src/main.rs +++ b/07_uart_chainloader/src/main.rs @@ -100,20 +100,17 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. -//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. -//! 3. Finally, [`runtime_init::runtime_init()`] is called. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] -#![feature(core_intrinsics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] @@ -126,7 +123,6 @@ mod driver; mod memory; mod panic_wait; mod print; -mod relocate; mod runtime_init; mod synchronization; diff --git a/07_uart_chainloader/src/relocate.rs b/07_uart_chainloader/src/relocate.rs deleted file mode 100644 index 6ac6bccea..000000000 --- a/07_uart_chainloader/src/relocate.rs +++ /dev/null @@ -1,49 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Relocation code. - -use crate::{bsp, cpu}; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Relocates the own binary from `bsp::memory::board_default_load_addr()` to the `__binary_start` -/// address from the linker script. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -/// - Function must not use the `bss` section. -#[inline(never)] -pub unsafe fn relocate_self() -> ! { - let range = bsp::memory::relocated_binary_range_inclusive(); - let mut relocated_binary_start_addr = *range.start(); - let relocated_binary_end_addr_inclusive = *range.end(); - - // The address of where the previous firmware loaded us. - let mut current_binary_start_addr = bsp::memory::board_default_load_addr(); - - // Copy the whole binary. - while relocated_binary_start_addr <= relocated_binary_end_addr_inclusive { - core::ptr::write_volatile( - relocated_binary_start_addr, - core::ptr::read_volatile(current_binary_start_addr), - ); - relocated_binary_start_addr = relocated_binary_start_addr.offset(1); - current_binary_start_addr = current_binary_start_addr.offset(1); - } - - // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by - // forcing an indirection through the global offset table (GOT), so that execution continues - // from the relocated binary. - // - // Without the indirection through the assembly, the address of `runtime_init()` would be - // calculated as a relative offset from the current program counter, since we are compiling as - // `position independent code`. This would cause us to keep executing from the address to which - // the firmware loaded us, instead of the relocated position. - let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; - cpu::branch_to_raw_addr(relocated_runtime_init_addr) -} diff --git a/07_uart_chainloader/src/runtime_init.rs b/07_uart_chainloader/src/runtime_init.rs index 5e9b74974..ee0946863 100644 --- a/07_uart_chainloader/src/runtime_init.rs +++ b/07_uart_chainloader/src/runtime_init.rs @@ -17,7 +17,7 @@ use crate::{bsp, memory}; /// - Must only be called pre `kernel_init()`. #[inline(always)] unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::relocated_bss_range_inclusive()); + memory::zero_volatile(bsp::memory::bss_range_inclusive()); } //-------------------------------------------------------------------------------------------------- @@ -30,7 +30,6 @@ unsafe fn zero_bss() { /// # Safety /// /// - Only a single core must be active and running this function. -#[no_mangle] pub unsafe fn runtime_init() -> ! { zero_bss(); diff --git a/08_timestamps/README.md b/08_timestamps/README.md index eebdd20cd..122308a6c 100644 --- a/08_timestamps/README.md +++ b/08_timestamps/README.md @@ -49,23 +49,19 @@ Binary files 07_uart_chainloader/demo_payload_rpi4.img and 08_timestamps/demo_pa diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile --- 07_uart_chainloader/Makefile +++ 08_timestamps/Makefile -@@ -24,8 +24,7 @@ - NM_BINARY = aarch64-none-elf-nm +@@ -25,7 +25,6 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld -- RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img -+ RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img -@@ -36,8 +35,7 @@ - NM_BINARY = aarch64-none-elf-nm +@@ -37,7 +36,6 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld -- RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 -C relocation-model=pic + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img -+ RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif # Export for build.rs @@ -97,23 +93,47 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs 08_timestamps/src/_arch/aarch64/cpu/boot.rs ---- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs -+++ 08_timestamps/src/_arch/aarch64/cpu/boot.rs -@@ -29,11 +29,11 @@ - /// actually set (`SP.set()`). - #[no_mangle] - pub unsafe fn _start() -> ! { -- use crate::relocate; -+ use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); -- relocate::relocate_self() -+ runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() +diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s 08_timestamps/src/_arch/aarch64/cpu/boot.s +--- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s ++++ 08_timestamps/src/_arch/aarch64/cpu/boot.s +@@ -22,31 +22,20 @@ + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 +- b.ne 2f ++ b.ne 1f + +- // If execution reaches here, it is the boot core. +- +- // Next, relocate the binary. +- adr x0, __binary_nonzero_start // The address the binary got loaded to. +- ldr x1, =__binary_nonzero_start // The address the binary was linked to. +- ldr x2, =__binary_nonzero_end_exclusive +- +-1: ldr x3, [x0], #8 +- str x3, [x1], #8 +- cmp x1, x2 +- b.lo 1b ++ // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + +- // Jump to the relocated Rust code. +- ldr x1, =_start_rust +- br x1 ++ // Jump to Rust code. ++ b _start_rust + + // Infinitely wait for events (aka "park the core"). +-2: wfe +- b 2b ++1: wfe ++ b 1b + + .size _start, . - _start + .type _start, function diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs --- 07_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -134,26 +154,6 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a /// Pause execution on the core. #[inline(always)] pub fn wait_forever() -> ! { -@@ -35,19 +26,3 @@ - asm::wfe() - } - } -- --/// Branch to a raw integer value. --/// --/// # Safety --/// --/// - This is highly unsafe. Use with care. --#[inline(always)] --pub unsafe fn branch_to_raw_addr(addr: usize) -> ! { -- asm!( -- "blr {destination:x}", -- destination = in(reg) addr, -- options(nomem, nostack) -- ); -- -- core::intrinsics::unreachable() --} diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/aarch64/time.rs --- 07_uart_chainloader/src/_arch/aarch64/time.rs @@ -361,58 +361,46 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08 diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/link.ld 08_timestamps/src/bsp/raspberrypi/link.ld --- 07_uart_chainloader/src/bsp/raspberrypi/link.ld +++ 08_timestamps/src/bsp/raspberrypi/link.ld -@@ -16,13 +16,12 @@ +@@ -16,8 +16,7 @@ SECTIONS { - /* Set the link address to 32 MiB */ - . = 0x2000000; + . = __rpi_load_addr; - + /* ^ */ + /* | stack */ + /* | growth */ +@@ -27,7 +26,6 @@ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ -- __binary_start = .; -+ __rx_start = .; +- __binary_nonzero_start = .; .text : { KEEP(*(.text._start)) -@@ -47,10 +46,4 @@ - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; - } :NONE -- +@@ -51,12 +49,8 @@ + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw + - /* Fill up to 8 byte, b/c relocating the binary is done in u64 chunks */ - . = ALIGN(8); -- __binary_end_inclusive = . - 8; +- __binary_nonzero_end_exclusive = .; - -- __runtime_init_reloc = runtime_init; - } + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ +- .bss : ++ .bss : ALIGN(8) + { + __bss_start = .; + *(.bss*); diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bsp/raspberrypi/memory.rs --- 07_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ 08_timestamps/src/bsp/raspberrypi/memory.rs -@@ -12,12 +12,10 @@ - - // Symbols from the linker script. - extern "Rust" { -- static __binary_start: UnsafeCell; -+ static __rx_start: UnsafeCell<()>; -+ - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -- static __binary_end_inclusive: UnsafeCell; -- -- static __runtime_init_reloc: UnsafeCell; - } - - //-------------------------------------------------------------------------------------------------- -@@ -27,12 +25,9 @@ +@@ -23,10 +23,9 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { -- pub const BOOT_CORE_STACK_END: usize = 0x8_0000; -- - pub const BOARD_DEFAULT_LOAD_ADDRESS: usize = 0x8_0000; - pub const GPIO_OFFSET: usize = 0x0020_0000; @@ -422,114 +410,53 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -56,44 +51,36 @@ - } - - //-------------------------------------------------------------------------------------------------- --// Public Code -+// Private Code +@@ -53,13 +52,7 @@ + // Public Code //-------------------------------------------------------------------------------------------------- --/// Exclusive end address of the boot core's stack. --#[inline(always)] --pub fn boot_core_stack_end() -> usize { -- map::BOOT_CORE_STACK_END --} -- -/// The address on which the Raspberry firmware loads every binary by default. -#[inline(always)] -pub fn board_default_load_addr() -> *const u64 { - map::BOARD_DEFAULT_LOAD_ADDRESS as _ -} - --/// Return the inclusive range spanning the relocated kernel binary. -+/// Start address of the Read+Execute (RX) range. - /// - /// # Safety - /// --/// - Values are provided by the linker script and must be trusted as-is. --/// - The linker-provided addresses must be u64 aligned. --pub fn relocated_binary_range_inclusive() -> RangeInclusive<*mut u64> { -- unsafe { RangeInclusive::new(__binary_start.get(), __binary_end_inclusive.get()) } -+/// - Value is provided by the linker script and must be trusted as-is. -+#[inline(always)] -+fn rx_start() -> usize { -+ unsafe { __rx_start.get() as usize } - } - --/// The relocated address of function `runtime_init()`. -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Exclusive end address of the boot core's stack. - #[inline(always)] --pub fn relocated_runtime_init_addr() -> *const u64 { -- unsafe { __runtime_init_reloc.get() as _ } -+pub fn boot_core_stack_end() -> usize { -+ rx_start() - } - -/// Return the inclusive range spanning the relocated .bss section. +/// Return the inclusive range spanning the .bss section. /// /// # Safety /// - /// - Values are provided by the linker script and must be trusted as-is. - /// - The linker-provided addresses must be u64 aligned. --pub fn relocated_bss_range_inclusive() -> RangeInclusive<*mut u64> { -+pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs --- 07_uart_chainloader/src/cpu.rs +++ 08_timestamps/src/cpu.rs -@@ -15,7 +15,4 @@ - //-------------------------------------------------------------------------------------------------- +@@ -14,6 +14,3 @@ // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- --pub use arch_cpu::{branch_to_raw_addr, nop, wait_forever}; + pub use arch_cpu::{nop, wait_forever}; - -#[cfg(feature = "bsp_rpi3")] -pub use arch_cpu::spin_for_cycles; -+pub use arch_cpu::{nop, wait_forever}; diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs --- 07_uart_chainloader/src/main.rs +++ 08_timestamps/src/main.rs -@@ -102,17 +102,13 @@ - //! - //! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. - //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. --//! 2. Once finished with architectural setup, the arch code calls [`relocate::relocate_self()`]. --//! 3. Finally, [`runtime_init::runtime_init()`] is called. -+//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. - //! - //! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html --//! [`relocate::relocate_self()`]: relocate/fn.relocate_self.html +@@ -107,7 +107,6 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] --#![feature(core_intrinsics)] #![feature(format_args_nl)] - #![feature(panic_info_message)] - #![feature(trait_alias)] -@@ -126,9 +122,9 @@ - mod memory; - mod panic_wait; + #![feature(global_asm)] +@@ -125,6 +124,7 @@ mod print; --mod relocate; mod runtime_init; mod synchronization; +mod time; /// Early init code. /// -@@ -153,51 +149,31 @@ +@@ -149,51 +149,31 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -680,81 +607,6 @@ diff -uNr 07_uart_chainloader/src/print.rs 08_timestamps/src/print.rs + }) +} -diff -uNr 07_uart_chainloader/src/relocate.rs 08_timestamps/src/relocate.rs ---- 07_uart_chainloader/src/relocate.rs -+++ 08_timestamps/src/relocate.rs -@@ -1,49 +0,0 @@ --// SPDX-License-Identifier: MIT OR Apache-2.0 --// --// Copyright (c) 2018-2021 Andre Richter -- --//! Relocation code. -- --use crate::{bsp, cpu}; -- --//-------------------------------------------------------------------------------------------------- --// Public Code --//-------------------------------------------------------------------------------------------------- -- --/// Relocates the own binary from `bsp::memory::board_default_load_addr()` to the `__binary_start` --/// address from the linker script. --/// --/// # Safety --/// --/// - Only a single core must be active and running this function. --/// - Function must not use the `bss` section. --#[inline(never)] --pub unsafe fn relocate_self() -> ! { -- let range = bsp::memory::relocated_binary_range_inclusive(); -- let mut relocated_binary_start_addr = *range.start(); -- let relocated_binary_end_addr_inclusive = *range.end(); -- -- // The address of where the previous firmware loaded us. -- let mut current_binary_start_addr = bsp::memory::board_default_load_addr(); -- -- // Copy the whole binary. -- while relocated_binary_start_addr <= relocated_binary_end_addr_inclusive { -- core::ptr::write_volatile( -- relocated_binary_start_addr, -- core::ptr::read_volatile(current_binary_start_addr), -- ); -- relocated_binary_start_addr = relocated_binary_start_addr.offset(1); -- current_binary_start_addr = current_binary_start_addr.offset(1); -- } -- -- // The following function calls realize an "absolute jump" to `runtime_init::runtime_init()` by -- // forcing an indirection through the global offset table (GOT), so that execution continues -- // from the relocated binary. -- // -- // Without the indirection through the assembly, the address of `runtime_init()` would be -- // calculated as a relative offset from the current program counter, since we are compiling as -- // `position independent code`. This would cause us to keep executing from the address to which -- // the firmware loaded us, instead of the relocated position. -- let relocated_runtime_init_addr = bsp::memory::relocated_runtime_init_addr() as usize; -- cpu::branch_to_raw_addr(relocated_runtime_init_addr) --} - -diff -uNr 07_uart_chainloader/src/runtime_init.rs 08_timestamps/src/runtime_init.rs ---- 07_uart_chainloader/src/runtime_init.rs -+++ 08_timestamps/src/runtime_init.rs -@@ -17,7 +17,7 @@ - /// - Must only be called pre `kernel_init()`. - #[inline(always)] - unsafe fn zero_bss() { -- memory::zero_volatile(bsp::memory::relocated_bss_range_inclusive()); -+ memory::zero_volatile(bsp::memory::bss_range_inclusive()); - } - - //-------------------------------------------------------------------------------------------------- -@@ -30,7 +30,6 @@ - /// # Safety - /// - /// - Only a single core must be active and running this function. --#[no_mangle] - pub unsafe fn runtime_init() -> ! { - zero_bss(); - - diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs --- 07_uart_chainloader/src/time.rs +++ 08_timestamps/src/time.rs diff --git a/08_timestamps/src/_arch/aarch64/cpu/boot.rs b/08_timestamps/src/_arch/aarch64/cpu/boot.rs index 549b5927a..c85bb94b8 100644 --- a/08_timestamps/src/_arch/aarch64/cpu/boot.rs +++ b/08_timestamps/src/_arch/aarch64/cpu/boot.rs @@ -11,31 +11,23 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; -use cortex_a::regs::*; +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/08_timestamps/src/_arch/aarch64/cpu/boot.s b/08_timestamps/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/08_timestamps/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/08_timestamps/src/_arch/aarch64/cpu/smp.rs b/08_timestamps/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/08_timestamps/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/08_timestamps/src/bsp/raspberrypi/cpu.rs b/08_timestamps/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/08_timestamps/src/bsp/raspberrypi/cpu.rs +++ b/08_timestamps/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/08_timestamps/src/bsp/raspberrypi/link.ld b/08_timestamps/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/08_timestamps/src/bsp/raspberrypi/link.ld +++ b/08_timestamps/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/08_timestamps/src/bsp/raspberrypi/memory.rs b/08_timestamps/src/bsp/raspberrypi/memory.rs index 56a3306e5..c6d65e36b 100644 --- a/08_timestamps/src/bsp/raspberrypi/memory.rs +++ b/08_timestamps/src/bsp/raspberrypi/memory.rs @@ -12,8 +12,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -50,30 +48,10 @@ pub(super) mod map { } } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/08_timestamps/src/cpu.rs b/08_timestamps/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/08_timestamps/src/cpu.rs +++ b/08_timestamps/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/08_timestamps/src/cpu/smp.rs b/08_timestamps/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/08_timestamps/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/08_timestamps/src/main.rs b/08_timestamps/src/main.rs index 14e1d0ab3..b7a74ea6b 100644 --- a/08_timestamps/src/main.rs +++ b/08_timestamps/src/main.rs @@ -100,16 +100,16 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/09_hw_debug_JTAG/README.md b/09_hw_debug_JTAG/README.md index b13e1fa7f..0c9fb497d 100644 --- a/09_hw_debug_JTAG/README.md +++ b/09_hw_debug_JTAG/README.md @@ -54,6 +54,7 @@ infrastructure for JTAG debugging around it. We need to add another line to the `config.txt` file from the SD Card: ```toml +arm_64bit=1 init_uart_clock=48000000 enable_jtag_gpio=1 ``` @@ -234,15 +235,16 @@ $ make gdb [...] >>> target remote :3333 # Connect to OpenOCD, core0 >>> load # Load the kernel into the RPi's DRAM over JTAG. -Loading section .text, size 0x2340 lma 0x80000 -Loading section .rodata, size 0xc2d lma 0x82340 -Loading section .data, size 0x20 lma 0x82f70 -Start address 0x80000, load size 12173 -Transfer rate: 65 KB/sec, 4057 bytes/write. +Loading section .text, size 0x2454 lma 0x80000 +Loading section .rodata, size 0xa1d lma 0x82460 +Loading section .got, size 0x10 lma 0x82e80 +Loading section .data, size 0x20 lma 0x82e90 +Start address 0x0000000000080000, load size 11937 +Transfer rate: 63 KB/sec, 2984 bytes/write. >>> set $pc = 0x80000 # Set RPI's program counter to the start of the # kernel binary. ->>> break main.rs:70 -Breakpoint 1 at 0x80108: file src/main.rs, line 153. +>>> break main.rs:158 +Breakpoint 1 at 0x8025c: file src/main.rs, line 158. >>> cont >>> step # Single-step through the kernel >>> step diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs index 549b5927a..c85bb94b8 100644 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs @@ -11,31 +11,23 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; -use cortex_a::regs::*; +use crate::runtime_init; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is -/// actually set (`SP.set()`). +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] -pub unsafe fn _start() -> ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs b/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs index 56a3306e5..c6d65e36b 100644 --- a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs +++ b/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs @@ -12,8 +12,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -50,30 +48,10 @@ pub(super) mod map { } } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/09_hw_debug_JTAG/src/cpu.rs b/09_hw_debug_JTAG/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/09_hw_debug_JTAG/src/cpu.rs +++ b/09_hw_debug_JTAG/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/09_hw_debug_JTAG/src/cpu/smp.rs b/09_hw_debug_JTAG/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/09_hw_debug_JTAG/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/09_hw_debug_JTAG/src/main.rs b/09_hw_debug_JTAG/src/main.rs index 14e1d0ab3..b7a74ea6b 100644 --- a/09_hw_debug_JTAG/src/main.rs +++ b/09_hw_debug_JTAG/src/main.rs @@ -100,16 +100,16 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/10_privilege_level/README.md b/10_privilege_level/README.md index 2c4355487..3e160e22d 100644 --- a/10_privilege_level/README.md +++ b/10_privilege_level/README.md @@ -12,7 +12,6 @@ - [Checking for EL2 in the entrypoint](#checking-for-el2-in-the-entrypoint) - [Transition preparation](#transition-preparation) - [Returning from an exception that never happened](#returning-from-an-exception-that-never-happened) -- [Are we stackless?](#are-we-stackless) - [Test it](#test-it) - [Diff to previous](#diff-to-previous) @@ -39,32 +38,35 @@ ARMv8-A] before you continue. It gives a concise overview about the topic. ## Scope of this tutorial -By default, the Rpi will always start executing in `EL2`. Since we are writing a traditional +By default, the Raspberry will always start executing in `EL2`. Since we are writing a traditional `Kernel`, we have to transition into the more appropriate `EL1`. ## Checking for EL2 in the entrypoint First of all, we need to ensure that we actually execute in `EL2` before we can call respective code -to transition to `EL1`: +to transition to `EL1`. Therefore, we add a new checkt to the top of `boot.s`, which parks the CPU +core should it not be in `EL2`. + +``` +// Only proceed if the core executes in EL2. Park it otherwise. +mrs x0, CurrentEL +cmp x0, _EL2 +b.ne 1f +``` + +Afterwards, we continue with preparing the `EL2` -> `EL1` transition by calling +`prepare_el2_to_el1_transition()` in `boot.rs`: ```rust #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } ``` -If this is the case, we continue with preparing the `EL2` -> `EL1` transition in -`el2_to_el1_transition()`. - ## Transition preparation Since `EL2` is more privileged than `EL1`, it has control over various processor features and can @@ -104,8 +106,8 @@ This instruction will copy the contents of the [Saved Program Status Register - Program Status Register - EL1` and jump to the instruction address that is stored in the [Exception Link Register - EL2]. -This is basically the reverse of what is happening when an exception is taken. You'll learn about it -in an upcoming tutorial. +This is basically the reverse of what is happening when an exception is taken. You'll learn about +that in an upcoming tutorial. [Saved Program Status Register - EL2]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/spsr_el2.rs.html [Exception Link Register - EL2]: https://docs.rs/cortex-a/5.1.2/src/cortex_a/regs/elr_el2.rs.html @@ -125,65 +127,39 @@ SPSR_EL2.write( // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + +// Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there +// are no plans to ever return to EL2, just re-use the same stack. +SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); ``` As you can see, we are populating `ELR_EL2` with the address of the [runtime_init()] function that -we earlier used to call directly from the entrypoint. - -Finally, we set the stack pointer for `SP_EL1` and call `ERET`: +we earlier used to call directly from the entrypoint. Finally, we set the stack pointer for +`SP_EL1`. [runtime_init()]: src/runtime_init.rs -```rust -// Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. -SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - -// Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. -asm::eret() -``` +You might have noticed that the stack's address was supplied as a function argument. As you might +remember, in `_start()` in `boot.s`, we are already setting up the stack for `EL2`. Since there +are no plans to ever return to `EL2`, we can just re-use the same stack for `EL1`, so its address is +forwarded using function arguments. -## Are we stackless? +Lastly, back in `_start_rust()` a call to `ERET` is made: -We just wrote a big inline rust function, `el2_to_el1_transition()`, that is executed in a context -where we do not have a stack yet. We should double-check the generated machine code: +```rust +#[no_mangle] +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); -```console -$ make objdump -[...] -Disassembly of section .text: - -0000000000080000 <_start>: - 80000: d53800a8 mrs x8, mpidr_el1 - 80004: f240051f tst x8, #0x3 - 80008: 54000081 b.ne 80018 <_start+0x18> // b.any - 8000c: d5384248 mrs x8, currentel - 80010: f100211f cmp x8, #0x8 - 80014: 54000060 b.eq 80020 <_start+0x20> // b.none - 80018: d503205f wfe - 8001c: 17ffffff b 80018 <_start+0x18> - 80020: aa1f03e8 mov x8, xzr - 80024: 52800069 mov w9, #0x3 // #3 - 80028: d51ce109 msr cnthctl_el2, x9 - 8002c: d51ce068 msr cntvoff_el2, x8 - 80030: d0000008 adrp x8, 82000 - 80034: 52b0000a mov w10, #0x80000000 // #-2147483648 - 80038: 528078ab mov w11, #0x3c5 // #965 - 8003c: 52a0010c mov w12, #0x80000 // #524288 - 80040: d51c110a msr hcr_el2, x10 - 80044: d51c400b msr spsr_el2, x11 - 80048: 9114c108 add x8, x8, #0x530 - 8004c: d51c4028 msr elr_el2, x8 - 80050: d51c410c msr sp_el1, x12 - 80054: d69f03e0 eret + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} ``` -Looks good! Thanks zero-overhead abstractions in the [cortex-a] crate! :heart_eyes: - -[cortex-a]: https://github.com/rust-embedded/cortex-a - ## Test it -In `main.rs`, we additionally inspect if the mask bits in `SPSR_EL2` made it to `EL1` as well: +In `main.rs`, we print the `current privilege level` and additionally inspect if the mask bits in +`SPSR_EL2` made it to `EL1` as well: ```console $ make chainboot @@ -225,30 +201,27 @@ Minipush 1.0 diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 10_privilege_level/src/_arch/aarch64/cpu/boot.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ 10_privilege_level/src/_arch/aarch64/cpu/boot.rs -@@ -12,7 +12,55 @@ +@@ -12,11 +12,53 @@ //! crate::cpu::boot::arch_boot - use crate::{bsp, cpu}; --use cortex_a::regs::*; + use crate::runtime_init; +use cortex_a::{asm, regs::*}; -+ -+//-------------------------------------------------------------------------------------------------- + + // Assembly counterpart to this file. + global_asm!(include_str!("boot.s")); + + //-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + -+/// Transition from EL2 to EL1. ++/// Prepares the transition from EL2 to EL1. +/// +/// # Safety +/// ++/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - The HW state of EL1 must be prepared in a sound way. -+/// - Exception return from EL2 must must continue execution in EL1 with -+/// `runtime_init::runtime_init()`. -+/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -+/// a stack for EL2. +#[inline(always)] -+unsafe fn el2_to_el1_transition() -> ! { -+ use crate::runtime_init; -+ ++unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + @@ -273,38 +246,67 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 10_privilege_level/src/ + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + -+ // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. -+ SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); -+ -+ // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. -+ asm::eret() ++ // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there ++ // are no plans to ever return to EL2, just re-use the same stack. ++ SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); +} - - //-------------------------------------------------------------------------------------------------- ++ ++//-------------------------------------------------------------------------------------------------- // Public Code -@@ -25,15 +73,15 @@ + //-------------------------------------------------------------------------------------------------- + +@@ -27,7 +69,11 @@ /// # Safety /// - /// - Linker script must ensure to place this function where it is expected by the target machine. --/// - We have to hope that the compiler omits any stack pointer usage before the stack pointer is --/// actually set (`SP.set()`). -+/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -+/// a stack for EL2. + /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. ++/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] - pub unsafe fn _start() -> ! { -- use crate::runtime_init; -- -- if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { -- SP.set(bsp::memory::boot_core_stack_end() as u64); -- runtime_init::runtime_init() -+ // Expect the boot core to start in EL2. -+ if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) -+ && (CurrentEL.get() == CurrentEL::EL::EL2.value) -+ { -+ el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() +-pub unsafe fn _start_rust() -> ! { +- runtime_init::runtime_init() ++pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { ++ prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); ++ ++ // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. ++ asm::eret() + } + +diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 10_privilege_level/src/_arch/aarch64/cpu/boot.s +--- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s ++++ 10_privilege_level/src/_arch/aarch64/cpu/boot.s +@@ -6,6 +6,7 @@ + // Definitions + //-------------------------------------------------------------------------------------------------- + ++.equ _EL2, 0x8 + .equ _core_id_mask, 0b11 + + //-------------------------------------------------------------------------------------------------- +@@ -17,6 +18,11 @@ + // fn _start() + //------------------------------------------------------------------------------ + _start: ++ // Only proceed if the core executes in EL2. Park it otherwise. ++ mrs x0, CurrentEL ++ cmp x0, _EL2 ++ b.ne 1f ++ + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask +@@ -26,11 +32,11 @@ + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + +- // Set the stack pointer. ++ // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + +- // Jump to Rust code. ++ // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs --- 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs diff --git a/10_privilege_level/src/_arch/aarch64/cpu/boot.rs b/10_privilege_level/src/_arch/aarch64/cpu/boot.rs index c2f5fe0b7..02798fdc0 100644 --- a/10_privilege_level/src/_arch/aarch64/cpu/boot.rs +++ b/10_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/10_privilege_level/src/_arch/aarch64/cpu/boot.s b/10_privilege_level/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/10_privilege_level/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs b/10_privilege_level/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/10_privilege_level/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/10_privilege_level/src/bsp/raspberrypi/cpu.rs b/10_privilege_level/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/10_privilege_level/src/bsp/raspberrypi/cpu.rs +++ b/10_privilege_level/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/10_privilege_level/src/bsp/raspberrypi/link.ld b/10_privilege_level/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/10_privilege_level/src/bsp/raspberrypi/link.ld +++ b/10_privilege_level/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/10_privilege_level/src/bsp/raspberrypi/memory.rs b/10_privilege_level/src/bsp/raspberrypi/memory.rs index 56a3306e5..c6d65e36b 100644 --- a/10_privilege_level/src/bsp/raspberrypi/memory.rs +++ b/10_privilege_level/src/bsp/raspberrypi/memory.rs @@ -12,8 +12,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -50,30 +48,10 @@ pub(super) mod map { } } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/10_privilege_level/src/cpu.rs b/10_privilege_level/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/10_privilege_level/src/cpu.rs +++ b/10_privilege_level/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/10_privilege_level/src/cpu/smp.rs b/10_privilege_level/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/10_privilege_level/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/10_privilege_level/src/main.rs b/10_privilege_level/src/main.rs index 181a9aa5b..0c3a06489 100644 --- a/10_privilege_level/src/main.rs +++ b/10_privilege_level/src/main.rs @@ -100,16 +100,16 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/11_virtual_mem_part1_identity_mapping/README.md index 8534ad881..b2313e1c2 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/11_virtual_mem_part1_identity_mapping/README.md @@ -813,7 +813,15 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld --- 10_privilege_level/src/bsp/raspberrypi/link.ld +++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld -@@ -31,6 +31,9 @@ +@@ -26,6 +26,7 @@ + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ ++ __rx_start = .; + .text : + { + KEEP(*(.text._start)) +@@ -44,6 +45,9 @@ .rodata : ALIGN(8) { *(.rodata*) } :segment_rx .got : ALIGN(8) { *(.got) } :segment_rx @@ -927,15 +935,17 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ use core::{cell::UnsafeCell, ops::RangeInclusive}; //-------------------------------------------------------------------------------------------------- -@@ -13,6 +15,7 @@ +@@ -12,6 +14,9 @@ + // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; ++ static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; - ++ static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; -@@ -25,6 +28,20 @@ + } +@@ -23,6 +28,20 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { @@ -956,7 +966,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; -@@ -37,6 +54,7 @@ +@@ -35,6 +54,7 @@ pub const START: usize = 0x3F00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -964,7 +974,7 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } /// Physical devices. -@@ -47,6 +65,7 @@ +@@ -45,10 +65,35 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -972,10 +982,20 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ } } -@@ -64,6 +83,16 @@ - unsafe { __rx_start.get() as usize } - } - + //-------------------------------------------------------------------------------------------------- ++// Private Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Start address of the Read+Execute (RX) range. ++/// ++/// # Safety ++/// ++/// - Value is provided by the linker script and must be trusted as-is. ++#[inline(always)] ++fn rx_start() -> usize { ++ unsafe { __rx_start.get() as usize } ++} ++ +/// Exclusive end address of the Read+Execute (RX) range. +/// +/// # Safety @@ -986,10 +1006,11 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ + unsafe { __rx_end_exclusive.get() as usize } +} + - //-------------------------------------------------------------------------------------------------- ++//-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- + diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/src/bsp.rs --- 10_privilege_level/src/bsp.rs +++ 11_virtual_mem_part1_identity_mapping/src/bsp.rs @@ -1006,7 +1027,7 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs --- 10_privilege_level/src/main.rs +++ 11_virtual_mem_part1_identity_mapping/src/main.rs -@@ -108,7 +108,11 @@ +@@ -107,7 +107,11 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] @@ -1016,8 +1037,8 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s +#![feature(const_panic)] +#![feature(core_intrinsics)] #![feature(format_args_nl)] + #![feature(global_asm)] #![feature(panic_info_message)] - #![feature(trait_alias)] @@ -132,9 +136,17 @@ /// # Safety /// diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs index c2f5fe0b7..02798fdc0 100644 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld index 6c63ba10f..854f5b388 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -25,7 +30,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs index bb18dd201..81233775d 100644 --- a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +++ b/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -97,12 +97,6 @@ fn rx_end_exclusive() -> usize { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu.rs b/11_virtual_mem_part1_identity_mapping/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/11_virtual_mem_part1_identity_mapping/src/cpu.rs +++ b/11_virtual_mem_part1_identity_mapping/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs b/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/11_virtual_mem_part1_identity_mapping/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/11_virtual_mem_part1_identity_mapping/src/main.rs index 9037541d1..8ce5d3d92 100644 --- a/11_virtual_mem_part1_identity_mapping/src/main.rs +++ b/11_virtual_mem_part1_identity_mapping/src/main.rs @@ -100,11 +100,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] @@ -114,6 +113,7 @@ #![feature(const_panic)] #![feature(core_intrinsics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/12_exceptions_part1_groundwork/README.md b/12_exceptions_part1_groundwork/README.md index 99838f3eb..cc4b043ff 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/12_exceptions_part1_groundwork/README.md @@ -177,47 +177,48 @@ handling code, restore the context, so that the processor can continue where it taking the exception. Context save and restore is one of the few places in system software where there is no way around -some hand-crafted assembly. Introducing `exception.S`: +some hand-crafted assembly. Introducing `exception.s`: ```asm /// Call the function provided by parameter `\handler` after saving the exception context. Provide /// the context as the first parameter to '\handler'. .macro CALL_WITH_CONTEXT handler - // Make room on the stack for the exception context. - sub sp, sp, #16 * 17 - - // Store all general purpose registers on the stack. - stp x0, x1, [sp, #16 * 0] - stp x2, x3, [sp, #16 * 1] - stp x4, x5, [sp, #16 * 2] - stp x6, x7, [sp, #16 * 3] - stp x8, x9, [sp, #16 * 4] - stp x10, x11, [sp, #16 * 5] - stp x12, x13, [sp, #16 * 6] - stp x14, x15, [sp, #16 * 7] - stp x16, x17, [sp, #16 * 8] - stp x18, x19, [sp, #16 * 9] - stp x20, x21, [sp, #16 * 10] - stp x22, x23, [sp, #16 * 11] - stp x24, x25, [sp, #16 * 12] - stp x26, x27, [sp, #16 * 13] - stp x28, x29, [sp, #16 * 14] - - // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). - mrs x1, ELR_EL1 - mrs x2, SPSR_EL1 - - stp lr, x1, [sp, #16 * 15] - str w2, [sp, #16 * 16] - - // x0 is the first argument for the function called through `\handler`. - mov x0, sp - - // Call `\handler`. - bl \handler - - // After returning from exception handling code, replay the saved context and return via `eret`. - b __exception_restore_context + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context .endm ``` @@ -238,8 +239,6 @@ because `Rust` has no stable convention ([yet](https://github.com/rust-lang/rfcs Next, we craft the exception vector table: ```asm -.section .text - // Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. .align 11 @@ -254,23 +253,23 @@ __exception_vector_start: // // - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. .org 0x000 - CALL_WITH_CONTEXT current_el0_synchronous + CALL_WITH_CONTEXT current_el0_synchronous .org 0x080 - CALL_WITH_CONTEXT current_el0_irq + CALL_WITH_CONTEXT current_el0_irq .org 0x100 - FIQ_SUSPEND + FIQ_SUSPEND .org 0x180 - CALL_WITH_CONTEXT current_el0_serror + CALL_WITH_CONTEXT current_el0_serror // Current exception level with SP_ELx, x > 0. .org 0x200 - CALL_WITH_CONTEXT current_elx_synchronous + CALL_WITH_CONTEXT current_elx_synchronous .org 0x280 - CALL_WITH_CONTEXT current_elx_irq + CALL_WITH_CONTEXT current_elx_irq .org 0x300 - FIQ_SUSPEND + FIQ_SUSPEND .org 0x380 - CALL_WITH_CONTEXT current_elx_serror + CALL_WITH_CONTEXT current_elx_serror [...] ``` @@ -490,7 +489,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 +use register::InMemoryRegister; + +// Assembly counterpart to this file. -+global_asm!(include_str!("exception.S")); ++global_asm!(include_str!("exception.s")); + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -731,64 +730,73 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 + barrier::isb(barrier::SY); +} -diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S ---- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S -+++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S -@@ -0,0 +1,138 @@ +diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s +--- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s ++++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s +@@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + ++//-------------------------------------------------------------------------------------------------- ++// Definitions ++//-------------------------------------------------------------------------------------------------- ++ +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler -+ // Make room on the stack for the exception context. -+ sub sp, sp, #16 * 17 -+ -+ // Store all general purpose registers on the stack. -+ stp x0, x1, [sp, #16 * 0] -+ stp x2, x3, [sp, #16 * 1] -+ stp x4, x5, [sp, #16 * 2] -+ stp x6, x7, [sp, #16 * 3] -+ stp x8, x9, [sp, #16 * 4] -+ stp x10, x11, [sp, #16 * 5] -+ stp x12, x13, [sp, #16 * 6] -+ stp x14, x15, [sp, #16 * 7] -+ stp x16, x17, [sp, #16 * 8] -+ stp x18, x19, [sp, #16 * 9] -+ stp x20, x21, [sp, #16 * 10] -+ stp x22, x23, [sp, #16 * 11] -+ stp x24, x25, [sp, #16 * 12] -+ stp x26, x27, [sp, #16 * 13] -+ stp x28, x29, [sp, #16 * 14] -+ -+ // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). -+ mrs x1, ELR_EL1 -+ mrs x2, SPSR_EL1 -+ -+ stp lr, x1, [sp, #16 * 15] -+ str x2, [sp, #16 * 16] -+ -+ // x0 is the first argument for the function called through `\handler`. -+ mov x0, sp -+ -+ // Call `\handler`. -+ bl \handler -+ -+ // After returning from exception handling code, replay the saved context and return via `eret`. -+ b __exception_restore_context ++ // Make room on the stack for the exception context. ++ sub sp, sp, #16 * 17 ++ ++ // Store all general purpose registers on the stack. ++ stp x0, x1, [sp, #16 * 0] ++ stp x2, x3, [sp, #16 * 1] ++ stp x4, x5, [sp, #16 * 2] ++ stp x6, x7, [sp, #16 * 3] ++ stp x8, x9, [sp, #16 * 4] ++ stp x10, x11, [sp, #16 * 5] ++ stp x12, x13, [sp, #16 * 6] ++ stp x14, x15, [sp, #16 * 7] ++ stp x16, x17, [sp, #16 * 8] ++ stp x18, x19, [sp, #16 * 9] ++ stp x20, x21, [sp, #16 * 10] ++ stp x22, x23, [sp, #16 * 11] ++ stp x24, x25, [sp, #16 * 12] ++ stp x26, x27, [sp, #16 * 13] ++ stp x28, x29, [sp, #16 * 14] ++ ++ // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). ++ mrs x1, ELR_EL1 ++ mrs x2, SPSR_EL1 ++ ++ stp lr, x1, [sp, #16 * 15] ++ str x2, [sp, #16 * 16] ++ ++ // x0 is the first argument for the function called through `\handler`. ++ mov x0, sp ++ ++ // Call `\handler`. ++ bl \handler ++ ++ // After returning from exception handling code, replay the saved context and return via ++ // `eret`. ++ b __exception_restore_context +.endm + +.macro FIQ_SUSPEND -+1: wfe -+ b 1b ++1: wfe ++ b 1b +.endm + +//-------------------------------------------------------------------------------------------------- -+// The exception vector table. ++// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + ++//------------------------------------------------------------------------------ ++// The exception vector table. ++//------------------------------------------------------------------------------ ++ +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + @@ -803,76 +811,77 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.S 12 +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 -+ CALL_WITH_CONTEXT current_el0_synchronous ++ CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 -+ CALL_WITH_CONTEXT current_el0_irq ++ CALL_WITH_CONTEXT current_el0_irq +.org 0x100 -+ FIQ_SUSPEND ++ FIQ_SUSPEND +.org 0x180 -+ CALL_WITH_CONTEXT current_el0_serror ++ CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 -+ CALL_WITH_CONTEXT current_elx_synchronous ++ CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 -+ CALL_WITH_CONTEXT current_elx_irq ++ CALL_WITH_CONTEXT current_elx_irq +.org 0x300 -+ FIQ_SUSPEND ++ FIQ_SUSPEND +.org 0x380 -+ CALL_WITH_CONTEXT current_elx_serror ++ CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 -+ CALL_WITH_CONTEXT lower_aarch64_synchronous ++ CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 -+ CALL_WITH_CONTEXT lower_aarch64_irq ++ CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 -+ FIQ_SUSPEND ++ FIQ_SUSPEND +.org 0x580 -+ CALL_WITH_CONTEXT lower_aarch64_serror ++ CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 -+ CALL_WITH_CONTEXT lower_aarch32_synchronous ++ CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 -+ CALL_WITH_CONTEXT lower_aarch32_irq ++ CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 -+ FIQ_SUSPEND ++ FIQ_SUSPEND +.org 0x780 -+ CALL_WITH_CONTEXT lower_aarch32_serror ++ CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + -+//-------------------------------------------------------------------------------------------------- -+// Helper functions -+//-------------------------------------------------------------------------------------------------- -+.section .text -+ ++//------------------------------------------------------------------------------ ++// fn __exception_restore_context() ++//------------------------------------------------------------------------------ +__exception_restore_context: -+ ldr w19, [sp, #16 * 16] -+ ldp lr, x20, [sp, #16 * 15] -+ -+ msr SPSR_EL1, x19 -+ msr ELR_EL1, x20 -+ -+ ldp x0, x1, [sp, #16 * 0] -+ ldp x2, x3, [sp, #16 * 1] -+ ldp x4, x5, [sp, #16 * 2] -+ ldp x6, x7, [sp, #16 * 3] -+ ldp x8, x9, [sp, #16 * 4] -+ ldp x10, x11, [sp, #16 * 5] -+ ldp x12, x13, [sp, #16 * 6] -+ ldp x14, x15, [sp, #16 * 7] -+ ldp x16, x17, [sp, #16 * 8] -+ ldp x18, x19, [sp, #16 * 9] -+ ldp x20, x21, [sp, #16 * 10] -+ ldp x22, x23, [sp, #16 * 11] -+ ldp x24, x25, [sp, #16 * 12] -+ ldp x26, x27, [sp, #16 * 13] -+ ldp x28, x29, [sp, #16 * 14] -+ -+ add sp, sp, #16 * 17 -+ -+ eret ++ ldr w19, [sp, #16 * 16] ++ ldp lr, x20, [sp, #16 * 15] ++ ++ msr SPSR_EL1, x19 ++ msr ELR_EL1, x20 ++ ++ ldp x0, x1, [sp, #16 * 0] ++ ldp x2, x3, [sp, #16 * 1] ++ ldp x4, x5, [sp, #16 * 2] ++ ldp x6, x7, [sp, #16 * 3] ++ ldp x8, x9, [sp, #16 * 4] ++ ldp x10, x11, [sp, #16 * 5] ++ ldp x12, x13, [sp, #16 * 6] ++ ldp x14, x15, [sp, #16 * 7] ++ ldp x16, x17, [sp, #16 * 8] ++ ldp x18, x19, [sp, #16 * 9] ++ ldp x20, x21, [sp, #16 * 10] ++ ldp x22, x23, [sp, #16 * 11] ++ ldp x24, x25, [sp, #16 * 12] ++ ldp x26, x27, [sp, #16 * 13] ++ ldp x28, x29, [sp, #16 * 14] ++ ++ add sp, sp, #16 * 17 ++ ++ eret ++ ++.size __exception_restore_context, . - __exception_restore_context ++.type __exception_restore_context, function diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs --- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -945,15 +954,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_p diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs --- 11_virtual_mem_part1_identity_mapping/src/main.rs +++ 12_exceptions_part1_groundwork/src/main.rs -@@ -114,6 +114,7 @@ - #![feature(const_panic)] - #![feature(core_intrinsics)] - #![feature(format_args_nl)] -+#![feature(global_asm)] - #![feature(panic_info_message)] - #![feature(trait_alias)] - #![no_main] -@@ -144,6 +145,8 @@ +@@ -144,6 +144,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -962,7 +963,7 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } -@@ -196,13 +199,28 @@ +@@ -196,13 +198,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs index c2f5fe0b7..02798fdc0 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S deleted file mode 100644 index fd7b1f930..000000000 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.S +++ /dev/null @@ -1,138 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -/// Call the function provided by parameter `\handler` after saving the exception context. Provide -/// the context as the first parameter to '\handler'. -.macro CALL_WITH_CONTEXT handler - // Make room on the stack for the exception context. - sub sp, sp, #16 * 17 - - // Store all general purpose registers on the stack. - stp x0, x1, [sp, #16 * 0] - stp x2, x3, [sp, #16 * 1] - stp x4, x5, [sp, #16 * 2] - stp x6, x7, [sp, #16 * 3] - stp x8, x9, [sp, #16 * 4] - stp x10, x11, [sp, #16 * 5] - stp x12, x13, [sp, #16 * 6] - stp x14, x15, [sp, #16 * 7] - stp x16, x17, [sp, #16 * 8] - stp x18, x19, [sp, #16 * 9] - stp x20, x21, [sp, #16 * 10] - stp x22, x23, [sp, #16 * 11] - stp x24, x25, [sp, #16 * 12] - stp x26, x27, [sp, #16 * 13] - stp x28, x29, [sp, #16 * 14] - - // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). - mrs x1, ELR_EL1 - mrs x2, SPSR_EL1 - - stp lr, x1, [sp, #16 * 15] - str x2, [sp, #16 * 16] - - // x0 is the first argument for the function called through `\handler`. - mov x0, sp - - // Call `\handler`. - bl \handler - - // After returning from exception handling code, replay the saved context and return via `eret`. - b __exception_restore_context -.endm - -.macro FIQ_SUSPEND -1: wfe - b 1b -.endm - -//-------------------------------------------------------------------------------------------------- -// The exception vector table. -//-------------------------------------------------------------------------------------------------- -.section .text - -// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. -.align 11 - -// Export a symbol for the Rust code to use. -__exception_vector_start: - -// Current exception level with SP_EL0. -// -// .org sets the offset relative to section start. -// -// # Safety -// -// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. -.org 0x000 - CALL_WITH_CONTEXT current_el0_synchronous -.org 0x080 - CALL_WITH_CONTEXT current_el0_irq -.org 0x100 - FIQ_SUSPEND -.org 0x180 - CALL_WITH_CONTEXT current_el0_serror - -// Current exception level with SP_ELx, x > 0. -.org 0x200 - CALL_WITH_CONTEXT current_elx_synchronous -.org 0x280 - CALL_WITH_CONTEXT current_elx_irq -.org 0x300 - FIQ_SUSPEND -.org 0x380 - CALL_WITH_CONTEXT current_elx_serror - -// Lower exception level, AArch64 -.org 0x400 - CALL_WITH_CONTEXT lower_aarch64_synchronous -.org 0x480 - CALL_WITH_CONTEXT lower_aarch64_irq -.org 0x500 - FIQ_SUSPEND -.org 0x580 - CALL_WITH_CONTEXT lower_aarch64_serror - -// Lower exception level, AArch32 -.org 0x600 - CALL_WITH_CONTEXT lower_aarch32_synchronous -.org 0x680 - CALL_WITH_CONTEXT lower_aarch32_irq -.org 0x700 - FIQ_SUSPEND -.org 0x780 - CALL_WITH_CONTEXT lower_aarch32_serror -.org 0x800 - -//-------------------------------------------------------------------------------------------------- -// Helper functions -//-------------------------------------------------------------------------------------------------- -.section .text - -__exception_restore_context: - ldr w19, [sp, #16 * 16] - ldp lr, x20, [sp, #16 * 15] - - msr SPSR_EL1, x19 - msr ELR_EL1, x20 - - ldp x0, x1, [sp, #16 * 0] - ldp x2, x3, [sp, #16 * 1] - ldp x4, x5, [sp, #16 * 2] - ldp x6, x7, [sp, #16 * 3] - ldp x8, x9, [sp, #16 * 4] - ldp x10, x11, [sp, #16 * 5] - ldp x12, x13, [sp, #16 * 6] - ldp x14, x15, [sp, #16 * 7] - ldp x16, x17, [sp, #16 * 8] - ldp x18, x19, [sp, #16 * 9] - ldp x20, x21, [sp, #16 * 10] - ldp x22, x23, [sp, #16 * 11] - ldp x24, x25, [sp, #16 * 12] - ldp x26, x27, [sp, #16 * 13] - ldp x28, x29, [sp, #16 * 14] - - add sp, sp, #16 * 17 - - eret diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs index f08d0ce4a..21f0c68cf 100644 --- a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -16,7 +16,7 @@ use cortex_a::{asm, barrier, regs::*}; use register::InMemoryRegister; // Assembly counterpart to this file. -global_asm!(include_str!("exception.S")); +global_asm!(include_str!("exception.s")); //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld index 6c63ba10f..854f5b388 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -25,7 +30,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs index bb18dd201..81233775d 100644 --- a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs +++ b/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs @@ -97,12 +97,6 @@ fn rx_end_exclusive() -> usize { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/12_exceptions_part1_groundwork/src/cpu.rs b/12_exceptions_part1_groundwork/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/12_exceptions_part1_groundwork/src/cpu.rs +++ b/12_exceptions_part1_groundwork/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/12_exceptions_part1_groundwork/src/cpu/smp.rs b/12_exceptions_part1_groundwork/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/12_exceptions_part1_groundwork/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/12_exceptions_part1_groundwork/src/main.rs b/12_exceptions_part1_groundwork/src/main.rs index aed3404a0..2af372ab9 100644 --- a/12_exceptions_part1_groundwork/src/main.rs +++ b/12_exceptions_part1_groundwork/src/main.rs @@ -100,11 +100,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] diff --git a/13_integrated_testing/README.md b/13_integrated_testing/README.md index 65423fc4c..046607ba7 100644 --- a/13_integrated_testing/README.md +++ b/13_integrated_testing/README.md @@ -1157,7 +1157,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_in diff -uNr 12_exceptions_part1_groundwork/src/cpu.rs 13_integrated_testing/src/cpu.rs --- 12_exceptions_part1_groundwork/src/cpu.rs +++ 13_integrated_testing/src/cpu.rs -@@ -16,3 +16,6 @@ +@@ -14,3 +14,6 @@ // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- pub use arch_cpu::{nop, wait_forever}; @@ -1194,7 +1194,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs --- 12_exceptions_part1_groundwork/src/lib.rs +++ 13_integrated_testing/src/lib.rs -@@ -0,0 +1,172 @@ +@@ -0,0 +1,171 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1299,11 +1299,10 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li +//! +//! # Boot flow +//! -+//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -+//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. ++//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. ++//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! -+//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + +#![allow(clippy::clippy::upper_case_acronyms)] @@ -1371,7 +1370,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs --- 12_exceptions_part1_groundwork/src/main.rs +++ 13_integrated_testing/src/main.rs -@@ -6,131 +6,12 @@ +@@ -6,130 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1469,11 +1468,10 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m -//! -//! # Boot flow -//! --//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. --//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +-//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +-//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! --//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - -#![allow(clippy::clippy::upper_case_acronyms)] @@ -1505,7 +1503,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m /// Early init code. /// -@@ -141,6 +22,7 @@ +@@ -140,6 +22,7 @@ /// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. @@ -1513,7 +1511,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -167,9 +49,7 @@ +@@ -166,9 +49,7 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1523,7 +1521,7 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); -@@ -196,31 +76,6 @@ +@@ -195,31 +76,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1797,7 +1795,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate +#![no_main] +#![no_std] + -+use libkernel::{bsp, console, exception, print}; ++use libkernel::{bsp, console, cpu, exception, print}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { @@ -1820,7 +1818,7 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. -+ loop {} ++ cpu::wait_forever() +} diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_testing/tests/01_timer_sanity.rs diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs b/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs index c2f5fe0b7..02798fdc0 100644 --- a/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs +++ b/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/boot.s b/13_integrated_testing/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/13_integrated_testing/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs b/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/13_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/13_integrated_testing/src/_arch/aarch64/exception.S b/13_integrated_testing/src/_arch/aarch64/exception.S deleted file mode 100644 index fd7b1f930..000000000 --- a/13_integrated_testing/src/_arch/aarch64/exception.S +++ /dev/null @@ -1,138 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -/// Call the function provided by parameter `\handler` after saving the exception context. Provide -/// the context as the first parameter to '\handler'. -.macro CALL_WITH_CONTEXT handler - // Make room on the stack for the exception context. - sub sp, sp, #16 * 17 - - // Store all general purpose registers on the stack. - stp x0, x1, [sp, #16 * 0] - stp x2, x3, [sp, #16 * 1] - stp x4, x5, [sp, #16 * 2] - stp x6, x7, [sp, #16 * 3] - stp x8, x9, [sp, #16 * 4] - stp x10, x11, [sp, #16 * 5] - stp x12, x13, [sp, #16 * 6] - stp x14, x15, [sp, #16 * 7] - stp x16, x17, [sp, #16 * 8] - stp x18, x19, [sp, #16 * 9] - stp x20, x21, [sp, #16 * 10] - stp x22, x23, [sp, #16 * 11] - stp x24, x25, [sp, #16 * 12] - stp x26, x27, [sp, #16 * 13] - stp x28, x29, [sp, #16 * 14] - - // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). - mrs x1, ELR_EL1 - mrs x2, SPSR_EL1 - - stp lr, x1, [sp, #16 * 15] - str x2, [sp, #16 * 16] - - // x0 is the first argument for the function called through `\handler`. - mov x0, sp - - // Call `\handler`. - bl \handler - - // After returning from exception handling code, replay the saved context and return via `eret`. - b __exception_restore_context -.endm - -.macro FIQ_SUSPEND -1: wfe - b 1b -.endm - -//-------------------------------------------------------------------------------------------------- -// The exception vector table. -//-------------------------------------------------------------------------------------------------- -.section .text - -// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. -.align 11 - -// Export a symbol for the Rust code to use. -__exception_vector_start: - -// Current exception level with SP_EL0. -// -// .org sets the offset relative to section start. -// -// # Safety -// -// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. -.org 0x000 - CALL_WITH_CONTEXT current_el0_synchronous -.org 0x080 - CALL_WITH_CONTEXT current_el0_irq -.org 0x100 - FIQ_SUSPEND -.org 0x180 - CALL_WITH_CONTEXT current_el0_serror - -// Current exception level with SP_ELx, x > 0. -.org 0x200 - CALL_WITH_CONTEXT current_elx_synchronous -.org 0x280 - CALL_WITH_CONTEXT current_elx_irq -.org 0x300 - FIQ_SUSPEND -.org 0x380 - CALL_WITH_CONTEXT current_elx_serror - -// Lower exception level, AArch64 -.org 0x400 - CALL_WITH_CONTEXT lower_aarch64_synchronous -.org 0x480 - CALL_WITH_CONTEXT lower_aarch64_irq -.org 0x500 - FIQ_SUSPEND -.org 0x580 - CALL_WITH_CONTEXT lower_aarch64_serror - -// Lower exception level, AArch32 -.org 0x600 - CALL_WITH_CONTEXT lower_aarch32_synchronous -.org 0x680 - CALL_WITH_CONTEXT lower_aarch32_irq -.org 0x700 - FIQ_SUSPEND -.org 0x780 - CALL_WITH_CONTEXT lower_aarch32_serror -.org 0x800 - -//-------------------------------------------------------------------------------------------------- -// Helper functions -//-------------------------------------------------------------------------------------------------- -.section .text - -__exception_restore_context: - ldr w19, [sp, #16 * 16] - ldp lr, x20, [sp, #16 * 15] - - msr SPSR_EL1, x19 - msr ELR_EL1, x20 - - ldp x0, x1, [sp, #16 * 0] - ldp x2, x3, [sp, #16 * 1] - ldp x4, x5, [sp, #16 * 2] - ldp x6, x7, [sp, #16 * 3] - ldp x8, x9, [sp, #16 * 4] - ldp x10, x11, [sp, #16 * 5] - ldp x12, x13, [sp, #16 * 6] - ldp x14, x15, [sp, #16 * 7] - ldp x16, x17, [sp, #16 * 8] - ldp x18, x19, [sp, #16 * 9] - ldp x20, x21, [sp, #16 * 10] - ldp x22, x23, [sp, #16 * 11] - ldp x24, x25, [sp, #16 * 12] - ldp x26, x27, [sp, #16 * 13] - ldp x28, x29, [sp, #16 * 14] - - add sp, sp, #16 * 17 - - eret diff --git a/13_integrated_testing/src/_arch/aarch64/exception.rs b/13_integrated_testing/src/_arch/aarch64/exception.rs index e67841f9f..1ee0a1982 100644 --- a/13_integrated_testing/src/_arch/aarch64/exception.rs +++ b/13_integrated_testing/src/_arch/aarch64/exception.rs @@ -16,7 +16,7 @@ use cortex_a::{barrier, regs::*}; use register::InMemoryRegister; // Assembly counterpart to this file. -global_asm!(include_str!("exception.S")); +global_asm!(include_str!("exception.s")); //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_integrated_testing/src/_arch/aarch64/exception.s b/13_integrated_testing/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/13_integrated_testing/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/13_integrated_testing/src/bsp/raspberrypi/link.ld b/13_integrated_testing/src/bsp/raspberrypi/link.ld index 6c63ba10f..854f5b388 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/link.ld +++ b/13_integrated_testing/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -25,7 +30,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory.rs b/13_integrated_testing/src/bsp/raspberrypi/memory.rs index bb18dd201..81233775d 100644 --- a/13_integrated_testing/src/bsp/raspberrypi/memory.rs +++ b/13_integrated_testing/src/bsp/raspberrypi/memory.rs @@ -97,12 +97,6 @@ fn rx_end_exclusive() -> usize { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/13_integrated_testing/src/cpu.rs b/13_integrated_testing/src/cpu.rs index 36b6a9d4c..61c411fcd 100644 --- a/13_integrated_testing/src/cpu.rs +++ b/13_integrated_testing/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/13_integrated_testing/src/cpu/smp.rs b/13_integrated_testing/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/13_integrated_testing/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/13_integrated_testing/src/lib.rs b/13_integrated_testing/src/lib.rs index b094daf3d..745122f85 100644 --- a/13_integrated_testing/src/lib.rs +++ b/13_integrated_testing/src/lib.rs @@ -102,11 +102,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] diff --git a/13_integrated_testing/tests/00_console_sanity.rs b/13_integrated_testing/tests/00_console_sanity.rs index c3754aa9f..ad7fd2bf5 100644 --- a/13_integrated_testing/tests/00_console_sanity.rs +++ b/13_integrated_testing/tests/00_console_sanity.rs @@ -8,7 +8,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, console, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { @@ -31,5 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - loop {} + cpu::wait_forever() } diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/14_exceptions_part2_peripheral_IRQs/README.md index 54e33071c..b78da7162 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/14_exceptions_part2_peripheral_IRQs/README.md @@ -744,6 +744,40 @@ Minipush 1.0 ## Diff to previous ```diff +diff -uNr 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +--- 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs ++++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +@@ -0,0 +1,29 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! Architectural symmetric multiprocessing. ++//! ++//! # Orientation ++//! ++//! Since arch modules are imported into generic modules using the path attribute, the path of this ++//! file is: ++//! ++//! crate::cpu::smp::arch_smp ++ ++use cortex_a::regs::*; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Return the executing core's id. ++#[inline(always)] ++pub fn core_id() -> T ++where ++ T: From, ++{ ++ const CORE_MASK: u64 = 0b11; ++ ++ T::from((MPIDR_EL1.get() & CORE_MASK) as u8) ++} + diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs --- 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs @@ -857,19 +891,6 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception.rs 14_exceptions_par #[no_mangle] -diff -uNr 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs ---- 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs -@@ -149,7 +149,7 @@ - barrier::isb(barrier::SY); - - // Enable the MMU and turn on data and instruction caching. -- SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); -+ SCTLR_EL1.modify(SCTLR_EL1::M::Enable); - - // Force MMU init to complete before next instruction. - barrier::isb(barrier::SY); - diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs --- 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs +++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -2136,6 +2157,38 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi.rs 14_exceptions_part2_perip //-------------------------------------------------------------------------------------------------- // Public Code +diff -uNr 13_integrated_testing/src/cpu/smp.rs 14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +--- 13_integrated_testing/src/cpu/smp.rs ++++ 14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +@@ -0,0 +1,14 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! Symmetric multiprocessing. ++ ++#[cfg(target_arch = "aarch64")] ++#[path = "../_arch/aarch64/cpu/smp.rs"] ++mod arch_smp; ++ ++//-------------------------------------------------------------------------------------------------- ++// Architectural Public Reexports ++//-------------------------------------------------------------------------------------------------- ++pub use arch_smp::core_id; + +diff -uNr 13_integrated_testing/src/cpu.rs 14_exceptions_part2_peripheral_IRQs/src/cpu.rs +--- 13_integrated_testing/src/cpu.rs ++++ 14_exceptions_part2_peripheral_IRQs/src/cpu.rs +@@ -10,6 +10,8 @@ + + mod boot; + ++pub mod smp; ++ + //-------------------------------------------------------------------------------------------------- + // Architectural Public Reexports + //-------------------------------------------------------------------------------------------------- + diff -uNr 13_integrated_testing/src/driver.rs 14_exceptions_part2_peripheral_IRQs/src/driver.rs --- 13_integrated_testing/src/driver.rs +++ 14_exceptions_part2_peripheral_IRQs/src/driver.rs @@ -2309,7 +2362,7 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs --- 13_integrated_testing/src/lib.rs +++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -111,6 +111,7 @@ +@@ -110,6 +110,7 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -2317,7 +2370,7 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -138,6 +139,7 @@ +@@ -137,6 +138,7 @@ pub mod exception; pub mod memory; pub mod print; diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs index c2f5fe0b7..02798fdc0 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S deleted file mode 100644 index fd7b1f930..000000000 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.S +++ /dev/null @@ -1,138 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -/// Call the function provided by parameter `\handler` after saving the exception context. Provide -/// the context as the first parameter to '\handler'. -.macro CALL_WITH_CONTEXT handler - // Make room on the stack for the exception context. - sub sp, sp, #16 * 17 - - // Store all general purpose registers on the stack. - stp x0, x1, [sp, #16 * 0] - stp x2, x3, [sp, #16 * 1] - stp x4, x5, [sp, #16 * 2] - stp x6, x7, [sp, #16 * 3] - stp x8, x9, [sp, #16 * 4] - stp x10, x11, [sp, #16 * 5] - stp x12, x13, [sp, #16 * 6] - stp x14, x15, [sp, #16 * 7] - stp x16, x17, [sp, #16 * 8] - stp x18, x19, [sp, #16 * 9] - stp x20, x21, [sp, #16 * 10] - stp x22, x23, [sp, #16 * 11] - stp x24, x25, [sp, #16 * 12] - stp x26, x27, [sp, #16 * 13] - stp x28, x29, [sp, #16 * 14] - - // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). - mrs x1, ELR_EL1 - mrs x2, SPSR_EL1 - - stp lr, x1, [sp, #16 * 15] - str x2, [sp, #16 * 16] - - // x0 is the first argument for the function called through `\handler`. - mov x0, sp - - // Call `\handler`. - bl \handler - - // After returning from exception handling code, replay the saved context and return via `eret`. - b __exception_restore_context -.endm - -.macro FIQ_SUSPEND -1: wfe - b 1b -.endm - -//-------------------------------------------------------------------------------------------------- -// The exception vector table. -//-------------------------------------------------------------------------------------------------- -.section .text - -// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. -.align 11 - -// Export a symbol for the Rust code to use. -__exception_vector_start: - -// Current exception level with SP_EL0. -// -// .org sets the offset relative to section start. -// -// # Safety -// -// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. -.org 0x000 - CALL_WITH_CONTEXT current_el0_synchronous -.org 0x080 - CALL_WITH_CONTEXT current_el0_irq -.org 0x100 - FIQ_SUSPEND -.org 0x180 - CALL_WITH_CONTEXT current_el0_serror - -// Current exception level with SP_ELx, x > 0. -.org 0x200 - CALL_WITH_CONTEXT current_elx_synchronous -.org 0x280 - CALL_WITH_CONTEXT current_elx_irq -.org 0x300 - FIQ_SUSPEND -.org 0x380 - CALL_WITH_CONTEXT current_elx_serror - -// Lower exception level, AArch64 -.org 0x400 - CALL_WITH_CONTEXT lower_aarch64_synchronous -.org 0x480 - CALL_WITH_CONTEXT lower_aarch64_irq -.org 0x500 - FIQ_SUSPEND -.org 0x580 - CALL_WITH_CONTEXT lower_aarch64_serror - -// Lower exception level, AArch32 -.org 0x600 - CALL_WITH_CONTEXT lower_aarch32_synchronous -.org 0x680 - CALL_WITH_CONTEXT lower_aarch32_irq -.org 0x700 - FIQ_SUSPEND -.org 0x780 - CALL_WITH_CONTEXT lower_aarch32_serror -.org 0x800 - -//-------------------------------------------------------------------------------------------------- -// Helper functions -//-------------------------------------------------------------------------------------------------- -.section .text - -__exception_restore_context: - ldr w19, [sp, #16 * 16] - ldp lr, x20, [sp, #16 * 15] - - msr SPSR_EL1, x19 - msr ELR_EL1, x20 - - ldp x0, x1, [sp, #16 * 0] - ldp x2, x3, [sp, #16 * 1] - ldp x4, x5, [sp, #16 * 2] - ldp x6, x7, [sp, #16 * 3] - ldp x8, x9, [sp, #16 * 4] - ldp x10, x11, [sp, #16 * 5] - ldp x12, x13, [sp, #16 * 6] - ldp x14, x15, [sp, #16 * 7] - ldp x16, x17, [sp, #16 * 8] - ldp x18, x19, [sp, #16 * 9] - ldp x20, x21, [sp, #16 * 10] - ldp x22, x23, [sp, #16 * 11] - ldp x24, x25, [sp, #16 * 12] - ldp x26, x27, [sp, #16 * 13] - ldp x28, x29, [sp, #16 * 14] - - add sp, sp, #16 * 17 - - eret diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs index de4f1f185..5c44abb70 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -17,7 +17,7 @@ use cortex_a::{barrier, regs::*}; use register::InMemoryRegister; // Assembly counterpart to this file. -global_asm!(include_str!("exception.S")); +global_asm!(include_str!("exception.s")); //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index cbf04a98e..29e8125d7 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -149,7 +149,7 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { barrier::isb(barrier::SY); // Enable the MMU and turn on data and instruction caching. - SCTLR_EL1.modify(SCTLR_EL1::M::Enable); + SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); // Force MMU init to complete before next instruction. barrier::isb(barrier::SY); diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld index 6c63ba10f..854f5b388 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld @@ -17,6 +17,11 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -25,7 +30,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs index d07f2dc34..8a58a17eb 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -101,12 +101,6 @@ fn rx_end_exclusive() -> usize { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/14_exceptions_part2_peripheral_IRQs/src/lib.rs index 4ea416fd5..c4e744608 100644 --- a/14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/14_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -102,11 +102,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs index c3754aa9f..ad7fd2bf5 100644 --- a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +++ b/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs @@ -8,7 +8,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, console, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { @@ -31,5 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - loop {} + cpu::wait_forever() } diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/15_virtual_mem_part2_mmio_remap/README.md index c512c51c6..1745d4b14 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/15_virtual_mem_part2_mmio_remap/README.md @@ -369,19 +369,6 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs -@@ -56,7 +56,7 @@ - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. -- SP_EL1.set(bsp::memory::boot_core_stack_end() as u64); -+ SP_EL1.set(bsp::memory::phys_boot_core_stack_end().into_usize() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() - diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs --- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -805,15 +792,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 self.configure_translation_control(); -@@ -149,7 +140,7 @@ - barrier::isb(barrier::SY); - - // Enable the MMU and turn on data and instruction caching. -- SCTLR_EL1.modify(SCTLR_EL1::M::Enable); -+ SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); - - // Force MMU init to complete before next instruction. - barrier::isb(barrier::SY); @@ -162,22 +153,3 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } @@ -1478,7 +1456,19 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs 15_v diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld -@@ -37,6 +37,7 @@ +@@ -17,11 +17,6 @@ + SECTIONS + { + . = __rpi_load_addr; +- /* ^ */ +- /* | stack */ +- /* | growth */ +- /* | direction */ +- __boot_core_stack_end_exclusive = .; /* | */ + + /*********************************************************************************************** + * Code + RO Data + Global Offset Table +@@ -51,6 +46,7 @@ /*********************************************************************************************** * Data + BSS ***********************************************************************************************/ @@ -1486,7 +1476,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -@@ -49,4 +50,21 @@ +@@ -63,4 +59,23 @@ . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; } :NONE @@ -1504,9 +1494,11 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir + /*********************************************************************************************** + * Boot Core Stack + ***********************************************************************************************/ -+ __boot_core_stack_start = .; -+ . += 512K; -+ __boot_core_stack_end_exclusive = .; ++ __boot_core_stack_start = .; /* ^ */ ++ /* | stack */ ++ . += 512K; /* | growth */ ++ /* | direction */ ++ __boot_core_stack_end_exclusive = .; /* | */ } diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -2002,19 +1994,6 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v } //-------------------------------------------------------------------------------------------------- -@@ -103,8 +193,10 @@ - - /// Exclusive end address of the boot core's stack. - #[inline(always)] --pub fn boot_core_stack_end() -> usize { -- rx_start() -+pub fn phys_boot_core_stack_end() -> Address { -+ // The binary is still identity mapped, so we don't need to convert here. -+ let end = virt_boot_core_stack_start().into_usize() + boot_core_stack_size(); -+ Address::new(end) - } - - /// Return the inclusive range spanning the .bss section. diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs --- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs @@ -2138,7 +2117,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs --- 14_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 15_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -112,6 +112,8 @@ +@@ -111,6 +111,8 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] @@ -2147,7 +2126,7 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -133,6 +135,7 @@ +@@ -132,6 +134,7 @@ mod synchronization; pub mod bsp; diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs index 276965146..02798fdc0 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs @@ -11,26 +11,24 @@ //! //! crate::cpu::boot::arch_boot -use crate::{bsp, cpu}; +use crate::runtime_init; use cortex_a::{asm, regs::*}; +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -/// Transition from EL2 to EL1. +/// Prepares the transition from EL2 to EL1. /// /// # Safety /// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with -/// `runtime_init::runtime_init()`. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. #[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - use crate::runtime_init; - +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -55,35 +53,27 @@ unsafe fn el2_to_el1_transition() -> ! { // Second, let the link register point to runtime_init(). ELR_EL2.set(runtime_init::runtime_init as *const () as u64); - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::memory::phys_boot_core_stack_end().into_usize() as u64); - - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. - asm::eret() + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); } //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// The entry of the `kernel` binary. +/// The Rust entry of the `kernel` binary. /// -/// The function must be named `_start`, because the linker is looking for this exact name. +/// The function is called from the assembly `_start` function. /// /// # Safety /// -/// - Linker script must ensure to place this function where it is expected by the target machine. -/// - We have to hope that the compiler omits any stack pointer usage, because we are not setting up -/// a stack for EL2. +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. #[no_mangle] -pub unsafe fn _start() -> ! { - // Expect the boot core to start in EL2. - if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() } diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..be81b20a2 --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. x0 holds the function argument provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S deleted file mode 100644 index fd7b1f930..000000000 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.S +++ /dev/null @@ -1,138 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -/// Call the function provided by parameter `\handler` after saving the exception context. Provide -/// the context as the first parameter to '\handler'. -.macro CALL_WITH_CONTEXT handler - // Make room on the stack for the exception context. - sub sp, sp, #16 * 17 - - // Store all general purpose registers on the stack. - stp x0, x1, [sp, #16 * 0] - stp x2, x3, [sp, #16 * 1] - stp x4, x5, [sp, #16 * 2] - stp x6, x7, [sp, #16 * 3] - stp x8, x9, [sp, #16 * 4] - stp x10, x11, [sp, #16 * 5] - stp x12, x13, [sp, #16 * 6] - stp x14, x15, [sp, #16 * 7] - stp x16, x17, [sp, #16 * 8] - stp x18, x19, [sp, #16 * 9] - stp x20, x21, [sp, #16 * 10] - stp x22, x23, [sp, #16 * 11] - stp x24, x25, [sp, #16 * 12] - stp x26, x27, [sp, #16 * 13] - stp x28, x29, [sp, #16 * 14] - - // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). - mrs x1, ELR_EL1 - mrs x2, SPSR_EL1 - - stp lr, x1, [sp, #16 * 15] - str x2, [sp, #16 * 16] - - // x0 is the first argument for the function called through `\handler`. - mov x0, sp - - // Call `\handler`. - bl \handler - - // After returning from exception handling code, replay the saved context and return via `eret`. - b __exception_restore_context -.endm - -.macro FIQ_SUSPEND -1: wfe - b 1b -.endm - -//-------------------------------------------------------------------------------------------------- -// The exception vector table. -//-------------------------------------------------------------------------------------------------- -.section .text - -// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. -.align 11 - -// Export a symbol for the Rust code to use. -__exception_vector_start: - -// Current exception level with SP_EL0. -// -// .org sets the offset relative to section start. -// -// # Safety -// -// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. -.org 0x000 - CALL_WITH_CONTEXT current_el0_synchronous -.org 0x080 - CALL_WITH_CONTEXT current_el0_irq -.org 0x100 - FIQ_SUSPEND -.org 0x180 - CALL_WITH_CONTEXT current_el0_serror - -// Current exception level with SP_ELx, x > 0. -.org 0x200 - CALL_WITH_CONTEXT current_elx_synchronous -.org 0x280 - CALL_WITH_CONTEXT current_elx_irq -.org 0x300 - FIQ_SUSPEND -.org 0x380 - CALL_WITH_CONTEXT current_elx_serror - -// Lower exception level, AArch64 -.org 0x400 - CALL_WITH_CONTEXT lower_aarch64_synchronous -.org 0x480 - CALL_WITH_CONTEXT lower_aarch64_irq -.org 0x500 - FIQ_SUSPEND -.org 0x580 - CALL_WITH_CONTEXT lower_aarch64_serror - -// Lower exception level, AArch32 -.org 0x600 - CALL_WITH_CONTEXT lower_aarch32_synchronous -.org 0x680 - CALL_WITH_CONTEXT lower_aarch32_irq -.org 0x700 - FIQ_SUSPEND -.org 0x780 - CALL_WITH_CONTEXT lower_aarch32_serror -.org 0x800 - -//-------------------------------------------------------------------------------------------------- -// Helper functions -//-------------------------------------------------------------------------------------------------- -.section .text - -__exception_restore_context: - ldr w19, [sp, #16 * 16] - ldp lr, x20, [sp, #16 * 15] - - msr SPSR_EL1, x19 - msr ELR_EL1, x20 - - ldp x0, x1, [sp, #16 * 0] - ldp x2, x3, [sp, #16 * 1] - ldp x4, x5, [sp, #16 * 2] - ldp x6, x7, [sp, #16 * 3] - ldp x8, x9, [sp, #16 * 4] - ldp x10, x11, [sp, #16 * 5] - ldp x12, x13, [sp, #16 * 6] - ldp x14, x15, [sp, #16 * 7] - ldp x16, x17, [sp, #16 * 8] - ldp x18, x19, [sp, #16 * 9] - ldp x20, x21, [sp, #16 * 10] - ldp x22, x23, [sp, #16 * 11] - ldp x24, x25, [sp, #16 * 12] - ldp x26, x27, [sp, #16 * 13] - ldp x28, x29, [sp, #16 * 14] - - add sp, sp, #16 * 17 - - eret diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs index b93387344..4a2c8de96 100644 --- a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -21,7 +21,7 @@ use cortex_a::{barrier, regs::*}; use register::InMemoryRegister; // Assembly counterpart to this file. -global_asm!(include_str!("exception.S")); +global_asm!(include_str!("exception.s")); //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld index f2b5d05d7..7fddf3ec5 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -25,7 +25,16 @@ SECTIONS .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx @@ -64,7 +73,9 @@ SECTIONS /*********************************************************************************************** * Boot Core Stack ***********************************************************************************************/ - __boot_core_stack_start = .; - . += 512K; - __boot_core_stack_end_exclusive = .; + __boot_core_stack_start = .; /* ^ */ + /* | stack */ + . += 512K; /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ } diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs index 19545c003..7f571e080 100644 --- a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +++ b/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -191,14 +191,6 @@ fn phys_addr_space_end() -> Address { // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn phys_boot_core_stack_end() -> Address { - // The binary is still identity mapped, so we don't need to convert here. - let end = virt_boot_core_stack_start().into_usize() + boot_core_stack_size(); - Address::new(end) -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/15_virtual_mem_part2_mmio_remap/src/lib.rs index 30f684af4..3346134ad 100644 --- a/15_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/15_virtual_mem_part2_mmio_remap/src/lib.rs @@ -102,11 +102,10 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs index c3754aa9f..ad7fd2bf5 100644 --- a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs +++ b/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs @@ -8,7 +8,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, console, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { @@ -31,5 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - loop {} + cpu::wait_forever() } diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index d45cc1ff0028c6d32ecd176a828fc440da668d97..8eed7dba0e7d56b27a14d14f981f77b599d9b6c3 100755 GIT binary patch delta 1590 zcmb7EUrbw782?UhYkM~k3jM=??zX~g3uY6_27^H_-6Ct9uvLs?QvpfX9}*+Phb@uN z8XuU!gyWe*At8mAd2vKZFBQ+?jD=`IOUA$XV6&9#Ilb3}?L|Gwmvg`G z{Jwv`@0@$ryY4+W0X26VTJb#}f;KN`R>A?q2b zq+mCMf{PwFICCV0G-65At0_YoC?!4ITPRFcxjHmY#&a%{IqjEw!?oDmSc;dO*QK}~ zNpTMFAuU|8gZ-yDuKj26i_ni9OEa5s^Dmo`ukMqNbq7#5x}-a$N6~0Q_I+eN;lVor zhO+=ZO&$i^bGr@?Es|$)UPSZJ@to60)8Qg|WB2}vTaTgje_JrKBEGWoiFHp#Qru8gH1!nKYKuVs9A*+6P%PzJgMuIZr@~}^pdH(a&zw!c=Z!bO6-FaPXj&MKRuTFDoN%EWt~sqtLg5AY-$~x zHWUTYeGAi$a*U{+JPomTD0Q1x^pb<7^W;}%hiY+`-Ge)Iq2SvQ(wE=0BjmykE)?{E zoBWn<8Avvzc%2yYn@t{G2eC6$7v6`MPnJj6HxC~K`41u1pz7>Qp!!zJ{6~ymCUlC! ztV+XTQ_B5EQ_71BD?EZ$I@@6l6$sViPAyGoZ@|AkkVn{T7@lFt8>T&)jtVknYA&Nq zC+Ya&`o|>Jj$^J0$LU0tom%NAZ7b>2Ng21sDw1#R-lMYT;=dWqnCVbfDU0{YK|1&O z#b6+7vhP+e88-`b0N2f4%lrWxpv)zTAQkxR@uCDx#aPO>RBq>_#bRlEh%^sg;t($1 zQtFmD&Fw7@r2cHtOLf!+dYHXLi(?g-e!tSUOryF_kQAN;rwp zZ=la$c|a#EeGS-|mPh7NauuVWqoSn(tR=Bdxd(G~nBitAj*Wuh5DEpMqaL$y#yMeX zkqY~+2M|-EY1wKh6(!2URGc%=Lw?G#C0lO*%+h|R+$OQcr-cpP3^fz7+{zr;yB5hz zfz|!GMtS8&)Yo34>b}Fz*^eK4x}w7VLSvBLm07Y;U`5vmzw3KrWcR~=hO;EGs~WWt zM`3lvUq#9pZ7s7-saSetGV4*_{p4by;QPN`57aEVYSf*@(PUvWa{M4D+0Pd%`(0I! zsCuSnmG34;q{p=D$1^KhTo>5%;B0`aG}inoRwoH@WY= zbIj?!)kE*!MnZUQuk3%>hFC`n|%6uIsXckgJa6uCzi4%V8~U$4PVsNi2G8 zAWbcas`*2f8fuAE!&|~h>VsTs8*PWhg7fgB_2XxJtt_}y&DQ-lRLxFRO(3k(O3r$T z_uGWf|6N3y`X-WmZ!_`g&CNvXBlz65lltZ}wnKK>IsbD0Z`8R~&+-VF*oH-I z*v3u|!r_9KX#4zP!BJ|_{7@|oIYWi4Tf8-$i?Ka4%pU0a!v_@Jl@vM%H>6rDDJ3ab zE#wuwLSKjBBEvCWL`hiqd1$F?Wzyc^%mFXdt2F&i}h^K z7K^^&g)_xR_L2be39)ELEWk3q>yw5Ynch#-Vmx5n8jBwLxhpN&BJpl#NrPx3al=;5 zk$8u$PiB4OLR6e0@!eS)c9$rjm9BK~PFFfi9UCGgeFRUk;3Q-h8^jYkF79GK@k|Dp z;2k(q;d#HOMrDH>25eRD**r(#SLvd);qUpF=JwzfH}8mKvbWIrJ2lEFgX{ zo!v&g@1u10YKc0$cq-bn?b66s6c(Kds{Xw*>}uc1dQh&Uo1}X_e8Qc?RE9KluKLl( zggDI0GfEe5T5*!UN!P&c3Y67mOu>veN@G#7(87#f(gbC84X5$13zp?f14%#XwX2O7 zPweFSS!80ATk^~0k##H~7^-4$)3rVS6b3j@(XsO+{w8d>_P7#N>{vO88{Csk=IsZU z+Mr*qpaT!rSl&dNp}o8* zwCXl}cqiG>#$&Nlvg4mvC#dC0$NxqBz_S$cQ#ai6x$b?`x1gBW4|>cs!R;8gvz(=v z%k)n0H2^anSsD<`-Pibvv6XV}ABE=PWr!! z@GQ(!+{-g0@7mwpOGpSCxTk`94#Je; Pmk7VsRqkDdTZ;cr4O^>0 diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 080ff5e6dc2f097df2754fccfcc5f3acd0ef3fbb..cbdd1b808832df622e2be50a4bb24e5437756355 100755 GIT binary patch delta 1876 zcmZ9MUu;uV9LIm>-WGb-!f4mKb{(*G14_#f<`uR|LT|f~xB-cjDi|NODr#U`p!i16 z+Xal#j-=Ov&fo(bJV=`nS;i6<|ImRiOEd-*eKTXm1X~CZ60;Jw>+jrq`6DN3PVf1B ze}B&Bch328OKFQT3LQ5B7Wtz%f&Ee7DnkItJiz`Y0Cg#VyVuX(&@{~hV1p+|$pWYS z;Qyt>_x`GN!gU+SRjn4LFh=?u#<(d#Bm|j6$O@B(F-{cxY0Qxf9bdAyW<@3uc7qgn z!>2sUgynXKbfb8Nhsgs>EX(P0C>Bu2#2bz~_J&)RNFj7e9c+B$^Vc?r97I2XgUOfC zFLd0c2e+e-gZXtiq*nCPlMSxa{VJI2Hu{yL5cvQUbQa)pSpv122w67ovVw|K+Bd8O zSDAMd9Yn5582l^|DtQi)JFRl0M*y{71Xt?{aBW20l_5TT0&TsfX16GJTtxez=#y9+ zWtvZ)H|?`52HKgws9@j<-xMzMLh;F9R~NKo*6EX2ayM%3n3^X8o6}e7Fi+!!X}|leZV`Sj%RtML}47Slb4R90}M!9Yi2Io#5(4n4P#D1)-9` zxz#Oq45MvK^eIv=l4PWrv4nFNjXh|52DjV)RKb;v5bIN(mG6f8~4IA7)+1F zKo1~9?6j_75X-3PM~GR3c!<=5dl6y=kE5M*Jf=8+6u(D`hmc|oLhM0cH3*RvmYUa+ z3>u?>m*x_b=xhA@Bt^^oHZnxd)P6$0wtVwUYKF3t1BsKB7_0^^1#KiNpbRPlu6 z;;>!~@R>yXUu}dv5V-#w^sP~OoL`3))q?ZMObebIqNc2wgk3Xq_`KwVJv*vlt2~vp zWD`Qe=)mG^`j!{Cu;9@e#41)7x6+-$AEs1glPc5ST)zBk1IrHIFOU3jBns2kr#- zoiO-4iXZyq5%0xh+x-x@h<@9Cpe`aTB$04($$Xp|4&_enqDxTM z?O<>7uMih2@_i)8R*hHKE#0RNyPBytTTAoq&yu-vqPFBU{@-gTnXY{!ZNI;ldHIXI z%<(_iC;D#UTHDhad?canKSYge{sJB*s=vui9PJbf*oH(;r_7jTqjY*~NMI zS(bmh{QIM<&ukatK3at?AH>=7<~9;%i{_?Lb!-TZ4WVxqO?aEo;m-qkdF3zpk^hvOYc7>M(V}7|^|8<#(x*^^RHg)-gySHj zeksZ}z&SdnN~9Cql;C`^u(g$VSlnv!_HuXtT&?(isY}}r430XVe7wD*!}(M&gsd#F ztkp)YvhS^5j|lY|w&Vfz%3lF4?3B8OW@gUM;4fa&*J}E^ar?DwulBJHp|0eP%~V&V zwnrw^b$2=go3#Dvx@s?ZO=##9ZGTbQtK|iHttHg3COul2b%(Jv`5a%uy`)JG%huV% z{pyl7&$jB^ ! { - use crate::runtime_init; - - if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { - SP.set(bsp::memory::boot_core_stack_end() as u64); - runtime_init::runtime_init() - } else { - // If not core0, infinitely wait for events. - cpu::wait_forever() - } +pub unsafe fn _start_rust() -> ! { + runtime_init::runtime_init() } diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..ad4a26892 --- /dev/null +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. + ldr x0, =__boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs deleted file mode 100644 index b9fdd0f73..000000000 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/smp.rs +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Architectural symmetric multiprocessing. -//! -//! # Orientation -//! -//! Since arch modules are imported into generic modules using the path attribute, the path of this -//! file is: -//! -//! crate::cpu::smp::arch_smp - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the executing core's id. -#[inline(always)] -pub fn core_id() -> T -where - T: From, -{ - const CORE_MASK: u64 = 0b11; - - T::from((MPIDR_EL1.get() & CORE_MASK) as u8) -} diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs index 16ab927d7..d09ff9568 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/cpu.rs @@ -9,4 +9,6 @@ //-------------------------------------------------------------------------------------------------- /// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld index 87e6a976f..3b73b3c7e 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld +++ b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld @@ -17,15 +17,28 @@ PHDRS SECTIONS { . = __rpi_load_addr; + /* ^ */ + /* | stack */ + /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ /*********************************************************************************************** * Code + RO Data + Global Offset Table ***********************************************************************************************/ - __rx_start = .; .text : { KEEP(*(.text._start)) - *(.text*) + + /* Special constants (or statics in Rust speak) needed by _start(). + * + * They are placed in close proximity to _start() from where they will be read. This ensures + * that position-independent, PC-relative loads can be emitted. + */ + *(.text._start_arguments) + + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs index 56a3306e5..c6d65e36b 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs @@ -12,8 +12,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; // Symbols from the linker script. extern "Rust" { - static __rx_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; static __bss_end_inclusive: UnsafeCell; } @@ -50,30 +48,10 @@ pub(super) mod map { } } -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Start address of the Read+Execute (RX) range. -/// -/// # Safety -/// -/// - Value is provided by the linker script and must be trusted as-is. -#[inline(always)] -fn rx_start() -> usize { - unsafe { __rx_start.get() as usize } -} - //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- -/// Exclusive end address of the boot core's stack. -#[inline(always)] -pub fn boot_core_stack_end() -> usize { - rx_start() -} - /// Return the inclusive range spanning the .bss section. /// /// # Safety diff --git a/X1_JTAG_boot/src/cpu.rs b/X1_JTAG_boot/src/cpu.rs index 3834f183c..7a095cb12 100644 --- a/X1_JTAG_boot/src/cpu.rs +++ b/X1_JTAG_boot/src/cpu.rs @@ -10,8 +10,6 @@ mod arch_cpu; mod boot; -pub mod smp; - //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- diff --git a/X1_JTAG_boot/src/cpu/smp.rs b/X1_JTAG_boot/src/cpu/smp.rs deleted file mode 100644 index 38230ce1e..000000000 --- a/X1_JTAG_boot/src/cpu/smp.rs +++ /dev/null @@ -1,14 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Symmetric multiprocessing. - -#[cfg(target_arch = "aarch64")] -#[path = "../_arch/aarch64/cpu/smp.rs"] -mod arch_smp; - -//-------------------------------------------------------------------------------------------------- -// Architectural Public Reexports -//-------------------------------------------------------------------------------------------------- -pub use arch_smp::core_id; diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index aeff7a4da..19c864c4d 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -100,16 +100,16 @@ //! //! # Boot flow //! -//! 1. The kernel's entry point is the function [`cpu::boot::arch_boot::_start()`]. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. //! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. //! -//! [`cpu::boot::arch_boot::_start()`]: cpu/boot/arch_boot/fn._start.html //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html #![allow(clippy::clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] +#![feature(global_asm)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] diff --git a/rust-toolchain b/rust-toolchain index 1b5ba1c03..e0e7993d0 100644 --- a/rust-toolchain +++ b/rust-toolchain @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2021-03-06" +channel = "nightly-2021-03-18" components = ["llvm-tools-preview"] targets = ["aarch64-unknown-none-softfloat"] From f81eb7428d06fa11b6788231ed0e84eea6fad59b Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sat, 20 Mar 2021 10:47:39 +0100 Subject: [PATCH 043/214] Adapt tutorial numbers --- .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 {05_safe_globals => 04_safe_globals}/Makefile | 0 .../README.md | 39 ++-- {05_safe_globals => 04_safe_globals}/build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/bsp.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 110 +++++------ .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 60 +++--- .../build.rs | 0 .../demo_payload_rpi3.img | Bin .../demo_payload_rpi4.img | Bin .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 06_uart_chainloader/update.sh | 8 + .../.vscode/settings.json | 0 {08_timestamps => 07_timestamps}/Cargo.lock | 0 {08_timestamps => 07_timestamps}/Cargo.toml | 0 {08_timestamps => 07_timestamps}/Makefile | 0 {08_timestamps => 07_timestamps}/README.md | 90 ++++----- {08_timestamps => 07_timestamps}/build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/time.rs | 0 {08_timestamps => 07_timestamps}/src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 {08_timestamps => 07_timestamps}/src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 {08_timestamps => 07_timestamps}/src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 {08_timestamps => 07_timestamps}/src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 {08_timestamps => 07_timestamps}/src/time.rs | 0 07_uart_chainloader/update.sh | 8 - .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 8 +- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 46 ++--- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/exception.rs | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 62 +++--- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/exception.rs | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/memory/mmu.rs | 0 .../aarch64/memory/mmu/translation_table.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/bsp/raspberrypi/memory/mmu.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/memory/mmu.rs | 0 .../src/memory/mmu/translation_table.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 38 ++-- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/exception.rs | 0 .../src/_arch/aarch64/exception.s | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/memory/mmu.rs | 0 .../aarch64/memory/mmu/translation_table.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/bsp/raspberrypi/memory/mmu.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/memory/mmu.rs | 0 .../src/memory/mmu/translation_table.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../.cargo/config.toml | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 166 ++++++++-------- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/exception.rs | 0 .../src/_arch/aarch64/exception.s | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/memory/mmu.rs | 0 .../aarch64/memory/mmu/translation_table.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/bsp/raspberrypi/memory/mmu.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/lib.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/memory/mmu.rs | 0 .../src/memory/mmu/translation_table.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../test-macros/Cargo.toml | 0 .../test-macros/src/lib.rs | 0 .../test-types/Cargo.toml | 0 .../test-types/src/lib.rs | 0 .../tests/00_console_sanity.rb | 0 .../tests/00_console_sanity.rs | 0 .../tests/01_timer_sanity.rs | 0 .../tests/02_exception_sync_page_fault.rs | 0 .../tests/panic_exit_success/mod.rs | 0 .../tests/runner.rb | 0 .../.cargo/config.toml | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 178 +++++++++--------- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/cpu/smp.rs | 0 .../src/_arch/aarch64/exception.rs | 0 .../src/_arch/aarch64/exception.s | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/memory/mmu.rs | 0 .../aarch64/memory/mmu/translation_table.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/arm.rs | 0 .../src/bsp/device_driver/arm/gicv2.rs | 0 .../src/bsp/device_driver/arm/gicv2/gicc.rs | 0 .../src/bsp/device_driver/arm/gicv2/gicd.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../bcm/bcm2xxx_interrupt_controller.rs | 0 .../peripheral_ic.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/exception.rs | 0 .../bsp/raspberrypi/exception/asynchronous.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/bsp/raspberrypi/memory/mmu.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/cpu/smp.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/lib.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/memory/mmu.rs | 0 .../src/memory/mmu/translation_table.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/state.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../test-macros/Cargo.toml | 0 .../test-macros/src/lib.rs | 0 .../test-types/Cargo.toml | 0 .../test-types/src/lib.rs | 0 .../tests/00_console_sanity.rb | 0 .../tests/00_console_sanity.rs | 0 .../tests/01_timer_sanity.rs | 0 .../tests/02_exception_sync_page_fault.rs | 0 .../tests/03_exception_irq_sanity.rs | 0 .../tests/panic_exit_success/mod.rs | 0 .../tests/runner.rb | 0 .../.cargo/config.toml | 0 .../.vscode/settings.json | 0 .../Cargo.lock | 0 .../Cargo.toml | 0 .../Makefile | 0 .../README.md | 158 ++++++++-------- .../build.rs | 0 .../src/_arch/aarch64/cpu.rs | 0 .../src/_arch/aarch64/cpu/boot.rs | 0 .../src/_arch/aarch64/cpu/boot.s | 0 .../src/_arch/aarch64/cpu/smp.rs | 0 .../src/_arch/aarch64/exception.rs | 0 .../src/_arch/aarch64/exception.s | 0 .../_arch/aarch64/exception/asynchronous.rs | 0 .../src/_arch/aarch64/memory/mmu.rs | 0 .../aarch64/memory/mmu/translation_table.rs | 0 .../src/_arch/aarch64/time.rs | 0 .../src/bsp.rs | 0 .../src/bsp/device_driver.rs | 0 .../src/bsp/device_driver/arm.rs | 0 .../src/bsp/device_driver/arm/gicv2.rs | 0 .../src/bsp/device_driver/arm/gicv2/gicc.rs | 0 .../src/bsp/device_driver/arm/gicv2/gicd.rs | 0 .../src/bsp/device_driver/bcm.rs | 0 .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 0 .../bcm/bcm2xxx_interrupt_controller.rs | 0 .../peripheral_ic.rs | 0 .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 0 .../src/bsp/device_driver/common.rs | 0 .../src/bsp/raspberrypi.rs | 0 .../src/bsp/raspberrypi/console.rs | 0 .../src/bsp/raspberrypi/cpu.rs | 0 .../src/bsp/raspberrypi/driver.rs | 0 .../src/bsp/raspberrypi/exception.rs | 0 .../bsp/raspberrypi/exception/asynchronous.rs | 0 .../src/bsp/raspberrypi/link.ld | 0 .../src/bsp/raspberrypi/memory.rs | 0 .../src/bsp/raspberrypi/memory/mmu.rs | 0 .../src/common.rs | 0 .../src/console.rs | 0 .../src/cpu.rs | 0 .../src/cpu/boot.rs | 0 .../src/cpu/smp.rs | 0 .../src/driver.rs | 0 .../src/exception.rs | 0 .../src/exception/asynchronous.rs | 0 .../src/lib.rs | 0 .../src/main.rs | 0 .../src/memory.rs | 0 .../src/memory/mmu.rs | 0 .../src/memory/mmu/mapping_record.rs | 0 .../src/memory/mmu/translation_table.rs | 0 .../src/memory/mmu/types.rs | 0 .../src/panic_wait.rs | 0 .../src/print.rs | 0 .../src/runtime_init.rs | 0 .../src/state.rs | 0 .../src/synchronization.rs | 0 .../src/time.rs | 0 .../test-macros/Cargo.toml | 0 .../test-macros/src/lib.rs | 0 .../test-types/Cargo.toml | 0 .../test-types/src/lib.rs | 0 .../tests/00_console_sanity.rb | 0 .../tests/00_console_sanity.rs | 0 .../tests/01_timer_sanity.rs | 0 .../tests/02_exception_sync_page_fault.rs | 0 .../tests/03_exception_irq_sanity.rs | 0 .../tests/panic_exit_success/mod.rs | 0 .../tests/runner.rb | 0 README.md | 11 +- utils/devtool.rb | 2 +- 472 files changed, 493 insertions(+), 491 deletions(-) rename {05_safe_globals => 04_safe_globals}/.vscode/settings.json (100%) rename {05_safe_globals => 04_safe_globals}/Cargo.lock (100%) rename {05_safe_globals => 04_safe_globals}/Cargo.toml (100%) rename {05_safe_globals => 04_safe_globals}/Makefile (100%) rename {05_safe_globals => 04_safe_globals}/README.md (92%) rename {05_safe_globals => 04_safe_globals}/build.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/_arch/aarch64/cpu.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/_arch/aarch64/cpu/boot.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/_arch/aarch64/cpu/boot.s (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp/raspberrypi.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp/raspberrypi/console.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp/raspberrypi/cpu.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp/raspberrypi/link.ld (100%) rename {05_safe_globals => 04_safe_globals}/src/bsp/raspberrypi/memory.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/console.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/cpu.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/cpu/boot.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/main.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/memory.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/panic_wait.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/print.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/runtime_init.rs (100%) rename {05_safe_globals => 04_safe_globals}/src/synchronization.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/.vscode/settings.json (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/Cargo.lock (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/Cargo.toml (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/Makefile (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/README.md (93%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/build.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/_arch/aarch64/cpu.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/_arch/aarch64/cpu/boot.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/_arch/aarch64/cpu/boot.s (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/device_driver.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/device_driver/bcm.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/device_driver/common.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi/console.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi/cpu.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi/driver.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi/link.ld (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/bsp/raspberrypi/memory.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/console.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/cpu.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/cpu/boot.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/driver.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/main.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/memory.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/panic_wait.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/print.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/runtime_init.rs (100%) rename {06_drivers_gpio_uart => 05_drivers_gpio_uart}/src/synchronization.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/.vscode/settings.json (100%) rename {07_uart_chainloader => 06_uart_chainloader}/Cargo.lock (100%) rename {07_uart_chainloader => 06_uart_chainloader}/Cargo.toml (100%) rename {07_uart_chainloader => 06_uart_chainloader}/Makefile (100%) rename {07_uart_chainloader => 06_uart_chainloader}/README.md (88%) rename {07_uart_chainloader => 06_uart_chainloader}/build.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/demo_payload_rpi3.img (100%) rename {07_uart_chainloader => 06_uart_chainloader}/demo_payload_rpi4.img (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/_arch/aarch64/cpu.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/_arch/aarch64/cpu/boot.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/_arch/aarch64/cpu/boot.s (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/device_driver.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/device_driver/bcm.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/device_driver/common.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi/console.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi/cpu.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi/driver.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi/link.ld (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/bsp/raspberrypi/memory.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/console.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/cpu.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/cpu/boot.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/driver.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/main.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/memory.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/panic_wait.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/print.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/runtime_init.rs (100%) rename {07_uart_chainloader => 06_uart_chainloader}/src/synchronization.rs (100%) create mode 100755 06_uart_chainloader/update.sh rename {08_timestamps => 07_timestamps}/.vscode/settings.json (100%) rename {08_timestamps => 07_timestamps}/Cargo.lock (100%) rename {08_timestamps => 07_timestamps}/Cargo.toml (100%) rename {08_timestamps => 07_timestamps}/Makefile (100%) rename {08_timestamps => 07_timestamps}/README.md (89%) rename {08_timestamps => 07_timestamps}/build.rs (100%) rename {08_timestamps => 07_timestamps}/src/_arch/aarch64/cpu.rs (100%) rename {08_timestamps => 07_timestamps}/src/_arch/aarch64/cpu/boot.rs (100%) rename {08_timestamps => 07_timestamps}/src/_arch/aarch64/cpu/boot.s (100%) rename {08_timestamps => 07_timestamps}/src/_arch/aarch64/time.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/device_driver.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/device_driver/bcm.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/device_driver/common.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi/console.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi/cpu.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi/driver.rs (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi/link.ld (100%) rename {08_timestamps => 07_timestamps}/src/bsp/raspberrypi/memory.rs (100%) rename {08_timestamps => 07_timestamps}/src/console.rs (100%) rename {08_timestamps => 07_timestamps}/src/cpu.rs (100%) rename {08_timestamps => 07_timestamps}/src/cpu/boot.rs (100%) rename {08_timestamps => 07_timestamps}/src/driver.rs (100%) rename {08_timestamps => 07_timestamps}/src/main.rs (100%) rename {08_timestamps => 07_timestamps}/src/memory.rs (100%) rename {08_timestamps => 07_timestamps}/src/panic_wait.rs (100%) rename {08_timestamps => 07_timestamps}/src/print.rs (100%) rename {08_timestamps => 07_timestamps}/src/runtime_init.rs (100%) rename {08_timestamps => 07_timestamps}/src/synchronization.rs (100%) rename {08_timestamps => 07_timestamps}/src/time.rs (100%) delete mode 100755 07_uart_chainloader/update.sh rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/.vscode/settings.json (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/Cargo.lock (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/Cargo.toml (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/Makefile (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/README.md (98%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/build.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/_arch/aarch64/cpu.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/_arch/aarch64/cpu/boot.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/_arch/aarch64/cpu/boot.s (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/_arch/aarch64/time.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/device_driver.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/device_driver/bcm.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/device_driver/common.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi/console.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi/cpu.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi/driver.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi/link.ld (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/bsp/raspberrypi/memory.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/console.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/cpu.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/cpu/boot.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/driver.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/main.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/memory.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/panic_wait.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/print.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/runtime_init.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/synchronization.rs (100%) rename {09_hw_debug_JTAG => 08_hw_debug_JTAG}/src/time.rs (100%) rename {10_privilege_level => 09_privilege_level}/.vscode/settings.json (100%) rename {10_privilege_level => 09_privilege_level}/Cargo.lock (100%) rename {10_privilege_level => 09_privilege_level}/Cargo.toml (100%) rename {10_privilege_level => 09_privilege_level}/Makefile (100%) rename {10_privilege_level => 09_privilege_level}/README.md (93%) rename {10_privilege_level => 09_privilege_level}/build.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/cpu.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/cpu/boot.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/cpu/boot.s (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/exception.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/_arch/aarch64/time.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/device_driver.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/device_driver/bcm.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/device_driver/common.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi/console.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi/cpu.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi/driver.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi/link.ld (100%) rename {10_privilege_level => 09_privilege_level}/src/bsp/raspberrypi/memory.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/console.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/cpu.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/cpu/boot.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/driver.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/exception.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/exception/asynchronous.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/main.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/memory.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/panic_wait.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/print.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/runtime_init.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/synchronization.rs (100%) rename {10_privilege_level => 09_privilege_level}/src/time.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/.vscode/settings.json (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/Cargo.lock (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/Cargo.toml (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/Makefile (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/README.md (96%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/build.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/cpu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/cpu/boot.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/cpu/boot.s (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/exception.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/memory/mmu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/memory/mmu/translation_table.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/_arch/aarch64/time.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/device_driver.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/device_driver/bcm.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/device_driver/common.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/console.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/cpu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/driver.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/link.ld (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/memory.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/bsp/raspberrypi/memory/mmu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/console.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/cpu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/cpu/boot.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/driver.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/exception.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/exception/asynchronous.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/main.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/memory.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/memory/mmu.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/memory/mmu/translation_table.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/panic_wait.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/print.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/runtime_init.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/synchronization.rs (100%) rename {11_virtual_mem_part1_identity_mapping => 10_virtual_mem_part1_identity_mapping}/src/time.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/.vscode/settings.json (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/Cargo.lock (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/Cargo.toml (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/Makefile (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/README.md (96%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/build.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/cpu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/cpu/boot.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/cpu/boot.s (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/exception.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/exception.s (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/memory/mmu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/memory/mmu/translation_table.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/_arch/aarch64/time.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/device_driver.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/device_driver/bcm.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/device_driver/common.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/console.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/cpu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/driver.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/link.ld (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/memory.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/bsp/raspberrypi/memory/mmu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/console.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/cpu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/cpu/boot.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/driver.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/exception.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/exception/asynchronous.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/main.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/memory.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/memory/mmu.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/memory/mmu/translation_table.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/panic_wait.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/print.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/runtime_init.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/synchronization.rs (100%) rename {12_exceptions_part1_groundwork => 11_exceptions_part1_groundwork}/src/time.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/.cargo/config.toml (100%) rename {13_integrated_testing => 12_integrated_testing}/.vscode/settings.json (100%) rename {13_integrated_testing => 12_integrated_testing}/Cargo.lock (100%) rename {13_integrated_testing => 12_integrated_testing}/Cargo.toml (100%) rename {13_integrated_testing => 12_integrated_testing}/Makefile (100%) rename {13_integrated_testing => 12_integrated_testing}/README.md (92%) rename {13_integrated_testing => 12_integrated_testing}/build.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/cpu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/cpu/boot.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/cpu/boot.s (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/exception.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/exception.s (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/memory/mmu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/memory/mmu/translation_table.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/_arch/aarch64/time.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/device_driver.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/device_driver/bcm.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/device_driver/common.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/console.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/cpu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/driver.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/link.ld (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/memory.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/bsp/raspberrypi/memory/mmu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/console.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/cpu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/cpu/boot.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/driver.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/exception.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/exception/asynchronous.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/lib.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/main.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/memory.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/memory/mmu.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/memory/mmu/translation_table.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/panic_wait.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/print.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/runtime_init.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/synchronization.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/src/time.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/test-macros/Cargo.toml (100%) rename {13_integrated_testing => 12_integrated_testing}/test-macros/src/lib.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/test-types/Cargo.toml (100%) rename {13_integrated_testing => 12_integrated_testing}/test-types/src/lib.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/00_console_sanity.rb (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/00_console_sanity.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/01_timer_sanity.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/02_exception_sync_page_fault.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/panic_exit_success/mod.rs (100%) rename {13_integrated_testing => 12_integrated_testing}/tests/runner.rb (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/.cargo/config.toml (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/.vscode/settings.json (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/Cargo.lock (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/Cargo.toml (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/Makefile (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/README.md (94%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/build.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/cpu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/cpu/boot.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/cpu/boot.s (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/cpu/smp.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/exception.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/exception.s (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/memory/mmu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/memory/mmu/translation_table.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/_arch/aarch64/time.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/arm.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/arm/gicv2.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/arm/gicv2/gicc.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/arm/gicv2/gicd.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/bcm.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/device_driver/common.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/console.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/cpu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/driver.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/exception.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/exception/asynchronous.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/link.ld (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/memory.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/bsp/raspberrypi/memory/mmu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/console.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/cpu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/cpu/boot.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/cpu/smp.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/driver.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/exception.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/exception/asynchronous.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/lib.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/main.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/memory.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/memory/mmu.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/memory/mmu/translation_table.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/panic_wait.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/print.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/runtime_init.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/state.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/synchronization.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/src/time.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/test-macros/Cargo.toml (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/test-macros/src/lib.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/test-types/Cargo.toml (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/test-types/src/lib.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/00_console_sanity.rb (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/00_console_sanity.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/01_timer_sanity.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/02_exception_sync_page_fault.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/03_exception_irq_sanity.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/panic_exit_success/mod.rs (100%) rename {14_exceptions_part2_peripheral_IRQs => 13_exceptions_part2_peripheral_IRQs}/tests/runner.rb (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/.cargo/config.toml (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/.vscode/settings.json (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/Cargo.lock (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/Cargo.toml (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/Makefile (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/README.md (95%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/build.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/cpu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/cpu/boot.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/cpu/boot.s (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/cpu/smp.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/exception.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/exception.s (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/exception/asynchronous.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/memory/mmu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/memory/mmu/translation_table.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/_arch/aarch64/time.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/arm.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/arm/gicv2.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/arm/gicv2/gicc.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/arm/gicv2/gicd.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/bcm.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/device_driver/common.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/console.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/cpu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/driver.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/exception.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/exception/asynchronous.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/link.ld (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/memory.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/bsp/raspberrypi/memory/mmu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/common.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/console.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/cpu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/cpu/boot.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/cpu/smp.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/driver.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/exception.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/exception/asynchronous.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/lib.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/main.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/memory.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/memory/mmu.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/memory/mmu/mapping_record.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/memory/mmu/translation_table.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/memory/mmu/types.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/panic_wait.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/print.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/runtime_init.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/state.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/synchronization.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/src/time.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/test-macros/Cargo.toml (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/test-macros/src/lib.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/test-types/Cargo.toml (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/test-types/src/lib.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/00_console_sanity.rb (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/00_console_sanity.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/01_timer_sanity.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/02_exception_sync_page_fault.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/03_exception_irq_sanity.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/panic_exit_success/mod.rs (100%) rename {15_virtual_mem_part2_mmio_remap => 14_virtual_mem_part2_mmio_remap}/tests/runner.rb (100%) diff --git a/05_safe_globals/.vscode/settings.json b/04_safe_globals/.vscode/settings.json similarity index 100% rename from 05_safe_globals/.vscode/settings.json rename to 04_safe_globals/.vscode/settings.json diff --git a/05_safe_globals/Cargo.lock b/04_safe_globals/Cargo.lock similarity index 100% rename from 05_safe_globals/Cargo.lock rename to 04_safe_globals/Cargo.lock diff --git a/05_safe_globals/Cargo.toml b/04_safe_globals/Cargo.toml similarity index 100% rename from 05_safe_globals/Cargo.toml rename to 04_safe_globals/Cargo.toml diff --git a/05_safe_globals/Makefile b/04_safe_globals/Makefile similarity index 100% rename from 05_safe_globals/Makefile rename to 04_safe_globals/Makefile diff --git a/05_safe_globals/README.md b/04_safe_globals/README.md similarity index 92% rename from 05_safe_globals/README.md rename to 04_safe_globals/README.md index ea61489e2..13b9e298a 100644 --- a/05_safe_globals/README.md +++ b/04_safe_globals/README.md @@ -1,4 +1,4 @@ -# Tutorial 05 - Safe Globals +# Tutorial 04 - Safe Globals ## tl;dr @@ -54,9 +54,9 @@ $ make qemu ## Diff to previous ```diff -diff -uNr 04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs 05_safe_globals/src/bsp/raspberrypi/console.rs ---- 04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs -+++ 05_safe_globals/src/bsp/raspberrypi/console.rs +diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/console.rs 04_safe_globals/src/bsp/raspberrypi/console.rs +--- 03_hacky_hello_world/src/bsp/raspberrypi/console.rs ++++ 04_safe_globals/src/bsp/raspberrypi/console.rs @@ -4,7 +4,7 @@ //! BSP console facilities. @@ -176,9 +176,9 @@ diff -uNr 04_zero_overhead_abstraction/src/bsp/raspberrypi/console.rs 05_safe_gl + } } -diff -uNr 04_zero_overhead_abstraction/src/console.rs 05_safe_globals/src/console.rs ---- 04_zero_overhead_abstraction/src/console.rs -+++ 05_safe_globals/src/console.rs +diff -uNr 03_hacky_hello_world/src/console.rs 04_safe_globals/src/console.rs +--- 03_hacky_hello_world/src/console.rs ++++ 04_safe_globals/src/console.rs @@ -10,10 +10,22 @@ /// Console interfaces. @@ -208,12 +208,12 @@ diff -uNr 04_zero_overhead_abstraction/src/console.rs 05_safe_globals/src/consol + pub trait All = Write + Statistics; } -diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs ---- 04_zero_overhead_abstraction/src/main.rs -+++ 05_safe_globals/src/main.rs +diff -uNr 03_hacky_hello_world/src/main.rs 04_safe_globals/src/main.rs +--- 03_hacky_hello_world/src/main.rs ++++ 04_safe_globals/src/main.rs @@ -109,6 +109,7 @@ - #![feature(format_args_nl)] + #![feature(global_asm)] #![feature(panic_info_message)] +#![feature(trait_alias)] #![no_main] @@ -227,27 +227,28 @@ diff -uNr 04_zero_overhead_abstraction/src/main.rs 05_safe_globals/src/main.rs /// Early init code. /// -@@ -126,8 +128,15 @@ +@@ -126,7 +128,15 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { +- println!("[0] Hello from Rust!"); + use console::interface::Statistics; -+ - println!("[0] Hello from pure Rust!"); -- println!("[1] Stopping here."); +- panic!("Stopping here.") ++ println!("[0] Hello from pure Rust!"); ++ + println!( + "[1] Chars written: {}", + bsp::console::console().chars_written() + ); + + println!("[2] Stopping here."); - cpu::wait_forever() ++ cpu::wait_forever() } -diff -uNr 04_zero_overhead_abstraction/src/synchronization.rs 05_safe_globals/src/synchronization.rs ---- 04_zero_overhead_abstraction/src/synchronization.rs -+++ 05_safe_globals/src/synchronization.rs +diff -uNr 03_hacky_hello_world/src/synchronization.rs 04_safe_globals/src/synchronization.rs +--- 03_hacky_hello_world/src/synchronization.rs ++++ 04_safe_globals/src/synchronization.rs @@ -0,0 +1,77 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// diff --git a/05_safe_globals/build.rs b/04_safe_globals/build.rs similarity index 100% rename from 05_safe_globals/build.rs rename to 04_safe_globals/build.rs diff --git a/05_safe_globals/src/_arch/aarch64/cpu.rs b/04_safe_globals/src/_arch/aarch64/cpu.rs similarity index 100% rename from 05_safe_globals/src/_arch/aarch64/cpu.rs rename to 04_safe_globals/src/_arch/aarch64/cpu.rs diff --git a/05_safe_globals/src/_arch/aarch64/cpu/boot.rs b/04_safe_globals/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 05_safe_globals/src/_arch/aarch64/cpu/boot.rs rename to 04_safe_globals/src/_arch/aarch64/cpu/boot.rs diff --git a/05_safe_globals/src/_arch/aarch64/cpu/boot.s b/04_safe_globals/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 05_safe_globals/src/_arch/aarch64/cpu/boot.s rename to 04_safe_globals/src/_arch/aarch64/cpu/boot.s diff --git a/05_safe_globals/src/bsp.rs b/04_safe_globals/src/bsp.rs similarity index 100% rename from 05_safe_globals/src/bsp.rs rename to 04_safe_globals/src/bsp.rs diff --git a/05_safe_globals/src/bsp/raspberrypi.rs b/04_safe_globals/src/bsp/raspberrypi.rs similarity index 100% rename from 05_safe_globals/src/bsp/raspberrypi.rs rename to 04_safe_globals/src/bsp/raspberrypi.rs diff --git a/05_safe_globals/src/bsp/raspberrypi/console.rs b/04_safe_globals/src/bsp/raspberrypi/console.rs similarity index 100% rename from 05_safe_globals/src/bsp/raspberrypi/console.rs rename to 04_safe_globals/src/bsp/raspberrypi/console.rs diff --git a/05_safe_globals/src/bsp/raspberrypi/cpu.rs b/04_safe_globals/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 05_safe_globals/src/bsp/raspberrypi/cpu.rs rename to 04_safe_globals/src/bsp/raspberrypi/cpu.rs diff --git a/05_safe_globals/src/bsp/raspberrypi/link.ld b/04_safe_globals/src/bsp/raspberrypi/link.ld similarity index 100% rename from 05_safe_globals/src/bsp/raspberrypi/link.ld rename to 04_safe_globals/src/bsp/raspberrypi/link.ld diff --git a/05_safe_globals/src/bsp/raspberrypi/memory.rs b/04_safe_globals/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 05_safe_globals/src/bsp/raspberrypi/memory.rs rename to 04_safe_globals/src/bsp/raspberrypi/memory.rs diff --git a/05_safe_globals/src/console.rs b/04_safe_globals/src/console.rs similarity index 100% rename from 05_safe_globals/src/console.rs rename to 04_safe_globals/src/console.rs diff --git a/05_safe_globals/src/cpu.rs b/04_safe_globals/src/cpu.rs similarity index 100% rename from 05_safe_globals/src/cpu.rs rename to 04_safe_globals/src/cpu.rs diff --git a/05_safe_globals/src/cpu/boot.rs b/04_safe_globals/src/cpu/boot.rs similarity index 100% rename from 05_safe_globals/src/cpu/boot.rs rename to 04_safe_globals/src/cpu/boot.rs diff --git a/05_safe_globals/src/main.rs b/04_safe_globals/src/main.rs similarity index 100% rename from 05_safe_globals/src/main.rs rename to 04_safe_globals/src/main.rs diff --git a/05_safe_globals/src/memory.rs b/04_safe_globals/src/memory.rs similarity index 100% rename from 05_safe_globals/src/memory.rs rename to 04_safe_globals/src/memory.rs diff --git a/05_safe_globals/src/panic_wait.rs b/04_safe_globals/src/panic_wait.rs similarity index 100% rename from 05_safe_globals/src/panic_wait.rs rename to 04_safe_globals/src/panic_wait.rs diff --git a/05_safe_globals/src/print.rs b/04_safe_globals/src/print.rs similarity index 100% rename from 05_safe_globals/src/print.rs rename to 04_safe_globals/src/print.rs diff --git a/05_safe_globals/src/runtime_init.rs b/04_safe_globals/src/runtime_init.rs similarity index 100% rename from 05_safe_globals/src/runtime_init.rs rename to 04_safe_globals/src/runtime_init.rs diff --git a/05_safe_globals/src/synchronization.rs b/04_safe_globals/src/synchronization.rs similarity index 100% rename from 05_safe_globals/src/synchronization.rs rename to 04_safe_globals/src/synchronization.rs diff --git a/06_drivers_gpio_uart/.vscode/settings.json b/05_drivers_gpio_uart/.vscode/settings.json similarity index 100% rename from 06_drivers_gpio_uart/.vscode/settings.json rename to 05_drivers_gpio_uart/.vscode/settings.json diff --git a/06_drivers_gpio_uart/Cargo.lock b/05_drivers_gpio_uart/Cargo.lock similarity index 100% rename from 06_drivers_gpio_uart/Cargo.lock rename to 05_drivers_gpio_uart/Cargo.lock diff --git a/06_drivers_gpio_uart/Cargo.toml b/05_drivers_gpio_uart/Cargo.toml similarity index 100% rename from 06_drivers_gpio_uart/Cargo.toml rename to 05_drivers_gpio_uart/Cargo.toml diff --git a/06_drivers_gpio_uart/Makefile b/05_drivers_gpio_uart/Makefile similarity index 100% rename from 06_drivers_gpio_uart/Makefile rename to 05_drivers_gpio_uart/Makefile diff --git a/06_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md similarity index 93% rename from 06_drivers_gpio_uart/README.md rename to 05_drivers_gpio_uart/README.md index bd7c02089..4ecdd41ad 100644 --- a/06_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -1,4 +1,4 @@ -# Tutorial 06 - Drivers: GPIO and UART +# Tutorial 05 - Drivers: GPIO and UART ## tl;dr @@ -108,9 +108,9 @@ Miniterm 1.0 ## Diff to previous ```diff -diff -uNr 05_safe_globals/Cargo.toml 06_drivers_gpio_uart/Cargo.toml ---- 05_safe_globals/Cargo.toml -+++ 06_drivers_gpio_uart/Cargo.toml +diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml +--- 04_safe_globals/Cargo.toml ++++ 05_drivers_gpio_uart/Cargo.toml @@ -9,8 +9,8 @@ [features] @@ -133,9 +133,9 @@ diff -uNr 05_safe_globals/Cargo.toml 06_drivers_gpio_uart/Cargo.toml [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } -diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile ---- 05_safe_globals/Makefile -+++ 06_drivers_gpio_uart/Makefile +diff -uNr 04_safe_globals/Makefile 05_drivers_gpio_uart/Makefile +--- 04_safe_globals/Makefile ++++ 05_drivers_gpio_uart/Makefile @@ -7,6 +7,12 @@ # Default to the RPi3 BSP ?= rpi3 @@ -186,9 +186,9 @@ diff -uNr 05_safe_globals/Makefile 06_drivers_gpio_uart/Makefile @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs ---- 05_safe_globals/src/_arch/aarch64/cpu.rs -+++ 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +diff -uNr 04_safe_globals/src/_arch/aarch64/cpu.rs 05_drivers_gpio_uart/src/_arch/aarch64/cpu.rs +--- 04_safe_globals/src/_arch/aarch64/cpu.rs ++++ 05_drivers_gpio_uart/src/_arch/aarch64/cpu.rs @@ -17,6 +17,17 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -208,9 +208,9 @@ diff -uNr 05_safe_globals/src/_arch/aarch64/cpu.rs 06_drivers_gpio_uart/src/_arc #[inline(always)] pub fn wait_forever() -> ! { -diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ---- 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -+++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -434,9 +434,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_drivers_g + } +} -diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ---- 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -+++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +--- 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ++++ 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -0,0 +1,403 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -842,9 +842,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_dri + } +} -diff -uNr 05_safe_globals/src/bsp/device_driver/bcm.rs 06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs ---- 05_safe_globals/src/bsp/device_driver/bcm.rs -+++ 06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs +diff -uNr 04_safe_globals/src/bsp/device_driver/bcm.rs 05_drivers_gpio_uart/src/bsp/device_driver/bcm.rs +--- 04_safe_globals/src/bsp/device_driver/bcm.rs ++++ 05_drivers_gpio_uart/src/bsp/device_driver/bcm.rs @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -858,9 +858,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/bcm.rs 06_drivers_gpio_uart/src/ +pub use bcm2xxx_gpio::*; +pub use bcm2xxx_pl011_uart::*; -diff -uNr 05_safe_globals/src/bsp/device_driver/common.rs 06_drivers_gpio_uart/src/bsp/device_driver/common.rs ---- 05_safe_globals/src/bsp/device_driver/common.rs -+++ 06_drivers_gpio_uart/src/bsp/device_driver/common.rs +diff -uNr 04_safe_globals/src/bsp/device_driver/common.rs 05_drivers_gpio_uart/src/bsp/device_driver/common.rs +--- 04_safe_globals/src/bsp/device_driver/common.rs ++++ 05_drivers_gpio_uart/src/bsp/device_driver/common.rs @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -901,9 +901,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver/common.rs 06_drivers_gpio_uart/s + } +} -diff -uNr 05_safe_globals/src/bsp/device_driver.rs 06_drivers_gpio_uart/src/bsp/device_driver.rs ---- 05_safe_globals/src/bsp/device_driver.rs -+++ 06_drivers_gpio_uart/src/bsp/device_driver.rs +diff -uNr 04_safe_globals/src/bsp/device_driver.rs 05_drivers_gpio_uart/src/bsp/device_driver.rs +--- 04_safe_globals/src/bsp/device_driver.rs ++++ 05_drivers_gpio_uart/src/bsp/device_driver.rs @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -918,9 +918,9 @@ diff -uNr 05_safe_globals/src/bsp/device_driver.rs 06_drivers_gpio_uart/src/bsp/ +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +pub use bcm::*; -diff -uNr 05_safe_globals/src/bsp/raspberrypi/console.rs 06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs ---- 05_safe_globals/src/bsp/raspberrypi/console.rs -+++ 06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs +diff -uNr 04_safe_globals/src/bsp/raspberrypi/console.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs +--- 04_safe_globals/src/bsp/raspberrypi/console.rs ++++ 05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs @@ -4,113 +4,34 @@ //! BSP console facilities. @@ -1054,9 +1054,9 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/console.rs 06_drivers_gpio_uart/sr + &super::PL011_UART } -diff -uNr 05_safe_globals/src/bsp/raspberrypi/driver.rs 06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs ---- 05_safe_globals/src/bsp/raspberrypi/driver.rs -+++ 06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs +diff -uNr 04_safe_globals/src/bsp/raspberrypi/driver.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs +--- 04_safe_globals/src/bsp/raspberrypi/driver.rs ++++ 05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1108,9 +1108,9 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/driver.rs 06_drivers_gpio_uart/src + } +} -diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs ---- 05_safe_globals/src/bsp/raspberrypi/memory.rs -+++ 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +diff -uNr 04_safe_globals/src/bsp/raspberrypi/memory.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +--- 04_safe_globals/src/bsp/raspberrypi/memory.rs ++++ 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -17,6 +17,38 @@ } @@ -1151,9 +1151,9 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi/memory.rs 06_drivers_gpio_uart/src //-------------------------------------------------------------------------------------------------- -diff -uNr 05_safe_globals/src/bsp/raspberrypi.rs 06_drivers_gpio_uart/src/bsp/raspberrypi.rs ---- 05_safe_globals/src/bsp/raspberrypi.rs -+++ 06_drivers_gpio_uart/src/bsp/raspberrypi.rs +diff -uNr 04_safe_globals/src/bsp/raspberrypi.rs 05_drivers_gpio_uart/src/bsp/raspberrypi.rs +--- 04_safe_globals/src/bsp/raspberrypi.rs ++++ 05_drivers_gpio_uart/src/bsp/raspberrypi.rs @@ -6,4 +6,33 @@ pub mod console; @@ -1189,9 +1189,9 @@ diff -uNr 05_safe_globals/src/bsp/raspberrypi.rs 06_drivers_gpio_uart/src/bsp/ra + } +} -diff -uNr 05_safe_globals/src/bsp.rs 06_drivers_gpio_uart/src/bsp.rs ---- 05_safe_globals/src/bsp.rs -+++ 06_drivers_gpio_uart/src/bsp.rs +diff -uNr 04_safe_globals/src/bsp.rs 05_drivers_gpio_uart/src/bsp.rs +--- 04_safe_globals/src/bsp.rs ++++ 05_drivers_gpio_uart/src/bsp.rs @@ -4,6 +4,8 @@ //! Conditional reexporting of Board Support Packages. @@ -1202,9 +1202,9 @@ diff -uNr 05_safe_globals/src/bsp.rs 06_drivers_gpio_uart/src/bsp.rs mod raspberrypi; -diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs ---- 05_safe_globals/src/console.rs -+++ 06_drivers_gpio_uart/src/console.rs +diff -uNr 04_safe_globals/src/console.rs 05_drivers_gpio_uart/src/console.rs +--- 04_safe_globals/src/console.rs ++++ 05_drivers_gpio_uart/src/console.rs @@ -14,8 +14,25 @@ /// Console write functions. @@ -1247,9 +1247,9 @@ diff -uNr 05_safe_globals/src/console.rs 06_drivers_gpio_uart/src/console.rs + pub trait All = Write + Read + Statistics; } -diff -uNr 05_safe_globals/src/cpu.rs 06_drivers_gpio_uart/src/cpu.rs ---- 05_safe_globals/src/cpu.rs -+++ 06_drivers_gpio_uart/src/cpu.rs +diff -uNr 04_safe_globals/src/cpu.rs 05_drivers_gpio_uart/src/cpu.rs +--- 04_safe_globals/src/cpu.rs ++++ 05_drivers_gpio_uart/src/cpu.rs @@ -13,4 +13,7 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -1260,9 +1260,9 @@ diff -uNr 05_safe_globals/src/cpu.rs 06_drivers_gpio_uart/src/cpu.rs +#[cfg(feature = "bsp_rpi3")] +pub use arch_cpu::spin_for_cycles; -diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs ---- 05_safe_globals/src/driver.rs -+++ 06_drivers_gpio_uart/src/driver.rs +diff -uNr 04_safe_globals/src/driver.rs 05_drivers_gpio_uart/src/driver.rs +--- 04_safe_globals/src/driver.rs ++++ 05_drivers_gpio_uart/src/driver.rs @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1309,9 +1309,9 @@ diff -uNr 05_safe_globals/src/driver.rs 06_drivers_gpio_uart/src/driver.rs + } +} -diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs ---- 05_safe_globals/src/main.rs -+++ 06_drivers_gpio_uart/src/main.rs +diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs +--- 04_safe_globals/src/main.rs ++++ 05_drivers_gpio_uart/src/main.rs @@ -106,6 +106,8 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html @@ -1385,9 +1385,9 @@ diff -uNr 05_safe_globals/src/main.rs 06_drivers_gpio_uart/src/main.rs + } } -diff -uNr 05_safe_globals/src/panic_wait.rs 06_drivers_gpio_uart/src/panic_wait.rs ---- 05_safe_globals/src/panic_wait.rs -+++ 06_drivers_gpio_uart/src/panic_wait.rs +diff -uNr 04_safe_globals/src/panic_wait.rs 05_drivers_gpio_uart/src/panic_wait.rs +--- 04_safe_globals/src/panic_wait.rs ++++ 05_drivers_gpio_uart/src/panic_wait.rs @@ -4,15 +4,35 @@ //! A panic handler that infinitely waits. diff --git a/06_drivers_gpio_uart/build.rs b/05_drivers_gpio_uart/build.rs similarity index 100% rename from 06_drivers_gpio_uart/build.rs rename to 05_drivers_gpio_uart/build.rs diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs b/05_drivers_gpio_uart/src/_arch/aarch64/cpu.rs similarity index 100% rename from 06_drivers_gpio_uart/src/_arch/aarch64/cpu.rs rename to 05_drivers_gpio_uart/src/_arch/aarch64/cpu.rs diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs rename to 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs diff --git a/06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s rename to 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s diff --git a/06_drivers_gpio_uart/src/bsp.rs b/05_drivers_gpio_uart/src/bsp.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp.rs rename to 05_drivers_gpio_uart/src/bsp.rs diff --git a/06_drivers_gpio_uart/src/bsp/device_driver.rs b/05_drivers_gpio_uart/src/bsp/device_driver.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/device_driver.rs rename to 05_drivers_gpio_uart/src/bsp/device_driver.rs diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs b/05_drivers_gpio_uart/src/bsp/device_driver/bcm.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/device_driver/bcm.rs rename to 05_drivers_gpio_uart/src/bsp/device_driver/bcm.rs diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/06_drivers_gpio_uart/src/bsp/device_driver/common.rs b/05_drivers_gpio_uart/src/bsp/device_driver/common.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/device_driver/common.rs rename to 05_drivers_gpio_uart/src/bsp/device_driver/common.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi.rs rename to 05_drivers_gpio_uart/src/bsp/raspberrypi.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi/console.rs rename to 05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs rename to 05_drivers_gpio_uart/src/bsp/raspberrypi/cpu.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs rename to 05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld rename to 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld diff --git a/06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs rename to 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs diff --git a/06_drivers_gpio_uart/src/console.rs b/05_drivers_gpio_uart/src/console.rs similarity index 100% rename from 06_drivers_gpio_uart/src/console.rs rename to 05_drivers_gpio_uart/src/console.rs diff --git a/06_drivers_gpio_uart/src/cpu.rs b/05_drivers_gpio_uart/src/cpu.rs similarity index 100% rename from 06_drivers_gpio_uart/src/cpu.rs rename to 05_drivers_gpio_uart/src/cpu.rs diff --git a/06_drivers_gpio_uart/src/cpu/boot.rs b/05_drivers_gpio_uart/src/cpu/boot.rs similarity index 100% rename from 06_drivers_gpio_uart/src/cpu/boot.rs rename to 05_drivers_gpio_uart/src/cpu/boot.rs diff --git a/06_drivers_gpio_uart/src/driver.rs b/05_drivers_gpio_uart/src/driver.rs similarity index 100% rename from 06_drivers_gpio_uart/src/driver.rs rename to 05_drivers_gpio_uart/src/driver.rs diff --git a/06_drivers_gpio_uart/src/main.rs b/05_drivers_gpio_uart/src/main.rs similarity index 100% rename from 06_drivers_gpio_uart/src/main.rs rename to 05_drivers_gpio_uart/src/main.rs diff --git a/06_drivers_gpio_uart/src/memory.rs b/05_drivers_gpio_uart/src/memory.rs similarity index 100% rename from 06_drivers_gpio_uart/src/memory.rs rename to 05_drivers_gpio_uart/src/memory.rs diff --git a/06_drivers_gpio_uart/src/panic_wait.rs b/05_drivers_gpio_uart/src/panic_wait.rs similarity index 100% rename from 06_drivers_gpio_uart/src/panic_wait.rs rename to 05_drivers_gpio_uart/src/panic_wait.rs diff --git a/06_drivers_gpio_uart/src/print.rs b/05_drivers_gpio_uart/src/print.rs similarity index 100% rename from 06_drivers_gpio_uart/src/print.rs rename to 05_drivers_gpio_uart/src/print.rs diff --git a/06_drivers_gpio_uart/src/runtime_init.rs b/05_drivers_gpio_uart/src/runtime_init.rs similarity index 100% rename from 06_drivers_gpio_uart/src/runtime_init.rs rename to 05_drivers_gpio_uart/src/runtime_init.rs diff --git a/06_drivers_gpio_uart/src/synchronization.rs b/05_drivers_gpio_uart/src/synchronization.rs similarity index 100% rename from 06_drivers_gpio_uart/src/synchronization.rs rename to 05_drivers_gpio_uart/src/synchronization.rs diff --git a/07_uart_chainloader/.vscode/settings.json b/06_uart_chainloader/.vscode/settings.json similarity index 100% rename from 07_uart_chainloader/.vscode/settings.json rename to 06_uart_chainloader/.vscode/settings.json diff --git a/07_uart_chainloader/Cargo.lock b/06_uart_chainloader/Cargo.lock similarity index 100% rename from 07_uart_chainloader/Cargo.lock rename to 06_uart_chainloader/Cargo.lock diff --git a/07_uart_chainloader/Cargo.toml b/06_uart_chainloader/Cargo.toml similarity index 100% rename from 07_uart_chainloader/Cargo.toml rename to 06_uart_chainloader/Cargo.toml diff --git a/07_uart_chainloader/Makefile b/06_uart_chainloader/Makefile similarity index 100% rename from 07_uart_chainloader/Makefile rename to 06_uart_chainloader/Makefile diff --git a/07_uart_chainloader/README.md b/06_uart_chainloader/README.md similarity index 88% rename from 07_uart_chainloader/README.md rename to 06_uart_chainloader/README.md index a60fe9051..457a96cd4 100644 --- a/07_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -1,4 +1,4 @@ -# Tutorial 07 - UART Chainloader +# Tutorial 06 - UART Chainloader ## tl;dr @@ -118,12 +118,12 @@ IN: ## Diff to previous ```diff -Binary files 06_drivers_gpio_uart/demo_payload_rpi3.img and 07_uart_chainloader/demo_payload_rpi3.img differ -Binary files 06_drivers_gpio_uart/demo_payload_rpi4.img and 07_uart_chainloader/demo_payload_rpi4.img differ +Binary files 05_drivers_gpio_uart/demo_payload_rpi3.img and 06_uart_chainloader/demo_payload_rpi3.img differ +Binary files 05_drivers_gpio_uart/demo_payload_rpi4.img and 06_uart_chainloader/demo_payload_rpi4.img differ -diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile ---- 06_drivers_gpio_uart/Makefile -+++ 07_uart_chainloader/Makefile +diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile +--- 05_drivers_gpio_uart/Makefile ++++ 06_uart_chainloader/Makefile @@ -25,6 +25,7 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -176,9 +176,9 @@ diff -uNr 06_drivers_gpio_uart/Makefile 07_uart_chainloader/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s ---- 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s -+++ 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s +diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s +--- 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s ++++ 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -22,20 +22,31 @@ and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs @@ -218,9 +218,9 @@ diff -uNr 06_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 07_uart_chainloader/ .size _start, . - _start .type _start, function -diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ---- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -+++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -144,7 +144,7 @@ // Make an educated guess for a good delay value (Sequence described in the BCM2837 // peripherals PDF). @@ -231,9 +231,9 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_uart // // So lets try to be on the safe side and default to 2000 cycles, which would equal 1 µs -diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ---- 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -+++ 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +--- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ++++ 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -279,7 +279,7 @@ } @@ -275,9 +275,9 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 {} } -diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 07_uart_chainloader/src/bsp/raspberrypi/link.ld ---- 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld -+++ 07_uart_chainloader/src/bsp/raspberrypi/link.ld +diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 06_uart_chainloader/src/bsp/raspberrypi/link.ld +--- 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld ++++ 06_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -16,7 +16,8 @@ SECTIONS @@ -311,9 +311,9 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 07_uart_chainloader/s __bss_start = .; *(.bss*); -diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader/src/bsp/raspberrypi/memory.rs ---- 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs -+++ 07_uart_chainloader/src/bsp/raspberrypi/memory.rs +diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 06_uart_chainloader/src/bsp/raspberrypi/memory.rs +--- 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs ++++ 06_uart_chainloader/src/bsp/raspberrypi/memory.rs @@ -23,9 +23,10 @@ /// The board's physical memory map. #[rustfmt::skip] @@ -343,9 +343,9 @@ diff -uNr 06_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 07_uart_chainloader /// # Safety /// -diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs ---- 06_drivers_gpio_uart/src/main.rs -+++ 07_uart_chainloader/src/main.rs +diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs +--- 05_drivers_gpio_uart/src/main.rs ++++ 06_uart_chainloader/src/main.rs @@ -107,6 +107,7 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html @@ -425,17 +425,17 @@ diff -uNr 06_drivers_gpio_uart/src/main.rs 07_uart_chainloader/src/main.rs + kernel() } -diff -uNr 06_drivers_gpio_uart/update.sh 07_uart_chainloader/update.sh ---- 06_drivers_gpio_uart/update.sh -+++ 07_uart_chainloader/update.sh +diff -uNr 05_drivers_gpio_uart/update.sh 06_uart_chainloader/update.sh +--- 05_drivers_gpio_uart/update.sh ++++ 06_uart_chainloader/update.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash + -+cd ../06_drivers_gpio_uart ++cd ../05_drivers_gpio_uart +BSP=rpi4 make -+cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img ++cp kernel8.img ../06_uart_chainloader/demo_payload_rpi4.img +make -+cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img ++cp kernel8.img ../06_uart_chainloader/demo_payload_rpi3.img +rm kernel8.img ``` diff --git a/07_uart_chainloader/build.rs b/06_uart_chainloader/build.rs similarity index 100% rename from 07_uart_chainloader/build.rs rename to 06_uart_chainloader/build.rs diff --git a/07_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img similarity index 100% rename from 07_uart_chainloader/demo_payload_rpi3.img rename to 06_uart_chainloader/demo_payload_rpi3.img diff --git a/07_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img similarity index 100% rename from 07_uart_chainloader/demo_payload_rpi4.img rename to 06_uart_chainloader/demo_payload_rpi4.img diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu.rs b/06_uart_chainloader/src/_arch/aarch64/cpu.rs similarity index 100% rename from 07_uart_chainloader/src/_arch/aarch64/cpu.rs rename to 06_uart_chainloader/src/_arch/aarch64/cpu.rs diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 07_uart_chainloader/src/_arch/aarch64/cpu/boot.rs rename to 06_uart_chainloader/src/_arch/aarch64/cpu/boot.rs diff --git a/07_uart_chainloader/src/_arch/aarch64/cpu/boot.s b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s rename to 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s diff --git a/07_uart_chainloader/src/bsp.rs b/06_uart_chainloader/src/bsp.rs similarity index 100% rename from 07_uart_chainloader/src/bsp.rs rename to 06_uart_chainloader/src/bsp.rs diff --git a/07_uart_chainloader/src/bsp/device_driver.rs b/06_uart_chainloader/src/bsp/device_driver.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/device_driver.rs rename to 06_uart_chainloader/src/bsp/device_driver.rs diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm.rs b/06_uart_chainloader/src/bsp/device_driver/bcm.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/device_driver/bcm.rs rename to 06_uart_chainloader/src/bsp/device_driver/bcm.rs diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/07_uart_chainloader/src/bsp/device_driver/common.rs b/06_uart_chainloader/src/bsp/device_driver/common.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/device_driver/common.rs rename to 06_uart_chainloader/src/bsp/device_driver/common.rs diff --git a/07_uart_chainloader/src/bsp/raspberrypi.rs b/06_uart_chainloader/src/bsp/raspberrypi.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi.rs rename to 06_uart_chainloader/src/bsp/raspberrypi.rs diff --git a/07_uart_chainloader/src/bsp/raspberrypi/console.rs b/06_uart_chainloader/src/bsp/raspberrypi/console.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi/console.rs rename to 06_uart_chainloader/src/bsp/raspberrypi/console.rs diff --git a/07_uart_chainloader/src/bsp/raspberrypi/cpu.rs b/06_uart_chainloader/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi/cpu.rs rename to 06_uart_chainloader/src/bsp/raspberrypi/cpu.rs diff --git a/07_uart_chainloader/src/bsp/raspberrypi/driver.rs b/06_uart_chainloader/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi/driver.rs rename to 06_uart_chainloader/src/bsp/raspberrypi/driver.rs diff --git a/07_uart_chainloader/src/bsp/raspberrypi/link.ld b/06_uart_chainloader/src/bsp/raspberrypi/link.ld similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi/link.ld rename to 06_uart_chainloader/src/bsp/raspberrypi/link.ld diff --git a/07_uart_chainloader/src/bsp/raspberrypi/memory.rs b/06_uart_chainloader/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 07_uart_chainloader/src/bsp/raspberrypi/memory.rs rename to 06_uart_chainloader/src/bsp/raspberrypi/memory.rs diff --git a/07_uart_chainloader/src/console.rs b/06_uart_chainloader/src/console.rs similarity index 100% rename from 07_uart_chainloader/src/console.rs rename to 06_uart_chainloader/src/console.rs diff --git a/07_uart_chainloader/src/cpu.rs b/06_uart_chainloader/src/cpu.rs similarity index 100% rename from 07_uart_chainloader/src/cpu.rs rename to 06_uart_chainloader/src/cpu.rs diff --git a/07_uart_chainloader/src/cpu/boot.rs b/06_uart_chainloader/src/cpu/boot.rs similarity index 100% rename from 07_uart_chainloader/src/cpu/boot.rs rename to 06_uart_chainloader/src/cpu/boot.rs diff --git a/07_uart_chainloader/src/driver.rs b/06_uart_chainloader/src/driver.rs similarity index 100% rename from 07_uart_chainloader/src/driver.rs rename to 06_uart_chainloader/src/driver.rs diff --git a/07_uart_chainloader/src/main.rs b/06_uart_chainloader/src/main.rs similarity index 100% rename from 07_uart_chainloader/src/main.rs rename to 06_uart_chainloader/src/main.rs diff --git a/07_uart_chainloader/src/memory.rs b/06_uart_chainloader/src/memory.rs similarity index 100% rename from 07_uart_chainloader/src/memory.rs rename to 06_uart_chainloader/src/memory.rs diff --git a/07_uart_chainloader/src/panic_wait.rs b/06_uart_chainloader/src/panic_wait.rs similarity index 100% rename from 07_uart_chainloader/src/panic_wait.rs rename to 06_uart_chainloader/src/panic_wait.rs diff --git a/07_uart_chainloader/src/print.rs b/06_uart_chainloader/src/print.rs similarity index 100% rename from 07_uart_chainloader/src/print.rs rename to 06_uart_chainloader/src/print.rs diff --git a/07_uart_chainloader/src/runtime_init.rs b/06_uart_chainloader/src/runtime_init.rs similarity index 100% rename from 07_uart_chainloader/src/runtime_init.rs rename to 06_uart_chainloader/src/runtime_init.rs diff --git a/07_uart_chainloader/src/synchronization.rs b/06_uart_chainloader/src/synchronization.rs similarity index 100% rename from 07_uart_chainloader/src/synchronization.rs rename to 06_uart_chainloader/src/synchronization.rs diff --git a/06_uart_chainloader/update.sh b/06_uart_chainloader/update.sh new file mode 100755 index 000000000..5fac48ba0 --- /dev/null +++ b/06_uart_chainloader/update.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash + +cd ../05_drivers_gpio_uart +BSP=rpi4 make +cp kernel8.img ../06_uart_chainloader/demo_payload_rpi4.img +make +cp kernel8.img ../06_uart_chainloader/demo_payload_rpi3.img +rm kernel8.img diff --git a/08_timestamps/.vscode/settings.json b/07_timestamps/.vscode/settings.json similarity index 100% rename from 08_timestamps/.vscode/settings.json rename to 07_timestamps/.vscode/settings.json diff --git a/08_timestamps/Cargo.lock b/07_timestamps/Cargo.lock similarity index 100% rename from 08_timestamps/Cargo.lock rename to 07_timestamps/Cargo.lock diff --git a/08_timestamps/Cargo.toml b/07_timestamps/Cargo.toml similarity index 100% rename from 08_timestamps/Cargo.toml rename to 07_timestamps/Cargo.toml diff --git a/08_timestamps/Makefile b/07_timestamps/Makefile similarity index 100% rename from 08_timestamps/Makefile rename to 07_timestamps/Makefile diff --git a/08_timestamps/README.md b/07_timestamps/README.md similarity index 89% rename from 08_timestamps/README.md rename to 07_timestamps/README.md index 122308a6c..b115dc09d 100644 --- a/08_timestamps/README.md +++ b/07_timestamps/README.md @@ -1,4 +1,4 @@ -# Tutorial 08 - Timestamps +# Tutorial 07 - Timestamps ## tl;dr @@ -43,12 +43,12 @@ Minipush 1.0 ## Diff to previous ```diff -Binary files 07_uart_chainloader/demo_payload_rpi3.img and 08_timestamps/demo_payload_rpi3.img differ -Binary files 07_uart_chainloader/demo_payload_rpi4.img and 08_timestamps/demo_payload_rpi4.img differ +Binary files 06_uart_chainloader/demo_payload_rpi3.img and 07_timestamps/demo_payload_rpi3.img differ +Binary files 06_uart_chainloader/demo_payload_rpi4.img and 07_timestamps/demo_payload_rpi4.img differ -diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile ---- 07_uart_chainloader/Makefile -+++ 08_timestamps/Makefile +diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile +--- 06_uart_chainloader/Makefile ++++ 07_timestamps/Makefile @@ -25,7 +25,6 @@ READELF_BINARY = aarch64-none-elf-readelf LINKER_FILE = src/bsp/raspberrypi/link.ld @@ -93,9 +93,9 @@ diff -uNr 07_uart_chainloader/Makefile 08_timestamps/Makefile clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) -diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s 08_timestamps/src/_arch/aarch64/cpu/boot.s ---- 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s -+++ 08_timestamps/src/_arch/aarch64/cpu/boot.s +diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_arch/aarch64/cpu/boot.s +--- 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s ++++ 07_timestamps/src/_arch/aarch64/cpu/boot.s @@ -22,31 +22,20 @@ and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs @@ -135,9 +135,9 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu/boot.s 08_timestamps/src/_ar .size _start, . - _start .type _start, function -diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/aarch64/cpu.rs ---- 07_uart_chainloader/src/_arch/aarch64/cpu.rs -+++ 08_timestamps/src/_arch/aarch64/cpu.rs +diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu.rs 07_timestamps/src/_arch/aarch64/cpu.rs +--- 06_uart_chainloader/src/_arch/aarch64/cpu.rs ++++ 07_timestamps/src/_arch/aarch64/cpu.rs @@ -19,15 +19,6 @@ pub use asm::nop; @@ -155,9 +155,9 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/cpu.rs 08_timestamps/src/_arch/a #[inline(always)] pub fn wait_forever() -> ! { -diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/aarch64/time.rs ---- 07_uart_chainloader/src/_arch/aarch64/time.rs -+++ 08_timestamps/src/_arch/aarch64/time.rs +diff -uNr 06_uart_chainloader/src/_arch/aarch64/time.rs 07_timestamps/src/_arch/aarch64/time.rs +--- 06_uart_chainloader/src/_arch/aarch64/time.rs ++++ 07_timestamps/src/_arch/aarch64/time.rs @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -278,9 +278,9 @@ diff -uNr 07_uart_chainloader/src/_arch/aarch64/time.rs 08_timestamps/src/_arch/ + } +} -diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ---- 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -+++ 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -139,25 +139,19 @@ /// Disable pull-up/down on pins 14 and 15. #[cfg(feature = "bsp_rpi3")] @@ -314,9 +314,9 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 08_times self.registers.GPPUD.write(GPPUD::PUD::Off); self.registers.GPPUDCLK0.set(0); -diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ---- 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -+++ 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +--- 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ++++ 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -279,7 +279,7 @@ } @@ -358,9 +358,9 @@ diff -uNr 07_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 08 {} } -diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/link.ld 08_timestamps/src/bsp/raspberrypi/link.ld ---- 07_uart_chainloader/src/bsp/raspberrypi/link.ld -+++ 08_timestamps/src/bsp/raspberrypi/link.ld +diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/link.ld 07_timestamps/src/bsp/raspberrypi/link.ld +--- 06_uart_chainloader/src/bsp/raspberrypi/link.ld ++++ 07_timestamps/src/bsp/raspberrypi/link.ld @@ -16,8 +16,7 @@ SECTIONS @@ -394,9 +394,9 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/link.ld 08_timestamps/src/bsp/ __bss_start = .; *(.bss*); -diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bsp/raspberrypi/memory.rs ---- 07_uart_chainloader/src/bsp/raspberrypi/memory.rs -+++ 08_timestamps/src/bsp/raspberrypi/memory.rs +diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/memory.rs 07_timestamps/src/bsp/raspberrypi/memory.rs +--- 06_uart_chainloader/src/bsp/raspberrypi/memory.rs ++++ 07_timestamps/src/bsp/raspberrypi/memory.rs @@ -23,10 +23,9 @@ /// The board's physical memory map. #[rustfmt::skip] @@ -426,9 +426,9 @@ diff -uNr 07_uart_chainloader/src/bsp/raspberrypi/memory.rs 08_timestamps/src/bs /// # Safety /// -diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs ---- 07_uart_chainloader/src/cpu.rs -+++ 08_timestamps/src/cpu.rs +diff -uNr 06_uart_chainloader/src/cpu.rs 07_timestamps/src/cpu.rs +--- 06_uart_chainloader/src/cpu.rs ++++ 07_timestamps/src/cpu.rs @@ -14,6 +14,3 @@ // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- @@ -437,9 +437,9 @@ diff -uNr 07_uart_chainloader/src/cpu.rs 08_timestamps/src/cpu.rs -#[cfg(feature = "bsp_rpi3")] -pub use arch_cpu::spin_for_cycles; -diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs ---- 07_uart_chainloader/src/main.rs -+++ 08_timestamps/src/main.rs +diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs +--- 06_uart_chainloader/src/main.rs ++++ 07_timestamps/src/main.rs @@ -107,7 +107,6 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html @@ -531,9 +531,9 @@ diff -uNr 07_uart_chainloader/src/main.rs 08_timestamps/src/main.rs + } } -diff -uNr 07_uart_chainloader/src/print.rs 08_timestamps/src/print.rs ---- 07_uart_chainloader/src/print.rs -+++ 08_timestamps/src/print.rs +diff -uNr 06_uart_chainloader/src/print.rs 07_timestamps/src/print.rs +--- 06_uart_chainloader/src/print.rs ++++ 07_timestamps/src/print.rs @@ -36,3 +36,71 @@ $crate::print::_print(format_args_nl!($($arg)*)); }) @@ -607,9 +607,9 @@ diff -uNr 07_uart_chainloader/src/print.rs 08_timestamps/src/print.rs + }) +} -diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs ---- 07_uart_chainloader/src/time.rs -+++ 08_timestamps/src/time.rs +diff -uNr 06_uart_chainloader/src/time.rs 07_timestamps/src/time.rs +--- 06_uart_chainloader/src/time.rs ++++ 07_timestamps/src/time.rs @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -649,17 +649,17 @@ diff -uNr 07_uart_chainloader/src/time.rs 08_timestamps/src/time.rs + } +} -diff -uNr 07_uart_chainloader/update.sh 08_timestamps/update.sh ---- 07_uart_chainloader/update.sh -+++ 08_timestamps/update.sh +diff -uNr 06_uart_chainloader/update.sh 07_timestamps/update.sh +--- 06_uart_chainloader/update.sh ++++ 07_timestamps/update.sh @@ -1,8 +0,0 @@ -#!/usr/bin/env bash - --cd ../06_drivers_gpio_uart +-cd ../05_drivers_gpio_uart -BSP=rpi4 make --cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img +-cp kernel8.img ../06_uart_chainloader/demo_payload_rpi4.img -make --cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img +-cp kernel8.img ../06_uart_chainloader/demo_payload_rpi3.img -rm kernel8.img ``` diff --git a/08_timestamps/build.rs b/07_timestamps/build.rs similarity index 100% rename from 08_timestamps/build.rs rename to 07_timestamps/build.rs diff --git a/08_timestamps/src/_arch/aarch64/cpu.rs b/07_timestamps/src/_arch/aarch64/cpu.rs similarity index 100% rename from 08_timestamps/src/_arch/aarch64/cpu.rs rename to 07_timestamps/src/_arch/aarch64/cpu.rs diff --git a/08_timestamps/src/_arch/aarch64/cpu/boot.rs b/07_timestamps/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 08_timestamps/src/_arch/aarch64/cpu/boot.rs rename to 07_timestamps/src/_arch/aarch64/cpu/boot.rs diff --git a/08_timestamps/src/_arch/aarch64/cpu/boot.s b/07_timestamps/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 08_timestamps/src/_arch/aarch64/cpu/boot.s rename to 07_timestamps/src/_arch/aarch64/cpu/boot.s diff --git a/08_timestamps/src/_arch/aarch64/time.rs b/07_timestamps/src/_arch/aarch64/time.rs similarity index 100% rename from 08_timestamps/src/_arch/aarch64/time.rs rename to 07_timestamps/src/_arch/aarch64/time.rs diff --git a/08_timestamps/src/bsp.rs b/07_timestamps/src/bsp.rs similarity index 100% rename from 08_timestamps/src/bsp.rs rename to 07_timestamps/src/bsp.rs diff --git a/08_timestamps/src/bsp/device_driver.rs b/07_timestamps/src/bsp/device_driver.rs similarity index 100% rename from 08_timestamps/src/bsp/device_driver.rs rename to 07_timestamps/src/bsp/device_driver.rs diff --git a/08_timestamps/src/bsp/device_driver/bcm.rs b/07_timestamps/src/bsp/device_driver/bcm.rs similarity index 100% rename from 08_timestamps/src/bsp/device_driver/bcm.rs rename to 07_timestamps/src/bsp/device_driver/bcm.rs diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 08_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/08_timestamps/src/bsp/device_driver/common.rs b/07_timestamps/src/bsp/device_driver/common.rs similarity index 100% rename from 08_timestamps/src/bsp/device_driver/common.rs rename to 07_timestamps/src/bsp/device_driver/common.rs diff --git a/08_timestamps/src/bsp/raspberrypi.rs b/07_timestamps/src/bsp/raspberrypi.rs similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi.rs rename to 07_timestamps/src/bsp/raspberrypi.rs diff --git a/08_timestamps/src/bsp/raspberrypi/console.rs b/07_timestamps/src/bsp/raspberrypi/console.rs similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi/console.rs rename to 07_timestamps/src/bsp/raspberrypi/console.rs diff --git a/08_timestamps/src/bsp/raspberrypi/cpu.rs b/07_timestamps/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi/cpu.rs rename to 07_timestamps/src/bsp/raspberrypi/cpu.rs diff --git a/08_timestamps/src/bsp/raspberrypi/driver.rs b/07_timestamps/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi/driver.rs rename to 07_timestamps/src/bsp/raspberrypi/driver.rs diff --git a/08_timestamps/src/bsp/raspberrypi/link.ld b/07_timestamps/src/bsp/raspberrypi/link.ld similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi/link.ld rename to 07_timestamps/src/bsp/raspberrypi/link.ld diff --git a/08_timestamps/src/bsp/raspberrypi/memory.rs b/07_timestamps/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 08_timestamps/src/bsp/raspberrypi/memory.rs rename to 07_timestamps/src/bsp/raspberrypi/memory.rs diff --git a/08_timestamps/src/console.rs b/07_timestamps/src/console.rs similarity index 100% rename from 08_timestamps/src/console.rs rename to 07_timestamps/src/console.rs diff --git a/08_timestamps/src/cpu.rs b/07_timestamps/src/cpu.rs similarity index 100% rename from 08_timestamps/src/cpu.rs rename to 07_timestamps/src/cpu.rs diff --git a/08_timestamps/src/cpu/boot.rs b/07_timestamps/src/cpu/boot.rs similarity index 100% rename from 08_timestamps/src/cpu/boot.rs rename to 07_timestamps/src/cpu/boot.rs diff --git a/08_timestamps/src/driver.rs b/07_timestamps/src/driver.rs similarity index 100% rename from 08_timestamps/src/driver.rs rename to 07_timestamps/src/driver.rs diff --git a/08_timestamps/src/main.rs b/07_timestamps/src/main.rs similarity index 100% rename from 08_timestamps/src/main.rs rename to 07_timestamps/src/main.rs diff --git a/08_timestamps/src/memory.rs b/07_timestamps/src/memory.rs similarity index 100% rename from 08_timestamps/src/memory.rs rename to 07_timestamps/src/memory.rs diff --git a/08_timestamps/src/panic_wait.rs b/07_timestamps/src/panic_wait.rs similarity index 100% rename from 08_timestamps/src/panic_wait.rs rename to 07_timestamps/src/panic_wait.rs diff --git a/08_timestamps/src/print.rs b/07_timestamps/src/print.rs similarity index 100% rename from 08_timestamps/src/print.rs rename to 07_timestamps/src/print.rs diff --git a/08_timestamps/src/runtime_init.rs b/07_timestamps/src/runtime_init.rs similarity index 100% rename from 08_timestamps/src/runtime_init.rs rename to 07_timestamps/src/runtime_init.rs diff --git a/08_timestamps/src/synchronization.rs b/07_timestamps/src/synchronization.rs similarity index 100% rename from 08_timestamps/src/synchronization.rs rename to 07_timestamps/src/synchronization.rs diff --git a/08_timestamps/src/time.rs b/07_timestamps/src/time.rs similarity index 100% rename from 08_timestamps/src/time.rs rename to 07_timestamps/src/time.rs diff --git a/07_uart_chainloader/update.sh b/07_uart_chainloader/update.sh deleted file mode 100755 index 7d57d0a62..000000000 --- a/07_uart_chainloader/update.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env bash - -cd ../06_drivers_gpio_uart -BSP=rpi4 make -cp kernel8.img ../07_uart_chainloader/demo_payload_rpi4.img -make -cp kernel8.img ../07_uart_chainloader/demo_payload_rpi3.img -rm kernel8.img diff --git a/09_hw_debug_JTAG/.vscode/settings.json b/08_hw_debug_JTAG/.vscode/settings.json similarity index 100% rename from 09_hw_debug_JTAG/.vscode/settings.json rename to 08_hw_debug_JTAG/.vscode/settings.json diff --git a/09_hw_debug_JTAG/Cargo.lock b/08_hw_debug_JTAG/Cargo.lock similarity index 100% rename from 09_hw_debug_JTAG/Cargo.lock rename to 08_hw_debug_JTAG/Cargo.lock diff --git a/09_hw_debug_JTAG/Cargo.toml b/08_hw_debug_JTAG/Cargo.toml similarity index 100% rename from 09_hw_debug_JTAG/Cargo.toml rename to 08_hw_debug_JTAG/Cargo.toml diff --git a/09_hw_debug_JTAG/Makefile b/08_hw_debug_JTAG/Makefile similarity index 100% rename from 09_hw_debug_JTAG/Makefile rename to 08_hw_debug_JTAG/Makefile diff --git a/09_hw_debug_JTAG/README.md b/08_hw_debug_JTAG/README.md similarity index 98% rename from 09_hw_debug_JTAG/README.md rename to 08_hw_debug_JTAG/README.md index 0c9fb497d..03d16a4f0 100644 --- a/09_hw_debug_JTAG/README.md +++ b/08_hw_debug_JTAG/README.md @@ -1,4 +1,4 @@ -# Tutorial 09 - Hardware Debugging using JTAG +# Tutorial 08 - Hardware Debugging using JTAG ## tl;dr @@ -305,9 +305,9 @@ Thanks to [@naotaco](https://github.com/naotaco) for laying the groundwork for t ## Diff to previous ```diff -diff -uNr 08_timestamps/Makefile 09_hw_debug_JTAG/Makefile ---- 08_timestamps/Makefile -+++ 09_hw_debug_JTAG/Makefile +diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile +--- 07_timestamps/Makefile ++++ 08_hw_debug_JTAG/Makefile @@ -23,6 +23,8 @@ OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm diff --git a/09_hw_debug_JTAG/build.rs b/08_hw_debug_JTAG/build.rs similarity index 100% rename from 09_hw_debug_JTAG/build.rs rename to 08_hw_debug_JTAG/build.rs diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs b/08_hw_debug_JTAG/src/_arch/aarch64/cpu.rs similarity index 100% rename from 09_hw_debug_JTAG/src/_arch/aarch64/cpu.rs rename to 08_hw_debug_JTAG/src/_arch/aarch64/cpu.rs diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs rename to 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s rename to 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s diff --git a/09_hw_debug_JTAG/src/_arch/aarch64/time.rs b/08_hw_debug_JTAG/src/_arch/aarch64/time.rs similarity index 100% rename from 09_hw_debug_JTAG/src/_arch/aarch64/time.rs rename to 08_hw_debug_JTAG/src/_arch/aarch64/time.rs diff --git a/09_hw_debug_JTAG/src/bsp.rs b/08_hw_debug_JTAG/src/bsp.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp.rs rename to 08_hw_debug_JTAG/src/bsp.rs diff --git a/09_hw_debug_JTAG/src/bsp/device_driver.rs b/08_hw_debug_JTAG/src/bsp/device_driver.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/device_driver.rs rename to 08_hw_debug_JTAG/src/bsp/device_driver.rs diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs b/08_hw_debug_JTAG/src/bsp/device_driver/bcm.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/device_driver/bcm.rs rename to 08_hw_debug_JTAG/src/bsp/device_driver/bcm.rs diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/09_hw_debug_JTAG/src/bsp/device_driver/common.rs b/08_hw_debug_JTAG/src/bsp/device_driver/common.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/device_driver/common.rs rename to 08_hw_debug_JTAG/src/bsp/device_driver/common.rs diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi.rs rename to 08_hw_debug_JTAG/src/bsp/raspberrypi.rs diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/console.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi/console.rs rename to 08_hw_debug_JTAG/src/bsp/raspberrypi/console.rs diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs rename to 08_hw_debug_JTAG/src/bsp/raspberrypi/cpu.rs diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs rename to 08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi/link.ld rename to 08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld diff --git a/09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 09_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs rename to 08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs diff --git a/09_hw_debug_JTAG/src/console.rs b/08_hw_debug_JTAG/src/console.rs similarity index 100% rename from 09_hw_debug_JTAG/src/console.rs rename to 08_hw_debug_JTAG/src/console.rs diff --git a/09_hw_debug_JTAG/src/cpu.rs b/08_hw_debug_JTAG/src/cpu.rs similarity index 100% rename from 09_hw_debug_JTAG/src/cpu.rs rename to 08_hw_debug_JTAG/src/cpu.rs diff --git a/09_hw_debug_JTAG/src/cpu/boot.rs b/08_hw_debug_JTAG/src/cpu/boot.rs similarity index 100% rename from 09_hw_debug_JTAG/src/cpu/boot.rs rename to 08_hw_debug_JTAG/src/cpu/boot.rs diff --git a/09_hw_debug_JTAG/src/driver.rs b/08_hw_debug_JTAG/src/driver.rs similarity index 100% rename from 09_hw_debug_JTAG/src/driver.rs rename to 08_hw_debug_JTAG/src/driver.rs diff --git a/09_hw_debug_JTAG/src/main.rs b/08_hw_debug_JTAG/src/main.rs similarity index 100% rename from 09_hw_debug_JTAG/src/main.rs rename to 08_hw_debug_JTAG/src/main.rs diff --git a/09_hw_debug_JTAG/src/memory.rs b/08_hw_debug_JTAG/src/memory.rs similarity index 100% rename from 09_hw_debug_JTAG/src/memory.rs rename to 08_hw_debug_JTAG/src/memory.rs diff --git a/09_hw_debug_JTAG/src/panic_wait.rs b/08_hw_debug_JTAG/src/panic_wait.rs similarity index 100% rename from 09_hw_debug_JTAG/src/panic_wait.rs rename to 08_hw_debug_JTAG/src/panic_wait.rs diff --git a/09_hw_debug_JTAG/src/print.rs b/08_hw_debug_JTAG/src/print.rs similarity index 100% rename from 09_hw_debug_JTAG/src/print.rs rename to 08_hw_debug_JTAG/src/print.rs diff --git a/09_hw_debug_JTAG/src/runtime_init.rs b/08_hw_debug_JTAG/src/runtime_init.rs similarity index 100% rename from 09_hw_debug_JTAG/src/runtime_init.rs rename to 08_hw_debug_JTAG/src/runtime_init.rs diff --git a/09_hw_debug_JTAG/src/synchronization.rs b/08_hw_debug_JTAG/src/synchronization.rs similarity index 100% rename from 09_hw_debug_JTAG/src/synchronization.rs rename to 08_hw_debug_JTAG/src/synchronization.rs diff --git a/09_hw_debug_JTAG/src/time.rs b/08_hw_debug_JTAG/src/time.rs similarity index 100% rename from 09_hw_debug_JTAG/src/time.rs rename to 08_hw_debug_JTAG/src/time.rs diff --git a/10_privilege_level/.vscode/settings.json b/09_privilege_level/.vscode/settings.json similarity index 100% rename from 10_privilege_level/.vscode/settings.json rename to 09_privilege_level/.vscode/settings.json diff --git a/10_privilege_level/Cargo.lock b/09_privilege_level/Cargo.lock similarity index 100% rename from 10_privilege_level/Cargo.lock rename to 09_privilege_level/Cargo.lock diff --git a/10_privilege_level/Cargo.toml b/09_privilege_level/Cargo.toml similarity index 100% rename from 10_privilege_level/Cargo.toml rename to 09_privilege_level/Cargo.toml diff --git a/10_privilege_level/Makefile b/09_privilege_level/Makefile similarity index 100% rename from 10_privilege_level/Makefile rename to 09_privilege_level/Makefile diff --git a/10_privilege_level/README.md b/09_privilege_level/README.md similarity index 93% rename from 10_privilege_level/README.md rename to 09_privilege_level/README.md index 3e160e22d..b8af30216 100644 --- a/10_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -1,4 +1,4 @@ -# Tutorial 10 - Privilege Level +# Tutorial 09 - Privilege Level ## tl;dr @@ -71,7 +71,7 @@ pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u6 Since `EL2` is more privileged than `EL1`, it has control over various processor features and can allow or disallow `EL1` code to use them. One such example is access to timer and counter registers. -We are already using them since [tutorial 08](../08_timestamps/), so of course we want to keep them. +We are already using them since [tutorial 07](../07_timestamps/), so of course we want to keep them. Therefore we set the respective flags in the [Counter-timer Hypervisor Control register] and additionally set the virtual offset to zero so that we get the real physical value everytime: @@ -198,9 +198,9 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 10_privilege_level/src/_arch/aarch64/cpu/boot.rs ---- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs -+++ 10_privilege_level/src/_arch/aarch64/cpu/boot.rs +diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/_arch/aarch64/cpu/boot.rs +--- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs ++++ 09_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -12,11 +12,53 @@ //! crate::cpu::boot::arch_boot @@ -270,9 +270,9 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 10_privilege_level/src/ + asm::eret() } -diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 10_privilege_level/src/_arch/aarch64/cpu/boot.s ---- 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s -+++ 10_privilege_level/src/_arch/aarch64/cpu/boot.s +diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_arch/aarch64/cpu/boot.s +--- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s ++++ 09_privilege_level/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,7 @@ // Definitions //-------------------------------------------------------------------------------------------------- @@ -308,9 +308,9 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 10_privilege_level/src/_ // Infinitely wait for events (aka "park the core"). -diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs ---- 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs -+++ 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs +diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs +--- 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs ++++ 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -394,9 +394,9 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 10_privil + info!(" FIQ: {}", to_mask_str(is_masked::())); +} -diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs 10_privilege_level/src/_arch/aarch64/exception.rs ---- 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs -+++ 10_privilege_level/src/_arch/aarch64/exception.rs +diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs 09_privilege_level/src/_arch/aarch64/exception.rs +--- 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs ++++ 09_privilege_level/src/_arch/aarch64/exception.rs @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -429,9 +429,9 @@ diff -uNr 09_hw_debug_JTAG/src/_arch/aarch64/exception.rs 10_privilege_level/src + } +} -diff -uNr 09_hw_debug_JTAG/src/exception/asynchronous.rs 10_privilege_level/src/exception/asynchronous.rs ---- 09_hw_debug_JTAG/src/exception/asynchronous.rs -+++ 10_privilege_level/src/exception/asynchronous.rs +diff -uNr 08_hw_debug_JTAG/src/exception/asynchronous.rs 09_privilege_level/src/exception/asynchronous.rs +--- 08_hw_debug_JTAG/src/exception/asynchronous.rs ++++ 09_privilege_level/src/exception/asynchronous.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -448,9 +448,9 @@ diff -uNr 09_hw_debug_JTAG/src/exception/asynchronous.rs 10_privilege_level/src/ +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::print_state; -diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs ---- 09_hw_debug_JTAG/src/exception.rs -+++ 10_privilege_level/src/exception.rs +diff -uNr 08_hw_debug_JTAG/src/exception.rs 09_privilege_level/src/exception.rs +--- 08_hw_debug_JTAG/src/exception.rs ++++ 09_privilege_level/src/exception.rs @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -483,9 +483,9 @@ diff -uNr 09_hw_debug_JTAG/src/exception.rs 10_privilege_level/src/exception.rs + Unknown, +} -diff -uNr 09_hw_debug_JTAG/src/main.rs 10_privilege_level/src/main.rs ---- 09_hw_debug_JTAG/src/main.rs -+++ 10_privilege_level/src/main.rs +diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs +--- 08_hw_debug_JTAG/src/main.rs ++++ 09_privilege_level/src/main.rs @@ -119,6 +119,7 @@ mod console; mod cpu; diff --git a/10_privilege_level/build.rs b/09_privilege_level/build.rs similarity index 100% rename from 10_privilege_level/build.rs rename to 09_privilege_level/build.rs diff --git a/10_privilege_level/src/_arch/aarch64/cpu.rs b/09_privilege_level/src/_arch/aarch64/cpu.rs similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/cpu.rs rename to 09_privilege_level/src/_arch/aarch64/cpu.rs diff --git a/10_privilege_level/src/_arch/aarch64/cpu/boot.rs b/09_privilege_level/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/cpu/boot.rs rename to 09_privilege_level/src/_arch/aarch64/cpu/boot.rs diff --git a/10_privilege_level/src/_arch/aarch64/cpu/boot.s b/09_privilege_level/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/cpu/boot.s rename to 09_privilege_level/src/_arch/aarch64/cpu/boot.s diff --git a/10_privilege_level/src/_arch/aarch64/exception.rs b/09_privilege_level/src/_arch/aarch64/exception.rs similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/exception.rs rename to 09_privilege_level/src/_arch/aarch64/exception.rs diff --git a/10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs b/09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/exception/asynchronous.rs rename to 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs diff --git a/10_privilege_level/src/_arch/aarch64/time.rs b/09_privilege_level/src/_arch/aarch64/time.rs similarity index 100% rename from 10_privilege_level/src/_arch/aarch64/time.rs rename to 09_privilege_level/src/_arch/aarch64/time.rs diff --git a/10_privilege_level/src/bsp.rs b/09_privilege_level/src/bsp.rs similarity index 100% rename from 10_privilege_level/src/bsp.rs rename to 09_privilege_level/src/bsp.rs diff --git a/10_privilege_level/src/bsp/device_driver.rs b/09_privilege_level/src/bsp/device_driver.rs similarity index 100% rename from 10_privilege_level/src/bsp/device_driver.rs rename to 09_privilege_level/src/bsp/device_driver.rs diff --git a/10_privilege_level/src/bsp/device_driver/bcm.rs b/09_privilege_level/src/bsp/device_driver/bcm.rs similarity index 100% rename from 10_privilege_level/src/bsp/device_driver/bcm.rs rename to 09_privilege_level/src/bsp/device_driver/bcm.rs diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 10_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/10_privilege_level/src/bsp/device_driver/common.rs b/09_privilege_level/src/bsp/device_driver/common.rs similarity index 100% rename from 10_privilege_level/src/bsp/device_driver/common.rs rename to 09_privilege_level/src/bsp/device_driver/common.rs diff --git a/10_privilege_level/src/bsp/raspberrypi.rs b/09_privilege_level/src/bsp/raspberrypi.rs similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi.rs rename to 09_privilege_level/src/bsp/raspberrypi.rs diff --git a/10_privilege_level/src/bsp/raspberrypi/console.rs b/09_privilege_level/src/bsp/raspberrypi/console.rs similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi/console.rs rename to 09_privilege_level/src/bsp/raspberrypi/console.rs diff --git a/10_privilege_level/src/bsp/raspberrypi/cpu.rs b/09_privilege_level/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi/cpu.rs rename to 09_privilege_level/src/bsp/raspberrypi/cpu.rs diff --git a/10_privilege_level/src/bsp/raspberrypi/driver.rs b/09_privilege_level/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi/driver.rs rename to 09_privilege_level/src/bsp/raspberrypi/driver.rs diff --git a/10_privilege_level/src/bsp/raspberrypi/link.ld b/09_privilege_level/src/bsp/raspberrypi/link.ld similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi/link.ld rename to 09_privilege_level/src/bsp/raspberrypi/link.ld diff --git a/10_privilege_level/src/bsp/raspberrypi/memory.rs b/09_privilege_level/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 10_privilege_level/src/bsp/raspberrypi/memory.rs rename to 09_privilege_level/src/bsp/raspberrypi/memory.rs diff --git a/10_privilege_level/src/console.rs b/09_privilege_level/src/console.rs similarity index 100% rename from 10_privilege_level/src/console.rs rename to 09_privilege_level/src/console.rs diff --git a/10_privilege_level/src/cpu.rs b/09_privilege_level/src/cpu.rs similarity index 100% rename from 10_privilege_level/src/cpu.rs rename to 09_privilege_level/src/cpu.rs diff --git a/10_privilege_level/src/cpu/boot.rs b/09_privilege_level/src/cpu/boot.rs similarity index 100% rename from 10_privilege_level/src/cpu/boot.rs rename to 09_privilege_level/src/cpu/boot.rs diff --git a/10_privilege_level/src/driver.rs b/09_privilege_level/src/driver.rs similarity index 100% rename from 10_privilege_level/src/driver.rs rename to 09_privilege_level/src/driver.rs diff --git a/10_privilege_level/src/exception.rs b/09_privilege_level/src/exception.rs similarity index 100% rename from 10_privilege_level/src/exception.rs rename to 09_privilege_level/src/exception.rs diff --git a/10_privilege_level/src/exception/asynchronous.rs b/09_privilege_level/src/exception/asynchronous.rs similarity index 100% rename from 10_privilege_level/src/exception/asynchronous.rs rename to 09_privilege_level/src/exception/asynchronous.rs diff --git a/10_privilege_level/src/main.rs b/09_privilege_level/src/main.rs similarity index 100% rename from 10_privilege_level/src/main.rs rename to 09_privilege_level/src/main.rs diff --git a/10_privilege_level/src/memory.rs b/09_privilege_level/src/memory.rs similarity index 100% rename from 10_privilege_level/src/memory.rs rename to 09_privilege_level/src/memory.rs diff --git a/10_privilege_level/src/panic_wait.rs b/09_privilege_level/src/panic_wait.rs similarity index 100% rename from 10_privilege_level/src/panic_wait.rs rename to 09_privilege_level/src/panic_wait.rs diff --git a/10_privilege_level/src/print.rs b/09_privilege_level/src/print.rs similarity index 100% rename from 10_privilege_level/src/print.rs rename to 09_privilege_level/src/print.rs diff --git a/10_privilege_level/src/runtime_init.rs b/09_privilege_level/src/runtime_init.rs similarity index 100% rename from 10_privilege_level/src/runtime_init.rs rename to 09_privilege_level/src/runtime_init.rs diff --git a/10_privilege_level/src/synchronization.rs b/09_privilege_level/src/synchronization.rs similarity index 100% rename from 10_privilege_level/src/synchronization.rs rename to 09_privilege_level/src/synchronization.rs diff --git a/10_privilege_level/src/time.rs b/09_privilege_level/src/time.rs similarity index 100% rename from 10_privilege_level/src/time.rs rename to 09_privilege_level/src/time.rs diff --git a/11_virtual_mem_part1_identity_mapping/.vscode/settings.json b/10_virtual_mem_part1_identity_mapping/.vscode/settings.json similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/.vscode/settings.json rename to 10_virtual_mem_part1_identity_mapping/.vscode/settings.json diff --git a/11_virtual_mem_part1_identity_mapping/Cargo.lock b/10_virtual_mem_part1_identity_mapping/Cargo.lock similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/Cargo.lock rename to 10_virtual_mem_part1_identity_mapping/Cargo.lock diff --git a/11_virtual_mem_part1_identity_mapping/Cargo.toml b/10_virtual_mem_part1_identity_mapping/Cargo.toml similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/Cargo.toml rename to 10_virtual_mem_part1_identity_mapping/Cargo.toml diff --git a/11_virtual_mem_part1_identity_mapping/Makefile b/10_virtual_mem_part1_identity_mapping/Makefile similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/Makefile rename to 10_virtual_mem_part1_identity_mapping/Makefile diff --git a/11_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md similarity index 96% rename from 11_virtual_mem_part1_identity_mapping/README.md rename to 10_virtual_mem_part1_identity_mapping/README.md index b2313e1c2..d2d72ea55 100644 --- a/11_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -1,4 +1,4 @@ -# Tutorial 11 - Virtual Memory Part 1: Identity Map All The Things! +# Tutorial 10 - Virtual Memory Part 1: Identity Map All The Things! ## tl;dr @@ -348,9 +348,9 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs ---- 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs -+++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,288 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -641,9 +641,9 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + } +} -diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs ---- 10_privilege_level/src/_arch/aarch64/memory/mmu.rs -+++ 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu.rs 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +--- 09_privilege_level/src/_arch/aarch64/memory/mmu.rs ++++ 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -810,9 +810,9 @@ diff -uNr 10_privilege_level/src/_arch/aarch64/memory/mmu.rs 11_virtual_mem_part + } +} -diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld ---- 10_privilege_level/src/bsp/raspberrypi/link.ld -+++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +diff -uNr 09_privilege_level/src/bsp/raspberrypi/link.ld 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +--- 09_privilege_level/src/bsp/raspberrypi/link.ld ++++ 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -26,6 +26,7 @@ /*********************************************************************************************** * Code + RO Data + Global Offset Table @@ -832,9 +832,9 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/link.ld 11_virtual_mem_part1_id * Data + BSS ***********************************************************************************************/ -diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs ---- 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs -+++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs +--- 09_privilege_level/src/bsp/raspberrypi/memory/mmu.rs ++++ 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -923,9 +923,9 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 11_virtual_mem_pa + &LAYOUT +} -diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs ---- 10_privilege_level/src/bsp/raspberrypi/memory.rs -+++ 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory.rs 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +--- 09_privilege_level/src/bsp/raspberrypi/memory.rs ++++ 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -4,6 +4,8 @@ //! BSP Memory Management. @@ -1011,9 +1011,9 @@ diff -uNr 10_privilege_level/src/bsp/raspberrypi/memory.rs 11_virtual_mem_part1_ //-------------------------------------------------------------------------------------------------- -diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/src/bsp.rs ---- 10_privilege_level/src/bsp.rs -+++ 11_virtual_mem_part1_identity_mapping/src/bsp.rs +diff -uNr 09_privilege_level/src/bsp.rs 10_virtual_mem_part1_identity_mapping/src/bsp.rs +--- 09_privilege_level/src/bsp.rs ++++ 10_virtual_mem_part1_identity_mapping/src/bsp.rs @@ -4,7 +4,7 @@ //! Conditional reexporting of Board Support Packages. @@ -1024,9 +1024,9 @@ diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_mem_part1_identity_mapping/sr #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; -diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/src/main.rs ---- 10_privilege_level/src/main.rs -+++ 11_virtual_mem_part1_identity_mapping/src/main.rs +diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/src/main.rs +--- 09_privilege_level/src/main.rs ++++ 10_virtual_mem_part1_identity_mapping/src/main.rs @@ -107,7 +107,11 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html @@ -1083,9 +1083,9 @@ diff -uNr 10_privilege_level/src/main.rs 11_virtual_mem_part1_identity_mapping/s // Discard any spurious received characters before going into echo mode. -diff -uNr 10_privilege_level/src/memory/mmu/translation_table.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs ---- 10_privilege_level/src/memory/mmu/translation_table.rs -+++ 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs +diff -uNr 09_privilege_level/src/memory/mmu/translation_table.rs 10_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs +--- 09_privilege_level/src/memory/mmu/translation_table.rs ++++ 10_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1102,9 +1102,9 @@ diff -uNr 10_privilege_level/src/memory/mmu/translation_table.rs 11_virtual_mem_ +//-------------------------------------------------------------------------------------------------- +pub use arch_translation_table::KernelTranslationTable; -diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs ---- 10_privilege_level/src/memory/mmu.rs -+++ 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs +diff -uNr 09_privilege_level/src/memory/mmu.rs 10_virtual_mem_part1_identity_mapping/src/memory/mmu.rs +--- 09_privilege_level/src/memory/mmu.rs ++++ 10_virtual_mem_part1_identity_mapping/src/memory/mmu.rs @@ -0,0 +1,264 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1371,9 +1371,9 @@ diff -uNr 10_privilege_level/src/memory/mmu.rs 11_virtual_mem_part1_identity_map + } +} -diff -uNr 10_privilege_level/src/memory.rs 11_virtual_mem_part1_identity_mapping/src/memory.rs ---- 10_privilege_level/src/memory.rs -+++ 11_virtual_mem_part1_identity_mapping/src/memory.rs +diff -uNr 09_privilege_level/src/memory.rs 10_virtual_mem_part1_identity_mapping/src/memory.rs +--- 09_privilege_level/src/memory.rs ++++ 10_virtual_mem_part1_identity_mapping/src/memory.rs @@ -4,6 +4,8 @@ //! Memory Management. diff --git a/11_virtual_mem_part1_identity_mapping/build.rs b/10_virtual_mem_part1_identity_mapping/build.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/build.rs rename to 10_virtual_mem_part1_identity_mapping/build.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs rename to 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp.rs b/10_virtual_mem_part1_identity_mapping/src/bsp.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/device_driver.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/common.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/cpu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs rename to 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/console.rs b/10_virtual_mem_part1_identity_mapping/src/console.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/console.rs rename to 10_virtual_mem_part1_identity_mapping/src/console.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu.rs b/10_virtual_mem_part1_identity_mapping/src/cpu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/cpu.rs rename to 10_virtual_mem_part1_identity_mapping/src/cpu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs b/10_virtual_mem_part1_identity_mapping/src/cpu/boot.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/cpu/boot.rs rename to 10_virtual_mem_part1_identity_mapping/src/cpu/boot.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/driver.rs b/10_virtual_mem_part1_identity_mapping/src/driver.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/driver.rs rename to 10_virtual_mem_part1_identity_mapping/src/driver.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/exception.rs b/10_virtual_mem_part1_identity_mapping/src/exception.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/exception.rs rename to 10_virtual_mem_part1_identity_mapping/src/exception.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs b/10_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs rename to 10_virtual_mem_part1_identity_mapping/src/exception/asynchronous.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/main.rs b/10_virtual_mem_part1_identity_mapping/src/main.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/main.rs rename to 10_virtual_mem_part1_identity_mapping/src/main.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/memory.rs b/10_virtual_mem_part1_identity_mapping/src/memory.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/memory.rs rename to 10_virtual_mem_part1_identity_mapping/src/memory.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs b/10_virtual_mem_part1_identity_mapping/src/memory/mmu.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/memory/mmu.rs rename to 10_virtual_mem_part1_identity_mapping/src/memory/mmu.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs b/10_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs rename to 10_virtual_mem_part1_identity_mapping/src/memory/mmu/translation_table.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/panic_wait.rs b/10_virtual_mem_part1_identity_mapping/src/panic_wait.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/panic_wait.rs rename to 10_virtual_mem_part1_identity_mapping/src/panic_wait.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/print.rs b/10_virtual_mem_part1_identity_mapping/src/print.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/print.rs rename to 10_virtual_mem_part1_identity_mapping/src/print.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/runtime_init.rs b/10_virtual_mem_part1_identity_mapping/src/runtime_init.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/runtime_init.rs rename to 10_virtual_mem_part1_identity_mapping/src/runtime_init.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/synchronization.rs b/10_virtual_mem_part1_identity_mapping/src/synchronization.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/synchronization.rs rename to 10_virtual_mem_part1_identity_mapping/src/synchronization.rs diff --git a/11_virtual_mem_part1_identity_mapping/src/time.rs b/10_virtual_mem_part1_identity_mapping/src/time.rs similarity index 100% rename from 11_virtual_mem_part1_identity_mapping/src/time.rs rename to 10_virtual_mem_part1_identity_mapping/src/time.rs diff --git a/12_exceptions_part1_groundwork/.vscode/settings.json b/11_exceptions_part1_groundwork/.vscode/settings.json similarity index 100% rename from 12_exceptions_part1_groundwork/.vscode/settings.json rename to 11_exceptions_part1_groundwork/.vscode/settings.json diff --git a/12_exceptions_part1_groundwork/Cargo.lock b/11_exceptions_part1_groundwork/Cargo.lock similarity index 100% rename from 12_exceptions_part1_groundwork/Cargo.lock rename to 11_exceptions_part1_groundwork/Cargo.lock diff --git a/12_exceptions_part1_groundwork/Cargo.toml b/11_exceptions_part1_groundwork/Cargo.toml similarity index 100% rename from 12_exceptions_part1_groundwork/Cargo.toml rename to 11_exceptions_part1_groundwork/Cargo.toml diff --git a/12_exceptions_part1_groundwork/Makefile b/11_exceptions_part1_groundwork/Makefile similarity index 100% rename from 12_exceptions_part1_groundwork/Makefile rename to 11_exceptions_part1_groundwork/Makefile diff --git a/12_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md similarity index 96% rename from 12_exceptions_part1_groundwork/README.md rename to 11_exceptions_part1_groundwork/README.md index cc4b043ff..878ea4362 100644 --- a/12_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -1,4 +1,4 @@ -# Tutorial 12 - Exceptions Part 1: Groundwork +# Tutorial 11 - Exceptions Part 1: Groundwork ## tl;dr @@ -476,9 +476,9 @@ General purpose register: ## Diff to previous ```diff -diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs ---- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs -+++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +diff -uNr 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +--- 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs ++++ 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -11,7 +11,224 @@ //! //! crate::exception::arch_exception @@ -730,9 +730,9 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 + barrier::isb(barrier::SY); +} -diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s ---- 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s -+++ 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s +diff -uNr 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.s +--- 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s ++++ 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -883,9 +883,9 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.s 12 +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function -diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs ---- 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs -+++ 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +diff -uNr 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs +--- 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.rs ++++ 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs @@ -15,7 +15,7 @@ /// The kernel's address space defined by this BSP. pub type KernelAddrSpace = AddressSpace<{ memory_map::END_INCLUSIVE + 1 }>; @@ -925,9 +925,9 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory/mmu.r RangeInclusive::new(memory_map::mmio::START, memory_map::mmio::END_INCLUSIVE) } -diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp.rs 12_exceptions_part1_groundwork/src/bsp.rs ---- 11_virtual_mem_part1_identity_mapping/src/bsp.rs -+++ 12_exceptions_part1_groundwork/src/bsp.rs +diff -uNr 10_virtual_mem_part1_identity_mapping/src/bsp.rs 11_exceptions_part1_groundwork/src/bsp.rs +--- 10_virtual_mem_part1_identity_mapping/src/bsp.rs ++++ 11_exceptions_part1_groundwork/src/bsp.rs @@ -4,7 +4,7 @@ //! Conditional reexporting of Board Support Packages. @@ -938,9 +938,9 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/bsp.rs 12_exceptions_part1_g #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod raspberrypi; -diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_part1_groundwork/src/exception.rs ---- 11_virtual_mem_part1_identity_mapping/src/exception.rs -+++ 12_exceptions_part1_groundwork/src/exception.rs +diff -uNr 10_virtual_mem_part1_identity_mapping/src/exception.rs 11_exceptions_part1_groundwork/src/exception.rs +--- 10_virtual_mem_part1_identity_mapping/src/exception.rs ++++ 11_exceptions_part1_groundwork/src/exception.rs @@ -13,7 +13,7 @@ //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -951,9 +951,9 @@ diff -uNr 11_virtual_mem_part1_identity_mapping/src/exception.rs 12_exceptions_p //-------------------------------------------------------------------------------------------------- // Public Definitions -diff -uNr 11_virtual_mem_part1_identity_mapping/src/main.rs 12_exceptions_part1_groundwork/src/main.rs ---- 11_virtual_mem_part1_identity_mapping/src/main.rs -+++ 12_exceptions_part1_groundwork/src/main.rs +diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_groundwork/src/main.rs +--- 10_virtual_mem_part1_identity_mapping/src/main.rs ++++ 11_exceptions_part1_groundwork/src/main.rs @@ -144,6 +144,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; diff --git a/12_exceptions_part1_groundwork/build.rs b/11_exceptions_part1_groundwork/build.rs similarity index 100% rename from 12_exceptions_part1_groundwork/build.rs rename to 11_exceptions_part1_groundwork/build.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.s similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.s rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.s diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs diff --git a/12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/_arch/aarch64/time.rs rename to 11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs diff --git a/12_exceptions_part1_groundwork/src/bsp.rs b/11_exceptions_part1_groundwork/src/bsp.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp.rs rename to 11_exceptions_part1_groundwork/src/bsp.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/device_driver.rs rename to 11_exceptions_part1_groundwork/src/bsp/device_driver.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs rename to 11_exceptions_part1_groundwork/src/bsp/device_driver/bcm.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/common.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/device_driver/common.rs rename to 11_exceptions_part1_groundwork/src/bsp/device_driver/common.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/cpu.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs diff --git a/12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs rename to 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs diff --git a/12_exceptions_part1_groundwork/src/console.rs b/11_exceptions_part1_groundwork/src/console.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/console.rs rename to 11_exceptions_part1_groundwork/src/console.rs diff --git a/12_exceptions_part1_groundwork/src/cpu.rs b/11_exceptions_part1_groundwork/src/cpu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/cpu.rs rename to 11_exceptions_part1_groundwork/src/cpu.rs diff --git a/12_exceptions_part1_groundwork/src/cpu/boot.rs b/11_exceptions_part1_groundwork/src/cpu/boot.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/cpu/boot.rs rename to 11_exceptions_part1_groundwork/src/cpu/boot.rs diff --git a/12_exceptions_part1_groundwork/src/driver.rs b/11_exceptions_part1_groundwork/src/driver.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/driver.rs rename to 11_exceptions_part1_groundwork/src/driver.rs diff --git a/12_exceptions_part1_groundwork/src/exception.rs b/11_exceptions_part1_groundwork/src/exception.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/exception.rs rename to 11_exceptions_part1_groundwork/src/exception.rs diff --git a/12_exceptions_part1_groundwork/src/exception/asynchronous.rs b/11_exceptions_part1_groundwork/src/exception/asynchronous.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/exception/asynchronous.rs rename to 11_exceptions_part1_groundwork/src/exception/asynchronous.rs diff --git a/12_exceptions_part1_groundwork/src/main.rs b/11_exceptions_part1_groundwork/src/main.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/main.rs rename to 11_exceptions_part1_groundwork/src/main.rs diff --git a/12_exceptions_part1_groundwork/src/memory.rs b/11_exceptions_part1_groundwork/src/memory.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/memory.rs rename to 11_exceptions_part1_groundwork/src/memory.rs diff --git a/12_exceptions_part1_groundwork/src/memory/mmu.rs b/11_exceptions_part1_groundwork/src/memory/mmu.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/memory/mmu.rs rename to 11_exceptions_part1_groundwork/src/memory/mmu.rs diff --git a/12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs b/11_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs rename to 11_exceptions_part1_groundwork/src/memory/mmu/translation_table.rs diff --git a/12_exceptions_part1_groundwork/src/panic_wait.rs b/11_exceptions_part1_groundwork/src/panic_wait.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/panic_wait.rs rename to 11_exceptions_part1_groundwork/src/panic_wait.rs diff --git a/12_exceptions_part1_groundwork/src/print.rs b/11_exceptions_part1_groundwork/src/print.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/print.rs rename to 11_exceptions_part1_groundwork/src/print.rs diff --git a/12_exceptions_part1_groundwork/src/runtime_init.rs b/11_exceptions_part1_groundwork/src/runtime_init.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/runtime_init.rs rename to 11_exceptions_part1_groundwork/src/runtime_init.rs diff --git a/12_exceptions_part1_groundwork/src/synchronization.rs b/11_exceptions_part1_groundwork/src/synchronization.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/synchronization.rs rename to 11_exceptions_part1_groundwork/src/synchronization.rs diff --git a/12_exceptions_part1_groundwork/src/time.rs b/11_exceptions_part1_groundwork/src/time.rs similarity index 100% rename from 12_exceptions_part1_groundwork/src/time.rs rename to 11_exceptions_part1_groundwork/src/time.rs diff --git a/13_integrated_testing/.cargo/config.toml b/12_integrated_testing/.cargo/config.toml similarity index 100% rename from 13_integrated_testing/.cargo/config.toml rename to 12_integrated_testing/.cargo/config.toml diff --git a/13_integrated_testing/.vscode/settings.json b/12_integrated_testing/.vscode/settings.json similarity index 100% rename from 13_integrated_testing/.vscode/settings.json rename to 12_integrated_testing/.vscode/settings.json diff --git a/13_integrated_testing/Cargo.lock b/12_integrated_testing/Cargo.lock similarity index 100% rename from 13_integrated_testing/Cargo.lock rename to 12_integrated_testing/Cargo.lock diff --git a/13_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml similarity index 100% rename from 13_integrated_testing/Cargo.toml rename to 12_integrated_testing/Cargo.toml diff --git a/13_integrated_testing/Makefile b/12_integrated_testing/Makefile similarity index 100% rename from 13_integrated_testing/Makefile rename to 12_integrated_testing/Makefile diff --git a/13_integrated_testing/README.md b/12_integrated_testing/README.md similarity index 92% rename from 13_integrated_testing/README.md rename to 12_integrated_testing/README.md index 046607ba7..6b76c1902 100644 --- a/13_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -1,4 +1,4 @@ -# Tutorial 13 - Integrated Testing +# Tutorial 12 - Integrated Testing ## tl;dr @@ -404,7 +404,7 @@ provided to it by `cargo`, and finally compiles a `docker` command to execute th reference, here it is fully resolved for an `RPi3 BSP`: ```bash -docker run -i --rm -v /opt/rust-raspberrypi-OS-tutorials/13_integrated_testing:/work/tutorial -w /work/tutorial rustembedded/osdev-utils ruby tests/runner.rb qemu-system-aarch64 -M raspi3 -serial stdio -display none -semihosting -kernel $TEST_BINARY +docker run -i --rm -v /opt/rust-raspberrypi-OS-tutorials/12_integrated_testing:/work/tutorial -w /work/tutorial rustembedded/osdev-utils ruby tests/runner.rb qemu-system-aarch64 -M raspi3 -serial stdio -display none -semihosting -kernel $TEST_BINARY ``` We're still not done with all the redirections. Spotted the `ruby tests/runner.rb` part that gets @@ -809,16 +809,16 @@ RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D ## Diff to previous ```diff -diff -uNr 12_exceptions_part1_groundwork/.cargo/config.toml 13_integrated_testing/.cargo/config.toml ---- 12_exceptions_part1_groundwork/.cargo/config.toml -+++ 13_integrated_testing/.cargo/config.toml +diff -uNr 11_exceptions_part1_groundwork/.cargo/config.toml 12_integrated_testing/.cargo/config.toml +--- 11_exceptions_part1_groundwork/.cargo/config.toml ++++ 12_integrated_testing/.cargo/config.toml @@ -0,0 +1,2 @@ +[target.'cfg(target_os = "none")'] +runner = "target/kernel_test_runner.sh" -diff -uNr 12_exceptions_part1_groundwork/Cargo.toml 13_integrated_testing/Cargo.toml ---- 12_exceptions_part1_groundwork/Cargo.toml -+++ 13_integrated_testing/Cargo.toml +diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo.toml +--- 11_exceptions_part1_groundwork/Cargo.toml ++++ 12_integrated_testing/Cargo.toml @@ -11,17 +11,45 @@ default = [] bsp_rpi3 = ["register"] @@ -867,9 +867,9 @@ diff -uNr 12_exceptions_part1_groundwork/Cargo.toml 13_integrated_testing/Cargo. +name = "02_exception_sync_page_fault" +harness = false -diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile ---- 12_exceptions_part1_groundwork/Makefile -+++ 13_integrated_testing/Makefile +diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile +--- 11_exceptions_part1_groundwork/Makefile ++++ 12_integrated_testing/Makefile @@ -20,6 +20,7 @@ QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = raspi3 @@ -963,9 +963,9 @@ diff -uNr 12_exceptions_part1_groundwork/Makefile 13_integrated_testing/Makefile chainboot: $(KERNEL_BIN) -diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_testing/src/_arch/aarch64/cpu.rs ---- 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs -+++ 13_integrated_testing/src/_arch/aarch64/cpu.rs +diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 12_integrated_testing/src/_arch/aarch64/cpu.rs +--- 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs ++++ 12_integrated_testing/src/_arch/aarch64/cpu.rs @@ -26,3 +26,24 @@ asm::wfe() } @@ -992,9 +992,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 13_integrated_ + QEMU_EXIT_HANDLE.exit_success() +} -diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integrated_testing/src/_arch/aarch64/exception.rs ---- 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs -+++ 13_integrated_testing/src/_arch/aarch64/exception.rs +diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 12_integrated_testing/src/_arch/aarch64/exception.rs +--- 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs ++++ 12_integrated_testing/src/_arch/aarch64/exception.rs @@ -12,7 +12,7 @@ //! crate::exception::arch_exception @@ -1022,9 +1022,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 13_integ } -diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs ---- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs -+++ 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs 12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -286,3 +286,31 @@ self.lvl2.phys_start_addr_u64() } @@ -1058,9 +1058,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translatio + } +} -diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs ---- 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs -+++ 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs +diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs +--- 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs ++++ 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -162,3 +162,22 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } @@ -1085,9 +1085,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 13_inte + } +} -diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 13_integrated_testing/src/bsp/raspberrypi/console.rs ---- 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs -+++ 13_integrated_testing/src/bsp/raspberrypi/console.rs +diff -uNr 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 12_integrated_testing/src/bsp/raspberrypi/console.rs +--- 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs ++++ 12_integrated_testing/src/bsp/raspberrypi/console.rs @@ -35,3 +35,13 @@ pub fn console() -> &'static impl console::interface::All { &super::PL011_UART @@ -1103,9 +1103,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 13_integ +/// For the RPi, nothing needs to be done. +pub fn qemu_bring_up_console() {} -diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs ---- 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs -+++ 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +diff -uNr 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 12_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs +--- 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs ++++ 12_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs @@ -69,3 +69,46 @@ pub fn virt_mem_layout() -> &'static KernelVirtualLayout { &LAYOUT @@ -1154,9 +1154,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 13_in + } +} -diff -uNr 12_exceptions_part1_groundwork/src/cpu.rs 13_integrated_testing/src/cpu.rs ---- 12_exceptions_part1_groundwork/src/cpu.rs -+++ 13_integrated_testing/src/cpu.rs +diff -uNr 11_exceptions_part1_groundwork/src/cpu.rs 12_integrated_testing/src/cpu.rs +--- 11_exceptions_part1_groundwork/src/cpu.rs ++++ 12_integrated_testing/src/cpu.rs @@ -14,3 +14,6 @@ // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- @@ -1165,9 +1165,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/cpu.rs 13_integrated_testing/src/cp +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; -diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/src/exception.rs ---- 12_exceptions_part1_groundwork/src/exception.rs -+++ 13_integrated_testing/src/exception.rs +diff -uNr 11_exceptions_part1_groundwork/src/exception.rs 12_integrated_testing/src/exception.rs +--- 11_exceptions_part1_groundwork/src/exception.rs ++++ 12_integrated_testing/src/exception.rs @@ -28,3 +28,21 @@ Hypervisor, Unknown, @@ -1191,9 +1191,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/exception.rs 13_integrated_testing/ + } +} -diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/lib.rs ---- 12_exceptions_part1_groundwork/src/lib.rs -+++ 13_integrated_testing/src/lib.rs +diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/lib.rs +--- 11_exceptions_part1_groundwork/src/lib.rs ++++ 12_integrated_testing/src/lib.rs @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1367,9 +1367,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/lib.rs 13_integrated_testing/src/li + cpu::qemu_exit_success() +} -diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/main.rs ---- 12_exceptions_part1_groundwork/src/main.rs -+++ 13_integrated_testing/src/main.rs +diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/main.rs +--- 11_exceptions_part1_groundwork/src/main.rs ++++ 12_integrated_testing/src/main.rs @@ -6,130 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] @@ -1554,9 +1554,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/main.rs 13_integrated_testing/src/m // Discard any spurious received characters before going into echo mode. -diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing/src/memory/mmu.rs ---- 12_exceptions_part1_groundwork/src/memory/mmu.rs -+++ 13_integrated_testing/src/memory/mmu.rs +diff -uNr 11_exceptions_part1_groundwork/src/memory/mmu.rs 12_integrated_testing/src/memory/mmu.rs +--- 11_exceptions_part1_groundwork/src/memory/mmu.rs ++++ 12_integrated_testing/src/memory/mmu.rs @@ -66,7 +66,6 @@ /// Architecture agnostic translation types. @@ -1576,9 +1576,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/memory/mmu.rs 13_integrated_testing + } } -diff -uNr 12_exceptions_part1_groundwork/src/memory.rs 13_integrated_testing/src/memory.rs ---- 12_exceptions_part1_groundwork/src/memory.rs -+++ 13_integrated_testing/src/memory.rs +diff -uNr 11_exceptions_part1_groundwork/src/memory.rs 12_integrated_testing/src/memory.rs +--- 11_exceptions_part1_groundwork/src/memory.rs ++++ 12_integrated_testing/src/memory.rs @@ -30,3 +30,40 @@ ptr = ptr.offset(1); } @@ -1621,9 +1621,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/memory.rs 13_integrated_testing/src + } +} -diff -uNr 12_exceptions_part1_groundwork/src/panic_wait.rs 13_integrated_testing/src/panic_wait.rs ---- 12_exceptions_part1_groundwork/src/panic_wait.rs -+++ 13_integrated_testing/src/panic_wait.rs +diff -uNr 11_exceptions_part1_groundwork/src/panic_wait.rs 12_integrated_testing/src/panic_wait.rs +--- 11_exceptions_part1_groundwork/src/panic_wait.rs ++++ 12_integrated_testing/src/panic_wait.rs @@ -17,6 +17,23 @@ unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } @@ -1656,9 +1656,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/panic_wait.rs 13_integrated_testing + _panic_exit() } -diff -uNr 12_exceptions_part1_groundwork/src/runtime_init.rs 13_integrated_testing/src/runtime_init.rs ---- 12_exceptions_part1_groundwork/src/runtime_init.rs -+++ 13_integrated_testing/src/runtime_init.rs +diff -uNr 11_exceptions_part1_groundwork/src/runtime_init.rs 12_integrated_testing/src/runtime_init.rs +--- 11_exceptions_part1_groundwork/src/runtime_init.rs ++++ 12_integrated_testing/src/runtime_init.rs @@ -31,7 +31,10 @@ /// /// - Only a single core must be active and running this function. @@ -1673,9 +1673,9 @@ diff -uNr 12_exceptions_part1_groundwork/src/runtime_init.rs 13_integrated_testi + kernel_init() } -diff -uNr 12_exceptions_part1_groundwork/test-macros/Cargo.toml 13_integrated_testing/test-macros/Cargo.toml ---- 12_exceptions_part1_groundwork/test-macros/Cargo.toml -+++ 13_integrated_testing/test-macros/Cargo.toml +diff -uNr 11_exceptions_part1_groundwork/test-macros/Cargo.toml 12_integrated_testing/test-macros/Cargo.toml +--- 11_exceptions_part1_groundwork/test-macros/Cargo.toml ++++ 12_integrated_testing/test-macros/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "test-macros" @@ -1692,9 +1692,9 @@ diff -uNr 12_exceptions_part1_groundwork/test-macros/Cargo.toml 13_integrated_te +syn = { version = "1.x", features = ["full"] } +test-types = { path = "../test-types" } -diff -uNr 12_exceptions_part1_groundwork/test-macros/src/lib.rs 13_integrated_testing/test-macros/src/lib.rs ---- 12_exceptions_part1_groundwork/test-macros/src/lib.rs -+++ 13_integrated_testing/test-macros/src/lib.rs +diff -uNr 11_exceptions_part1_groundwork/test-macros/src/lib.rs 12_integrated_testing/test-macros/src/lib.rs +--- 11_exceptions_part1_groundwork/test-macros/src/lib.rs ++++ 12_integrated_testing/test-macros/src/lib.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1726,9 +1726,9 @@ diff -uNr 12_exceptions_part1_groundwork/test-macros/src/lib.rs 13_integrated_te + .into() +} -diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rb 13_integrated_testing/tests/00_console_sanity.rb ---- 12_exceptions_part1_groundwork/tests/00_console_sanity.rb -+++ 13_integrated_testing/tests/00_console_sanity.rb +diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrated_testing/tests/00_console_sanity.rb +--- 11_exceptions_part1_groundwork/tests/00_console_sanity.rb ++++ 12_integrated_testing/tests/00_console_sanity.rb @@ -0,0 +1,50 @@ +# frozen_string_literal: true + @@ -1781,9 +1781,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rb 13_integrate + [TxRxHandshake.new, TxStatistics.new, RxStatistics.new] +end -diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrated_testing/tests/00_console_sanity.rs ---- 12_exceptions_part1_groundwork/tests/00_console_sanity.rs -+++ 13_integrated_testing/tests/00_console_sanity.rs +diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrated_testing/tests/00_console_sanity.rs +--- 11_exceptions_part1_groundwork/tests/00_console_sanity.rs ++++ 12_integrated_testing/tests/00_console_sanity.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1821,9 +1821,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/00_console_sanity.rs 13_integrate + cpu::wait_forever() +} -diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_testing/tests/01_timer_sanity.rs ---- 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs -+++ 13_integrated_testing/tests/01_timer_sanity.rs +diff -uNr 11_exceptions_part1_groundwork/tests/01_timer_sanity.rs 12_integrated_testing/tests/01_timer_sanity.rs +--- 11_exceptions_part1_groundwork/tests/01_timer_sanity.rs ++++ 12_integrated_testing/tests/01_timer_sanity.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1875,9 +1875,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/01_timer_sanity.rs 13_integrated_ + assert_eq!((t2 - t1).as_secs(), 1) +} -diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 13_integrated_testing/tests/02_exception_sync_page_fault.rs ---- 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs -+++ 13_integrated_testing/tests/02_exception_sync_page_fault.rs +diff -uNr 11_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 12_integrated_testing/tests/02_exception_sync_page_fault.rs +--- 11_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs ++++ 12_integrated_testing/tests/02_exception_sync_page_fault.rs @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1923,9 +1923,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 + cpu::qemu_exit_failure() +} -diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 13_integrated_testing/tests/panic_exit_success/mod.rs ---- 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs -+++ 13_integrated_testing/tests/panic_exit_success/mod.rs +diff -uNr 11_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 12_integrated_testing/tests/panic_exit_success/mod.rs +--- 11_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs ++++ 12_integrated_testing/tests/panic_exit_success/mod.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1937,9 +1937,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 13_inte + libkernel::cpu::qemu_exit_success() +} -diff -uNr 12_exceptions_part1_groundwork/tests/runner.rb 13_integrated_testing/tests/runner.rb ---- 12_exceptions_part1_groundwork/tests/runner.rb -+++ 13_integrated_testing/tests/runner.rb +diff -uNr 11_exceptions_part1_groundwork/tests/runner.rb 12_integrated_testing/tests/runner.rb +--- 11_exceptions_part1_groundwork/tests/runner.rb ++++ 12_integrated_testing/tests/runner.rb @@ -0,0 +1,143 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true @@ -2085,9 +2085,9 @@ diff -uNr 12_exceptions_part1_groundwork/tests/runner.rb 13_integrated_testing/t + +test_runner.exec -diff -uNr 12_exceptions_part1_groundwork/test-types/Cargo.toml 13_integrated_testing/test-types/Cargo.toml ---- 12_exceptions_part1_groundwork/test-types/Cargo.toml -+++ 13_integrated_testing/test-types/Cargo.toml +diff -uNr 11_exceptions_part1_groundwork/test-types/Cargo.toml 12_integrated_testing/test-types/Cargo.toml +--- 11_exceptions_part1_groundwork/test-types/Cargo.toml ++++ 12_integrated_testing/test-types/Cargo.toml @@ -0,0 +1,5 @@ +[package] +name = "test-types" @@ -2095,9 +2095,9 @@ diff -uNr 12_exceptions_part1_groundwork/test-types/Cargo.toml 13_integrated_tes +authors = ["Andre Richter "] +edition = "2018" -diff -uNr 12_exceptions_part1_groundwork/test-types/src/lib.rs 13_integrated_testing/test-types/src/lib.rs ---- 12_exceptions_part1_groundwork/test-types/src/lib.rs -+++ 13_integrated_testing/test-types/src/lib.rs +diff -uNr 11_exceptions_part1_groundwork/test-types/src/lib.rs 12_integrated_testing/test-types/src/lib.rs +--- 11_exceptions_part1_groundwork/test-types/src/lib.rs ++++ 12_integrated_testing/test-types/src/lib.rs @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// diff --git a/13_integrated_testing/build.rs b/12_integrated_testing/build.rs similarity index 100% rename from 13_integrated_testing/build.rs rename to 12_integrated_testing/build.rs diff --git a/13_integrated_testing/src/_arch/aarch64/cpu.rs b/12_integrated_testing/src/_arch/aarch64/cpu.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/cpu.rs rename to 12_integrated_testing/src/_arch/aarch64/cpu.rs diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/boot.rs b/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/cpu/boot.rs rename to 12_integrated_testing/src/_arch/aarch64/cpu/boot.rs diff --git a/13_integrated_testing/src/_arch/aarch64/cpu/boot.s b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/cpu/boot.s rename to 12_integrated_testing/src/_arch/aarch64/cpu/boot.s diff --git a/13_integrated_testing/src/_arch/aarch64/exception.rs b/12_integrated_testing/src/_arch/aarch64/exception.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/exception.rs rename to 12_integrated_testing/src/_arch/aarch64/exception.rs diff --git a/13_integrated_testing/src/_arch/aarch64/exception.s b/12_integrated_testing/src/_arch/aarch64/exception.s similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/exception.s rename to 12_integrated_testing/src/_arch/aarch64/exception.s diff --git a/13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs b/12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs rename to 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/memory/mmu.rs rename to 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs diff --git a/13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs rename to 12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs diff --git a/13_integrated_testing/src/_arch/aarch64/time.rs b/12_integrated_testing/src/_arch/aarch64/time.rs similarity index 100% rename from 13_integrated_testing/src/_arch/aarch64/time.rs rename to 12_integrated_testing/src/_arch/aarch64/time.rs diff --git a/13_integrated_testing/src/bsp.rs b/12_integrated_testing/src/bsp.rs similarity index 100% rename from 13_integrated_testing/src/bsp.rs rename to 12_integrated_testing/src/bsp.rs diff --git a/13_integrated_testing/src/bsp/device_driver.rs b/12_integrated_testing/src/bsp/device_driver.rs similarity index 100% rename from 13_integrated_testing/src/bsp/device_driver.rs rename to 12_integrated_testing/src/bsp/device_driver.rs diff --git a/13_integrated_testing/src/bsp/device_driver/bcm.rs b/12_integrated_testing/src/bsp/device_driver/bcm.rs similarity index 100% rename from 13_integrated_testing/src/bsp/device_driver/bcm.rs rename to 12_integrated_testing/src/bsp/device_driver/bcm.rs diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/13_integrated_testing/src/bsp/device_driver/common.rs b/12_integrated_testing/src/bsp/device_driver/common.rs similarity index 100% rename from 13_integrated_testing/src/bsp/device_driver/common.rs rename to 12_integrated_testing/src/bsp/device_driver/common.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi.rs b/12_integrated_testing/src/bsp/raspberrypi.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi.rs rename to 12_integrated_testing/src/bsp/raspberrypi.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi/console.rs b/12_integrated_testing/src/bsp/raspberrypi/console.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/console.rs rename to 12_integrated_testing/src/bsp/raspberrypi/console.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi/cpu.rs b/12_integrated_testing/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/cpu.rs rename to 12_integrated_testing/src/bsp/raspberrypi/cpu.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi/driver.rs b/12_integrated_testing/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/driver.rs rename to 12_integrated_testing/src/bsp/raspberrypi/driver.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi/link.ld b/12_integrated_testing/src/bsp/raspberrypi/link.ld similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/link.ld rename to 12_integrated_testing/src/bsp/raspberrypi/link.ld diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory.rs b/12_integrated_testing/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/memory.rs rename to 12_integrated_testing/src/bsp/raspberrypi/memory.rs diff --git a/13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs b/12_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs similarity index 100% rename from 13_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs rename to 12_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs diff --git a/13_integrated_testing/src/console.rs b/12_integrated_testing/src/console.rs similarity index 100% rename from 13_integrated_testing/src/console.rs rename to 12_integrated_testing/src/console.rs diff --git a/13_integrated_testing/src/cpu.rs b/12_integrated_testing/src/cpu.rs similarity index 100% rename from 13_integrated_testing/src/cpu.rs rename to 12_integrated_testing/src/cpu.rs diff --git a/13_integrated_testing/src/cpu/boot.rs b/12_integrated_testing/src/cpu/boot.rs similarity index 100% rename from 13_integrated_testing/src/cpu/boot.rs rename to 12_integrated_testing/src/cpu/boot.rs diff --git a/13_integrated_testing/src/driver.rs b/12_integrated_testing/src/driver.rs similarity index 100% rename from 13_integrated_testing/src/driver.rs rename to 12_integrated_testing/src/driver.rs diff --git a/13_integrated_testing/src/exception.rs b/12_integrated_testing/src/exception.rs similarity index 100% rename from 13_integrated_testing/src/exception.rs rename to 12_integrated_testing/src/exception.rs diff --git a/13_integrated_testing/src/exception/asynchronous.rs b/12_integrated_testing/src/exception/asynchronous.rs similarity index 100% rename from 13_integrated_testing/src/exception/asynchronous.rs rename to 12_integrated_testing/src/exception/asynchronous.rs diff --git a/13_integrated_testing/src/lib.rs b/12_integrated_testing/src/lib.rs similarity index 100% rename from 13_integrated_testing/src/lib.rs rename to 12_integrated_testing/src/lib.rs diff --git a/13_integrated_testing/src/main.rs b/12_integrated_testing/src/main.rs similarity index 100% rename from 13_integrated_testing/src/main.rs rename to 12_integrated_testing/src/main.rs diff --git a/13_integrated_testing/src/memory.rs b/12_integrated_testing/src/memory.rs similarity index 100% rename from 13_integrated_testing/src/memory.rs rename to 12_integrated_testing/src/memory.rs diff --git a/13_integrated_testing/src/memory/mmu.rs b/12_integrated_testing/src/memory/mmu.rs similarity index 100% rename from 13_integrated_testing/src/memory/mmu.rs rename to 12_integrated_testing/src/memory/mmu.rs diff --git a/13_integrated_testing/src/memory/mmu/translation_table.rs b/12_integrated_testing/src/memory/mmu/translation_table.rs similarity index 100% rename from 13_integrated_testing/src/memory/mmu/translation_table.rs rename to 12_integrated_testing/src/memory/mmu/translation_table.rs diff --git a/13_integrated_testing/src/panic_wait.rs b/12_integrated_testing/src/panic_wait.rs similarity index 100% rename from 13_integrated_testing/src/panic_wait.rs rename to 12_integrated_testing/src/panic_wait.rs diff --git a/13_integrated_testing/src/print.rs b/12_integrated_testing/src/print.rs similarity index 100% rename from 13_integrated_testing/src/print.rs rename to 12_integrated_testing/src/print.rs diff --git a/13_integrated_testing/src/runtime_init.rs b/12_integrated_testing/src/runtime_init.rs similarity index 100% rename from 13_integrated_testing/src/runtime_init.rs rename to 12_integrated_testing/src/runtime_init.rs diff --git a/13_integrated_testing/src/synchronization.rs b/12_integrated_testing/src/synchronization.rs similarity index 100% rename from 13_integrated_testing/src/synchronization.rs rename to 12_integrated_testing/src/synchronization.rs diff --git a/13_integrated_testing/src/time.rs b/12_integrated_testing/src/time.rs similarity index 100% rename from 13_integrated_testing/src/time.rs rename to 12_integrated_testing/src/time.rs diff --git a/13_integrated_testing/test-macros/Cargo.toml b/12_integrated_testing/test-macros/Cargo.toml similarity index 100% rename from 13_integrated_testing/test-macros/Cargo.toml rename to 12_integrated_testing/test-macros/Cargo.toml diff --git a/13_integrated_testing/test-macros/src/lib.rs b/12_integrated_testing/test-macros/src/lib.rs similarity index 100% rename from 13_integrated_testing/test-macros/src/lib.rs rename to 12_integrated_testing/test-macros/src/lib.rs diff --git a/13_integrated_testing/test-types/Cargo.toml b/12_integrated_testing/test-types/Cargo.toml similarity index 100% rename from 13_integrated_testing/test-types/Cargo.toml rename to 12_integrated_testing/test-types/Cargo.toml diff --git a/13_integrated_testing/test-types/src/lib.rs b/12_integrated_testing/test-types/src/lib.rs similarity index 100% rename from 13_integrated_testing/test-types/src/lib.rs rename to 12_integrated_testing/test-types/src/lib.rs diff --git a/13_integrated_testing/tests/00_console_sanity.rb b/12_integrated_testing/tests/00_console_sanity.rb similarity index 100% rename from 13_integrated_testing/tests/00_console_sanity.rb rename to 12_integrated_testing/tests/00_console_sanity.rb diff --git a/13_integrated_testing/tests/00_console_sanity.rs b/12_integrated_testing/tests/00_console_sanity.rs similarity index 100% rename from 13_integrated_testing/tests/00_console_sanity.rs rename to 12_integrated_testing/tests/00_console_sanity.rs diff --git a/13_integrated_testing/tests/01_timer_sanity.rs b/12_integrated_testing/tests/01_timer_sanity.rs similarity index 100% rename from 13_integrated_testing/tests/01_timer_sanity.rs rename to 12_integrated_testing/tests/01_timer_sanity.rs diff --git a/13_integrated_testing/tests/02_exception_sync_page_fault.rs b/12_integrated_testing/tests/02_exception_sync_page_fault.rs similarity index 100% rename from 13_integrated_testing/tests/02_exception_sync_page_fault.rs rename to 12_integrated_testing/tests/02_exception_sync_page_fault.rs diff --git a/13_integrated_testing/tests/panic_exit_success/mod.rs b/12_integrated_testing/tests/panic_exit_success/mod.rs similarity index 100% rename from 13_integrated_testing/tests/panic_exit_success/mod.rs rename to 12_integrated_testing/tests/panic_exit_success/mod.rs diff --git a/13_integrated_testing/tests/runner.rb b/12_integrated_testing/tests/runner.rb similarity index 100% rename from 13_integrated_testing/tests/runner.rb rename to 12_integrated_testing/tests/runner.rb diff --git a/14_exceptions_part2_peripheral_IRQs/.cargo/config.toml b/13_exceptions_part2_peripheral_IRQs/.cargo/config.toml similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/.cargo/config.toml rename to 13_exceptions_part2_peripheral_IRQs/.cargo/config.toml diff --git a/14_exceptions_part2_peripheral_IRQs/.vscode/settings.json b/13_exceptions_part2_peripheral_IRQs/.vscode/settings.json similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/.vscode/settings.json rename to 13_exceptions_part2_peripheral_IRQs/.vscode/settings.json diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.lock b/13_exceptions_part2_peripheral_IRQs/Cargo.lock similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/Cargo.lock rename to 13_exceptions_part2_peripheral_IRQs/Cargo.lock diff --git a/14_exceptions_part2_peripheral_IRQs/Cargo.toml b/13_exceptions_part2_peripheral_IRQs/Cargo.toml similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/Cargo.toml rename to 13_exceptions_part2_peripheral_IRQs/Cargo.toml diff --git a/14_exceptions_part2_peripheral_IRQs/Makefile b/13_exceptions_part2_peripheral_IRQs/Makefile similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/Makefile rename to 13_exceptions_part2_peripheral_IRQs/Makefile diff --git a/14_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md similarity index 94% rename from 14_exceptions_part2_peripheral_IRQs/README.md rename to 13_exceptions_part2_peripheral_IRQs/README.md index b78da7162..ea327b77e 100644 --- a/14_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -1,4 +1,4 @@ -# Tutorial 14 - Exceptions Part 2: Peripheral IRQs +# Tutorial 13 - Exceptions Part 2: Peripheral IRQs ## tl;dr @@ -35,11 +35,11 @@ ## Introduction -In [tutorial 12], we laid the groundwork for exception handling from the processor architecture +In [tutorial 11], we laid the groundwork for exception handling from the processor architecture side. Handler stubs for the different exception types were set up, and a first glimpse at exception handling was presented by causing a `synchronous` exception by means of a `page fault`. -[tutorial 12]: ../12_exceptions_part1_groundwork +[tutorial 11]: ../11_exceptions_part1_groundwork In this tutorial, we will add a first level of support for one of the three types of `asynchronous` exceptions that are defined for `AArch64`: `IRQs`. The overall goal for this tutorial is to get rid @@ -350,13 +350,13 @@ fn handle_pending_irqs<'irq_context>( An important aspect of this function signature is that we want to ensure that IRQ handling is only possible from IRQ context. Part of the reason is that this invariant allows us to make some implicit assumptions (which might depend on the target architecture, though). For example, as we have learned -in [tutorial 12], in `AArch64`, _"all kinds of exceptions are turned off upon taking an exception, +in [tutorial 11], in `AArch64`, _"all kinds of exceptions are turned off upon taking an exception, so that by default, exception handlers can not get interrupted themselves"_ (note that an IRQ is an exception). This is a useful property that relieves us from explicitly protecting IRQ handling from being interrupted itself. Another reason would be that calling IRQ handling functions from arbitrary execution contexts just doesn't make a lot of sense. -[tutorial 12]: ../12_exceptions_part1_groundwork/ +[tutorial 11]: ../11_exceptions_part1_groundwork/ So in order to ensure that this function is only being called from IRQ context, we borrow a technique that I first saw in the [Rust embedded WG]'s [bare-metal crate]. It uses Rust's type @@ -744,9 +744,9 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs ---- 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +--- 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -778,9 +778,9 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/cpu/smp.rs 14_exceptions_part2 + T::from((MPIDR_EL1.get() & CORE_MASK) as u8) +} -diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs ---- 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs +diff -uNr 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs +--- 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs @@ -17,6 +17,10 @@ // Private Definitions //-------------------------------------------------------------------------------------------------- @@ -865,9 +865,9 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 14_e #[rustfmt::skip] pub fn print_state() { -diff -uNr 13_integrated_testing/src/_arch/aarch64/exception.rs 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs ---- 13_integrated_testing/src/_arch/aarch64/exception.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +diff -uNr 12_integrated_testing/src/_arch/aarch64/exception.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +--- 12_integrated_testing/src/_arch/aarch64/exception.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -11,6 +11,7 @@ //! //! crate::exception::arch_exception @@ -891,9 +891,9 @@ diff -uNr 13_integrated_testing/src/_arch/aarch64/exception.rs 14_exceptions_par #[no_mangle] -diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs ---- 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs +--- 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1033,9 +1033,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 14_excep + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs ---- 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs +--- 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1233,9 +1233,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 14_excep + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs ---- 13_integrated_testing/src/bsp/device_driver/arm/gicv2.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs +--- 12_integrated_testing/src/bsp/device_driver/arm/gicv2.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1457,9 +1457,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm/gicv2.rs 14_exceptions + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/arm.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs ---- 13_integrated_testing/src/bsp/device_driver/arm.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/arm.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs +--- 12_integrated_testing/src/bsp/device_driver/arm.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1471,9 +1471,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/arm.rs 14_exceptions_part2 + +pub use gicv2::*; -diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ---- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -6,7 +6,7 @@ use crate::{ @@ -1502,9 +1502,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 14_exc } -diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs ---- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +--- 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1670,9 +1670,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs ---- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +--- 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -1806,9 +1806,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ---- 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +--- 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,8 +10,8 @@ //! - @@ -1974,9 +1974,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs + } +} -diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs ---- 13_integrated_testing/src/bsp/device_driver/bcm.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs +--- 12_integrated_testing/src/bsp/device_driver/bcm.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs @@ -5,7 +5,11 @@ //! BCM driver top level. @@ -1990,9 +1990,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver/bcm.rs 14_exceptions_part2 +pub use bcm2xxx_interrupt_controller::*; pub use bcm2xxx_pl011_uart::*; -diff -uNr 13_integrated_testing/src/bsp/device_driver.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs ---- 13_integrated_testing/src/bsp/device_driver.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs +diff -uNr 12_integrated_testing/src/bsp/device_driver.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs +--- 12_integrated_testing/src/bsp/device_driver.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs @@ -4,9 +4,13 @@ //! Device driver. @@ -2008,9 +2008,9 @@ diff -uNr 13_integrated_testing/src/bsp/device_driver.rs 14_exceptions_part2_per #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] pub use bcm::*; -diff -uNr 13_integrated_testing/src/bsp/raspberrypi/driver.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs ---- 13_integrated_testing/src/bsp/raspberrypi/driver.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs +diff -uNr 12_integrated_testing/src/bsp/raspberrypi/driver.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs +--- 12_integrated_testing/src/bsp/raspberrypi/driver.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs @@ -12,7 +12,7 @@ /// Device Driver Manager type. @@ -2034,9 +2034,9 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/driver.rs 14_exceptions_part //-------------------------------------------------------------------------------------------------- -diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs ---- 13_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs +diff -uNr 12_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs +--- 12_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2075,9 +2075,9 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception/asynchronous.rs 14 + &super::super::INTERRUPT_CONTROLLER +} -diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs ---- 13_integrated_testing/src/bsp/raspberrypi/exception.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs +diff -uNr 12_integrated_testing/src/bsp/raspberrypi/exception.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs +--- 12_integrated_testing/src/bsp/raspberrypi/exception.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2087,9 +2087,9 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/exception.rs 14_exceptions_p + +pub mod asynchronous; -diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs ---- 13_integrated_testing/src/bsp/raspberrypi/memory.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +diff -uNr 12_integrated_testing/src/bsp/raspberrypi/memory.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +--- 12_integrated_testing/src/bsp/raspberrypi/memory.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -51,10 +51,12 @@ pub mod mmio { use super::*; @@ -2117,9 +2117,9 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi/memory.rs 14_exceptions_part } } -diff -uNr 13_integrated_testing/src/bsp/raspberrypi.rs 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs ---- 13_integrated_testing/src/bsp/raspberrypi.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs +diff -uNr 12_integrated_testing/src/bsp/raspberrypi.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs +--- 12_integrated_testing/src/bsp/raspberrypi.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs @@ -7,6 +7,7 @@ pub mod console; pub mod cpu; @@ -2157,9 +2157,9 @@ diff -uNr 13_integrated_testing/src/bsp/raspberrypi.rs 14_exceptions_part2_perip //-------------------------------------------------------------------------------------------------- // Public Code -diff -uNr 13_integrated_testing/src/cpu/smp.rs 14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs ---- 13_integrated_testing/src/cpu/smp.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +diff -uNr 12_integrated_testing/src/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs +--- 12_integrated_testing/src/cpu/smp.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2176,9 +2176,9 @@ diff -uNr 13_integrated_testing/src/cpu/smp.rs 14_exceptions_part2_peripheral_IR +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; -diff -uNr 13_integrated_testing/src/cpu.rs 14_exceptions_part2_peripheral_IRQs/src/cpu.rs ---- 13_integrated_testing/src/cpu.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/cpu.rs +diff -uNr 12_integrated_testing/src/cpu.rs 13_exceptions_part2_peripheral_IRQs/src/cpu.rs +--- 12_integrated_testing/src/cpu.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/cpu.rs @@ -10,6 +10,8 @@ mod boot; @@ -2189,9 +2189,9 @@ diff -uNr 13_integrated_testing/src/cpu.rs 14_exceptions_part2_peripheral_IRQs/s // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- -diff -uNr 13_integrated_testing/src/driver.rs 14_exceptions_part2_peripheral_IRQs/src/driver.rs ---- 13_integrated_testing/src/driver.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/driver.rs +diff -uNr 12_integrated_testing/src/driver.rs 13_exceptions_part2_peripheral_IRQs/src/driver.rs +--- 12_integrated_testing/src/driver.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/driver.rs @@ -23,6 +23,14 @@ unsafe fn init(&self) -> Result<(), &'static str> { Ok(()) @@ -2208,9 +2208,9 @@ diff -uNr 13_integrated_testing/src/driver.rs 14_exceptions_part2_peripheral_IRQ /// Device driver management functions. -diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs ---- 13_integrated_testing/src/exception/asynchronous.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs +diff -uNr 12_integrated_testing/src/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs +--- 12_integrated_testing/src/exception/asynchronous.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs @@ -8,7 +8,145 @@ #[path = "../_arch/aarch64/exception/asynchronous.rs"] mod arch_asynchronous; @@ -2359,9 +2359,9 @@ diff -uNr 13_integrated_testing/src/exception/asynchronous.rs 14_exceptions_part + ret +} -diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/src/lib.rs ---- 13_integrated_testing/src/lib.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/lib.rs +diff -uNr 12_integrated_testing/src/lib.rs 13_exceptions_part2_peripheral_IRQs/src/lib.rs +--- 12_integrated_testing/src/lib.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -110,6 +110,7 @@ #![allow(clippy::clippy::upper_case_acronyms)] @@ -2379,9 +2379,9 @@ diff -uNr 13_integrated_testing/src/lib.rs 14_exceptions_part2_peripheral_IRQs/s //-------------------------------------------------------------------------------------------------- -diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/src/main.rs ---- 13_integrated_testing/src/main.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/main.rs +diff -uNr 12_integrated_testing/src/main.rs 13_exceptions_part2_peripheral_IRQs/src/main.rs +--- 12_integrated_testing/src/main.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/main.rs @@ -11,7 +11,7 @@ #![no_main] #![no_std] @@ -2448,9 +2448,9 @@ diff -uNr 13_integrated_testing/src/main.rs 14_exceptions_part2_peripheral_IRQs/ + cpu::wait_forever(); } -diff -uNr 13_integrated_testing/src/panic_wait.rs 14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs ---- 13_integrated_testing/src/panic_wait.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +diff -uNr 12_integrated_testing/src/panic_wait.rs 13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +--- 12_integrated_testing/src/panic_wait.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs @@ -4,7 +4,7 @@ //! A panic handler that infinitely waits. @@ -2470,9 +2470,9 @@ diff -uNr 13_integrated_testing/src/panic_wait.rs 14_exceptions_part2_peripheral panic_println!("\nKernel panic: {}", args); } else { -diff -uNr 13_integrated_testing/src/state.rs 14_exceptions_part2_peripheral_IRQs/src/state.rs ---- 13_integrated_testing/src/state.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/state.rs +diff -uNr 12_integrated_testing/src/state.rs 13_exceptions_part2_peripheral_IRQs/src/state.rs +--- 12_integrated_testing/src/state.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/state.rs @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2567,9 +2567,9 @@ diff -uNr 13_integrated_testing/src/state.rs 14_exceptions_part2_peripheral_IRQs + } +} -diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_peripheral_IRQs/src/synchronization.rs ---- 13_integrated_testing/src/synchronization.rs -+++ 14_exceptions_part2_peripheral_IRQs/src/synchronization.rs +diff -uNr 12_integrated_testing/src/synchronization.rs 13_exceptions_part2_peripheral_IRQs/src/synchronization.rs +--- 12_integrated_testing/src/synchronization.rs ++++ 13_exceptions_part2_peripheral_IRQs/src/synchronization.rs @@ -28,6 +28,21 @@ /// Locks the mutex and grants the closure temporary mutable access to the wrapped data. fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; @@ -2702,9 +2702,9 @@ diff -uNr 13_integrated_testing/src/synchronization.rs 14_exceptions_part2_perip + } } -diff -uNr 13_integrated_testing/tests/03_exception_irq_sanity.rs 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs ---- 13_integrated_testing/tests/03_exception_irq_sanity.rs -+++ 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs +diff -uNr 12_integrated_testing/tests/03_exception_irq_sanity.rs 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs +--- 12_integrated_testing/tests/03_exception_irq_sanity.rs ++++ 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// diff --git a/14_exceptions_part2_peripheral_IRQs/build.rs b/13_exceptions_part2_peripheral_IRQs/build.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/build.rs rename to 13_exceptions_part2_peripheral_IRQs/build.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.s diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs rename to 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/common.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/cpu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/exception/asynchronous.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/console.rs b/13_exceptions_part2_peripheral_IRQs/src/console.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/console.rs rename to 13_exceptions_part2_peripheral_IRQs/src/console.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu.rs b/13_exceptions_part2_peripheral_IRQs/src/cpu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/cpu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/cpu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs b/13_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs rename to 13_exceptions_part2_peripheral_IRQs/src/cpu/boot.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs b/13_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs rename to 13_exceptions_part2_peripheral_IRQs/src/cpu/smp.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/driver.rs b/13_exceptions_part2_peripheral_IRQs/src/driver.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/driver.rs rename to 13_exceptions_part2_peripheral_IRQs/src/driver.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception.rs b/13_exceptions_part2_peripheral_IRQs/src/exception.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/exception.rs rename to 13_exceptions_part2_peripheral_IRQs/src/exception.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs rename to 13_exceptions_part2_peripheral_IRQs/src/exception/asynchronous.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/lib.rs rename to 13_exceptions_part2_peripheral_IRQs/src/lib.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/main.rs b/13_exceptions_part2_peripheral_IRQs/src/main.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/main.rs rename to 13_exceptions_part2_peripheral_IRQs/src/main.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory.rs b/13_exceptions_part2_peripheral_IRQs/src/memory.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/memory.rs rename to 13_exceptions_part2_peripheral_IRQs/src/memory.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs b/13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs rename to 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs b/13_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs rename to 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs b/13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/panic_wait.rs rename to 13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/print.rs b/13_exceptions_part2_peripheral_IRQs/src/print.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/print.rs rename to 13_exceptions_part2_peripheral_IRQs/src/print.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs b/13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/runtime_init.rs rename to 13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/state.rs b/13_exceptions_part2_peripheral_IRQs/src/state.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/state.rs rename to 13_exceptions_part2_peripheral_IRQs/src/state.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/synchronization.rs b/13_exceptions_part2_peripheral_IRQs/src/synchronization.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/synchronization.rs rename to 13_exceptions_part2_peripheral_IRQs/src/synchronization.rs diff --git a/14_exceptions_part2_peripheral_IRQs/src/time.rs b/13_exceptions_part2_peripheral_IRQs/src/time.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/src/time.rs rename to 13_exceptions_part2_peripheral_IRQs/src/time.rs diff --git a/14_exceptions_part2_peripheral_IRQs/test-macros/Cargo.toml b/13_exceptions_part2_peripheral_IRQs/test-macros/Cargo.toml similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/test-macros/Cargo.toml rename to 13_exceptions_part2_peripheral_IRQs/test-macros/Cargo.toml diff --git a/14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs rename to 13_exceptions_part2_peripheral_IRQs/test-macros/src/lib.rs diff --git a/14_exceptions_part2_peripheral_IRQs/test-types/Cargo.toml b/13_exceptions_part2_peripheral_IRQs/test-types/Cargo.toml similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/test-types/Cargo.toml rename to 13_exceptions_part2_peripheral_IRQs/test-types/Cargo.toml diff --git a/14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs rename to 13_exceptions_part2_peripheral_IRQs/test-types/src/lib.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb rename to 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb diff --git a/14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs rename to 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs b/13_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs rename to 13_exceptions_part2_peripheral_IRQs/tests/01_timer_sanity.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs b/13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs rename to 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs b/13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs rename to 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs b/13_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs rename to 13_exceptions_part2_peripheral_IRQs/tests/panic_exit_success/mod.rs diff --git a/14_exceptions_part2_peripheral_IRQs/tests/runner.rb b/13_exceptions_part2_peripheral_IRQs/tests/runner.rb similarity index 100% rename from 14_exceptions_part2_peripheral_IRQs/tests/runner.rb rename to 13_exceptions_part2_peripheral_IRQs/tests/runner.rb diff --git a/15_virtual_mem_part2_mmio_remap/.cargo/config.toml b/14_virtual_mem_part2_mmio_remap/.cargo/config.toml similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/.cargo/config.toml rename to 14_virtual_mem_part2_mmio_remap/.cargo/config.toml diff --git a/15_virtual_mem_part2_mmio_remap/.vscode/settings.json b/14_virtual_mem_part2_mmio_remap/.vscode/settings.json similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/.vscode/settings.json rename to 14_virtual_mem_part2_mmio_remap/.vscode/settings.json diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.lock b/14_virtual_mem_part2_mmio_remap/Cargo.lock similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/Cargo.lock rename to 14_virtual_mem_part2_mmio_remap/Cargo.lock diff --git a/15_virtual_mem_part2_mmio_remap/Cargo.toml b/14_virtual_mem_part2_mmio_remap/Cargo.toml similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/Cargo.toml rename to 14_virtual_mem_part2_mmio_remap/Cargo.toml diff --git a/15_virtual_mem_part2_mmio_remap/Makefile b/14_virtual_mem_part2_mmio_remap/Makefile similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/Makefile rename to 14_virtual_mem_part2_mmio_remap/Makefile diff --git a/15_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md similarity index 95% rename from 15_virtual_mem_part2_mmio_remap/README.md rename to 14_virtual_mem_part2_mmio_remap/README.md index 1745d4b14..cf387da92 100644 --- a/15_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -1,4 +1,4 @@ -# Tutorial 15 - Virtual Memory Part 2: MMIO Remap +# Tutorial 14 - Virtual Memory Part 2: MMIO Remap ## tl;dr @@ -369,9 +369,9 @@ Minipush 1.0 ## Diff to previous ```diff -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +--- 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs ++++ 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -11,7 +11,11 @@ //! //! crate::exception::arch_exception @@ -418,9 +418,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 15_ } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -15,9 +15,12 @@ use crate::{ @@ -730,9 +730,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans use super::*; use test_macros::kernel_test; -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs ---- 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs -+++ 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +--- 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs ++++ 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -15,7 +15,7 @@ use crate::{ @@ -816,9 +816,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 15 - } -} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -4,7 +4,9 @@ //! GICC Driver - GIC CPU interface. @@ -905,9 +905,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -8,8 +8,9 @@ //! - SPI - Shared Peripheral Interrupt. @@ -981,9 +981,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi _ => { let enable_reg_index_shared = enable_reg_index - 1; -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs @@ -79,7 +79,8 @@ mod gicc; mod gicd; @@ -1059,9 +1059,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2.rs self.gicd.boot_core_init(); } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -5,9 +5,10 @@ //! GPIO Driver. @@ -1147,9 +1147,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + } } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -2,12 +2,12 @@ // // Copyright (c) 2020-2021 Andre Richter @@ -1240,9 +1240,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ type IRQNumberType = PeripheralIRQ; -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -6,7 +6,7 @@ mod peripheral_ic; @@ -1281,9 +1281,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ impl exception::asynchronous::interface::IRQManager for InterruptController { -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,10 +10,13 @@ //! - @@ -1390,9 +1390,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ impl console::interface::Write for PL011Uart { -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs @@ -5,7 +5,7 @@ //! BSP console facilities. @@ -1432,9 +1432,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs 15_ } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs @@ -46,7 +46,15 @@ &self.device_drivers[..] } @@ -1453,9 +1453,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/driver.rs 15_v super::GPIO.map_pl011_uart(); } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -17,11 +17,6 @@ SECTIONS { @@ -1501,9 +1501,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 15_vir + __boot_core_stack_end_exclusive = .; /* | */ } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -4,70 +4,164 @@ //! BSP Memory Management Unit. @@ -1779,9 +1779,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + } } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -3,9 +3,40 @@ // Copyright (c) 2018-2021 Andre Richter @@ -1995,9 +1995,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 15_v //-------------------------------------------------------------------------------------------------- -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs ---- 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs -+++ 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs +--- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs ++++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs @@ -10,17 +10,20 @@ pub mod exception; pub mod memory; @@ -2044,9 +2044,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs 15_virtual_ //-------------------------------------------------------------------------------------------------- -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/common.rs 15_virtual_mem_part2_mmio_remap/src/common.rs ---- 14_exceptions_part2_peripheral_IRQs/src/common.rs -+++ 15_virtual_mem_part2_mmio_remap/src/common.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/common.rs 14_virtual_mem_part2_mmio_remap/src/common.rs +--- 13_exceptions_part2_peripheral_IRQs/src/common.rs ++++ 14_virtual_mem_part2_mmio_remap/src/common.rs @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2070,9 +2070,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/common.rs 15_virtual_mem_part2 + value & !(alignment - 1) +} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2_mmio_remap/src/driver.rs ---- 14_exceptions_part2_peripheral_IRQs/src/driver.rs -+++ 15_virtual_mem_part2_mmio_remap/src/driver.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/driver.rs 14_virtual_mem_part2_mmio_remap/src/driver.rs +--- 13_exceptions_part2_peripheral_IRQs/src/driver.rs ++++ 14_virtual_mem_part2_mmio_remap/src/driver.rs @@ -31,6 +31,14 @@ fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { Ok(()) @@ -2114,9 +2114,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/driver.rs 15_virtual_mem_part2 } } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mmio_remap/src/lib.rs ---- 14_exceptions_part2_peripheral_IRQs/src/lib.rs -+++ 15_virtual_mem_part2_mmio_remap/src/lib.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/lib.rs 14_virtual_mem_part2_mmio_remap/src/lib.rs +--- 13_exceptions_part2_peripheral_IRQs/src/lib.rs ++++ 14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -111,6 +111,8 @@ #![allow(clippy::clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -2135,9 +2135,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/lib.rs 15_virtual_mem_part2_mm pub mod cpu; pub mod driver; -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_mmio_remap/src/main.rs ---- 14_exceptions_part2_peripheral_IRQs/src/main.rs -+++ 15_virtual_mem_part2_mmio_remap/src/main.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/main.rs 14_virtual_mem_part2_mmio_remap/src/main.rs +--- 13_exceptions_part2_peripheral_IRQs/src/main.rs ++++ 14_virtual_mem_part2_mmio_remap/src/main.rs @@ -25,21 +25,39 @@ #[no_mangle] unsafe fn kernel_init() -> ! { @@ -2196,9 +2196,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/main.rs 15_virtual_mem_part2_m let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +--- 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs ++++ 14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2422,9 +2422,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + KERNEL_MAPPING_RECORD.read(|mr| mr.print()); +} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs 14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs +--- 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.rs ++++ 14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs @@ -8,7 +8,105 @@ #[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] mod arch_translation_table; @@ -2533,9 +2533,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/translation_table.r + } +} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs +--- 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs ++++ 14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs @@ -0,0 +1,210 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// @@ -2748,9 +2748,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu/types.rs 15_virtual + } +} -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +--- 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs ++++ 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -3,29 +3,23 @@ // Copyright (c) 2020-2021 Andre Richter @@ -3119,9 +3119,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 15_virtual_mem_p + mapping_record::kernel_print() } -diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2_mmio_remap/src/memory.rs ---- 14_exceptions_part2_peripheral_IRQs/src/memory.rs -+++ 15_virtual_mem_part2_mmio_remap/src/memory.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory.rs 14_virtual_mem_part2_mmio_remap/src/memory.rs +--- 13_exceptions_part2_peripheral_IRQs/src/memory.rs ++++ 14_virtual_mem_part2_mmio_remap/src/memory.rs @@ -6,12 +6,136 @@ pub mod mmu; @@ -3261,9 +3261,9 @@ diff -uNr 14_exceptions_part2_peripheral_IRQs/src/memory.rs 15_virtual_mem_part2 /// /// # Safety -diff -uNr 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs 15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs ---- 14_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs -+++ 15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +diff -uNr 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +--- 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs ++++ 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs @@ -21,7 +21,7 @@ #[no_mangle] diff --git a/15_virtual_mem_part2_mmio_remap/build.rs b/14_virtual_mem_part2_mmio_remap/build.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/build.rs rename to 14_virtual_mem_part2_mmio_remap/build.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.s diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs rename to 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp.rs b/14_virtual_mem_part2_mmio_remap/src/bsp.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/common.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/cpu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/driver.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/exception/asynchronous.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs rename to 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/common.rs b/14_virtual_mem_part2_mmio_remap/src/common.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/common.rs rename to 14_virtual_mem_part2_mmio_remap/src/common.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/console.rs b/14_virtual_mem_part2_mmio_remap/src/console.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/console.rs rename to 14_virtual_mem_part2_mmio_remap/src/console.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu.rs b/14_virtual_mem_part2_mmio_remap/src/cpu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/cpu.rs rename to 14_virtual_mem_part2_mmio_remap/src/cpu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs b/14_virtual_mem_part2_mmio_remap/src/cpu/boot.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/cpu/boot.rs rename to 14_virtual_mem_part2_mmio_remap/src/cpu/boot.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs b/14_virtual_mem_part2_mmio_remap/src/cpu/smp.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/cpu/smp.rs rename to 14_virtual_mem_part2_mmio_remap/src/cpu/smp.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/driver.rs b/14_virtual_mem_part2_mmio_remap/src/driver.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/driver.rs rename to 14_virtual_mem_part2_mmio_remap/src/driver.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/exception.rs b/14_virtual_mem_part2_mmio_remap/src/exception.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/exception.rs rename to 14_virtual_mem_part2_mmio_remap/src/exception.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs rename to 14_virtual_mem_part2_mmio_remap/src/exception/asynchronous.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/lib.rs rename to 14_virtual_mem_part2_mmio_remap/src/lib.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/main.rs b/14_virtual_mem_part2_mmio_remap/src/main.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/main.rs rename to 14_virtual_mem_part2_mmio_remap/src/main.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/memory.rs b/14_virtual_mem_part2_mmio_remap/src/memory.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/memory.rs rename to 14_virtual_mem_part2_mmio_remap/src/memory.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/memory/mmu.rs rename to 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs rename to 14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs rename to 14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs rename to 14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/panic_wait.rs b/14_virtual_mem_part2_mmio_remap/src/panic_wait.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/panic_wait.rs rename to 14_virtual_mem_part2_mmio_remap/src/panic_wait.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/print.rs b/14_virtual_mem_part2_mmio_remap/src/print.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/print.rs rename to 14_virtual_mem_part2_mmio_remap/src/print.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/runtime_init.rs b/14_virtual_mem_part2_mmio_remap/src/runtime_init.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/runtime_init.rs rename to 14_virtual_mem_part2_mmio_remap/src/runtime_init.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/state.rs b/14_virtual_mem_part2_mmio_remap/src/state.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/state.rs rename to 14_virtual_mem_part2_mmio_remap/src/state.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/synchronization.rs b/14_virtual_mem_part2_mmio_remap/src/synchronization.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/synchronization.rs rename to 14_virtual_mem_part2_mmio_remap/src/synchronization.rs diff --git a/15_virtual_mem_part2_mmio_remap/src/time.rs b/14_virtual_mem_part2_mmio_remap/src/time.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/src/time.rs rename to 14_virtual_mem_part2_mmio_remap/src/time.rs diff --git a/15_virtual_mem_part2_mmio_remap/test-macros/Cargo.toml b/14_virtual_mem_part2_mmio_remap/test-macros/Cargo.toml similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/test-macros/Cargo.toml rename to 14_virtual_mem_part2_mmio_remap/test-macros/Cargo.toml diff --git a/15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs b/14_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs rename to 14_virtual_mem_part2_mmio_remap/test-macros/src/lib.rs diff --git a/15_virtual_mem_part2_mmio_remap/test-types/Cargo.toml b/14_virtual_mem_part2_mmio_remap/test-types/Cargo.toml similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/test-types/Cargo.toml rename to 14_virtual_mem_part2_mmio_remap/test-types/Cargo.toml diff --git a/15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs b/14_virtual_mem_part2_mmio_remap/test-types/src/lib.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/test-types/src/lib.rs rename to 14_virtual_mem_part2_mmio_remap/test-types/src/lib.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb rename to 14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb diff --git a/15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs rename to 14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs b/14_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs rename to 14_virtual_mem_part2_mmio_remap/tests/01_timer_sanity.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs b/14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs rename to 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs b/14_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs rename to 14_virtual_mem_part2_mmio_remap/tests/03_exception_irq_sanity.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs b/14_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs rename to 14_virtual_mem_part2_mmio_remap/tests/panic_exit_success/mod.rs diff --git a/15_virtual_mem_part2_mmio_remap/tests/runner.rb b/14_virtual_mem_part2_mmio_remap/tests/runner.rb similarity index 100% rename from 15_virtual_mem_part2_mmio_remap/tests/runner.rb rename to 14_virtual_mem_part2_mmio_remap/tests/runner.rb diff --git a/README.md b/README.md index daf3b7d74..183b33d4e 100644 --- a/README.md +++ b/README.md @@ -18,8 +18,9 @@ Have fun! _Best regards,
Andre ([@andre-richter])_ -P.S.: In the future, Chinese :cn: versions of the tutorials will be maintained as -[`README.CN.md`](README.CN.md) by [@colachg] and [@readlnh]. +P.S.: Chinese :cn: versions of the tutorials were started by [@colachg] and [@readlnh]. You can find +them as [`README.CN.md`](README.CN.md) in the respective folders. They are a bit out-of-date at the +moment though. [ARMv8-A architecture]: https://developer.arm.com/products/architecture/cpu-architecture/a-profile/docs [monolithic]: https://en.wikipedia.org/wiki/Monolithic_kernel @@ -40,7 +41,7 @@ P.S.: In the future, Chinese :cn: versions of the tutorials will be maintained a - The code written in these tutorials supports and runs on the **Raspberry Pi 3** and the **Raspberry Pi 4**. - Tutorials 1 till 5 are groundwork code which only makes sense to run in `QEMU`. - - Starting with [tutorial 6](06_drivers_gpio_uart), you can load and run the kernel on the real + - Starting with [tutorial 5](05_drivers_gpio_uart), you can load and run the kernel on the real Raspberrys and observe output over `UART`. - Although the Raspberry Pi 3 and 4 are the main target boards, the code is written in a modular fashion which allows for easy porting to other CPU architectures and/or boards. @@ -125,9 +126,9 @@ get a USB serial cable to get the full experience. - You can find USB-to-serial cables that should work right away at [\[1\]] [\[2\]], but many others will work too. Ideally, your cable is based on the `CP2102` chip. - You connect it to `GND` and GPIO pins `14/15` as shown below. -- [Tutorial 6](06_drivers_gpio_uart) is the first where you can use it. Check it out for +- [Tutorial 5](05_drivers_gpio_uart) is the first where you can use it. Check it out for instructions on how to prepare the SD card to boot your self-made kernel from it. -- Starting with [tutorial 7](07_uart_chainloader), booting kernels on your Raspberry is getting +- Starting with [tutorial 6](06_uart_chainloader), booting kernels on your Raspberry is getting _really_ comfortable. In this tutorial, a so-called `chainloader` is developed, which will be the last file you need to manually copy on the SD card for a while. It will enable you to load the tutorial kernels during boot on demand over `UART`. diff --git a/utils/devtool.rb b/utils/devtool.rb index 66271202f..4f8700694 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -147,7 +147,7 @@ def make_xtra return if @user_has_supplied_crates puts 'Make Xtra stuff'.light_blue - system('cd 07_uart_chainloader && bash update.sh') + system('cd *_uart_chainloader && bash update.sh') system('cd X1_JTAG_boot && bash update.sh') end From b4ed84dd77196a7eb9f599dc6cd55d87888ea4e4 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 22 Mar 2021 22:54:08 +0100 Subject: [PATCH 044/214] Replace ldr pseudo-instruction with adrp/add --- 02_runtime_init/README.md | 18 +++++++-- 02_runtime_init/src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- 04_safe_globals/src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- 06_uart_chainloader/README.md | 35 ++++++++++++++---- 06_uart_chainloader/demo_payload_rpi3.img | Bin 6840 -> 6840 bytes 06_uart_chainloader/demo_payload_rpi4.img | Bin 6680 -> 6680 bytes .../src/_arch/aarch64/cpu/boot.s | 33 ++++++++++++++--- 07_timestamps/README.md | 35 ++++++++++++++---- 07_timestamps/src/_arch/aarch64/cpu/boot.s | 14 ++++++- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s | 14 ++++++- 09_privilege_level/README.md | 12 +++--- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- .../src/_arch/aarch64/cpu/boot.s | 14 ++++++- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8176 -> 8176 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6848 -> 6848 bytes X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s | 14 ++++++- 22 files changed, 272 insertions(+), 43 deletions(-) diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 64e36440e..811433040 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -84,20 +84,32 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.rs 02_runtime_init/src/_arc diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch/aarch64/cpu/boot.s --- 01_wait_forever/src/_arch/aarch64/cpu/boot.s +++ 02_runtime_init/src/_arch/aarch64/cpu/boot.s -@@ -3,6 +3,12 @@ +@@ -3,6 +3,24 @@ // Copyright (c) 2021 Andre Richter //-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + ++// Load the address of a symbol into a register, PC-relative. ++// ++// The symbol must lie within +/- 4 GiB of the Program Counter. ++// ++// # Resources ++// ++// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html ++.macro ADR_REL register, symbol ++ adrp \register, \symbol ++ add \register, \register, #:lo12:\symbol ++.endm ++ +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- .section .text._start -@@ -11,6 +17,22 @@ +@@ -11,6 +29,22 @@ // fn _start() //------------------------------------------------------------------------------ _start: @@ -111,7 +123,7 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Set the stack pointer. -+ ldr x0, =__boot_core_stack_end_exclusive ++ ADR_REL x0, __boot_core_stack_end_exclusive + mov sp, x0 + + // Jump to Rust code. diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.s b/02_runtime_init/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.s +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/04_safe_globals/src/_arch/aarch64/cpu/boot.s b/04_safe_globals/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/04_safe_globals/src/_arch/aarch64/cpu/boot.s +++ b/04_safe_globals/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s +++ b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 457a96cd4..89a4c796e 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -179,20 +179,38 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s --- 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s +++ 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s -@@ -22,20 +22,31 @@ +@@ -18,6 +18,17 @@ + add \register, \register, #:lo12:\symbol + .endm + ++// Load the address of a symbol into a register, absolute. ++// ++// # Resources ++// ++// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html ++.macro ADR_ABS register, symbol ++ movz \register, #:abs_g2:\symbol ++ movk \register, #:abs_g1_nc:\symbol ++ movk \register, #:abs_g0_nc:\symbol ++.endm ++ + .equ _core_id_mask, 0b11 + + //-------------------------------------------------------------------------------------------------- +@@ -34,20 +45,31 @@ and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne 2f ++ ++ // If execution reaches here, it is the boot core. - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. -+ // If execution reaches here, it is the boot core. -+ + // Next, relocate the binary. -+ adr x0, __binary_nonzero_start // The address the binary got loaded to. -+ ldr x1, =__binary_nonzero_start // The address the binary was linked to. -+ ldr x2, =__binary_nonzero_end_exclusive ++ ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. ++ ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. ++ ADR_ABS x2, __binary_nonzero_end_exclusive + +1: ldr x3, [x0], #8 + str x3, [x1], #8 @@ -200,13 +218,14 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ + b.lo 1b // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive +- ADR_REL x0, __boot_core_stack_end_exclusive ++ ADR_ABS x0, __boot_core_stack_end_exclusive mov sp, x0 - // Jump to Rust code. - b _start_rust + // Jump to the relocated Rust code. -+ ldr x1, =_start_rust ++ ADR_ABS x1, _start_rust + br x1 // Infinitely wait for events (aka "park the core"). diff --git a/06_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img index 91602b45b02de681e2e3f0706864f16396592fa9..203ebb7652369e33837c1da09e9b7b7998e4bcf4 100755 GIT binary patch delta 53 zcmdmCy2DgwA%n$LMHYuiPK*o@_6$s~7cww}FfcGo0OE=AK+ML#AQG>@eD(kT|Kb~U GdZYk}WDh|A delta 53 zcmdmCy2DgwA%n$LMHYuiPK*o@_6$s~8yOfv7BDbG$TKiZWM^OyiC190`v3obaRvqs JhK)KsQUIEt4*>uG diff --git a/06_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img index b1060665b4af3eadced31cec72de42265985d110..b06166dcbae527c6f1cc42c11d9200ed60f9dabf 100755 GIT binary patch delta 53 zcmbPXGQ&h?A%n$LMHYuiPK*o@_6$s~7cww}FfcGo0OE=AK+ML#AQG>@eD(kT|Kb~U H-bexfe7z4| delta 53 zcmbPXGQ&h?A%n$LMHYuiPK*o@_6$s~8yOfv7BDbG$TKiZWM^OyiC190`v3obaRvqs JhK)LJBms^14;}yj diff --git a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s index 4f45ebd12..10aebb343 100644 --- a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s +++ b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,29 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + +// Load the address of a symbol into a register, absolute. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_ABS register, symbol + movz \register, #:abs_g2:\symbol + movk \register, #:abs_g1_nc:\symbol + movk \register, #:abs_g0_nc:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,9 +50,9 @@ _start: // If execution reaches here, it is the boot core. // Next, relocate the binary. - adr x0, __binary_nonzero_start // The address the binary got loaded to. - ldr x1, =__binary_nonzero_start // The address the binary was linked to. - ldr x2, =__binary_nonzero_end_exclusive + ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. + ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. + ADR_ABS x2, __binary_nonzero_end_exclusive 1: ldr x3, [x0], #8 str x3, [x1], #8 @@ -37,11 +60,11 @@ _start: b.lo 1b // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_ABS x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to the relocated Rust code. - ldr x1, =_start_rust + ADR_ABS x1, _start_rust br x1 // Infinitely wait for events (aka "park the core"). diff --git a/07_timestamps/README.md b/07_timestamps/README.md index b115dc09d..b898aa409 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -96,19 +96,37 @@ diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_arch/aarch64/cpu/boot.s --- 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s +++ 07_timestamps/src/_arch/aarch64/cpu/boot.s -@@ -22,31 +22,20 @@ +@@ -18,17 +18,6 @@ + add \register, \register, #:lo12:\symbol + .endm + +-// Load the address of a symbol into a register, absolute. +-// +-// # Resources +-// +-// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +-.macro ADR_ABS register, symbol +- movz \register, #:abs_g2:\symbol +- movk \register, #:abs_g1_nc:\symbol +- movk \register, #:abs_g0_nc:\symbol +-.endm +- + .equ _core_id_mask, 0b11 + + //-------------------------------------------------------------------------------------------------- +@@ -45,31 +34,20 @@ and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 2f +- +- // If execution reaches here, it is the boot core. + b.ne 1f -- // If execution reaches here, it is the boot core. -- - // Next, relocate the binary. -- adr x0, __binary_nonzero_start // The address the binary got loaded to. -- ldr x1, =__binary_nonzero_start // The address the binary was linked to. -- ldr x2, =__binary_nonzero_end_exclusive +- ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. +- ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. +- ADR_ABS x2, __binary_nonzero_end_exclusive - -1: ldr x3, [x0], #8 - str x3, [x1], #8 @@ -117,11 +135,12 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_ar + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive +- ADR_ABS x0, __boot_core_stack_end_exclusive ++ ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 - // Jump to the relocated Rust code. -- ldr x1, =_start_rust +- ADR_ABS x1, _start_rust - br x1 + // Jump to Rust code. + b _start_rust diff --git a/07_timestamps/src/_arch/aarch64/cpu/boot.s b/07_timestamps/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/07_timestamps/src/_arch/aarch64/cpu/boot.s +++ b/07_timestamps/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s +++ b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index b8af30216..b786bf33c 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -273,15 +273,15 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_arch/aarch64/cpu/boot.s --- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s +++ 09_privilege_level/src/_arch/aarch64/cpu/boot.s -@@ -6,6 +6,7 @@ - // Definitions - //-------------------------------------------------------------------------------------------------- +@@ -18,6 +18,7 @@ + add \register, \register, #:lo12:\symbol + .endm +.equ _EL2, 0x8 .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- -@@ -17,6 +18,11 @@ +@@ -29,6 +30,11 @@ // fn _start() //------------------------------------------------------------------------------ _start: @@ -293,13 +293,13 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_ // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask -@@ -26,11 +32,11 @@ +@@ -38,11 +44,11 @@ // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. - // Set the stack pointer. + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 - // Jump to Rust code. diff --git a/09_privilege_level/src/_arch/aarch64/cpu/boot.s b/09_privilege_level/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/09_privilege_level/src/_arch/aarch64/cpu/boot.s +++ b/09_privilege_level/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s +++ b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s index be81b20a2..5696220d0 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _EL2, 0x8 .equ _core_id_mask, 0b11 @@ -33,7 +45,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. x0 holds the function argument provided to _start_rust(). diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 8eed7dba0e7d56b27a14d14f981f77b599d9b6c3..53be6bb3a5207dad2a75a526053ad11b367272c0 100755 GIT binary patch delta 53 zcmexh|G{2oA%n$LMHYuiPK*o@_6$s~7cww}FfcGo0OE=AK+ML#AQG>@eD(kT|Kb~U Hj>rQ5q?Hf? delta 53 zcmexh|G{2oA%n$LMHYuiPK*o@_6$s~8yOfv7BDbG$TKiZWM^OyiC190`v3obaRvqs JhK)K$wN55fQd diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index cbdd1b808832df622e2be50a4bb24e5437756355..5e1e1bf7d6716daa5b88a735299e89c268faf47d 100755 GIT binary patch delta 53 zcmX?Ldcag?A%n$LMHYuiPK*o@_6$s~7cww}FfcGo0OE=AK+ML#AQG>@eD(kT|Kb~U Hrbqz*jI<9~ delta 53 zcmX?Ldcag?A%n$LMHYuiPK*o@_6$s~8yOfv7BDbG$TKiZWM^OyiC190`v3obaRvqs JhK)K?qyU}o4;cUe diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s index ad4a26892..bfa94abf3 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s @@ -6,6 +6,18 @@ // Definitions //-------------------------------------------------------------------------------------------------- +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- @@ -27,7 +39,7 @@ _start: // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. // Set the stack pointer. - ldr x0, =__boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 // Jump to Rust code. From c0da8ab1b70cb5fc34652cbd547337046716d83e Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 24 Mar 2021 00:00:21 +0100 Subject: [PATCH 045/214] Remove confusing statement --- 02_runtime_init/README.md | 15 ++++----------- 02_runtime_init/src/bsp/raspberrypi/link.ld | 13 +++---------- 03_hacky_hello_world/src/bsp/raspberrypi/link.ld | 13 +++---------- 04_safe_globals/src/bsp/raspberrypi/link.ld | 13 +++---------- 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld | 13 +++---------- 06_uart_chainloader/README.md | 2 +- 06_uart_chainloader/src/bsp/raspberrypi/link.ld | 13 +++---------- 07_timestamps/README.md | 2 +- 07_timestamps/src/bsp/raspberrypi/link.ld | 13 +++---------- 08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld | 13 +++---------- 09_privilege_level/src/bsp/raspberrypi/link.ld | 13 +++---------- 10_virtual_mem_part1_identity_mapping/README.md | 2 +- .../src/bsp/raspberrypi/link.ld | 13 +++---------- .../src/bsp/raspberrypi/link.ld | 13 +++---------- 12_integrated_testing/src/bsp/raspberrypi/link.ld | 13 +++---------- .../src/bsp/raspberrypi/link.ld | 13 +++---------- 14_virtual_mem_part2_mmio_remap/README.md | 4 ++-- .../src/bsp/raspberrypi/link.ld | 13 +++---------- X1_JTAG_boot/src/bsp/raspberrypi/link.ld | 13 +++---------- 19 files changed, 51 insertions(+), 156 deletions(-) diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 811433040..b294e2e1f 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -186,7 +186,7 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/cpu.rs 02_runtime_init/src/bsp/ras diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld -@@ -11,17 +11,52 @@ +@@ -11,17 +11,45 @@ PHDRS { segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ @@ -209,16 +209,9 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/ra .text : { KEEP(*(.text._start)) -+ -+ /* Special constants (or statics in Rust speak) needed by _start(). -+ * -+ * They are placed in close proximity to _start() from where they will be read. This ensures -+ * that position-independent, PC-relative loads can be emitted. -+ */ -+ *(.text._start_arguments) -+ -+ *(.text._start_rust) /* The Rust entry point */ -+ *(.text*) /* Everything else */ ++ *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ ++ *(.text._start_rust) /* The Rust entry point */ ++ *(.text*) /* Everything else */ } :segment_rx + + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/02_runtime_init/src/bsp/raspberrypi/link.ld b/02_runtime_init/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/02_runtime_init/src/bsp/raspberrypi/link.ld +++ b/02_runtime_init/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld +++ b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/04_safe_globals/src/bsp/raspberrypi/link.ld b/04_safe_globals/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/04_safe_globals/src/bsp/raspberrypi/link.ld +++ b/04_safe_globals/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ b/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 89a4c796e..c05390460 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -315,7 +315,7 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 06_uart_chainloader/s .text : { KEEP(*(.text._start)) -@@ -49,8 +51,12 @@ +@@ -42,8 +44,12 @@ ***********************************************************************************************/ .data : { *(.data*) } :segment_rw diff --git a/06_uart_chainloader/src/bsp/raspberrypi/link.ld b/06_uart_chainloader/src/bsp/raspberrypi/link.ld index f75239061..324172b19 100644 --- a/06_uart_chainloader/src/bsp/raspberrypi/link.ld +++ b/06_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -31,16 +31,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/07_timestamps/README.md b/07_timestamps/README.md index b898aa409..dc40f8048 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -398,7 +398,7 @@ diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/link.ld 07_timestamps/src/bsp/ .text : { KEEP(*(.text._start)) -@@ -51,12 +49,8 @@ +@@ -44,12 +42,8 @@ ***********************************************************************************************/ .data : { *(.data*) } :segment_rw diff --git a/07_timestamps/src/bsp/raspberrypi/link.ld b/07_timestamps/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/07_timestamps/src/bsp/raspberrypi/link.ld +++ b/07_timestamps/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld +++ b/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/09_privilege_level/src/bsp/raspberrypi/link.ld b/09_privilege_level/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/09_privilege_level/src/bsp/raspberrypi/link.ld +++ b/09_privilege_level/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index d2d72ea55..3161e6b01 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -821,7 +821,7 @@ diff -uNr 09_privilege_level/src/bsp/raspberrypi/link.ld 10_virtual_mem_part1_id .text : { KEEP(*(.text._start)) -@@ -44,6 +45,9 @@ +@@ -37,6 +38,9 @@ .rodata : ALIGN(8) { *(.rodata*) } :segment_rx .got : ALIGN(8) { *(.got) } :segment_rx diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld index 854f5b388..bda0da5e9 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -30,16 +30,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld index 854f5b388..bda0da5e9 100644 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld +++ b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld @@ -30,16 +30,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/12_integrated_testing/src/bsp/raspberrypi/link.ld b/12_integrated_testing/src/bsp/raspberrypi/link.ld index 854f5b388..bda0da5e9 100644 --- a/12_integrated_testing/src/bsp/raspberrypi/link.ld +++ b/12_integrated_testing/src/bsp/raspberrypi/link.ld @@ -30,16 +30,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld index 854f5b388..bda0da5e9 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld @@ -30,16 +30,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index cf387da92..ca7905f10 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -1468,7 +1468,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 14_vir /*********************************************************************************************** * Code + RO Data + Global Offset Table -@@ -51,6 +46,7 @@ +@@ -44,6 +39,7 @@ /*********************************************************************************************** * Data + BSS ***********************************************************************************************/ @@ -1476,7 +1476,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 14_vir .data : { *(.data*) } :segment_rw /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -@@ -63,4 +59,23 @@ +@@ -56,4 +52,23 @@ . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ __bss_end_inclusive = . - 8; } :NONE diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld index 7fddf3ec5..64545edf2 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -25,16 +25,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld index 3b73b3c7e..97ea6d69c 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld +++ b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld @@ -29,16 +29,9 @@ SECTIONS .text : { KEEP(*(.text._start)) - - /* Special constants (or statics in Rust speak) needed by _start(). - * - * They are placed in close proximity to _start() from where they will be read. This ensures - * that position-independent, PC-relative loads can be emitted. - */ - *(.text._start_arguments) - - *(.text._start_rust) /* The Rust entry point */ - *(.text*) /* Everything else */ + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ } :segment_rx .rodata : ALIGN(8) { *(.rodata*) } :segment_rx From fd30d911d813ff3d58e91b3b760c5f7873c8b6cc Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 24 Mar 2021 00:00:40 +0100 Subject: [PATCH 046/214] Run cargo fmt in parallel --- utils/devtool.rb | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/utils/devtool.rb b/utils/devtool.rb index 4f8700694..c29987bba 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -42,10 +42,6 @@ def clippy(bsp) end def fmt_cargo_rust(args) - print 'Rust cargo fmt '.light_blue - print "#{args} ".light_blue unless args.nil? - puts @folder - Dir.chdir(@folder) { exit(1) unless system("cargo fmt #{args}") } end @@ -216,7 +212,14 @@ def bsp_from_env def fmt_cargo_rust(check: false) args = '-- --check' if check - @crates.each { |c| c.fmt_cargo_rust(args) } + @crates.each do |c| + print 'Rust cargo fmt '.light_blue + print "#{args} ".light_blue unless args.nil? + puts c.folder + + Process.fork { c.fmt_cargo_rust(args) } + end + Process.waitall end def fmt_prettier(check: false) From f51c1afeeb0fee9a440ae56c727d42aa9941e93b Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 24 Mar 2021 09:11:07 +0100 Subject: [PATCH 047/214] Workaround for failing test --- 12_integrated_testing/Cargo.toml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/12_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml index 206dfc878..06c10cf07 100644 --- a/12_integrated_testing/Cargo.toml +++ b/12_integrated_testing/Cargo.toml @@ -4,8 +4,10 @@ version = "0.1.0" authors = ["Andre Richter "] edition = "2018" +# TODO: Fixme +# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. [profile.release] -lto = true +lto = false [features] default = [] From b70b08d92ed94b6bfbc222996d9a37ac1aca9388 Mon Sep 17 00:00:00 2001 From: Pius Friesch Date: Sat, 27 Mar 2021 20:44:50 +0100 Subject: [PATCH 048/214] Update README.md --- 06_uart_chainloader/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index c05390460..040c498d3 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -37,7 +37,7 @@ Our chainloader is called `MiniLoad` and is inspired by [raspbootin]. You can try it with this tutorial already: 1. Depending on your target hardware, run:`make` or `BSP=rpi4 make`. -1. Copy `kernel8.img` to the SD card. +1. Copy `kernel8.img` to the SD card and put the SD card back into your RPi. 1. Run `make chainboot` or `BSP=rpi4 make chainboot`. 1. Connect the USB serial to your host PC. - Wiring diagram at [top-level README](../README.md#-usb-serial-output). From 02602a00df2639b72555598e37568f0469b76b6d Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 22:12:14 +0200 Subject: [PATCH 049/214] Update readmes --- 12_integrated_testing/README.md | 12 +++++++++++- 13_exceptions_part2_peripheral_IRQs/README.md | 16 ++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index 6b76c1902..eefbfa4da 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -819,7 +819,17 @@ diff -uNr 11_exceptions_part1_groundwork/.cargo/config.toml 12_integrated_testin diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo.toml --- 11_exceptions_part1_groundwork/Cargo.toml +++ 12_integrated_testing/Cargo.toml -@@ -11,17 +11,45 @@ +@@ -4,24 +4,54 @@ + authors = ["Andre Richter "] + edition = "2018" + ++# TODO: Fixme ++# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. + [profile.release] +-lto = true ++lto = false + + [features] default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index ea327b77e..18b3cf812 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -744,6 +744,22 @@ Minipush 1.0 ## Diff to previous ```diff +diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/Cargo.toml +--- 12_integrated_testing/Cargo.toml ++++ 13_exceptions_part2_peripheral_IRQs/Cargo.toml +@@ -4,10 +4,8 @@ + authors = ["Andre Richter "] + edition = "2018" + +-# TODO: Fixme +-# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. + [profile.release] +-lto = false ++lto = true + + [features] + default = [] + diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs --- 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs From 7c9b472e1e1746b47a6cb1cd45e14a5b6da9ed7a Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 22:13:18 +0200 Subject: [PATCH 050/214] Update readme --- 14_virtual_mem_part2_mmio_remap/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index ca7905f10..4c1c940c3 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -17,7 +17,7 @@ - [Implementation](#implementation) - [A New Mapping API in `src/memory/mmu.rs`](#a-new-mapping-api-in-srcmemorymmutranslationtablers) - [The new APIs in action](#the-new-apis-in-action) - - [Additional Changes](#additional-changes) + - [Supporting Changes](#supporting-changes) - [Test it](#test-it) - [Diff to previous](#diff-to-previous) From 035dd5421c8190b8088c27434d19642521ff04b1 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 22:46:06 +0200 Subject: [PATCH 051/214] Add tutorial 15 --- .rubocop.yml | 1 + .../.cargo/config.toml | 2 + .../.vscode/settings.json | 7 + .../Cargo.lock | 93 + .../Cargo.toml | 55 + .../Makefile | 193 ++ .../README.md | 2254 +++++++++++++++++ .../build.rs | 8 + .../src/_arch/aarch64/cpu.rs | 49 + .../src/_arch/aarch64/cpu/boot.rs | 89 + .../src/_arch/aarch64/cpu/boot.s | 63 + .../src/_arch/aarch64/cpu/smp.rs | 29 + .../src/_arch/aarch64/exception.rs | 281 ++ .../src/_arch/aarch64/exception.s | 148 ++ .../_arch/aarch64/exception/asynchronous.rs | 150 ++ .../src/_arch/aarch64/memory/mmu.rs | 182 ++ .../aarch64/memory/mmu/translation_table.rs | 474 ++++ .../src/_arch/aarch64/time.rs | 118 + .../src/bsp.rs | 13 + .../src/bsp/device_driver.rs | 16 + .../src/bsp/device_driver/arm.rs | 9 + .../src/bsp/device_driver/arm/gicv2.rs | 248 ++ .../src/bsp/device_driver/arm/gicv2/gicc.rs | 152 ++ .../src/bsp/device_driver/arm/gicv2/gicd.rs | 205 ++ .../src/bsp/device_driver/bcm.rs | 15 + .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 255 ++ .../bcm/bcm2xxx_interrupt_controller.rs | 138 + .../peripheral_ic.rs | 188 ++ .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 537 ++++ .../src/bsp/device_driver/common.rs | 38 + .../src/bsp/raspberrypi.rs | 62 + .../src/bsp/raspberrypi/console.rs | 88 + .../src/bsp/raspberrypi/cpu.rs | 14 + .../src/bsp/raspberrypi/driver.rs | 61 + .../src/bsp/raspberrypi/exception.rs | 7 + .../bsp/raspberrypi/exception/asynchronous.rs | 36 + .../kernel_virt_addr_space_size.ld | 1 + .../src/bsp/raspberrypi/link.ld | 77 + .../src/bsp/raspberrypi/memory.rs | 208 ++ .../src/bsp/raspberrypi/memory/mmu.rs | 187 ++ .../src/common.rs | 21 + .../src/console.rs | 53 + .../src/cpu.rs | 21 + .../src/cpu/boot.rs | 9 + .../src/cpu/smp.rs | 14 + .../src/driver.rs | 62 + .../src/exception.rs | 48 + .../src/exception/asynchronous.rs | 152 ++ .../src/lib.rs | 176 ++ .../src/main.rs | 105 + .../src/memory.rs | 202 ++ .../src/memory/mmu.rs | 265 ++ .../src/memory/mmu/mapping_record.rs | 221 ++ .../src/memory/mmu/translation_table.rs | 109 + .../src/memory/mmu/types.rs | 217 ++ .../src/panic_wait.rs | 58 + .../src/print.rs | 106 + .../src/runtime_init.rs | 40 + .../src/state.rs | 92 + .../src/synchronization.rs | 159 ++ .../src/time.rs | 37 + .../test-macros/Cargo.toml | 14 + .../test-macros/src/lib.rs | 29 + .../test-types/Cargo.toml | 5 + .../test-types/src/lib.rs | 16 + .../tests/00_console_sanity.rb | 50 + .../tests/00_console_sanity.rs | 35 + .../tests/01_timer_sanity.rs | 49 + .../tests/02_exception_sync_page_fault.rs | 36 + .../tests/03_exception_irq_sanity.rs | 66 + .../tests/panic_exit_success/mod.rs | 9 + .../tests/runner.rb | 143 ++ .../translation_table_tool/arch.rb | 323 +++ .../translation_table_tool/bsp.rb | 177 ++ .../translation_table_tool/generic.rb | 125 + .../translation_table_tool/main.rb | 47 + 76 files changed, 10042 insertions(+) create mode 100644 15_virtual_mem_part3_precomputed_tables/.cargo/config.toml create mode 100644 15_virtual_mem_part3_precomputed_tables/.vscode/settings.json create mode 100644 15_virtual_mem_part3_precomputed_tables/Cargo.lock create mode 100644 15_virtual_mem_part3_precomputed_tables/Cargo.toml create mode 100644 15_virtual_mem_part3_precomputed_tables/Makefile create mode 100644 15_virtual_mem_part3_precomputed_tables/README.md create mode 100644 15_virtual_mem_part3_precomputed_tables/build.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.s create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/common.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/cpu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/driver.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception/asynchronous.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/common.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/console.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/cpu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/cpu/boot.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/cpu/smp.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/driver.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/exception.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/exception/asynchronous.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/lib.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/main.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/memory.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/print.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/state.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/synchronization.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/src/time.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/test-macros/Cargo.toml create mode 100644 15_virtual_mem_part3_precomputed_tables/test-macros/src/lib.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/test-types/Cargo.toml create mode 100644 15_virtual_mem_part3_precomputed_tables/test-types/src/lib.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/01_timer_sanity.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/03_exception_irq_sanity.rs create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/panic_exit_success/mod.rs create mode 100755 15_virtual_mem_part3_precomputed_tables/tests/runner.rb create mode 100644 15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb create mode 100644 15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb create mode 100644 15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb create mode 100755 15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb diff --git a/.rubocop.yml b/.rubocop.yml index 78ba1c1d6..a1687fd16 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -11,6 +11,7 @@ Layout/IndentationWidth: Width: 4 + IgnoredPatterns: ['^\s*module'] Layout/LineLength: Max: 100 diff --git a/15_virtual_mem_part3_precomputed_tables/.cargo/config.toml b/15_virtual_mem_part3_precomputed_tables/.cargo/config.toml new file mode 100644 index 000000000..e34764853 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/.cargo/config.toml @@ -0,0 +1,2 @@ +[target.'cfg(target_os = "none")'] +runner = "target/kernel_test_runner.sh" diff --git a/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json b/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json new file mode 100644 index 000000000..a0d6a9202 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json @@ -0,0 +1,7 @@ +{ + "editor.formatOnSave": true, + "editor.rulers": [100], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", + "rust-analyzer.cargo.features": ["bsp_rpi3"] +} diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.lock b/15_virtual_mem_part3_precomputed_tables/Cargo.lock new file mode 100644 index 000000000..069559203 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.lock @@ -0,0 +1,93 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "cortex-a" +version = "5.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +dependencies = [ + "register", +] + +[[package]] +name = "kernel" +version = "0.1.0" +dependencies = [ + "cortex-a", + "qemu-exit", + "register", + "test-macros", + "test-types", +] + +[[package]] +name = "proc-macro2" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +dependencies = [ + "unicode-xid", +] + +[[package]] +name = "qemu-exit" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" + +[[package]] +name = "quote" +version = "1.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "register" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" +dependencies = [ + "tock-registers", +] + +[[package]] +name = "syn" +version = "1.0.62" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +dependencies = [ + "proc-macro2", + "quote", + "unicode-xid", +] + +[[package]] +name = "test-macros" +version = "0.1.0" +dependencies = [ + "proc-macro2", + "quote", + "syn", + "test-types", +] + +[[package]] +name = "test-types" +version = "0.1.0" + +[[package]] +name = "tock-registers" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" + +[[package]] +name = "unicode-xid" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f7fe0bb3479651439c9112f72b6c505038574c9fbb575ed1bf3b797fa39dd564" diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.toml b/15_virtual_mem_part3_precomputed_tables/Cargo.toml new file mode 100644 index 000000000..206dfc878 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.toml @@ -0,0 +1,55 @@ +[package] +name = "kernel" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" + +[profile.release] +lto = true + +[features] +default = [] +bsp_rpi3 = ["register"] +bsp_rpi4 = ["register"] +test_build = ["qemu-exit"] + +##-------------------------------------------------------------------------------------------------- +## Dependencies +##-------------------------------------------------------------------------------------------------- + +[dependencies] +test-types = { path = "test-types" } + +# Optional dependencies +register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } +qemu-exit = { version = "1.x.x", optional = true } + +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } + +##-------------------------------------------------------------------------------------------------- +## Testing +##-------------------------------------------------------------------------------------------------- + +[dev-dependencies] +test-macros = { path = "test-macros" } + +# Unit tests are done in the library part of the kernel. +[lib] +name = "libkernel" +test = true + +# Disable unit tests for the kernel binary. +[[bin]] +name = "kernel" +test = false + +# List of tests without harness. +[[test]] +name = "00_console_sanity" +harness = false + +[[test]] +name = "02_exception_sync_page_fault" +harness = false diff --git a/15_virtual_mem_part3_precomputed_tables/Makefile b/15_virtual_mem_part3_precomputed_tables/Makefile new file mode 100644 index 000000000..feb65cef7 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/Makefile @@ -0,0 +1,193 @@ +## SPDX-License-Identifier: MIT OR Apache-2.0 +## +## Copyright (c) 2018-2021 Andre Richter + +include ../utils/color.mk.in + +# Default to the RPi3 +BSP ?= rpi3 + +# Default to a serial device name that is common in Linux. +DEV_SERIAL ?= /dev/ttyUSB0 + +# Query the host system's kernel name +UNAME_S = $(shell uname -s) + +# BSP-specific arguments +ifeq ($(BSP),rpi3) + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = raspi3 + QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 +else ifeq ($(BSP),rpi4) + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = + QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 +endif + +# Export for build.rs +export LINKER_FILE + +# Testing-specific arguments +ifdef TEST + ifeq ($(TEST),unit) + TEST_ARG = --lib + else + TEST_ARG = --test $(TEST) + endif +endif + +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) +RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs + +FEATURES = --features bsp_$(BSP) +COMPILER_ARGS = --target=$(TARGET) \ + $(FEATURES) \ + --release + +RUSTC_CMD = cargo rustc $(COMPILER_ARGS) +DOC_CMD = cargo doc $(COMPILER_ARGS) +CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) +CHECK_CMD = cargo check $(COMPILER_ARGS) +TEST_CMD = cargo test $(COMPILER_ARGS) +OBJCOPY_CMD = rust-objcopy \ + --strip-all \ + -O binary + +KERNEL_ELF = target/$(TARGET)/release/kernel + +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host + +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + +# Dockerize commands that require USB device passthrough only on Linux +ifeq ($(UNAME_S),Linux) + DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) + + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +else + DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# +endif + +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_MINIPUSH = ruby ../utils/minipush.rb + +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ + clippy clean readelf objdump nm check + +all: $(KERNEL_BIN) + +$(KERNEL_ELF): + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + +$(KERNEL_BIN): $(KERNEL_ELF) + @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) + +doc: + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open + +ifeq ($(QEMU_MACHINE_TYPE),) +qemu test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") +else +qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") + @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build +test: + $(call colorecho, "\nCompiling test(s) - $(BSP)") + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) +endif + +chainboot: $(KERNEL_BIN) + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) + +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + +clippy: + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + +clean: + rm -rf target $(KERNEL_BIN) + +readelf: $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) + +objdump: $(KERNEL_ELF) + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + --section .got \ + $(KERNEL_ELF) | rustfilt + +nm: $(KERNEL_ELF) + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + +# For rust-analyzer +check: + @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md new file mode 100644 index 000000000..8de51b1bb --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -0,0 +1,2254 @@ +# Tutorial 15 - Virtual Memory Part 3: Precomputed Translation Tables + +## tl;dr + +- We are making the next baby-steps towards mapping the kernel to the most significant area of the + virtual memory space. +- Instead of dynamically computing the kernel's translation tables during runtime while booting, we + are precomputing them in advance just after kernel compilation, and patch them into the kernel's + binary ahead of time. +- For now, we are still `identity-mapping` the kernel binary. + - However, after this tutorial, we have all the infrastructure in place to easily map it + elsewhere. + +## Table of Contents + +- [Introduction](#introduction) +- [When Load Address != Link Address, Funny Things Can Happen](#when-load-address--link-address-funny-things-can-happen) + * [Interim Conclusion](#interim-conclusion) + * [Doing the Same Thing - Expecting Different Results](#doing-the-same-thing---expecting-different-results) +- [Position-Independent Code (PIC)](#position-independent-code-pic) + * [Using PIC during kernel startup](#using-pic-during-kernel-startup) +- [Precomputed Translation Tables](#precomputed-translation-tables) +- [Implementation](#implementation) + * [Preparing the Kernel Tables](#preparing-the-kernel-tables) + * [Turning on the MMU Before Switching to EL1](#turning-on-the-mmu-before-switching-to-el1) + * [The Translation Table Tool](#the-translation-table-tool) + * [Other changes](#other-changes) +- [Discussion](#discussion) +- [Test it](#test-it) +- [Diff to previous](#diff-to-previous) + +## Introduction + +This tutorial is another preparatory step for our overall goal of mapping the kernel to the most +significant area of the virtual memory space. + +The reasoning of why we want to do this was given in the previous tutorial's introduction. But lets +for a quick moment think about what it actually means in practice: Currently, the kernel's binary is +loaded by the Raspberry's firmware at address `0x8_0000`. + +In decimal, this address is at `512 KiB`, and therefore well within the _least significant part_ of +the address space. Let's have a look at the picture from the [ARM Cortex-A Series Programmer’s Guide +for ARMv8-A] again to understand in which virtual address space region the kernel would ideally be +mapped to: + +

+ +

+ +As we can see, the architecture proposes somewhere between addresses `0xffff_0000_0000_0000` and +`0xffff_ffff_ffff_ffff`. Once we succeed in mapping the kernel there, the whole lower range between +`0x0` and `0xffff_ffff_ffff` would be free for future applications to use. + +[ARM Cortex-A Series Programmer’s Guide for ARMv8-A]: https://developer.arm.com/documentation/den0024/latest/ + +Now, how can we get there? + +## When Load Address != Link Address, Funny Things Can Happen + +Imagine that, using the linker script, we link the kernel so that its `_start()` function is located +at address `0xffff_0000_0000_0000`. What hasn't changed is that the Raspberry's firmware will still +load the kernel binary at address `0x8_0000`, and the kernel will still start executing from there +with the `MMU` disabled. + +So one of the very first things the kernel must achieve during its boot to function correctly, is to +somehow enable the `MMU` together with `translation tables` that account for the address offset +(`0xffff_0000_0000_0000 -> 0x8_0000`). In previous tutorials, we already generated translation +tables during the kernel's boot, so lets quickly remember how we did that: + +In `src/bsp/__board_name__/memory/mmu.rs` we have a static (or "global" in non-Rust speak) instance +of `struct KernelTranslationTable`: + +```rust +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new()); +``` + +In other parts of the kernel code, this instance would be referenced one way or the other, and its +member functions would be called, for example, when mapping a range of pages. At the end of the day, +after multiple layers of indirection, what happens at the most basic level is that a `piece of code` +manipulates some `global data`. So part of the job of the code is to retrieve the data's constant +address before it can manipulate it. + +Let's simplify the address-retrieval to the most basic code example possible. The example will be +presented as `C` code. Don't ask yet why `C` is chosen. It will get clear as the tutorial develops. + +```c +#include + +uint64_t global_data_word = 0x11223344; + +uint64_t* get_address_of_global(void) { + return &global_data_word; +} +``` + +Let's compile and link this using the following linker script: + +```lds +SECTIONS +{ + . = 0x80000; + + .text : { + QUAD(0); /* Intentional fill word */ + QUAD(0); /* Intentional fill word */ + KEEP(*(.text*)) + } + .got : ALIGN(8) { *(.got) } + .data : ALIGN(64K) { + QUAD(0); /* Intentional fill word */ + *(.data*) + } +} +``` + +Here are the compilation steps and the corresponding `objdump` for `AArch64`: + +```console +$ clang --target=aarch64-none-elf -Iinclude -Wall -c start.c -o start.o +$ ld.lld start.o -T link.ld -o example.elf +``` + +```c-objdump +Disassembly of section .text: + +0000000000080010 get_address_of_global: + 80010: 80 00 00 90 adrp x0, #0x10000 + 80014: 00 20 00 91 add x0, x0, #0x8 + 80018: c0 03 5f d6 ret + +Disassembly of section .data: + +0000000000090008 global_data_word: + 90008: 44 33 22 11 + 9000c: 00 00 00 00 +``` + +As you can see, the address of function `get_address_of_global()` is `0x8_0010` and +`global_data_word` got address `0x9_0008`. In the function body, the compiler emitted an [`ADRP`] +and `ADD` instruction pair, which means that the global's address is calculated as a `PC-relative +offset`. `PC` means program counter, aka the current position of where the CPU core is currently +executing from. + +Without going in too much detail, what the instruction basically does is: It retrieves the `4 KiB` +page address that belongs to the program counter's (PC) current position (PC is at `0x8_0010`, so +the page address is `0x8_0000`), and adds adds `0x1_0000`. So after the `ADRP` instruction, register +`x0` holds the value `0x9_0000`. To this value, `8` is added in the next instruction, resulting in +the overall address of `0x9_0008`, which is exactly where `global_data_word` is located. This works, +because after linking a `static executable binary` like we do since `tutorial 01`, relative +positions of code and data are fixed, and not supposed to change during runtime. + +[`ADRP`]: https://developer.arm.com/documentation/dui0802/b/A64-General-Instructions/ADRP + +If the Raspberry's firmware now loads this binary at address `0x8_0000`, as always, we can be sure +that our function returns the correct address of our global data word. + +> So far, all looks good, doesn't it? However, this was a lot to digest already, and we're far from +> finished. So take a minute or two and clear your mind before we continue. 🧘 + +Now lets link this to the most significant area of memory: + +```lds +SECTIONS +{ + . = 0xffff000000000000; <--- Only line changed in the linker script! + + .text : { + + /* omitted for brevity */ +} +``` + +And compile again: + +```c-objdump +Disassembly of section .text: + +ffff000000000010 get_address_of_global: +ffff000000000010: 80 00 00 90 adrp x0, #0x10000 +ffff000000000014: 00 20 00 91 add x0, x0, #0x8 +ffff000000000018: c0 03 5f d6 ret + +Disassembly of section .data: + +ffff000000010008 global_data_word: +ffff000000010008: 44 33 22 11 +ffff00000001000c: 00 00 00 00 +``` + +And let the Raspberry's firmware load the binary at address `0x8_0000` again (we couldn't load it to +`0xffff_0000_0000_0000` even if we wanted to. That address is `15 Exbibyte`. A Raspberry Pi with +that much RAM won't exist for some time to come 😉). + +Let's try to answer the same question again: Would `get_address_of_global()` return the value for +`global_data_word` that we expect to see (`0xffff_0000_0001_0008` as shown in the objdump)? This +time, the answer is **no**. It would again return `0x9_0008`. + +Why is that? Don't let yourself be distracted by the addresses the `objdump` above is showing. When +the Raspberry's firmware loads this binary at `0x8_0000`, then the Program Counter value when +`get_address_of_global()` executes is again `0x8_0010`. So **the PC-relative calculation** will not +result in the expected value, which would be the **absolute** (alternatively: **link-time**) address +of `global_data_word`. + +### Interim Conclusion + +What have we learned so far? We wrote a little piece of code in a high-level language that retrieves +an address, and we naively expected to retrieve an **absolute** address. + +But compiler and linker conspired against us, and machine code was emitted that uses a PC-relative +addressing scheme, so our expectation is not matched when **load address != link address**. If you +compile for `AArch64`, you'll see relative addressing schemes a lot, because it is natural to the +architecture. + +If you now say: Wait a second, how is this a problem? It actually helps! After all, since the code +is loaded at address `0x8_0000`, this relative addressing scheme will ensure that the processor +accesses the global data word at the correct address! + +Yes, in this particular, constrained demo case, it worked out for us. But have a look at the +following. + +### Doing the Same Thing - Expecting Different Results + +Let's take a quick detour and see what happens if we compile **the exactly same code** for the +`x86_64` processor architecture. First when linked to `0x8_0000`: + +```c-objdump +Disassembly of section .text: + +0000000000080070 get_address_of_global: + 80070: 55 push rbp + 80071: 48 89 e5 mov rbp, rsp + 80074: 48 b8 08 00 09 00 00 00 00 00 movabs rax, 0x90008 + 8007e: 5d pop rbp + 8007f: c3 ret + +Disassembly of section .data: + +ffff000000010008 global_data_word: +ffff000000010008: 44 33 22 11 +ffff00000001000c: 00 00 00 00 +``` + +And now linked to `0xffff_0000_0000_0000`: + +```c-objdump +Disassembly of section .text: + +ffff000000000070 get_address_of_global: +ffff000000000070: 55 push rbp +ffff000000000071: 48 89 e5 mov rbp, rsp +ffff000000000074: 48 b8 08 00 01 00 00 00 ff ff movabs rax, 0xffff000000010008 +ffff00000000007e: 5d pop rbp +ffff00000000007f: c3 ret + +Disassembly of section .data: + +ffff000000010008 global_data_word: +ffff000000010008: 44 33 22 11 +ffff00000001000c: 00 00 00 00 +``` + +Both times, the `movabsq` instruction gets emitted. It means that the address is put into the target +register using hardcoded `immediate values`. PC-relative address calculation is not used here. +Hence, this code would return the `absolute` address in both cases. Which means in the second case, +even when the binary would be loaded at `0x8_0000`, the return value would be +`0xffff_0000_0001_0008`. + +**In summary, we get a different result for the same piece of `C` code, depending on the target +processor architecture**. What do we learn from this little detour? + +First, you cannot naively compile and run `Rust` or `C` statically linked binaries when there will +be a **load address != link address** situation. You'll run into undefined behavior very fast. It is +kinda expected and obvious, but hopefully it helped to see it fail in action. + +Furthermore, it is important to understand that there are of course ways to load a symbol's absolute +address into `AArch64` registers using `immediate values` as well. Likewise, you can also do +PC-relative addressing in `x86`. We just looked at a tiny example. Maybe the next line of code would +be compiled into the opposite behavior on the two architectures, so that the `x86` code would do a +PC-relative calculation while the `AArch64` code goes for absolute. + +At the end of the day, what is needed to solve our task at hand (bringup of virtual memory, while +being linked to one address and executing from another), is tight control over the machine +instructions that get emitted for **those pieces of code** that generate the `translation tables` +and enable the `MMU`. + +What we need is called [position-independent code]. + +[position-independent code]: https://en.wikipedia.org/wiki/Position-independent_code + +## Position-Independent Code (PIC) + +As describend by Wikipedia, position-independent code + +> is a body of machine code that, being placed somewhere in the primary memory, **executes +> properly** regardless of its absolute address. + +Your safest bet is to write the pieces that need to be position-independent in `assembly`, because +this gives you full control over when relative or absolute addresses are being generated or used. +You will see this approach often in big projects like the Linux kernel, for example. The downside of +that approach is that the programmer needs good domain knowledge. + +If you feel more adventurous and don't want to go completely without high-level code, you can try to +make use of suitable compiler flags such as `-fpic`, and only use `assembly` where absolutely +needed. Here is the [`-fpic` description for GCC]: + +> -fpic +> +> Generate position-independent code (PIC) suitable for use in a shared library, if supported for +> the target machine. Such code accesses all constant addresses through a global offset table (GOT). +> The dynamic loader resolves the GOT entries when the program starts (the dynamic loader is not +> part of GCC; it is part of the operating system). + +[`-fpic` description for GCC]: https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#Code-Gen-Options + +However, it is very important to understand that this flag **is not** a ready-made solution for our +particular problem (and wasn't invented for that case either). There is a hint in the quoted text +above that gives it away: "_The dynamic loader resolves the GOT entries when the program starts (the +dynamic loader is not part of GCC; it is part of the operating system)_". + +Well, we are a booting kernel, and not some (userspace) program running on top of an operating +system. Hence, there is no dynamic loader available. However, it is still possible to benefit from +`-fpic` even in our case. Lets have a look at what happens if we compile the earlier piece of `C` +code for `AArch64` using `-fpic`, still linking the output to the most signifcant part of the memory +space: + +```console +$ clang --target=aarch64-none-elf -Iinclude -Wall -fpic -c start.c -o start.o +$ ld.lld start.o -T link.ld -o example.elf +``` + +```c-objdump +Disassembly of section .text: + +ffff000000000010 get_address_of_global: +ffff000000000010: 00 00 00 90 adrp x0, #0x0 +ffff000000000014: 00 28 40 f9 ldr x0, [x0, #0x50] +ffff000000000018: c0 03 5f d6 ret + +Disassembly of section .got: + +ffff000000000050 .got: +ffff000000000050: 08 00 01 00 +ffff000000000054: 00 00 ff ff + +Disassembly of section .data: + +ffff000000010008 global_data_word: +ffff000000010008: 44 33 22 11 +ffff00000001000c: 00 00 00 00 +``` + +What changed compared to earlier is that `rx_start_address()` now indirects through the `Global +Offset Table`, as has been promised by the compiler's documentation. Specifically, +`rx_start_address()` addresses the `GOT` using PC-relative addressing (distance from code to `GOT` +must always be fixed), and loads the first 64 bit word from the start of the `GOT`, which happens to +be `0xffff_0000_0001_0008`. + +Okay okay okay... So when we use `-fpic`, we get the **absolute** address of `global_data_word` even +on `AArch64` now. How does this help when the code executes from `0x8_0000`? + +Well, this is the part where the `dynamic loader` quoted above would come into picture if this was a +userspace program: "_The dynamic loader resolves the GOT entries when the program starts_". The +`-fpic` flag is normally used to compile _shared libraries_. Suppose we have a program that uses one +or more shared library. For various reasons, it happens that the shared library is loaded at a +different address than where the userspace program would initially expect it. In our example, +`global_data_word` could be supplied by such a shared library, and the userspace program is only +referencing it. The dynamic loader would know where the shared library was loaded into memory, and +therefore know the real address of `global_data_word`. So before the userspace program starts, the +loader would overwrite the `GOT` entry with the correct location. Et voilà, the compiled high-level +code would execute properly. + +### Using PIC during kernel startup + +If you think about it, our problem is a special case of what we just learned. We have a single +statically linked binary, where everything is dislocated by a fixed offset. In our case, it is +`0xffff_0000_0000_0000 - 0x8_0000 = 0x0fff_efff_ffff8_0000`. If we write some PIC-`assembly` code +which loops over the `GOT` and subtracts `0x0fff_efff_ffff8_0000` from every entry as the very first +thing when our kernel boots, any high-level code compiled with `-fpic` would work correctly +afterwards. + +Moreover, this approach would be portable! Here's the output of our code compiled with `-fpic` for +`x86_64`: + +```c-objdump +Disassembly of section .text: + +ffff000000000070 get_address_of_global: +ffff000000000070: 55 push rbp +ffff000000000071: 48 89 e5 mov rbp, rsp +ffff000000000074: 48 8b 05 2d 00 00 00 mov rax, qword ptr [rip + 0x2d] +ffff00000000007b: 5d pop rbp +ffff00000000007c: c3 ret +ffff00000000007d: 0f 1f 00 nop dword ptr [rax] + +Disassembly of section .got: + +ffff0000000000a8 .got: +ffff0000000000a8: 08 00 01 00 +ffff0000000000ac: 00 00 ff ff + +Disassembly of section .data: + +ffff000000010008 global_data_word: +ffff000000010008: 44 33 22 11 +ffff00000001000c: 00 00 00 00 +``` + +As you can see, the `x86_64` code indirects through the `GOT` now same as the `AArch64` code. + +Of course, indirecting through the `GOT` would be detrimental to performance, so you would restrict +`-fpic` compilation only to the code that is needed to enable the `MMU`. Everything else can be +compiled `non-relocatable` as always, because the translation tables naturally resolve the **load +address != link address** situation once they are live. + +With `C/C++` compilers, this can be done rather easily. The compilers support compilation of +PIC-code on a per-[translation-unit] basis. Think of it as telling the compiler to compile this `.c` +file as `PIC`, but this other `.c` file not. + +[translation-unit]: https://en.wikipedia.org/wiki/Translation_unit_(programming) + +With `Rust`, unfortunately, the [relocation model] can only be set on a per-`crate` basis at the +moment (IINM), so that makes it difficult for us to put this approach to use. + +[relocation model]: https://doc.rust-lang.org/rustc/codegen-options/index.html#relocation-model + +## Precomputed Translation Tables + +As we have just seen, going the `-fpic` way isn't a feasible solution at the time of writing this +text. On the other hand, writing the code to set up the initial page tables in `assembly` isn't that +attractive either, because writing larger pieces of assembly is an error-prone and delicate task. + +Fortunately, there is a third way. We are writing an embedded kernel, and therefore the execution +environment is way more static and deterministic as compared to a general-purpose kernel that can be +deployed on a wide variety of targets. Specifically, for the Raspberrypi, we exactly know the **load +address** of the kernel in advance, and we know about the capabilites of the `MMU`. So there is +nothing stopping us from precomputing the kernel's translation tables ahead of time. + +A disadvantage of this approach is an increased binary size, but this is not a deal breaker in our +case. + +## Implementation + +As stated in the initial `tl;dr`, we're not yet mapping the kernel to the most significant area of +virtual memory. This tutorial will keep the binary `identity-mapped`, and focuses only on the +infrastructure changes which enable the kernel to use `precomputed translation tables`. The actual +switch to high memory will happen in the next tutorial. + +The changes needed are as follows: + +1. Make preparations so that precomputed tables are supported by the kernel's memory subsystem code. +2. Change the boot code of the kernel so that the `MMU` is enabled with the precomputed tables as + soon as possible. +3. Write a `translation table tool` that precomputes the translation tables from the generated + `kernel.elf` file, and patches the tables back into the same. + +### Preparing the Kernel Tables + +The tables must be linked into the `.data` section now so that they become part of the final binary. +This is ensured using an attribute on the table's instance definition in +`bsp/__board_name__/memory/mmu.rs`: + +```rust +#[link_section = ".data"] +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new_for_precompute()); +``` + +The `new_for_precompute()` is a new constructor in the the respective `_arch` code that ensures some +struct members that are not the translation table entries themselves are initialized properly for +the precompute use-case. + +In the `BSP` code, there is also a new file called `kernel_virt_addr_space_size.ld`, which contains +the kernel's virtual address space size. This file gets included in both, the `link.ld` linker +script and `mmu.rs`. We need this value both as a symbol in the kernel's ELF (for the `translation +table tool` to parse it later) and as a constant in the `Rust` code. This inclusion approach is just +a convenience hack that turned out working well. + +One critical parameter that the kernel's boot code needs in order to enable the precomputed tables +is the `translation table base address` which must be programmed into the MMU's `TTBR` register. To +make it accessible easily, it is added to the `.text._start_arguments` section. The definition is +just below the definition of the kernel table instance in the `BSP` code: + +```rust +/// This value is needed during early boot for MMU setup. +/// +/// This will be patched to the correct value by the "translation table tool" after linking. This +/// given value here is just a dummy. +#[link_section = ".text._start_arguments"] +#[no_mangle] +static PHYS_KERNEL_TABLES_BASE_ADDR: u64 = 0xCCCCAAAAFFFFEEEE; +``` + +### Turning on the MMU Before Switching to EL1 + +Since the Raspberry Pi starts execution in the `EL2` privilege level, one of the first things we do +during boot since `tutorial 09` is to context-switch to the appropriate `EL1`. The `EL2` boot code +is a great place to set up virtual memory for `EL1`. It will allow execution in `EL1` to start with +virtual memory enabled since the very first instruction. The tweaks to `boot.s` are minimal: + +```asm +// Load the base address of the kernel's translation tables. +ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + +// Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. +ADR_REL x1, __boot_core_stack_end_exclusive +mov sp, x1 + +// Jump to Rust code. x0 and x1 hold the function arguments provided to _start_rust(). +b _start_rust +``` + +In addition to the stack's address, we are now reading _the value_ of +`PHYS_KERNEL_TABLES_BASE_ADDR`. The `ldr` instruction addresses the value-to-be-read using a +PC-relative offset, so this is a `position-independent` operation and will therefore be future +proof. The retrieved value is supplied as an argument to function `_start_rust()`, which is defined +in `_arch/__arch_name__/cpu/boot.rs`: + +```rust +#[no_mangle] +pub unsafe extern "C" fn _start_rust( + phys_kernel_tables_base_addr: u64, + phys_boot_core_stack_end_exclusive_addr: u64, +) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Turn on the MMU for EL1. + let addr = Address::new(phys_kernel_tables_base_addr as usize); + if unlikely(memory::mmu::enable_mmu_and_caching(addr).is_err()) { + cpu::wait_forever(); + } + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} +``` + +You can also see that we now turn on the `MMU` just before returning to `EL1`. That's basically it +already, the only missing piece that's left is the offline computation of the translation tables. + +### The Translation Table Tool + +The tool for translation table computation is located in the folder `translation_table_tool` in the +root directory. For ease of use, it is written in `Ruby` 💎. The code is organized into `BSP` and +`arch` parts just like the kernel's `Rust` code: + +```console +$ tree translation_table_tool + +translation_table_tool +├── arch.rb +├── bsp.rb +├── generic.rb +└── main.rb + +0 directories, 4 files +``` + +Especially the `arch` part, which deals with compiling the translation table entries, will contain +some overlap with the `Rust` code present in `_arch/aarch64/memory/mmu/translation_table.rs`. It +might have been possible to write this tool in Rust as well, and borrow/share these pieces of code +with the kernel. But in the end, I found it not worth the effort for the few lines of code. + +In the `Makefile`, the tool is invoked after compiling and linking the kernel, and before the +`objcopy`. It's command line arguments are the target `BSP` type and the path to the kernel's `ELF` +file: + +```Makefile +all: $(KERNEL_BIN) + +$(KERNEL_ELF): + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + +$(KERNEL_BIN): $(KERNEL_ELF) + @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +``` + +In `main.rb`, the `BSP`-part of the tool is executed first. This part is mostly concerned with +parsing symbols from the ELF file that are needed for the computation of the tables. Here is an +excerpt of the symbols in question: + +```ruby +class RaspberryPi + + # omitted + + def initialize(kernel_elf) + # omitted + + @virt_addresses = { + boot_core_stack_start: /__boot_core_stack_start/, + boot_core_stack_end_exclusive: /__boot_core_stack_end_exclusive/, + + rx_start: /__rx_start/, + rx_end_exclusive: /__rx_end_exclusive/, + + rw_start: /__rw_start/, + rw_end_exclusive: /__rw_end_exclusive/, + + table_struct_start_addr: /bsp::.*::memory::mmu::KERNEL_TABLES/, + phys_tables_base_addr: /PHYS_KERNEL_TABLES_BASE_ADDR/ + } +``` + +Using those values, the `BSP` code goes on to create descriptors for each of the three sections that +whose mappings need to be computed: + +- The RX section. +- The RW section. +- The boot-core's stack. + +After the `BSP` setup, `main.rb` creates an instance of the translation tables and the kernel binary +is mapped using the same: + +```ruby +TRANSLATION_TABLES = case TARGET + when :aarch64 + Arch::ARMv8::TranslationTable.new + else + raise + end + +BSP.kernel_map_binary +``` + +As initially said, the `arch` code works rather similar to the mapping code in the actual kernel's +driver. `BSP.kernel_map_binary` internally calls `map_pages_at` of the `TRANSLATION_TABLE` instance: + +```ruby +def kernel_map_binary + MappingDescriptor.print_header + + @descriptors.each do |i| + print 'Generating'.rjust(12).green.bold + print ' ' + puts i.to_s + + TRANSLATION_TABLES.map_pages_at(i.virt_pages, i.phys_pages, i.attributes) + end + + MappingDescriptor.print_divider +end +``` + +For `map_pages_at()` to work, the `arch` code needs knowledge about **location** and **layout** of +the kernels table structure. Location will be queried from the `BSP` code, which itself retrieves it +during the symbol parsing. The layout, on the other hand, is hardcoded and as such must be kept in +sync with the structure definition in `translation_tables.rs`. + +Finally, after the mappings have been created, it is time to _patch_ them back into the kernel ELF +file. This is initiated from `main.rb` again: + +```ruby +kernel_patch_tables(kernel_elf) +kernel_patch_base_addr(kernel_elf) +``` + +The tool prints some information on the fly. Here's the console output of a successful run: + +```console +$ make + +Compiling kernel - rpi3 + Finished release [optimized] target(s) in 0.01s + +Precomputing kernel translation tables and patching kernel ELF + -------------------------------------------------- + Section Start Virt Addr Size + -------------------------------------------------- + Generating Code and RO data | 0x0000_0000_0008_0000 | 64 KiB + Generating Data and bss | 0x0000_0000_0009_0000 | 384 KiB + Generating Boot-core stack | 0x0000_0000_0010_0000 | 512 KiB + -------------------------------------------------- + Patching Kernel table struct at physical 0x9_0000 + Patching Value of kernel table physical base address (0xd_0000) at physical 0x8_0040 + Finished in 0.03s +``` + +Please note how only the kernel's binary is precomputed! Thanks to the changes made in the last +tutorial, anything else, like `MMIO-remapping`, can and will happen lazily during runtime. + +### Other changes + +Two more things that changed in this tutorial, but won't be explained in detail: + +- Since virtual memory in `EL1` is now active from the start, any attempt to convert from + `Address` to `Address` is now done through the `TryFrom` trait, which + internally uses HW-supported address translation of the CPU. So either the translation succeeds + because there is a valid virtual-to-physical mapping in the currently used translation tables, or + an `Err()` is returned. +- The precomputed translation table mappings won't automatically have entries in the kernel's + `mapping info record`, which is used to print mapping info during boot. Mapping record entries are + not computed offline in order to reduce complexity. For this reason, the `BSP` code, which in + earlier tutorials would have called `kernel_map_pages_at()` (which implicitly would have generated + mapping record entries), now only calls `kernel_add_mapping_record()`, since the mappings are + already in place. + +## Discussion + +It is understood that there is room for optimizations in the presented approach. For example, the +generated kernel binary contains the _complete_ array of translation tables for the whole kernel +virtual address space (`2 GiB`). However, most of the entries are empty initially, because the +kernel binary only occupies a small area in the tables. It would make sense to add some smarts so +that only the non-zero entries are packed into binary. + +On the other hand, this would add complexity to the code. The increased size doesn't hurt too much +yet, so the reduced complexity in the code is a tradeoff made willingly to keep everything concise +and focused on the high-level concepts. + +## Test it + +```console +$ make chainboot +[...] + +Precomputing kernel translation tables and patching kernel ELF + -------------------------------------------------- + Section Start Virt Addr Size + -------------------------------------------------- + Generating Code and RO data | 0x0000_0000_0008_0000 | 64 KiB + Generating Data and bss | 0x0000_0000_0009_0000 | 384 KiB + Generating Boot-core stack | 0x0000_0000_0010_0000 | 512 KiB + -------------------------------------------------- + Patching Kernel table struct at physical 0x9_0000 + Patching Value of kernel table physical base address (0xd_0000) at physical 0x8_0040 + Finished in 0.03s + +Minipush 1.0 + +[MP] ⏳ Waiting for /dev/ttyUSB0 +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now + __ __ _ _ _ _ +| \/ (_)_ _ (_) | ___ __ _ __| | +| |\/| | | ' \| | |__/ _ \/ _` / _` | +|_| |_|_|_||_|_|____\___/\__,_\__,_| + + Raspberry Pi 3 + +[ML] Requesting binary +[MP] ⏩ Pushing 387 KiB =======================================🦀 100% 96 KiB/s Time: 00:00:04 +[ML] Loaded! Executing the payload now + +[ 4.319874] Booting on: Raspberry Pi 3 +[ 4.320147] MMU online: +[ 4.320439] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.322183] Virtual Physical Size Attr Entity +[ 4.323927] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.325674] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 4.327288] 0x0000_0000_0009_0000..0x0000_0000_000e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss +[ 4.328858] 0x0000_0000_0010_0000..0x0000_0000_0017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 4.330461] 0x0000_0000_7000_0000..0x0000_0000_7000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 4.331912] | BCM PL011 UART +[ 4.333430] 0x0000_0000_7001_0000..0x0000_0000_7001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 4.335173] ------------------------------------------------------------------------------------------------------------------------------------------- +``` + +## Diff to previous +```diff + +diff -uNr 14_virtual_mem_part2_mmio_remap/Makefile 15_virtual_mem_part3_precomputed_tables/Makefile +--- 14_virtual_mem_part2_mmio_remap/Makefile ++++ 15_virtual_mem_part3_precomputed_tables/Makefile +@@ -112,6 +112,7 @@ + $(KERNEL_ELF): + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) ++ @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + + $(KERNEL_BIN): $(KERNEL_ELF) + @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +@@ -134,6 +135,7 @@ + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + ++ $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY + endef + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +--- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +@@ -11,7 +11,8 @@ + //! + //! crate::cpu::boot::arch_boot + +-use crate::runtime_init; ++use crate::{cpu, memory, memory::Address, runtime_init}; ++use core::intrinsics::unlikely; + use cortex_a::{asm, regs::*}; + + // Assembly counterpart to this file. +@@ -71,9 +72,18 @@ + /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. + /// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. + #[no_mangle] +-pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { ++pub unsafe extern "C" fn _start_rust( ++ phys_kernel_tables_base_addr: u64, ++ phys_boot_core_stack_end_exclusive_addr: u64, ++) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + ++ // Turn on the MMU for EL1. ++ let addr = Address::new(phys_kernel_tables_base_addr as usize); ++ if unlikely(memory::mmu::enable_mmu_and_caching(addr).is_err()) { ++ cpu::wait_forever(); ++ } ++ + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() + } + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s +--- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s ++++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s +@@ -44,11 +44,14 @@ + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + ++ // Load the base address of the kernel's translation tables. ++ ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs ++ + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. +- ADR_REL x0, __boot_core_stack_end_exclusive +- mov sp, x0 ++ ADR_REL x1, __boot_core_stack_end_exclusive ++ mov sp, x1 + +- // Jump to Rust code. x0 holds the function argument provided to _start_rust(). ++ // Jump to Rust code. x0 and x1 hold the function arguments provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs +@@ -23,7 +23,7 @@ + Address, Physical, Virtual, + }, + }; +-use core::convert; ++use core::convert::{self, TryInto}; + use register::{register_bitfields, InMemoryRegister}; + + //-------------------------------------------------------------------------------------------------- +@@ -120,7 +120,7 @@ + } + + trait StartAddr { +- fn phys_start_addr(&self) -> Address; ++ fn virt_start_addr(&self) -> Address; + } + + //-------------------------------------------------------------------------------------------------- +@@ -149,9 +149,8 @@ + // Private Code + //-------------------------------------------------------------------------------------------------- + +-// The binary is still identity mapped, so we don't need to convert here. + impl StartAddr for [T; N] { +- fn phys_start_addr(&self) -> Address { ++ fn virt_start_addr(&self) -> Address { + Address::new(self as *const _ as usize) + } + } +@@ -269,7 +268,7 @@ + + /// Create an instance. + #[allow(clippy::assertions_on_constants)] +- pub const fn new() -> Self { ++ const fn _new(for_precompute: bool) -> Self { + assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); + + // Can't have a zero-sized address space. +@@ -278,11 +277,20 @@ + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], +- cur_l3_mmio_index: 0, +- initialized: false, ++ cur_l3_mmio_index: Self::L3_MMIO_START_INDEX, ++ initialized: for_precompute, + } + } + ++ pub const fn new_for_precompute() -> Self { ++ Self::_new(true) ++ } ++ ++ #[cfg(test)] ++ pub fn new_for_runtime() -> Self { ++ Self::_new(false) ++ } ++ + /// The start address of the table's MMIO range. + #[inline(always)] + fn mmio_start_addr(&self) -> Address { +@@ -338,24 +346,26 @@ + impl memory::mmu::translation_table::interface::TranslationTable + for FixedSizeTranslationTable + { +- fn init(&mut self) { ++ fn init(&mut self) -> Result<(), &'static str> { + if self.initialized { +- return; ++ return Ok(()); + } + + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { +- let desc = +- TableDescriptor::from_next_lvl_table_addr(self.lvl3[lvl2_nr].phys_start_addr()); ++ let addr = self.lvl3[lvl2_nr] ++ .virt_start_addr() ++ .try_into() ++ .map_err(|_| "Translation error")?; ++ ++ let desc = TableDescriptor::from_next_lvl_table_addr(addr); + *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; +- } + +- fn phys_base_address(&self) -> Address { +- self.lvl2.phys_start_addr() ++ Ok(()) + } + + unsafe fn map_pages_at( + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs +--- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs +@@ -15,7 +15,7 @@ + + use crate::{ + bsp, memory, +- memory::{mmu::TranslationGranule, Address, Physical}, ++ memory::{mmu::TranslationGranule, Address, Physical, Virtual}, + }; + use core::intrinsics::unlikely; + use cortex_a::{barrier, regs::*}; +@@ -108,7 +108,7 @@ + //------------------------------------------------------------------------------ + // OS Interface Code + //------------------------------------------------------------------------------ +-use memory::mmu::MMUEnableError; ++use memory::mmu::{MMUEnableError, TranslationError}; + + impl memory::mmu::interface::MMU for MemoryManagementUnit { + unsafe fn enable_mmu_and_caching( +@@ -152,4 +152,31 @@ + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } ++ ++ fn try_virt_to_phys( ++ &self, ++ virt: Address, ++ ) -> Result, TranslationError> { ++ if !self.is_enabled() { ++ return Err(TranslationError::MMUDisabled); ++ } ++ ++ let addr = virt.into_usize() as u64; ++ unsafe { ++ asm!( ++ "AT S1E1R, {0}", ++ in(reg) addr, ++ options(readonly, nostack, preserves_flags) ++ ); ++ } ++ ++ let par_el1 = PAR_EL1.extract(); ++ if par_el1.matches_all(PAR_EL1::F::TranslationAborted) { ++ return Err(TranslationError::Aborted); ++ } ++ ++ let phys_addr = (par_el1.read(PAR_EL1::PA) << 12) | (addr & 0xFFF); ++ ++ Ok(Address::new(phys_addr as usize)) ++ } + } + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs +--- 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs +@@ -22,6 +22,7 @@ + /// # Safety + /// + /// - Use only for printing during a panic. ++#[cfg(not(feature = "test_build"))] + pub unsafe fn panic_console_out() -> impl fmt::Write { + use driver::interface::DeviceDriver; + +@@ -45,6 +46,23 @@ + panic_uart + } + ++/// Reduced version for test builds. ++#[cfg(feature = "test_build")] ++pub unsafe fn panic_console_out() -> impl fmt::Write { ++ use driver::interface::DeviceDriver; ++ ++ let mut panic_uart = ++ device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START.into_usize()); ++ ++ let maybe_uart_mmio_start_addr = super::PL011_UART.virt_mmio_start_addr(); ++ ++ panic_uart ++ .init(maybe_uart_mmio_start_addr) ++ .unwrap_or_else(|_| cpu::qemu_exit_failure()); ++ ++ panic_uart ++} ++ + /// Return a reference to the console. + pub fn console() -> &'static impl console::interface::All { + &super::PL011_UART +@@ -56,6 +74,15 @@ + + /// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps + /// than on real hardware due to QEMU's abstractions. +-/// +-/// For the RPi, nothing needs to be done. +-pub fn qemu_bring_up_console() {} ++#[cfg(feature = "test_build")] ++pub fn qemu_bring_up_console() { ++ use driver::interface::DeviceDriver; ++ ++ // Calling the UART's init ensures that the BSP's instance of the UART does remap the MMIO ++ // addresses. ++ unsafe { ++ super::PL011_UART ++ .init() ++ .unwrap_or_else(|_| cpu::qemu_exit_failure()); ++ } ++} + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld +--- 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld ++++ 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld +@@ -0,0 +1 @@ ++__kernel_virt_addr_space_size = 2 * 1024 * 1024 * 1024 + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld +--- 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld ++++ 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld +@@ -3,6 +3,9 @@ + * Copyright (c) 2018-2021 Andre Richter + */ + ++/* This file provides __kernel_virt_addr_space_size */ ++INCLUDE src/bsp/raspberrypi/kernel_virt_addr_space_size.ld; ++ + /* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ + __rpi_load_addr = 0x80000; + + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs +--- 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs +@@ -16,6 +16,7 @@ + }, + synchronization::InitStateLock, + }; ++use core::convert::TryInto; + + //-------------------------------------------------------------------------------------------------- + // Private Definitions +@@ -33,7 +34,7 @@ + pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; + + /// The kernel's virtual address space defined by this BSP. +-pub type KernelVirtAddrSpace = AddressSpace<{ 8 * 1024 * 1024 * 1024 }>; ++pub type KernelVirtAddrSpace = AddressSpace<{ get_virt_addr_space_size() }>; + + //-------------------------------------------------------------------------------------------------- + // Global instances +@@ -45,13 +46,33 @@ + /// + /// That is, `size_of(InitStateLock) == size_of(KernelTranslationTable)`. + /// There is a unit tests that checks this porperty. ++#[link_section = ".data"] + static KERNEL_TABLES: InitStateLock = +- InitStateLock::new(KernelTranslationTable::new()); ++ InitStateLock::new(KernelTranslationTable::new_for_precompute()); ++ ++/// This value is needed during early boot for MMU setup. ++/// ++/// This will be patched to the correct value by the "translation table tool" after linking. This ++/// given value here is just a dummy. ++#[link_section = ".text._start_arguments"] ++#[no_mangle] ++static PHYS_KERNEL_TABLES_BASE_ADDR: u64 = 0xCCCCAAAAFFFFEEEE; + + //-------------------------------------------------------------------------------------------------- + // Private Code + //-------------------------------------------------------------------------------------------------- + ++/// This is a hack for retrieving the value for the kernel's virtual address space size as a ++/// constant from a common place, since it is needed as a compile-time/link-time constant in both, ++/// the linker script and the Rust sources. ++const fn get_virt_addr_space_size() -> usize { ++ let __kernel_virt_addr_space_size; ++ ++ include!("../kernel_virt_addr_space_size.ld"); ++ ++ __kernel_virt_addr_space_size ++} ++ + /// Helper function for calculating the number of pages the given parameter spans. + const fn size_to_num_pages(size: usize) -> usize { + assert!(size > 0); +@@ -81,21 +102,22 @@ + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) + } + +-// The binary is still identity mapped, so we don't need to convert in the following. ++// There is no reason to expect the following conversions to fail, since they were generated offline ++// by the `translation table tool`. If it doesn't work, a panic due to the unwrap is justified. + + /// The Read+Execute (RX) pages of the kernel binary. + fn phys_rx_page_desc() -> PageSliceDescriptor { +- virt_rx_page_desc().into() ++ virt_rx_page_desc().try_into().unwrap() + } + + /// The Read+Write (RW) pages of the kernel binary. + fn phys_rw_page_desc() -> PageSliceDescriptor { +- virt_rw_page_desc().into() ++ virt_rw_page_desc().try_into().unwrap() + } + + /// The boot core's stack. + fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { +- virt_boot_core_stack_page_desc().into() ++ virt_boot_core_stack_page_desc().try_into().unwrap() + } + + //-------------------------------------------------------------------------------------------------- +@@ -122,13 +144,15 @@ + ) as *const Page<_> + } + +-/// Map the kernel binary. ++/// Add mapping records for the kernel binary. + /// +-/// # Safety ++/// The actual translation table entries for the kernel binary are generated using the offline ++/// `translation table tool` and patched into the kernel binary. This function just adds the mapping ++/// record entries. + /// +-/// - Any miscalculation or attribute error will likely be fatal. Needs careful manual checking. +-pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { +- generic_mmu::kernel_map_pages_at( ++/// It must be ensured that these entries are in sync with the offline tool. ++pub fn kernel_add_mapping_records_for_precomputed() { ++ generic_mmu::kernel_add_mapping_record( + "Kernel code and RO data", + &virt_rx_page_desc(), + &phys_rx_page_desc(), +@@ -137,9 +161,9 @@ + acc_perms: AccessPermissions::ReadOnly, + execute_never: false, + }, +- )?; ++ ); + +- generic_mmu::kernel_map_pages_at( ++ generic_mmu::kernel_add_mapping_record( + "Kernel data and bss", + &virt_rw_page_desc(), + &phys_rw_page_desc(), +@@ -148,9 +172,9 @@ + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, +- )?; ++ ); + +- generic_mmu::kernel_map_pages_at( ++ generic_mmu::kernel_add_mapping_record( + "Kernel boot-core stack", + &virt_boot_core_stack_page_desc(), + &phys_boot_core_stack_page_desc(), +@@ -159,64 +183,5 @@ + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, +- )?; +- +- Ok(()) +-} +- +-//-------------------------------------------------------------------------------------------------- +-// Testing +-//-------------------------------------------------------------------------------------------------- +- +-#[cfg(test)] +-mod tests { +- use super::*; +- use test_macros::kernel_test; +- +- /// Check alignment of the kernel's virtual memory layout sections. +- #[kernel_test] +- fn virt_mem_layout_sections_are_64KiB_aligned() { +- for i in [ +- virt_rx_page_desc, +- virt_rw_page_desc, +- virt_boot_core_stack_page_desc, +- ] +- .iter() +- { +- let start: usize = i().start_addr().into_usize(); +- let end: usize = i().end_addr().into_usize(); +- +- assert_eq!(start modulo KernelGranule::SIZE, 0); +- assert_eq!(end modulo KernelGranule::SIZE, 0); +- assert!(end >= start); +- } +- } +- +- /// Ensure the kernel's virtual memory layout is free of overlaps. +- #[kernel_test] +- fn virt_mem_layout_has_no_overlaps() { +- let layout = [ +- virt_rx_page_desc(), +- virt_rw_page_desc(), +- virt_boot_core_stack_page_desc(), +- ]; +- +- for (i, first_range) in layout.iter().enumerate() { +- for second_range in layout.iter().skip(i + 1) { +- assert!(!first_range.contains(second_range.start_addr())); +- assert!(!first_range.contains(second_range.end_addr_inclusive())); +- assert!(!second_range.contains(first_range.start_addr())); +- assert!(!second_range.contains(first_range.end_addr_inclusive())); +- } +- } +- } +- +- /// Check if KERNEL_TABLES is in .bss. +- #[kernel_test] +- fn kernel_tables_in_bss() { +- let bss_range = super::super::bss_range_inclusive(); +- let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; +- +- assert!(bss_range.contains(&kernel_tables_addr)); +- } ++ ); + } + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/main.rs 15_virtual_mem_part3_precomputed_tables/src/main.rs +--- 14_virtual_mem_part2_mmio_remap/src/main.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/main.rs +@@ -18,25 +18,16 @@ + /// # Safety + /// + /// - Only a single core must be active and running this function. +-/// - The init calls in this function must appear in the correct order: +-/// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, +-/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +-/// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. ++/// - Printing will not work until the respective driver's MMIO is remapped. + #[no_mangle] + unsafe fn kernel_init() -> ! { + use driver::interface::DriverManager; + + exception::handling_init(); + +- let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { +- Err(string) => panic!("Error mapping kernel binary: {}", string), +- Ok(addr) => addr, +- }; +- +- if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { +- panic!("Enabling MMU failed: {}", e); +- } +- // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. ++ // Add the mapping records for the precomputed entries first, so that they appear on the top of ++ // the list. ++ bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); + + // Bring up the drivers needed for printing first. + for i in bsp::driver::driver_manager() +@@ -47,7 +38,7 @@ + i.init().unwrap_or_else(|_| cpu::wait_forever()); + } + bsp::driver::driver_manager().post_early_print_device_driver_init(); +- // Printing available again from here on. ++ // Printing available from here on. + + // Now bring up the remaining drivers. + for i in bsp::driver::driver_manager() + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs +--- 14_virtual_mem_part2_mmio_remap/src/memory/mmu/translation_table.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs +@@ -10,7 +10,7 @@ + + use crate::memory::{ + mmu::{AttributeFields, PageSliceDescriptor}, +- Address, Physical, Virtual, ++ Physical, Virtual, + }; + + //-------------------------------------------------------------------------------------------------- +@@ -35,10 +35,7 @@ + /// + /// - Implementor must ensure that this function can run only once or is harmless if invoked + /// multiple times. +- fn init(&mut self); +- +- /// The translation table's base address to be used for programming the MMU. +- fn phys_base_address(&self) -> Address; ++ fn init(&mut self) -> Result<(), &'static str>; + + /// Map the given virtual pages to the given physical pages. + /// +@@ -80,7 +77,7 @@ + #[cfg(test)] + mod tests { + use super::*; +- use crate::bsp; ++ use crate::{bsp, memory::Address}; + use arch_translation_table::MinSizeTranslationTable; + use interface::TranslationTable; + use test_macros::kernel_test; +@@ -89,9 +86,9 @@ + #[kernel_test] + fn translationtable_implementation_sanity() { + // This will occupy a lot of space on the stack. +- let mut tables = MinSizeTranslationTable::new(); ++ let mut tables = MinSizeTranslationTable::new_for_runtime(); + +- tables.init(); ++ assert!(tables.init().is_ok()); + + let x = tables.next_mmio_virt_page_slice(0); + assert!(x.is_err()); + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs +--- 14_virtual_mem_part2_mmio_remap/src/memory/mmu/types.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs +@@ -8,7 +8,10 @@ + bsp, common, + memory::{Address, AddressType, Physical, Virtual}, + }; +-use core::{convert::From, marker::PhantomData}; ++use core::{ ++ convert::{From, TryFrom}, ++ marker::PhantomData, ++}; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions +@@ -136,12 +139,16 @@ + } + } + +-impl From> for PageSliceDescriptor { +- fn from(desc: PageSliceDescriptor) -> Self { +- Self { +- start: Address::new(desc.start.into_usize()), ++impl TryFrom> for PageSliceDescriptor { ++ type Error = super::TranslationError; ++ ++ fn try_from(desc: PageSliceDescriptor) -> Result { ++ let phys_start = super::try_virt_to_phys(desc.start)?; ++ ++ Ok(Self { ++ start: phys_start, + num_pages: desc.num_pages, +- } ++ }) + } + } + + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs +--- 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs +@@ -33,6 +33,14 @@ + Other(&'static str), + } + ++/// Translation error variants. ++#[allow(missing_docs)] ++#[derive(Debug)] ++pub enum TranslationError { ++ MMUDisabled, ++ Aborted, ++} ++ + /// Memory Management interfaces. + pub mod interface { + use super::*; +@@ -51,6 +59,14 @@ + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; ++ ++ /// Try to translate a virtual address to a physical address. ++ /// ++ /// Will only succeed if there exists a valid mapping for the input VA. ++ fn try_virt_to_phys( ++ &self, ++ virt: Address, ++ ) -> Result, TranslationError>; + } + } + +@@ -92,9 +108,7 @@ + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; + +- if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { +- warn!("{}", x); +- } ++ kernel_add_mapping_record(name, virt_pages, phys_pages, attr); + + Ok(()) + } +@@ -146,6 +160,18 @@ + } + } + ++/// Add an entry to the mapping info record. ++pub fn kernel_add_mapping_record( ++ name: &'static str, ++ virt_pages: &PageSliceDescriptor, ++ phys_pages: &PageSliceDescriptor, ++ attr: &AttributeFields, ++) { ++ if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { ++ warn!("{}", x); ++ } ++} ++ + /// Raw mapping of virtual to physical pages in the kernel translation tables. + /// + /// Prevents mapping into the MMIO range of the tables. +@@ -214,21 +240,11 @@ + Ok(virt_addr + offset_into_start_page) + } + +-/// Map the kernel's binary. Returns the translation table's base address. +-/// +-/// # Safety ++/// Try to translate a virtual address to a physical address. + /// +-/// - See [`bsp::memory::mmu::kernel_map_binary()`]. +-pub unsafe fn kernel_map_binary() -> Result, &'static str> { +- let phys_kernel_tables_base_addr = +- bsp::memory::mmu::kernel_translation_tables().write(|tables| { +- tables.init(); +- tables.phys_base_address() +- }); +- +- bsp::memory::mmu::kernel_map_binary()?; +- +- Ok(phys_kernel_tables_base_addr) ++/// Will only succeed if there exists a valid mapping for the input VA. ++pub fn try_virt_to_phys(virt: Address) -> Result, TranslationError> { ++ arch_mmu::mmu().try_virt_to_phys(virt) + } + + /// Enable the MMU and data + instruction caching. +@@ -236,6 +252,7 @@ + /// # Safety + /// + /// - Crucial function during kernel init. Changes the the complete memory view of the processor. ++#[inline(always)] + pub unsafe fn enable_mmu_and_caching( + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError> { + +diff -uNr 14_virtual_mem_part2_mmio_remap/src/memory.rs 15_virtual_mem_part3_precomputed_tables/src/memory.rs +--- 14_virtual_mem_part2_mmio_remap/src/memory.rs ++++ 15_virtual_mem_part3_precomputed_tables/src/memory.rs +@@ -8,6 +8,7 @@ + + use crate::common; + use core::{ ++ convert::TryFrom, + fmt, + marker::PhantomData, + ops::{AddAssign, RangeInclusive, SubAssign}, +@@ -67,6 +68,14 @@ + } + } + ++impl TryFrom> for Address { ++ type Error = mmu::TranslationError; ++ ++ fn try_from(virt: Address) -> Result { ++ mmu::try_virt_to_phys(virt) ++ } ++} ++ + impl core::ops::Add for Address { + type Output = Self; + + +diff -uNr 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs +--- 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs ++++ 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs +@@ -17,43 +17,16 @@ + /// or indirectly. + mod panic_exit_success; + +-use libkernel::{bsp, cpu, exception, memory, println}; ++use libkernel::{bsp, cpu, exception, println}; + + #[no_mangle] + unsafe fn kernel_init() -> ! { +- use libkernel::driver::interface::DriverManager; +- + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + +- let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { +- Err(string) => { +- println!("Error mapping kernel binary: {}", string); +- cpu::qemu_exit_failure() +- } +- Ok(addr) => addr, +- }; +- +- if let Err(e) = memory::mmu::enable_mmu_and_caching(phys_kernel_tables_base_addr) { +- println!("Enabling MMU failed: {}", e); +- cpu::qemu_exit_failure() +- } +- // Printing will silently fail from here on, because the driver's MMIO is not remapped yet. +- +- // Bring up the drivers needed for printing first. +- for i in bsp::driver::driver_manager() +- .early_print_device_drivers() +- .iter() +- { +- // Any encountered errors cannot be printed yet, obviously, so just safely park the CPU. +- i.init().unwrap_or_else(|_| cpu::qemu_exit_failure()); +- } +- bsp::driver::driver_manager().post_early_print_device_driver_init(); +- // Printing available again from here on. +- + println!("Writing beyond mapped area to address 9 GiB..."); + let big_addr: u64 = 9 * 1024 * 1024 * 1024; + core::ptr::read_volatile(big_addr as *mut u64); + +diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb +--- 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb ++++ 15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb +@@ -0,0 +1,323 @@ ++# frozen_string_literal: true ++ ++# SPDX-License-Identifier: MIT OR Apache-2.0 ++# ++# Copyright (c) 2021 Andre Richter ++ ++# Bitfield manipulation. ++class BitField ++ def initialize ++ @value = 0 ++ end ++ ++ def self.attr_bitfield(name, offset, num_bits) ++ define_method("#{name}=") do |bits| ++ mask = 2**num_bits - 1 ++ ++ raise "Input out of range: #{name} = 0x#{bits.to_s(16)}" if (bits & ~mask).positive? ++ ++ # Clear bitfield ++ @value &= ~(mask << offset) ++ ++ # Set it ++ @value |= (bits << offset) ++ end ++ end ++ ++ def to_i ++ @value ++ end ++ ++ def size_in_byte ++ 8 ++ end ++end ++ ++# An array class that knows its memory location. ++class CArray < Array ++ attr_reader :phys_start_addr ++ ++ def initialize(phys_start_addr, size, &block) ++ @phys_start_addr = phys_start_addr ++ ++ super(size, &block) ++ end ++ ++ def size_in_byte ++ inject(0) { |sum, n| sum + n.size_in_byte } ++ end ++end ++ ++#--------------------------------------------------------------------------------------------------- ++# Arch:: ++#--------------------------------------------------------------------------------------------------- ++module Arch ++FALSE = 0b0 ++TRUE = 0b1 ++ ++#--------------------------------------------------------------------------------------------------- ++# Arch::ARMv8 ++#--------------------------------------------------------------------------------------------------- ++module ARMv8 ++# ARMv8 Table Descriptor. ++class Stage1TableDescriptor < BitField ++ module NextLevelTableAddr ++ OFFSET = 16 ++ NUMBITS = 32 ++ end ++ ++ module Type ++ OFFSET = 1 ++ NUMBITS = 1 ++ ++ BLOCK = 0 ++ TABLE = 1 ++ end ++ ++ module Valid ++ OFFSET = 0 ++ NUMBITS = 1 ++ end ++ ++ attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) ++ attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) ++ attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) ++ ++ def next_level_table_addr=(addr) ++ addr = addr >> Granule64KiB::SHIFT ++ ++ self.__next_level_table_addr = addr ++ end ++ ++ private :__next_level_table_addr= ++end ++ ++# ARMv8 level 3 page descriptor. ++class Stage1PageDescriptor < BitField ++ module UXN ++ OFFSET = 54 ++ NUMBITS = 1 ++ end ++ ++ module PXN ++ OFFSET = 53 ++ NUMBITS = 1 ++ end ++ ++ module OutputAddr ++ OFFSET = 16 ++ NUMBITS = 32 ++ end ++ ++ module AF ++ OFFSET = 10 ++ NUMBITS = 1 ++ end ++ ++ module SH ++ OFFSET = 8 ++ NUMBITS = 2 ++ ++ INNER_SHAREABLE = 0b11 ++ end ++ ++ module AP ++ OFFSET = 6 ++ NUMBITS = 2 ++ ++ RW_EL1 = 0b00 ++ RO_EL1 = 0b10 ++ end ++ ++ module AttrIndx ++ OFFSET = 2 ++ NUMBITS = 3 ++ end ++ ++ module Type ++ OFFSET = 1 ++ NUMBITS = 1 ++ ++ RESERVED_INVALID = 0 ++ PAGE = 1 ++ end ++ ++ module Valid ++ OFFSET = 0 ++ NUMBITS = 1 ++ end ++ ++ attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) ++ attr_bitfield(:pxn, PXN::OFFSET, PXN::NUMBITS) ++ attr_bitfield(:__output_addr, OutputAddr::OFFSET, OutputAddr::NUMBITS) ++ attr_bitfield(:af, AF::OFFSET, AF::NUMBITS) ++ attr_bitfield(:sh, SH::OFFSET, SH::NUMBITS) ++ attr_bitfield(:ap, AP::OFFSET, AP::NUMBITS) ++ attr_bitfield(:attr_indx, AttrIndx::OFFSET, AttrIndx::NUMBITS) ++ attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) ++ attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) ++ ++ def output_addr=(addr) ++ addr = addr >> Granule64KiB::SHIFT ++ ++ self.__output_addr = addr ++ end ++ ++ private :__output_addr= ++end ++ ++# Translation table representing the structure defined in translation_table.rs. ++class TranslationTable ++ MMIO_APERTURE_MiB = 256 * 1024 * 1024 ++ ++ module MAIR ++ NORMAL = 1 ++ end ++ ++ def initialize ++ @virt_mmio_start_addr = (BSP.kernel_virt_addr_space_size - MMIO_APERTURE_MiB) + ++ BSP.kernel_virt_start_addr ++ ++ do_sanity_checks ++ ++ num_lvl2_tables = BSP.kernel_virt_addr_space_size >> Granule512MiB::SHIFT ++ ++ @lvl3 = new_lvl3(num_lvl2_tables, BSP.phys_table_struct_start_addr) ++ ++ @lvl2_phys_start_addr = @lvl3.phys_start_addr + @lvl3.size_in_byte ++ @lvl2 = new_lvl2(num_lvl2_tables, @lvl2_phys_start_addr) ++ ++ populate_lvl2_entries ++ end ++ ++ def map_pages_at(virt_pages, phys_pages, attributes) ++ return if virt_pages.empty? ++ ++ raise if virt_pages.size != phys_pages.size ++ raise if phys_pages.last > BSP.phys_addr_space_end_page ++ ++ virt_pages.zip(phys_pages).each do |virt_page, phys_page| ++ desc = page_descriptor_from(virt_page) ++ set_lvl3_entry(desc, phys_page, attributes) ++ end ++ end ++ ++ def to_binary ++ data = @lvl3.flatten.map(&:to_i) + @lvl2.map(&:to_i) ++ data.pack('Q<*') # "Q" == uint64_t, "<" == little endian ++ end ++ ++ def phys_tables_base_addr_binary ++ [@lvl2_phys_start_addr].pack('Q<*') # "Q" == uint64_t, "<" == little endian ++ end ++ ++ def phys_tables_base_addr ++ @lvl2_phys_start_addr ++ end ++ ++ private ++ ++ def binary_with_mmio_clash? ++ BSP.rw_end_exclusive >= @virt_mmio_start_addr ++ end ++ ++ def do_sanity_checks ++ raise unless BSP.kernel_granule::SIZE == Granule64KiB::SIZE ++ raise unless (BSP.kernel_virt_addr_space_size modulo Granule512MiB::SIZE).zero? ++ ++ # Need to ensure that that the kernel binary does not clash with the upmost 256 MiB of the ++ # virtual address space, which is reserved for runtime-remapping of MMIO. ++ return unless binary_with_mmio_clash? ++ ++ puts format('__data_end_exclusive: 0xmodulo16x', BSP.data_end_exclusive) ++ puts format('MMIO start: 0xmodulo16x', @virt_mmio_start_addr) ++ ++ raise 'Kernel virtual addresses clash with 256 MiB MMIO window' ++ end ++ ++ def new_lvl3(num_lvl2_tables, start_addr) ++ CArray.new(start_addr, num_lvl2_tables) do ++ temp = CArray.new(start_addr, 8192) do ++ Stage1PageDescriptor.new ++ end ++ start_addr += temp.size_in_byte ++ ++ temp ++ end ++ end ++ ++ def new_lvl2(num_lvl2_tables, start_addr) ++ CArray.new(start_addr, num_lvl2_tables) do ++ Stage1TableDescriptor.new ++ end ++ end ++ ++ def populate_lvl2_entries ++ @lvl2.each_with_index do |descriptor, i| ++ descriptor.next_level_table_addr = @lvl3[i].phys_start_addr ++ descriptor.type = Stage1TableDescriptor::Type::TABLE ++ descriptor.valid = TRUE ++ end ++ end ++ ++ def lvl2_lvl3_index_from(addr) ++ addr -= BSP.kernel_virt_start_addr ++ ++ lvl2_index = addr >> Granule512MiB::SHIFT ++ lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT ++ ++ raise unless lvl2_index < @lvl2.size ++ ++ [lvl2_index, lvl3_index] ++ end ++ ++ def page_descriptor_from(virt_addr) ++ lvl2_index, lvl3_index = lvl2_lvl3_index_from(virt_addr) ++ ++ @lvl3[lvl2_index][lvl3_index] ++ end ++ ++ # rubocop:disable Metrics/MethodLength ++ def set_attributes(desc, attributes) ++ case attributes.mem_attributes ++ when :CacheableDRAM ++ desc.sh = Stage1PageDescriptor::SH::INNER_SHAREABLE ++ desc.attr_indx = MAIR::NORMAL ++ else ++ raise 'Invalid input' ++ end ++ ++ desc.ap = case attributes.acc_perms ++ when :ReadOnly ++ Stage1PageDescriptor::AP::RO_EL1 ++ when :ReadWrite ++ Stage1PageDescriptor::AP::RW_EL1 ++ else ++ raise 'Invalid input' ++ ++ end ++ ++ desc.pxn = case attributes.execute_never ++ when :XN ++ TRUE ++ when :X ++ FALSE ++ else ++ raise 'Invalid input' ++ end ++ ++ desc.uxn = TRUE ++ end ++ # rubocop:enable Metrics/MethodLength ++ ++ def set_lvl3_entry(desc, output_addr, attributes) ++ desc.output_addr = output_addr ++ desc.af = TRUE ++ desc.type = Stage1PageDescriptor::Type::PAGE ++ desc.valid = TRUE ++ ++ set_attributes(desc, attributes) ++ end ++end ++end ++end + +diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/bsp.rb 15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb +--- 14_virtual_mem_part2_mmio_remap/translation_table_tool/bsp.rb ++++ 15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb +@@ -0,0 +1,177 @@ ++# frozen_string_literal: true ++ ++# SPDX-License-Identifier: MIT OR Apache-2.0 ++# ++# Copyright (c) 2021 Andre Richter ++ ++# Raspberry Pi 3 + 4 ++class RaspberryPi ++ attr_reader :kernel_granule, :kernel_virt_addr_space_size, :kernel_virt_start_addr ++ ++ NM_BINARY = 'aarch64-none-elf-nm' ++ READELF_BINARY = 'aarch64-none-elf-readelf' ++ MEMORY_SRC = File.read('src/bsp/raspberrypi/memory.rs').split("\n") ++ ++ def initialize(kernel_elf) ++ @kernel_granule = Granule64KiB ++ ++ @virt_addresses = { ++ boot_core_stack_start: /__boot_core_stack_start/, ++ boot_core_stack_end_exclusive: /__boot_core_stack_end_exclusive/, ++ ++ rx_start: /__rx_start/, ++ rx_end_exclusive: /__rx_end_exclusive/, ++ ++ rw_start: /__rw_start/, ++ rw_end_exclusive: /__rw_end_exclusive/, ++ ++ table_struct_start_addr: /bsp::.*::memory::mmu::KERNEL_TABLES/, ++ phys_tables_base_addr: /PHYS_KERNEL_TABLES_BASE_ADDR/ ++ } ++ ++ symbols = `#{NM_BINARY} --demangle #{kernel_elf}`.split("\n") ++ @kernel_virt_addr_space_size = parse_from_symbols(symbols, /__kernel_virt_addr_space_size/) ++ @kernel_virt_start_addr = 0 ++ @virt_addresses = parse_from_symbols(symbols, @virt_addresses) ++ @phys_addresses = virt_to_phys(@virt_addresses) ++ ++ @descriptors = parse_descriptors ++ update_max_descriptor_name_length ++ ++ @text_section_offset_in_elf = parse_text_section_offset_in_elf(kernel_elf) ++ end ++ ++ def rw_end_exclusive ++ @virt_addresses[:rw_end_exclusive] ++ end ++ ++ def phys_table_struct_start_addr ++ @phys_addresses[:table_struct_start_addr] ++ end ++ ++ def table_struct_offset_in_kernel_elf ++ (@virt_addresses[:table_struct_start_addr] - @virt_addresses[:rx_start]) + ++ @text_section_offset_in_elf ++ end ++ ++ def phys_tables_base_addr ++ @phys_addresses[:phys_tables_base_addr] ++ end ++ ++ def phys_tables_base_addr_offset_in_kernel_elf ++ (@virt_addresses[:phys_tables_base_addr] - @virt_addresses[:rx_start]) + ++ @text_section_offset_in_elf ++ end ++ ++ def phys_addr_space_end_page ++ x = MEMORY_SRC.grep(/pub const END/) ++ x = case BSP_TYPE ++ when :rpi3 ++ x[0] ++ when :rpi4 ++ x[1] ++ else ++ raise ++ end ++ ++ x.scan(/\d+/).join.to_i(16) ++ end ++ ++ def kernel_map_binary ++ MappingDescriptor.print_header ++ ++ @descriptors.each do |i| ++ print 'Generating'.rjust(12).green.bold ++ print ' ' ++ puts i.to_s ++ ++ TRANSLATION_TABLES.map_pages_at(i.virt_pages, i.phys_pages, i.attributes) ++ end ++ ++ MappingDescriptor.print_divider ++ end ++ ++ private ++ ++ def parse_from_symbols(symbols, input) ++ case input.class.to_s ++ when 'Regexp' ++ symbols.grep(input).first.split.first.to_i(16) ++ when 'Hash' ++ input.transform_values do |val| ++ symbols.grep(val).first.split.first.to_i(16) ++ end ++ else ++ raise ++ end ++ end ++ ++ def parse_text_section_offset_in_elf(kernel_elf) ++ `#{READELF_BINARY} --sections #{kernel_elf}`.scan(/.text.*/).first.split.last.to_i(16) ++ end ++ ++ def virt_to_phys(input) ++ case input.class.to_s ++ when 'Integer' ++ input - @kernel_virt_start_addr ++ when 'Hash' ++ input.transform_values do |val| ++ val - @kernel_virt_start_addr ++ end ++ else ++ raise ++ end ++ end ++ ++ def descriptor_ro ++ name = 'Code and RO data' ++ ++ ro_size = @virt_addresses[:rx_end_exclusive] - ++ @virt_addresses[:rx_start] ++ ++ virt_ro_pages = PageArray.new(@virt_addresses[:rx_start], ro_size, @kernel_granule::SIZE) ++ phys_ro_pages = PageArray.new(@phys_addresses[:rx_start], ro_size, @kernel_granule::SIZE) ++ ro_attribues = AttributeFields.new(:CacheableDRAM, :ReadOnly, :X) ++ ++ MappingDescriptor.new(name, virt_ro_pages, phys_ro_pages, ro_attribues) ++ end ++ ++ def descriptor_data ++ name = 'Data and bss' ++ ++ data_size = @virt_addresses[:rw_end_exclusive] - ++ @virt_addresses[:rw_start] ++ ++ virt_data_pages = PageArray.new(@virt_addresses[:rw_start], data_size, ++ @kernel_granule::SIZE) ++ phys_data_pages = PageArray.new(@phys_addresses[:rw_start], data_size, ++ @kernel_granule::SIZE) ++ data_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) ++ ++ MappingDescriptor.new(name, virt_data_pages, phys_data_pages, data_attribues) ++ end ++ ++ def descriptor_boot_core_stack ++ name = 'Boot-core stack' ++ ++ boot_core_stack_size = @virt_addresses[:boot_core_stack_end_exclusive] - ++ @virt_addresses[:boot_core_stack_start] ++ ++ virt_boot_core_stack_pages = PageArray.new(@virt_addresses[:boot_core_stack_start], ++ boot_core_stack_size, @kernel_granule::SIZE) ++ phys_boot_core_stack_pages = PageArray.new(@phys_addresses[:boot_core_stack_start], ++ boot_core_stack_size, @kernel_granule::SIZE) ++ boot_core_stack_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) ++ ++ MappingDescriptor.new(name, virt_boot_core_stack_pages, phys_boot_core_stack_pages, ++ boot_core_stack_attribues) ++ end ++ ++ def parse_descriptors ++ [descriptor_ro, descriptor_data, descriptor_boot_core_stack] ++ end ++ ++ def update_max_descriptor_name_length ++ MappingDescriptor.max_descriptor_name_length = @descriptors.map { |i| i.name.size }.max ++ end ++end + +diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/generic.rb 15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb +--- 14_virtual_mem_part2_mmio_remap/translation_table_tool/generic.rb ++++ 15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb +@@ -0,0 +1,125 @@ ++# frozen_string_literal: true ++ ++# SPDX-License-Identifier: MIT OR Apache-2.0 ++# ++# Copyright (c) 2021 Andre Richter ++ ++module Granule64KiB ++ SIZE = 64 * 1024 ++ SHIFT = Math.log2(SIZE).to_i ++end ++ ++module Granule512MiB ++ SIZE = 512 * 1024 * 1024 ++ SHIFT = Math.log2(SIZE).to_i ++ MASK = SIZE - 1 ++end ++ ++# Monkey-patch Integer with some helper functions. ++class Integer ++ def power_of_two? ++ self[0].zero? ++ end ++ ++ def aligned?(alignment) ++ raise unless alignment.power_of_two? ++ ++ (self & (alignment - 1)).zero? ++ end ++ ++ def to_hex_underscore(with_leading_zeros: false) ++ fmt = with_leading_zeros ? 'modulo016x' : 'modulox' ++ value = format(fmt, self).to_s.reverse.scan(/.{4}|.+/).join('_').reverse ++ ++ format('0xmodulos', value) ++ end ++end ++ ++# An array where each value is the start address of a Page. ++class PageArray < Array ++ def initialize(start_addr, size, granule_size) ++ raise unless start_addr.aligned?(granule_size) ++ raise unless size.positive? ++ raise unless (size modulo granule_size).zero? ++ ++ num_pages = size / granule_size ++ super(num_pages) do |i| ++ i * granule_size + start_addr ++ end ++ end ++end ++ ++# Collection of memory attributes. ++class AttributeFields ++ attr_reader :mem_attributes, :acc_perms, :execute_never ++ ++ def initialize(mem_attributes, acc_perms, execute_never) ++ @mem_attributes = mem_attributes ++ @acc_perms = acc_perms ++ @execute_never = execute_never ++ end ++end ++ ++# A container that describes a one- or many-page virt-to-phys mapping. ++class MappingDescriptor ++ @max_descriptor_name_length = 0 ++ ++ class << self ++ attr_accessor :max_descriptor_name_length ++ end ++ ++ attr_reader :name, :virt_pages, :phys_pages, :attributes ++ ++ def initialize(name, virt_pages, phys_pages, attributes) ++ @name = name ++ @virt_pages = virt_pages ++ @phys_pages = phys_pages ++ @attributes = attributes ++ end ++ ++ def to_s ++ name = @name.ljust(self.class.max_descriptor_name_length) ++ virt_start = @virt_pages.first.to_hex_underscore(with_leading_zeros: true) ++ size = ((@virt_pages.size * 65_536) / 1024).to_s.rjust(3) ++ ++ "#{name} | #{virt_start} | #{size} KiB" ++ end ++ ++ def self.print_divider ++ print ' ' ++ print '-' * max_descriptor_name_length ++ puts '----------------------------------' ++ end ++ ++ def self.print_header ++ print_divider ++ print ' ' ++ print 'Section'.center(max_descriptor_name_length) ++ print ' ' ++ print 'Start Virt Addr'.center(21) ++ print ' ' ++ print 'Size'.center(7) ++ puts ++ print_divider ++ end ++end ++ ++def kernel_patch_tables(kernel_binary) ++ print 'Patching'.rjust(12).green.bold ++ print ' Kernel table struct at physical ' ++ puts BSP.phys_table_struct_start_addr.to_hex_underscore ++ ++ IO.binwrite(kernel_binary, TRANSLATION_TABLES.to_binary, ++ BSP.table_struct_offset_in_kernel_elf) ++end ++ ++def kernel_patch_base_addr(kernel_binary) ++ print 'Patching'.rjust(12).green.bold ++ print ' Value of kernel table physical base address (' ++ print TRANSLATION_TABLES.phys_tables_base_addr.to_hex_underscore ++ print ') at physical ' ++ puts BSP.phys_tables_base_addr.to_hex_underscore ++ ++ IO.binwrite(kernel_binary, TRANSLATION_TABLES.phys_tables_base_addr_binary, ++ BSP.phys_tables_base_addr_offset_in_kernel_elf) ++end + +diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/main.rb 15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb +--- 14_virtual_mem_part2_mmio_remap/translation_table_tool/main.rb ++++ 15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb +@@ -0,0 +1,47 @@ ++#!/usr/bin/env ruby ++# frozen_string_literal: true ++ ++# SPDX-License-Identifier: MIT OR Apache-2.0 ++# ++# Copyright (c) 2021 Andre Richter ++ ++TARGET = ARGV[0].split('-').first.to_sym ++BSP_TYPE = ARGV[1].to_sym ++kernel_elf = ARGV[2] ++ ++require 'rubygems' ++require 'bundler/setup' ++require 'colorize' ++ ++require_relative 'generic' ++require_relative 'bsp' ++require_relative 'arch' ++ ++puts ++puts 'Precomputing kernel translation tables and patching kernel ELF'.cyan ++ ++start = Time.now ++ ++BSP = case BSP_TYPE ++ when :rpi3, :rpi4 ++ RaspberryPi.new(kernel_elf) ++ else ++ raise ++ end ++ ++TRANSLATION_TABLES = case TARGET ++ when :aarch64 ++ Arch::ARMv8::TranslationTable.new ++ else ++ raise ++ end ++ ++BSP.kernel_map_binary ++ ++kernel_patch_tables(kernel_elf) ++kernel_patch_base_addr(kernel_elf) ++ ++elapsed = Time.now - start ++ ++print 'Finished'.rjust(12).green.bold ++puts " in #{elapsed.round(2)}s" + +``` diff --git a/15_virtual_mem_part3_precomputed_tables/build.rs b/15_virtual_mem_part3_precomputed_tables/build.rs new file mode 100644 index 000000000..3f0a29110 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/build.rs @@ -0,0 +1,8 @@ +use std::env; + +fn main() { + let linker_file = env::var("LINKER_FILE").unwrap(); + + println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu.rs new file mode 100644 index 000000000..56443865a --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural processor code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +pub use asm::nop; + +/// Pause execution on the core. +#[inline(always)] +pub fn wait_forever() -> ! { + loop { + asm::wfe() + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] +use qemu_exit::QEMUExit; + +#[cfg(feature = "test_build")] +const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); + +/// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] +pub fn qemu_exit_failure() -> ! { + QEMU_EXIT_HANDLE.exit_failure() +} + +/// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] +pub fn qemu_exit_success() -> ! { + QEMU_EXIT_HANDLE.exit_success() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..5e0ecfc6c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{cpu, memory, memory::Address, runtime_init}; +use core::intrinsics::unlikely; +use cortex_a::{asm, regs::*}; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Prepares the transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - The HW state of EL1 must be prepared in a sound way. +#[inline(always)] +unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The Rust entry of the `kernel` binary. +/// +/// The function is called from the assembly `_start` function. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +#[no_mangle] +pub unsafe extern "C" fn _start_rust( + phys_kernel_tables_base_addr: u64, + phys_boot_core_stack_end_exclusive_addr: u64, +) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + + // Turn on the MMU for EL1. + let addr = Address::new(phys_kernel_tables_base_addr as usize); + if unlikely(memory::mmu::enable_mmu_and_caching(addr).is_err()) { + cpu::wait_forever(); + } + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..286462616 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Load the base address of the kernel's translation tables. + ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. + ADR_REL x1, __boot_core_stack_end_exclusive + mov sp, x1 + + // Jump to Rust code. x0 and x1 hold the function arguments provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs new file mode 100644 index 000000000..b9fdd0f73 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the executing core's id. +#[inline(always)] +pub fn core_id() -> T +where + T: From, +{ + const CORE_MASK: u64 = 0b11; + + T::from((MPIDR_EL1.get() & CORE_MASK) as u8) +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs new file mode 100644 index 000000000..4a2c8de96 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs @@ -0,0 +1,281 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception + +use crate::{ + bsp::{self}, + exception, + memory::Address, +}; +use core::{cell::UnsafeCell, fmt}; +use cortex_a::{barrier, regs::*}; +use register::InMemoryRegister; + +// Assembly counterpart to this file. +global_asm!(include_str!("exception.s")); + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Wrapper struct for memory copy of SPSR_EL1. +#[repr(transparent)] +struct SpsrEL1(InMemoryRegister); + +/// The exception context as it is stored on the stack on exception entry. +#[repr(C)] +struct ExceptionContext { + /// General Purpose Registers. + gpr: [u64; 30], + + /// The link register, aka x30. + lr: u64, + + /// Exception link register. The program counter at the time the exception happened. + elr_el1: u64, + + /// Saved program status. + spsr_el1: SpsrEL1, +} + +/// Wrapper struct for pretty printing ESR_EL1. +struct EsrEL1; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Check if additional context can be derived from a data abort. +fn inspect_data_abort(f: &mut fmt::Formatter) -> fmt::Result { + let fault_addr = Address::new(FAR_EL1.get() as usize); + + if bsp::memory::mmu::virt_boot_core_stack_guard_page_desc().contains(fault_addr) { + writeln!( + f, + "\n\n >> Attempted to access the guard page of the kernel's boot core stack <<" + )?; + } + + Ok(()) +} + +/// Prints verbose information about the exception and then panics. +fn default_exception_handler(e: &ExceptionContext) { + panic!( + "\n\nCPU Exception!\n\ + FAR_EL1: {:#018x}\n\ + {}\n\ + {}", + FAR_EL1.get(), + EsrEL1 {}, + e + ); +} + +//------------------------------------------------------------------------------ +// Current, EL0 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +//------------------------------------------------------------------------------ +// Current, ELx +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn current_elx_irq(_e: &mut ExceptionContext) { + use exception::asynchronous::interface::IRQManager; + + let token = &exception::asynchronous::IRQContext::new(); + bsp::exception::asynchronous::irq_manager().handle_pending_irqs(token); +} + +#[no_mangle] +unsafe extern "C" fn current_elx_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Lower, AArch64 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_irq(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Lower, AArch32 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_irq(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Pretty printing +//------------------------------------------------------------------------------ + +/// Human readable ESR_EL1. +#[rustfmt::skip] +impl fmt::Display for EsrEL1 { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let esr_el1 = ESR_EL1.extract(); + + // Raw print of whole register. + writeln!(f, "ESR_EL1: {:#010x}", esr_el1.get())?; + + // Raw print of exception class. + write!(f, " Exception Class (EC) : {:#x}", esr_el1.read(ESR_EL1::EC))?; + + // Exception class, translation. + let ec_translation = match esr_el1.read_as_enum(ESR_EL1::EC) { + Some(ESR_EL1::EC::Value::DataAbortCurrentEL) => "Data Abort, current EL", + _ => "N/A", + }; + writeln!(f, " - {}", ec_translation)?; + + // Raw print of instruction specific syndrome. + write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; + + inspect_data_abort(f) + } +} + +/// Human readable SPSR_EL1. +#[rustfmt::skip] +impl fmt::Display for SpsrEL1 { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + // Raw value. + writeln!(f, "SPSR_EL1: {:#010x}", self.0.get())?; + + let to_flag_str = |x| -> _ { + if x { "Set" } else { "Not set" } + }; + + writeln!(f, " Flags:")?; + writeln!(f, " Negative (N): {}", to_flag_str(self.0.is_set(SPSR_EL1::N)))?; + writeln!(f, " Zero (Z): {}", to_flag_str(self.0.is_set(SPSR_EL1::Z)))?; + writeln!(f, " Carry (C): {}", to_flag_str(self.0.is_set(SPSR_EL1::C)))?; + writeln!(f, " Overflow (V): {}", to_flag_str(self.0.is_set(SPSR_EL1::V)))?; + + let to_mask_str = |x| -> _ { + if x { "Masked" } else { "Unmasked" } + }; + + writeln!(f, " Exception handling state:")?; + writeln!(f, " Debug (D): {}", to_mask_str(self.0.is_set(SPSR_EL1::D)))?; + writeln!(f, " SError (A): {}", to_mask_str(self.0.is_set(SPSR_EL1::A)))?; + writeln!(f, " IRQ (I): {}", to_mask_str(self.0.is_set(SPSR_EL1::I)))?; + writeln!(f, " FIQ (F): {}", to_mask_str(self.0.is_set(SPSR_EL1::F)))?; + + write!(f, " Illegal Execution State (IL): {}", + to_flag_str(self.0.is_set(SPSR_EL1::IL)) + ) + } +} + +/// Human readable print of the exception context. +impl fmt::Display for ExceptionContext { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + writeln!(f, "ELR_EL1: {:#018x}", self.elr_el1)?; + writeln!(f, "{}", self.spsr_el1)?; + writeln!(f)?; + writeln!(f, "General purpose register:")?; + + #[rustfmt::skip] + let alternating = |x| -> _ { + if x % 2 == 0 { " " } else { "\n" } + }; + + // Print two registers per line. + for (i, reg) in self.gpr.iter().enumerate() { + write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; + } + write!(f, " lr : {:#018x}", self.lr) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::exception::PrivilegeLevel; + +/// The processing element's current privilege level. +pub fn current_privilege_level() -> (PrivilegeLevel, &'static str) { + let el = CurrentEL.read_as_enum(CurrentEL::EL); + match el { + Some(CurrentEL::EL::Value::EL2) => (PrivilegeLevel::Hypervisor, "EL2"), + Some(CurrentEL::EL::Value::EL1) => (PrivilegeLevel::Kernel, "EL1"), + Some(CurrentEL::EL::Value::EL0) => (PrivilegeLevel::User, "EL0"), + _ => (PrivilegeLevel::Unknown, "Unknown"), + } +} + +/// Init exception handling by setting the exception vector base address register. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +/// - The vector table and the symbol `__exception_vector_table_start` from the linker script must +/// adhere to the alignment and size constraints demanded by the ARMv8-A Architecture Reference +/// Manual. +pub unsafe fn handling_init() { + // Provided by exception.S. + extern "Rust" { + static __exception_vector_start: UnsafeCell<()>; + } + + VBAR_EL1.set(__exception_vector_start.get() as u64); + + // Force VBAR update to complete before next instruction. + barrier::isb(barrier::SY); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.s b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs new file mode 100644 index 000000000..a4b1a548e --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +mod daif_bits { + pub const IRQ: u8 = 0b0010; +} + +trait DaifField { + fn daif_field() -> register::Field; +} + +struct Debug; +struct SError; +struct IRQ; +struct FIQ; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DaifField for Debug { + fn daif_field() -> register::Field { + DAIF::D + } +} + +impl DaifField for SError { + fn daif_field() -> register::Field { + DAIF::A + } +} + +impl DaifField for IRQ { + fn daif_field() -> register::Field { + DAIF::I + } +} + +impl DaifField for FIQ { + fn daif_field() -> register::Field { + DAIF::F + } +} + +fn is_masked() -> bool +where + T: DaifField, +{ + DAIF.is_set(T::daif_field()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Returns whether IRQs are masked on the executing core. +pub fn is_local_irq_masked() -> bool { + !is_masked::() +} + +/// Unmask IRQs on the executing core. +/// +/// It is not needed to place an explicit instruction synchronization barrier after the `msr`. +/// Quoting the Architecture Reference Manual for ARMv8-A, section C5.1.3: +/// +/// "Writes to PSTATE.{PAN, D, A, I, F} occur in program order without the need for additional +/// synchronization." +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_unmask() { + #[rustfmt::skip] + asm!( + "msr DAIFClr, {arg}", + arg = const daif_bits::IRQ, + options(nomem, nostack, preserves_flags) + ); +} + +/// Mask IRQs on the executing core. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_mask() { + #[rustfmt::skip] + asm!( + "msr DAIFSet, {arg}", + arg = const daif_bits::IRQ, + options(nomem, nostack, preserves_flags) + ); +} + +/// Mask IRQs on the executing core and return the previously saved interrupt mask bits (DAIF). +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_mask_save() -> u64 { + let saved = DAIF.get(); + local_irq_mask(); + + saved +} + +/// Restore the interrupt mask bits (DAIF) using the callee's argument. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +/// - No sanity checks on the input. +#[inline(always)] +pub unsafe fn local_irq_restore(saved: u64) { + DAIF.set(saved); +} + +/// Print the AArch64 exceptions status. +#[rustfmt::skip] +pub fn print_state() { + use crate::info; + + let to_mask_str = |x| -> _ { + if x { "Masked" } else { "Unmasked" } + }; + + info!(" Debug: {}", to_mask_str(is_masked::())); + info!(" SError: {}", to_mask_str(is_masked::())); + info!(" IRQ: {}", to_mask_str(is_masked::())); + info!(" FIQ: {}", to_mask_str(is_masked::())); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs new file mode 100644 index 000000000..49e6630ed --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management Unit Driver. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu + +use crate::{ + bsp, memory, + memory::{mmu::TranslationGranule, Address, Physical, Virtual}, +}; +use core::intrinsics::unlikely; +use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Memory Management Unit type. +struct MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; + +/// Constants for indexing the MAIR_EL1. +#[allow(dead_code)] +pub mod mair { + pub const DEVICE: u64 = 0; + pub const NORMAL: u64 = 1; +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static MMU: MemoryManagementUnit = MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + + + // Attribute 0 - Device. + MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, + ); + } + + /// Configure various settings of stage 1 of the EL1 translation regime. + fn configure_translation_control(&self) { + let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI0::Used + + TCR_EL1::IPS::Bits_40 + + TCR_EL1::TG0::KiB_64 + + TCR_EL1::SH0::Inner + + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD0::EnableTTBR0Walks + + TCR_EL1::A1::TTBR0 + + TCR_EL1::T0SZ.val(t0sz) + + TCR_EL1::EPD1::DisableTTBR1Walks, + ); + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the MMU instance. +pub fn mmu() -> &'static impl memory::mmu::interface::MMU { + &MMU +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use memory::mmu::{MMUEnableError, TranslationError}; + +impl memory::mmu::interface::MMU for MemoryManagementUnit { + unsafe fn enable_mmu_and_caching( + &self, + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); + } + + // Prepare the memory attribute indirection register. + self.set_up_mair(); + + // Set the "Translation Table Base Register". + TTBR0_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); + + self.configure_translation_control(); + + // Switch the MMU on. + // + // First, force all previous changes to be seen before the MMU is enabled. + barrier::isb(barrier::SY); + + // Enable the MMU and turn on data and instruction caching. + SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); + + // Force MMU init to complete before next instruction. + barrier::isb(barrier::SY); + + Ok(()) + } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } + + fn try_virt_to_phys( + &self, + virt: Address, + ) -> Result, TranslationError> { + if !self.is_enabled() { + return Err(TranslationError::MMUDisabled); + } + + let addr = virt.into_usize() as u64; + unsafe { + asm!( + "AT S1E1R, {0}", + in(reg) addr, + options(readonly, nostack, preserves_flags) + ); + } + + let par_el1 = PAR_EL1.extract(); + if par_el1.matches_all(PAR_EL1::F::TranslationAborted) { + return Err(TranslationError::Aborted); + } + + let phys_addr = (par_el1.read(PAR_EL1::PA) << 12) | (addr & 0xFFF); + + Ok(Address::new(phys_addr as usize)) + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..5f781bdd5 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,474 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::{ + mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, + }, + Address, Physical, Virtual, + }, +}; +use core::convert::{self, TryInto}; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Reserved_Invalid = 0, + Page = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait StartAddr { + fn virt_start_addr(&self) -> Address; +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, so the lvl3 is put first. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], + + /// Index of the next free MMIO page. + cur_l3_mmio_index: usize, + + /// Have the tables been initialized? + initialized: bool, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl StartAddr for [T; N] { + fn virt_start_addr(&self) -> Address { + Address::new(self as *const _ as usize) + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: Address) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = phys_next_lvl_table_addr.into_usize() >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::VALID::True, + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr( + phys_output_addr: *const Page, + attribute_fields: &AttributeFields, + ) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + + STAGE1_PAGE_DESCRIPTOR::AF::True + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), + ); + + Self { value: val.get() } + } + + /// Returns the valid bit. + fn is_valid(&self) -> bool { + InMemoryRegister::::new(self.value) + .is_set(STAGE1_PAGE_DESCRIPTOR::VALID) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl memory::mmu::AssociatedTranslationTable + for memory::mmu::AddressSpace +where + [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, +{ + type TableStartFromBottom = FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }>; +} + +impl FixedSizeTranslationTable { + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + const fn _new(for_precompute: bool) -> Self { + assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); + + // Can't have a zero-sized address space. + assert!(NUM_TABLES > 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + cur_l3_mmio_index: Self::L3_MMIO_START_INDEX, + initialized: for_precompute, + } + } + + pub const fn new_for_precompute() -> Self { + Self::_new(true) + } + + #[cfg(test)] + pub fn new_for_runtime() -> Self { + Self::_new(false) + } + + /// The start address of the table's MMIO range. + #[inline(always)] + fn mmio_start_addr(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), + ) + } + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] + fn mmio_end_addr_inclusive(&self) -> Address { + Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) + | (Granule64KiB::SIZE - 1), + ) + } + + /// Helper to calculate the lvl2 and lvl3 indices from an address. + #[inline(always)] + fn lvl2_lvl3_index_from( + &self, + addr: *const Page, + ) -> Result<(usize, usize), &'static str> { + let addr = addr as usize; + let lvl2_index = addr >> Granule512MiB::SHIFT; + let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + + if lvl2_index > (NUM_TABLES - 1) { + return Err("Virtual page is out of bounds of translation table"); + } + + Ok((lvl2_index, lvl3_index)) + } + + /// Returns the PageDescriptor corresponding to the supplied Page. + #[inline(always)] + fn page_descriptor_from( + &mut self, + addr: *const Page, + ) -> Result<&mut PageDescriptor, &'static str> { + let (lvl2_index, lvl3_index) = self.lvl2_lvl3_index_from(addr)?; + + Ok(&mut self.lvl3[lvl2_index][lvl3_index]) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl memory::mmu::translation_table::interface::TranslationTable + for FixedSizeTranslationTable +{ + fn init(&mut self) -> Result<(), &'static str> { + if self.initialized { + return Ok(()); + } + + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { + let addr = self.lvl3[lvl2_nr] + .virt_start_addr() + .try_into() + .map_err(|_| "Translation error")?; + + let desc = TableDescriptor::from_next_lvl_table_addr(addr); + *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; + + Ok(()) + } + + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + let p = phys_pages.as_slice(); + let v = virt_pages.as_slice(); + + // No work to do for empty slices. + if v.is_empty() { + return Ok(()); + } + + if v.len() != p.len() { + return Err("Tried to map page slices with unequal sizes"); + } + + if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { + return Err("Tried to map outside of physical address space"); + } + + let iter = p.iter().zip(v.iter()); + for (phys_page, virt_page) in iter { + let page_descriptor = self.page_descriptor_from(virt_page.as_ptr())?; + if page_descriptor.is_valid() { + return Err("Virtual page is already mapped"); + } + + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + } + + Ok(()) + } + + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + if num_pages == 0 { + return Err("num_pages == 0"); + } + + if (self.cur_l3_mmio_index + num_pages) > 8191 { + return Err("Not enough MMIO space left"); + } + + let addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (self.cur_l3_mmio_index << Granule64KiB::SHIFT), + ); + self.cur_l3_mmio_index += num_pages; + + Ok(PageSliceDescriptor::from_addr(addr, num_pages)) + } + + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { + let start_addr = virt_pages.start_addr(); + let end_addr_inclusive = virt_pages.end_addr_inclusive(); + + for i in [start_addr, end_addr_inclusive].iter() { + if (*i >= self.mmio_start_addr()) && (*i <= self.mmio_end_addr_inclusive()) { + return true; + } + } + + false + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +pub type MinSizeTranslationTable = FixedSizeTranslationTable<1>; + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct TableDescriptor` is as expected. + #[kernel_test] + fn size_of_tabledescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } + + /// Check if the size of `struct PageDescriptor` is as expected. + #[kernel_test] + fn size_of_pagedescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs new file mode 100644 index 000000000..63017305b --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time + +use crate::{time, warn}; +use core::time::Duration; +use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NS_PER_S: u64 = 1_000_000_000; + +/// ARMv8 Generic Timer. +struct GenericTimer; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static TIME_MANAGER: GenericTimer = GenericTimer; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the time manager. +pub fn time_manager() -> &'static impl time::interface::TimeManager { + &TIME_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl time::interface::TimeManager for GenericTimer { + fn resolution(&self) -> Duration { + Duration::from_nanos(NS_PER_S / (CNTFRQ_EL0.get() as u64)) + } + + fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; + let frq: u64 = CNTFRQ_EL0.get() as u64; + + Duration::from_nanos(current_count / frq) + } + + fn spin_for(&self, duration: Duration) { + // Instantly return on zero. + if duration.as_nanos() == 0 { + return; + } + + // Calculate the register compare value. + let frq = CNTFRQ_EL0.get(); + let x = match frq.checked_mul(duration.as_nanos() as u64) { + None => { + warn!("Spin duration too long, skipping"); + return; + } + Some(val) => val, + }; + let tval = x / NS_PER_S; + + // Check if it is within supported bounds. + let warn: Option<&str> = if tval == 0 { + Some("smaller") + // The upper 32 bits of CNTP_TVAL_EL0 are reserved. + } else if tval > u32::max_value().into() { + Some("bigger") + } else { + None + }; + + if let Some(w) = warn { + warn!( + "Spin duration {} than architecturally supported, skipping", + w + ); + return; + } + + // Set the compare value register. + CNTP_TVAL_EL0.set(tval); + + // Kick off the counting. // Disable timer interrupt. + CNTP_CTL_EL0.modify(CNTP_CTL_EL0::ENABLE::SET + CNTP_CTL_EL0::IMASK::SET); + + // ISTATUS will be '1' when cval ticks have passed. Busy-check it. + while !CNTP_CTL_EL0.matches_all(CNTP_CTL_EL0::ISTATUS::SET) {} + + // Disable counting again. + CNTP_CTL_EL0.modify(CNTP_CTL_EL0::ENABLE::CLEAR); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp.rs new file mode 100644 index 000000000..c558922f7 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp.rs @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Conditional reexporting of Board Support Packages. + +mod device_driver; + +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +mod raspberrypi; + +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +pub use raspberrypi::*; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver.rs new file mode 100644 index 000000000..eac6bdd16 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver.rs @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Device driver. + +#[cfg(feature = "bsp_rpi4")] +mod arm; +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +mod bcm; +mod common; + +#[cfg(feature = "bsp_rpi4")] +pub use arm::*; +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +pub use bcm::*; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm.rs new file mode 100644 index 000000000..0bb1bd4b6 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! ARM driver top level. + +pub mod gicv2; + +pub use gicv2::*; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2.rs new file mode 100644 index 000000000..51c321868 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2.rs @@ -0,0 +1,248 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICv2 Driver - ARM Generic Interrupt Controller v2. +//! +//! The following is a collection of excerpts with useful information from +//! - `Programmer's Guide for ARMv8-A` +//! - `ARM Generic Interrupt Controller Architecture Specification` +//! +//! # Programmer's Guide - 10.6.1 Configuration +//! +//! The GIC is accessed as a memory-mapped peripheral. +//! +//! All cores can access the common Distributor, but the CPU interface is banked, that is, each core +//! uses the same address to access its own private CPU interface. +//! +//! It is not possible for a core to access the CPU interface of another core. +//! +//! # Architecture Specification - 10.6.2 Initialization +//! +//! Both the Distributor and the CPU interfaces are disabled at reset. The GIC must be initialized +//! after reset before it can deliver interrupts to the core. +//! +//! In the Distributor, software must configure the priority, target, security and enable individual +//! interrupts. The Distributor must subsequently be enabled through its control register +//! (GICD_CTLR). For each CPU interface, software must program the priority mask and preemption +//! settings. +//! +//! Each CPU interface block itself must be enabled through its control register (GICD_CTLR). This +//! prepares the GIC to deliver interrupts to the core. +//! +//! Before interrupts are expected in the core, software prepares the core to take interrupts by +//! setting a valid interrupt vector in the vector table, and clearing interrupt mask bits in +//! PSTATE, and setting the routing controls. +//! +//! The entire interrupt mechanism in the system can be disabled by disabling the Distributor. +//! Interrupt delivery to an individual core can be disabled by disabling its CPU interface. +//! Individual interrupts can also be disabled (or enabled) in the distributor. +//! +//! For an interrupt to reach the core, the individual interrupt, Distributor and CPU interface must +//! all be enabled. The interrupt also needs to be of sufficient priority, that is, higher than the +//! core's priority mask. +//! +//! # Architecture Specification - 1.4.2 Interrupt types +//! +//! - Peripheral interrupt +//! - Private Peripheral Interrupt (PPI) +//! - This is a peripheral interrupt that is specific to a single processor. +//! - Shared Peripheral Interrupt (SPI) +//! - This is a peripheral interrupt that the Distributor can route to any of a specified +//! combination of processors. +//! +//! - Software-generated interrupt (SGI) +//! - This is an interrupt generated by software writing to a GICD_SGIR register in the GIC. The +//! system uses SGIs for interprocessor communication. +//! - An SGI has edge-triggered properties. The software triggering of the interrupt is +//! equivalent to the edge transition of the interrupt request signal. +//! - When an SGI occurs in a multiprocessor implementation, the CPUID field in the Interrupt +//! Acknowledge Register, GICC_IAR, or the Aliased Interrupt Acknowledge Register, GICC_AIAR, +//! identifies the processor that requested the interrupt. +//! +//! # Architecture Specification - 2.2.1 Interrupt IDs +//! +//! Interrupts from sources are identified using ID numbers. Each CPU interface can see up to 1020 +//! interrupts. The banking of SPIs and PPIs increases the total number of interrupts supported by +//! the Distributor. +//! +//! The GIC assigns interrupt ID numbers ID0-ID1019 as follows: +//! - Interrupt numbers 32..1019 are used for SPIs. +//! - Interrupt numbers 0..31 are used for interrupts that are private to a CPU interface. These +//! interrupts are banked in the Distributor. +//! - A banked interrupt is one where the Distributor can have multiple interrupts with the +//! same ID. A banked interrupt is identified uniquely by its ID number and its associated +//! CPU interface number. Of the banked interrupt IDs: +//! - 00..15 SGIs +//! - 16..31 PPIs + +mod gicc; +mod gicd; + +use crate::{bsp, cpu, driver, exception, memory, synchronization, synchronization::InitStateLock}; +use core::sync::atomic::{AtomicBool, Ordering}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +type HandlerTable = [Option; GICv2::NUM_IRQS]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; + +/// Representation of the GIC. +pub struct GICv2 { + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + + /// The Distributor. + gicd: gicd::GICD, + + /// The CPU Interface. + gicc: gicc::GICC, + + /// Have the MMIO regions been remapped yet? + is_mmio_remapped: AtomicBool, + + /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. + handler_table: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl GICv2 { + const MAX_IRQ_NUMBER: usize = 300; // Normally 1019, but keep it lower to save some space. + const NUM_IRQS: usize = Self::MAX_IRQ_NUMBER + 1; + + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { + Self { + gicd_mmio_descriptor, + gicc_mmio_descriptor, + gicd: gicd::GICD::new(gicd_mmio_descriptor.start_addr().into_usize()), + gicc: gicc::GICC::new(gicc_mmio_descriptor.start_addr().into_usize()), + is_mmio_remapped: AtomicBool::new(false), + handler_table: InitStateLock::new([None; Self::NUM_IRQS]), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::ReadWriteEx; + +impl driver::interface::DeviceDriver for GICv2 { + fn compatible(&self) -> &'static str { + "GICv2 (ARM Generic Interrupt Controller v2)" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let remapped = self.is_mmio_remapped.load(Ordering::Relaxed); + if !remapped { + let mut virt_addr; + + // GICD + virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_mmio_descriptor)?; + self.gicd.set_mmio(virt_addr.into_usize()); + + // GICC + virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_mmio_descriptor)?; + self.gicc.set_mmio(virt_addr.into_usize()); + + // Conclude remapping. + self.is_mmio_remapped.store(true, Ordering::Relaxed); + } + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + self.gicd.boot_core_init(); + } + + self.gicc.priority_accept_all(); + self.gicc.enable(); + + Ok(()) + } +} + +impl exception::asynchronous::interface::IRQManager for GICv2 { + type IRQNumberType = IRQNumber; + + fn register_handler( + &self, + irq_number: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { + let irq_number = irq_number.get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + + table[irq_number] = Some(descriptor); + + Ok(()) + }) + } + + fn enable(&self, irq_number: Self::IRQNumberType) { + self.gicd.enable(irq_number); + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + // Extract the highest priority pending IRQ number from the Interrupt Acknowledge Register + // (IAR). + let irq_number = self.gicc.pending_irq_number(ic); + + // Guard against spurious interrupts. + if irq_number > GICv2::MAX_IRQ_NUMBER { + return; + } + + // Call the IRQ handler. Panic if there is none. + self.handler_table.read(|table| { + match table[irq_number] { + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. + descriptor.handler.handle().expect("Error handling IRQ"); + } + } + }); + + // Signal completion of handling. + self.gicc.mark_comleted(irq_number as u32, ic); + } + + fn print_handler(&self) { + use crate::info; + + info!(" Peripheral handler:"); + + self.handler_table.read(|table| { + for (i, opt) in table.iter().skip(32).enumerate() { + if let Some(handler) = opt { + info!(" {: >3}. {}", i + 32, handler.name); + } + } + }); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs new file mode 100644 index 000000000..a1279f956 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICC Driver - GIC CPU interface. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_bitfields! { + u32, + + /// CPU Interface Control Register + CTLR [ + Enable OFFSET(0) NUMBITS(1) [] + ], + + /// Interrupt Priority Mask Register + PMR [ + Priority OFFSET(0) NUMBITS(8) [] + ], + + /// Interrupt Acknowledge Register + IAR [ + InterruptID OFFSET(0) NUMBITS(10) [] + ], + + /// End of Interrupt Register + EOIR [ + EOIINTID OFFSET(0) NUMBITS(10) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + pub RegisterBlock { + (0x000 => CTLR: ReadWrite), + (0x004 => PMR: ReadWrite), + (0x008 => _reserved1), + (0x00C => IAR: ReadWrite), + (0x010 => EOIR: ReadWrite), + (0x014 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the GIC CPU interface. +pub struct GICC { + registers: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::synchronization::interface::ReadWriteEx; + +impl GICC { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: InitStateLock::new(Registers::new(mmio_start_addr)), + } + } + + pub unsafe fn set_mmio(&self, new_mmio_start_addr: usize) { + self.registers + .write(|regs| *regs = Registers::new(new_mmio_start_addr)); + } + + /// Accept interrupts of any priority. + /// + /// Quoting the GICv2 Architecture Specification: + /// + /// "Writing 255 to the GICC_PMR always sets it to the largest supported priority field + /// value." + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + pub fn priority_accept_all(&self) { + self.registers.read(|regs| { + regs.PMR.write(PMR::Priority.val(255)); // Comment in arch spec. + }); + } + + /// Enable the interface - start accepting IRQs. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + pub fn enable(&self) { + self.registers.read(|regs| { + regs.CTLR.write(CTLR::Enable::SET); + }); + } + + /// Extract the number of the highest-priority pending IRQ. + /// + /// Can only be called from IRQ context, which is ensured by taking an `IRQContext` token. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + #[allow(clippy::trivially_copy_pass_by_ref)] + pub fn pending_irq_number<'irq_context>( + &self, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) -> usize { + self.registers + .read(|regs| regs.IAR.read(IAR::InterruptID) as usize) + } + + /// Complete handling of the currently active IRQ. + /// + /// Can only be called from IRQ context, which is ensured by taking an `IRQContext` token. + /// + /// To be called after `pending_irq_number()`. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + #[allow(clippy::trivially_copy_pass_by_ref)] + pub fn mark_comleted<'irq_context>( + &self, + irq_number: u32, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + self.registers.read(|regs| { + regs.EOIR.write(EOIR::EOIINTID.val(irq_number)); + }); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs new file mode 100644 index 000000000..5f56e54a1 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -0,0 +1,205 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICD Driver - GIC Distributor. +//! +//! # Glossary +//! - SPI - Shared Peripheral Interrupt. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, + state, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_bitfields! { + u32, + + /// Distributor Control Register + CTLR [ + Enable OFFSET(0) NUMBITS(1) [] + ], + + /// Interrupt Controller Type Register + TYPER [ + ITLinesNumber OFFSET(0) NUMBITS(5) [] + ], + + /// Interrupt Processor Targets Registers + ITARGETSR [ + Offset3 OFFSET(24) NUMBITS(8) [], + Offset2 OFFSET(16) NUMBITS(8) [], + Offset1 OFFSET(8) NUMBITS(8) [], + Offset0 OFFSET(0) NUMBITS(8) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + SharedRegisterBlock { + (0x000 => CTLR: ReadWrite), + (0x004 => TYPER: ReadOnly), + (0x008 => _reserved1), + (0x104 => ISENABLER: [ReadWrite; 31]), + (0x108 => _reserved2), + (0x820 => ITARGETSR: [ReadWrite; 248]), + (0x824 => @END), + } +} + +register_structs! { + #[allow(non_snake_case)] + BankedRegisterBlock { + (0x000 => _reserved1), + (0x100 => ISENABLER: ReadWrite), + (0x104 => _reserved2), + (0x800 => ITARGETSR: [ReadOnly; 8]), + (0x804 => @END), + } +} + +/// Abstraction for the non-banked parts of the associated MMIO registers. +type SharedRegisters = MMIODerefWrapper; + +/// Abstraction for the banked parts of the associated MMIO registers. +type BankedRegisters = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the GIC Distributor. +pub struct GICD { + /// Access to shared registers is guarded with a lock. + shared_registers: IRQSafeNullLock, + + /// Access to banked registers is unguarded. + banked_registers: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl SharedRegisters { + /// Return the number of IRQs that this HW implements. + #[inline(always)] + fn num_irqs(&mut self) -> usize { + // Query number of implemented IRQs. + // + // Refer to GICv2 Architecture Specification, Section 4.3.2. + ((self.TYPER.read(TYPER::ITLinesNumber) as usize) + 1) * 32 + } + + /// Return a slice of the implemented ITARGETSR. + #[inline(always)] + fn implemented_itargets_slice(&mut self) -> &[ReadWrite] { + assert!(self.num_irqs() >= 36); + + // Calculate the max index of the shared ITARGETSR array. + // + // The first 32 IRQs are private, so not included in `shared_registers`. Each ITARGETS + // register has four entries, so shift right by two. Subtract one because we start + // counting at zero. + let spi_itargetsr_max_index = ((self.num_irqs() - 32) >> 2) - 1; + + // Rust automatically inserts slice range sanity check, i.e. max >= min. + &self.ITARGETSR[0..spi_itargetsr_max_index] + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::synchronization::interface::ReadWriteEx; +use synchronization::interface::Mutex; + +impl GICD { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + shared_registers: IRQSafeNullLock::new(SharedRegisters::new(mmio_start_addr)), + banked_registers: InitStateLock::new(BankedRegisters::new(mmio_start_addr)), + } + } + + pub unsafe fn set_mmio(&self, new_mmio_start_addr: usize) { + self.shared_registers + .lock(|regs| *regs = SharedRegisters::new(new_mmio_start_addr)); + self.banked_registers + .write(|regs| *regs = BankedRegisters::new(new_mmio_start_addr)); + } + + /// Use a banked ITARGETSR to retrieve the executing core's GIC target mask. + /// + /// Quoting the GICv2 Architecture Specification: + /// + /// "GICD_ITARGETSR0 to GICD_ITARGETSR7 are read-only, and each field returns a value that + /// corresponds only to the processor reading the register." + fn local_gic_target_mask(&self) -> u32 { + self.banked_registers + .read(|regs| regs.ITARGETSR[0].read(ITARGETSR::Offset0)) + } + + /// Route all SPIs to the boot core and enable the distributor. + pub fn boot_core_init(&self) { + assert!( + state::state_manager().is_init(), + "Only allowed during kernel init phase" + ); + + // Target all SPIs to the boot core only. + let mask = self.local_gic_target_mask(); + + self.shared_registers.lock(|regs| { + for i in regs.implemented_itargets_slice().iter() { + i.write( + ITARGETSR::Offset3.val(mask) + + ITARGETSR::Offset2.val(mask) + + ITARGETSR::Offset1.val(mask) + + ITARGETSR::Offset0.val(mask), + ); + } + + regs.CTLR.write(CTLR::Enable::SET); + }); + } + + /// Enable an interrupt. + pub fn enable(&self, irq_num: super::IRQNumber) { + let irq_num = irq_num.get(); + + // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 + // (division by 32) and arrive at the index for the respective ISENABLER[i]. + let enable_reg_index = irq_num >> 5; + let enable_bit: u32 = 1u32 << (irq_num % 32); + + // Check if we are handling a private or shared IRQ. + match irq_num { + // Private. + 0..=31 => self.banked_registers.read(|regs| { + let enable_reg = ®s.ISENABLER; + enable_reg.set(enable_reg.get() | enable_bit); + }), + // Shared. + _ => { + let enable_reg_index_shared = enable_reg_index - 1; + + self.shared_registers.lock(|regs| { + let enable_reg = ®s.ISENABLER[enable_reg_index_shared]; + enable_reg.set(enable_reg.get() | enable_bit); + }); + } + } + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm.rs new file mode 100644 index 000000000..01bcaa895 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BCM driver top level. + +mod bcm2xxx_gpio; +#[cfg(feature = "bsp_rpi3")] +mod bcm2xxx_interrupt_controller; +mod bcm2xxx_pl011_uart; + +pub use bcm2xxx_gpio::*; +#[cfg(feature = "bsp_rpi3")] +pub use bcm2xxx_interrupt_controller::*; +pub use bcm2xxx_pl011_uart::*; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs new file mode 100644 index 000000000..e444812ba --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -0,0 +1,255 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! GPIO Driver. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, driver, memory, synchronization, + synchronization::IRQSafeNullLock, +}; +use core::sync::atomic::{AtomicUsize, Ordering}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// GPIO registers. +// +// Descriptions taken from +// - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// - https://datasheets.raspberrypi.org/bcm2711/bcm2711-peripherals.pdf +register_bitfields! { + u32, + + /// GPIO Function Select 1 + GPFSEL1 [ + /// Pin 15 + FSEL15 OFFSET(15) NUMBITS(3) [ + Input = 0b000, + Output = 0b001, + AltFunc0 = 0b100 // PL011 UART RX + + ], + + /// Pin 14 + FSEL14 OFFSET(12) NUMBITS(3) [ + Input = 0b000, + Output = 0b001, + AltFunc0 = 0b100 // PL011 UART TX + ] + ], + + /// GPIO Pull-up/down Register + /// + /// BCM2837 only. + GPPUD [ + /// Controls the actuation of the internal pull-up/down control line to ALL the GPIO pins. + PUD OFFSET(0) NUMBITS(2) [ + Off = 0b00, + PullDown = 0b01, + PullUp = 0b10 + ] + ], + + /// GPIO Pull-up/down Clock Register 0 + /// + /// BCM2837 only. + GPPUDCLK0 [ + /// Pin 15 + PUDCLK15 OFFSET(15) NUMBITS(1) [ + NoEffect = 0, + AssertClock = 1 + ], + + /// Pin 14 + PUDCLK14 OFFSET(14) NUMBITS(1) [ + NoEffect = 0, + AssertClock = 1 + ] + ], + + /// GPIO Pull-up / Pull-down Register 0 + /// + /// BCM2711 only. + GPIO_PUP_PDN_CNTRL_REG0 [ + /// Pin 15 + GPIO_PUP_PDN_CNTRL15 OFFSET(30) NUMBITS(2) [ + NoResistor = 0b00, + PullUp = 0b01 + ], + + /// Pin 14 + GPIO_PUP_PDN_CNTRL14 OFFSET(28) NUMBITS(2) [ + NoResistor = 0b00, + PullUp = 0b01 + ] + ] +} + +register_structs! { + #[allow(non_snake_case)] + RegisterBlock { + (0x00 => _reserved1), + (0x04 => GPFSEL1: ReadWrite), + (0x08 => _reserved2), + (0x94 => GPPUD: ReadWrite), + (0x98 => GPPUDCLK0: ReadWrite), + (0x9C => _reserved3), + (0xE4 => GPIO_PUP_PDN_CNTRL_REG0: ReadWrite), + (0xE8 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct GPIOInner { + registers: Registers, +} + +// Export the inner struct so that BSPs can use it for the panic handler. +pub use GPIOInner as PanicGPIO; + +/// Representation of the GPIO HW. +pub struct GPIO { + mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, + inner: IRQSafeNullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl GPIOInner { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: Registers::new(mmio_start_addr), + } + } + + /// Init code. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub unsafe fn init(&mut self, new_mmio_start_addr: Option) -> Result<(), &'static str> { + if let Some(addr) = new_mmio_start_addr { + self.registers = Registers::new(addr); + } + + Ok(()) + } + + /// Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi3")] + fn disable_pud_14_15_bcm2837(&mut self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The Linux 2837 GPIO driver waits 1 µs between the steps. + const DELAY: Duration = Duration::from_micros(1); + + self.registers.GPPUD.write(GPPUD::PUD::Off); + time::time_manager().spin_for(DELAY); + + self.registers + .GPPUDCLK0 + .write(GPPUDCLK0::PUDCLK15::AssertClock + GPPUDCLK0::PUDCLK14::AssertClock); + time::time_manager().spin_for(DELAY); + + self.registers.GPPUD.write(GPPUD::PUD::Off); + self.registers.GPPUDCLK0.set(0); + } + + /// Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi4")] + fn disable_pud_14_15_bcm2711(&mut self) { + self.registers.GPIO_PUP_PDN_CNTRL_REG0.write( + GPIO_PUP_PDN_CNTRL_REG0::GPIO_PUP_PDN_CNTRL15::PullUp + + GPIO_PUP_PDN_CNTRL_REG0::GPIO_PUP_PDN_CNTRL14::PullUp, + ); + } + + /// Map PL011 UART as standard output. + /// + /// TX to pin 14 + /// RX to pin 15 + pub fn map_pl011_uart(&mut self) { + // Select the UART on pins 14 and 15. + self.registers + .GPFSEL1 + .modify(GPFSEL1::FSEL15::AltFunc0 + GPFSEL1::FSEL14::AltFunc0); + + // Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi3")] + self.disable_pud_14_15_bcm2837(); + + #[cfg(feature = "bsp_rpi4")] + self.disable_pud_14_15_bcm2711(); + } +} + +impl GPIO { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + Self { + mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), + inner: IRQSafeNullLock::new(GPIOInner::new(mmio_descriptor.start_addr().into_usize())), + } + } + + /// Concurrency safe version of `GPIOInner.map_pl011_uart()` + pub fn map_pl011_uart(&self) { + self.inner.lock(|inner| inner.map_pl011_uart()) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; + +impl driver::interface::DeviceDriver for GPIO { + fn compatible(&self) -> &'static str { + "BCM GPIO" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; + + self.virt_mmio_start_addr + .store(virt_addr.into_usize(), Ordering::Relaxed); + + Ok(()) + } + + fn virt_mmio_start_addr(&self) -> Option { + let addr = self.virt_mmio_start_addr.load(Ordering::Relaxed); + + if addr == 0 { + return None; + } + + Some(addr) + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs new file mode 100644 index 000000000..0bd84e3cf --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Interrupt Controller Driver. + +mod peripheral_ic; + +use crate::{driver, exception, memory}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Wrapper struct for a bitmask indicating pending IRQ numbers. +struct PendingIRQs { + bitmask: u64, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub type LocalIRQ = + exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = + exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +#[derive(Copy, Clone)] +pub enum IRQNumber { + Local(LocalIRQ), + Peripheral(PeripheralIRQ), +} + +/// Representation of the Interrupt Controller. +pub struct InterruptController { + periph: peripheral_ic::PeripheralIC, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl PendingIRQs { + pub fn new(bitmask: u64) -> Self { + Self { bitmask } + } +} + +impl Iterator for PendingIRQs { + type Item = usize; + + fn next(&mut self) -> Option { + use core::intrinsics::cttz; + + let next = cttz(self.bitmask); + if next == 64 { + return None; + } + + self.bitmask &= !(1 << next); + + Some(next as usize) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl InterruptController { + const MAX_LOCAL_IRQ_NUMBER: usize = 11; + const MAX_PERIPHERAL_IRQ_NUMBER: usize = 63; + const NUM_PERIPHERAL_IRQS: usize = Self::MAX_PERIPHERAL_IRQ_NUMBER + 1; + + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( + _local_mmio_descriptor: memory::mmu::MMIODescriptor, + periph_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { + Self { + periph: peripheral_ic::PeripheralIC::new(periph_mmio_descriptor), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl driver::interface::DeviceDriver for InterruptController { + fn compatible(&self) -> &'static str { + "BCM Interrupt Controller" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + self.periph.init() + } +} + +impl exception::asynchronous::interface::IRQManager for InterruptController { + type IRQNumberType = IRQNumber; + + fn register_handler( + &self, + irq: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + match irq { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), + IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + } + } + + fn enable(&self, irq: Self::IRQNumberType) { + match irq { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), + IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), + } + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + // It can only be a peripheral IRQ pending because enable() does not support local IRQs yet. + self.periph.handle_pending_irqs(ic) + } + + fn print_handler(&self) { + self.periph.print_handler(); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs new file mode 100644 index 000000000..abc3af711 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -0,0 +1,188 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Peripheral Interrupt Controller Driver. + +use super::{InterruptController, PendingIRQs, PeripheralIRQ}; +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, + driver, exception, memory, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, +}; +use register::{mmio::*, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_structs! { + #[allow(non_snake_case)] + WORegisterBlock { + (0x00 => _reserved1), + (0x10 => ENABLE_1: WriteOnly), + (0x14 => ENABLE_2: WriteOnly), + (0x24 => @END), + } +} + +register_structs! { + #[allow(non_snake_case)] + RORegisterBlock { + (0x00 => _reserved1), + (0x04 => PENDING_1: ReadOnly), + (0x08 => PENDING_2: ReadOnly), + (0x0c => @END), + } +} + +/// Abstraction for the WriteOnly parts of the associated MMIO registers. +type WriteOnlyRegisters = MMIODerefWrapper; + +/// Abstraction for the ReadOnly parts of the associated MMIO registers. +type ReadOnlyRegisters = MMIODerefWrapper; + +type HandlerTable = + [Option; InterruptController::NUM_PERIPHERAL_IRQS]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the peripheral interrupt controller. +pub struct PeripheralIC { + mmio_descriptor: memory::mmu::MMIODescriptor, + + /// Access to write registers is guarded with a lock. + wo_registers: IRQSafeNullLock, + + /// Register read access is unguarded. + ro_registers: InitStateLock, + + /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. + handler_table: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl PeripheralIC { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + let addr = mmio_descriptor.start_addr().into_usize(); + + Self { + mmio_descriptor, + wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(addr)), + ro_registers: InitStateLock::new(ReadOnlyRegisters::new(addr)), + handler_table: InitStateLock::new([None; InterruptController::NUM_PERIPHERAL_IRQS]), + } + } + + /// Query the list of pending IRQs. + fn pending_irqs(&self) -> PendingIRQs { + self.ro_registers.read(|regs| { + let pending_mask: u64 = + (u64::from(regs.PENDING_2.get()) << 32) | u64::from(regs.PENDING_1.get()); + + PendingIRQs::new(pending_mask) + }) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::{Mutex, ReadWriteEx}; + +impl driver::interface::DeviceDriver for PeripheralIC { + fn compatible(&self) -> &'static str { + "BCM Peripheral Interrupt Controller" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = + memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?.into_usize(); + + self.wo_registers + .lock(|regs| *regs = WriteOnlyRegisters::new(virt_addr)); + self.ro_registers + .write(|regs| *regs = ReadOnlyRegisters::new(virt_addr)); + + Ok(()) + } +} + +impl exception::asynchronous::interface::IRQManager for PeripheralIC { + type IRQNumberType = PeripheralIRQ; + + fn register_handler( + &self, + irq: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { + let irq_number = irq.get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + + table[irq_number] = Some(descriptor); + + Ok(()) + }) + } + + fn enable(&self, irq: Self::IRQNumberType) { + self.wo_registers.lock(|regs| { + let enable_reg = if irq.get() <= 31 { + ®s.ENABLE_1 + } else { + ®s.ENABLE_2 + }; + + let enable_bit: u32 = 1 << (irq.get() % 32); + + // Writing a 1 to a bit will set the corresponding IRQ enable bit. All other IRQ enable + // bits are unaffected. So we don't need read and OR'ing here. + enable_reg.set(enable_bit); + }); + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + self.handler_table.read(|table| { + for irq_number in self.pending_irqs() { + match table[irq_number] { + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. + descriptor.handler.handle().expect("Error handling IRQ"); + } + } + } + }) + } + + fn print_handler(&self) { + use crate::info; + + info!(" Peripheral handler:"); + + self.handler_table.read(|table| { + for (i, opt) in table.iter().enumerate() { + if let Some(handler) = opt { + info!(" {: >3}. {}", i, handler.name); + } + } + }); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs new file mode 100644 index 000000000..3adb9137d --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -0,0 +1,537 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! PL011 UART driver. +//! +//! # Resources +//! +//! - +//! - + +use crate::{ + bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, + synchronization, synchronization::IRQSafeNullLock, +}; +use core::{ + fmt, + sync::atomic::{AtomicUsize, Ordering}, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// PL011 UART registers. +// +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. +register_bitfields! { + u32, + + /// Flag Register. + FR [ + /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// Line Control Register, LCR_H. + /// + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. + TXFE OFFSET(7) NUMBITS(1) [], + + /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + TXFF OFFSET(5) NUMBITS(1) [], + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If + /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] + ], + + /// Integer Baud Rate Divisor. + IBRD [ + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] + ], + + /// Fractional Baud Rate Divisor. + FBRD [ + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] + ], + + /// Line Control Register. + LCR_H [ + /// Word length. These bits indicate the number of data bits transmitted or received in a + /// frame. + WLEN OFFSET(5) NUMBITS(2) [ + FiveBit = 0b00, + SixBit = 0b01, + SevenBit = 0b10, + EightBit = 0b11 + ], + + /// Enable FIFOs: + /// + /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding + /// registers. + /// + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). + FEN OFFSET(4) NUMBITS(1) [ + FifosDisabled = 0, + FifosEnabled = 1 + ] + ], + + /// Control Register. + CR [ + /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit + UARTEN OFFSET(0) NUMBITS(1) [ + /// If the UART is disabled in the middle of transmission or reception, it completes the + /// current character before stopping. + Disabled = 0, + Enabled = 1 + ] + ], + + /// Interrupt FIFO Level Select Register. + IFLS [ + /// Receive interrupt FIFO level select. The trigger points for the receive interrupt are as + /// follows. + RXIFLSEL OFFSET(3) NUMBITS(5) [ + OneEigth = 0b000, + OneQuarter = 0b001, + OneHalf = 0b010, + ThreeQuarters = 0b011, + SevenEights = 0b100 + ] + ], + + /// Interrupt Mask Set/Clear Register. + IMSC [ + /// Receive timeout interrupt mask. A read returns the current mask for the UARTRTINTR + /// interrupt. + /// + /// - On a write of 1, the mask of the UARTRTINTR interrupt is set. + /// - A write of 0 clears the mask. + RTIM OFFSET(6) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. + /// + /// - On a write of 1, the mask of the UARTRXINTR interrupt is set. + /// - A write of 0 clears the mask. + RXIM OFFSET(4) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ] + ], + + /// Masked Interrupt Status Register. + MIS [ + /// Receive timeout masked interrupt status. Returns the masked interrupt state of the + /// UARTRTINTR interrupt. + RTMIS OFFSET(6) NUMBITS(1) [], + + /// Receive masked interrupt status. Returns the masked interrupt state of the UARTRXINTR + /// interrupt. + RXMIS OFFSET(4) NUMBITS(1) [] + ], + + /// Interrupt Clear Register. + ICR [ + /// Meta field for all pending interrupts. + ALL OFFSET(0) NUMBITS(11) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + pub RegisterBlock { + (0x00 => DR: ReadWrite), + (0x04 => _reserved1), + (0x18 => FR: ReadOnly), + (0x1c => _reserved2), + (0x24 => IBRD: WriteOnly), + (0x28 => FBRD: WriteOnly), + (0x2c => LCR_H: WriteOnly), + (0x30 => CR: WriteOnly), + (0x34 => IFLS: ReadWrite), + (0x38 => IMSC: ReadWrite), + (0x3C => _reserved3), + (0x40 => MIS: ReadOnly), + (0x44 => ICR: WriteOnly), + (0x48 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct PL011UartInner { + registers: Registers, + chars_written: usize, + chars_read: usize, +} + +// Export the inner struct so that BSPs can use it for the panic handler. +pub use PL011UartInner as PanicUart; + +/// Representation of the UART. +pub struct PL011Uart { + mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, + inner: IRQSafeNullLock, + irq_number: bsp::device_driver::IRQNumber, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl PL011UartInner { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: Registers::new(mmio_start_addr), + chars_written: 0, + chars_read: 0, + } + } + + /// Set up baud rate and characteristics. + /// + /// This results in 8N1 and 921_600 baud. + /// + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. + /// + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub unsafe fn init(&mut self, new_mmio_start_addr: Option) -> Result<(), &'static str> { + if let Some(addr) = new_mmio_start_addr { + self.registers = Registers::new(addr); + } + + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. + self.registers.CR.set(0); + + // Clear all pending interrupts. + self.registers.ICR.write(ICR::ALL::CLEAR); + + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); + self.registers + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); + + // Set RX FIFO fill level at 1/8. + self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); + + // Enable RX IRQ + RX timeout IRQ. + self.registers + .IMSC + .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); + + // Turn the UART on. + self.registers + .CR + .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); + + Ok(()) + } + + /// Send a character. + fn write_char(&mut self, c: char) { + // Spin while TX FIFO full is set, waiting for an empty slot. + while self.registers.FR.matches_all(FR::TXFF::SET) { + cpu::nop(); + } + + // Write the character to the buffer. + self.registers.DR.set(c as u32); + + self.chars_written += 1; + } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { + cpu::nop(); + } + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } +} + +/// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are +/// used to implement the `kernel`'s `print!` and `println!` macros. By implementing `write_str()`, +/// we get `write_fmt()` automatically. +/// +/// The function takes an `&mut self`, so it must be implemented for the inner struct. +/// +/// See [`src/print.rs`]. +/// +/// [`src/print.rs`]: ../../print/index.html +impl fmt::Write for PL011UartInner { + fn write_str(&mut self, s: &str) -> fmt::Result { + for c in s.chars() { + self.write_char(c); + } + + Ok(()) + } +} + +impl PL011Uart { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + /// - The user must ensure to provide correct IRQ numbers. + pub const unsafe fn new( + mmio_descriptor: memory::mmu::MMIODescriptor, + irq_number: bsp::device_driver::IRQNumber, + ) -> Self { + Self { + mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), + inner: IRQSafeNullLock::new(PL011UartInner::new( + mmio_descriptor.start_addr().into_usize(), + )), + irq_number, + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; + +impl driver::interface::DeviceDriver for PL011Uart { + fn compatible(&self) -> &'static str { + "BCM PL011 UART" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; + + self.virt_mmio_start_addr + .store(virt_addr.into_usize(), Ordering::Relaxed); + + Ok(()) + } + + fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { + use bsp::exception::asynchronous::irq_manager; + use exception::asynchronous::{interface::IRQManager, IRQDescriptor}; + + let descriptor = IRQDescriptor { + name: "BCM PL011 UART", + handler: self, + }; + + irq_manager().register_handler(self.irq_number, descriptor)?; + irq_manager().enable(self.irq_number); + + Ok(()) + } + + fn virt_mmio_start_addr(&self) -> Option { + let addr = self.virt_mmio_start_addr.load(Ordering::Relaxed); + + if addr == 0 { + return None; + } + + Some(addr) + } +} + +impl console::interface::Write for PL011Uart { + /// Passthrough of `args` to the `core::fmt::Write` implementation, but guarded by a Mutex to + /// serialize access. + fn write_char(&self, c: char) { + self.inner.lock(|inner| inner.write_char(c)); + } + + fn write_fmt(&self, args: core::fmt::Arguments) -> fmt::Result { + // Fully qualified syntax for the call to `core::fmt::Write::write:fmt()` to increase + // readability. + self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) + } + + fn flush(&self) { + // Spin until TX FIFO empty is set. + self.inner.lock(|inner| inner.flush()); + } +} + +impl console::interface::Read for PL011Uart { + fn read_char(&self) -> char { + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) + } + + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} + } +} + +impl console::interface::Statistics for PL011Uart { + fn chars_written(&self) -> usize { + self.inner.lock(|inner| inner.chars_written) + } + + fn chars_read(&self) -> usize { + self.inner.lock(|inner| inner.chars_read) + } +} + +impl exception::asynchronous::interface::IRQHandler for PL011Uart { + fn handle(&self) -> Result<(), &'static str> { + self.inner.lock(|inner| { + let pending = inner.registers.MIS.extract(); + + // Clear all pending IRQs. + inner.registers.ICR.write(ICR::ALL::CLEAR); + + // Check for any kind of RX interrupt. + if pending.matches_any(MIS::RXMIS::SET + MIS::RTMIS::SET) { + // Echo any received characters. + while let Some(c) = inner.read_char_converting(BlockingMode::NonBlocking) { + inner.write_char(c) + } + } + }); + + Ok(()) + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/common.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/common.rs new file mode 100644 index 000000000..41553b7a5 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/common.rs @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Common device driver code. + +use core::{marker::PhantomData, ops}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct MMIODerefWrapper { + start_addr: usize, + phantom: PhantomData T>, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl MMIODerefWrapper { + /// Create an instance. + pub const unsafe fn new(start_addr: usize) -> Self { + Self { + start_addr, + phantom: PhantomData, + } + } +} + +impl ops::Deref for MMIODerefWrapper { + type Target = T; + + fn deref(&self) -> &Self::Target { + unsafe { &*(self.start_addr as *const _) } + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi.rs new file mode 100644 index 000000000..10d482781 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi.rs @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Top-level BSP file for the Raspberry Pi 3 and 4. + +pub mod console; +pub mod cpu; +pub mod driver; +pub mod exception; +pub mod memory; + +use super::device_driver; +use crate::memory::mmu::MMIODescriptor; +use memory::map::mmio; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static GPIO: device_driver::GPIO = + unsafe { device_driver::GPIO::new(MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE)) }; + +static PL011_UART: device_driver::PL011Uart = unsafe { + device_driver::PL011Uart::new( + MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE), + exception::asynchronous::irq_map::PL011_UART, + ) +}; + +#[cfg(feature = "bsp_rpi3")] +static INTERRUPT_CONTROLLER: device_driver::InterruptController = unsafe { + device_driver::InterruptController::new( + MMIODescriptor::new(mmio::LOCAL_IC_START, mmio::LOCAL_IC_SIZE), + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE), + ) +}; + +#[cfg(feature = "bsp_rpi4")] +static INTERRUPT_CONTROLLER: device_driver::GICv2 = unsafe { + device_driver::GICv2::new( + MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE), + MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE), + ) +}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Board identification. +pub fn board_name() -> &'static str { + #[cfg(feature = "bsp_rpi3")] + { + "Raspberry Pi 3" + } + + #[cfg(feature = "bsp_rpi4")] + { + "Raspberry Pi 4" + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs new file mode 100644 index 000000000..e16b3a3d6 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/console.rs @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP console facilities. + +use super::memory; +use crate::{bsp::device_driver, console, cpu, driver}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// In case of a panic, the panic handler uses this function to take a last shot at printing +/// something before the system is halted. +/// +/// We try to init panic-versions of the GPIO and the UART. The panic versions are not protected +/// with synchronization primitives, which increases chances that we get to print something, even +/// when the kernel's default GPIO or UART instances happen to be locked at the time of the panic. +/// +/// # Safety +/// +/// - Use only for printing during a panic. +#[cfg(not(feature = "test_build"))] +pub unsafe fn panic_console_out() -> impl fmt::Write { + use driver::interface::DeviceDriver; + + let mut panic_gpio = device_driver::PanicGPIO::new(memory::map::mmio::GPIO_START.into_usize()); + let mut panic_uart = + device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START.into_usize()); + + // If remapping of the driver's MMIO already happened, take the remapped start address. + // Otherwise, take a chance with the default physical address. + let maybe_gpio_mmio_start_addr = super::GPIO.virt_mmio_start_addr(); + let maybe_uart_mmio_start_addr = super::PL011_UART.virt_mmio_start_addr(); + + panic_gpio + .init(maybe_gpio_mmio_start_addr) + .unwrap_or_else(|_| cpu::wait_forever()); + panic_gpio.map_pl011_uart(); + panic_uart + .init(maybe_uart_mmio_start_addr) + .unwrap_or_else(|_| cpu::wait_forever()); + + panic_uart +} + +/// Reduced version for test builds. +#[cfg(feature = "test_build")] +pub unsafe fn panic_console_out() -> impl fmt::Write { + use driver::interface::DeviceDriver; + + let mut panic_uart = + device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START.into_usize()); + + let maybe_uart_mmio_start_addr = super::PL011_UART.virt_mmio_start_addr(); + + panic_uart + .init(maybe_uart_mmio_start_addr) + .unwrap_or_else(|_| cpu::qemu_exit_failure()); + + panic_uart +} + +/// Return a reference to the console. +pub fn console() -> &'static impl console::interface::All { + &super::PL011_UART +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use driver::interface::DeviceDriver; + + // Calling the UART's init ensures that the BSP's instance of the UART does remap the MMIO + // addresses. + unsafe { + super::PL011_UART + .init() + .unwrap_or_else(|_| cpu::qemu_exit_failure()); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/cpu.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/cpu.rs new file mode 100644 index 000000000..d09ff9568 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/cpu.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Processor code. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used by `arch` code to find the early boot core. +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/driver.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/driver.rs new file mode 100644 index 000000000..4d5007e4e --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/driver.rs @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP driver support. + +use crate::driver; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Device Driver Manager type. +struct BSPDriverManager { + device_drivers: [&'static (dyn DeviceDriver + Sync); 3], +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { + device_drivers: [ + &super::GPIO, + &super::PL011_UART, + &super::INTERRUPT_CONTROLLER, + ], +}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the driver manager. +pub fn driver_manager() -> &'static impl driver::interface::DriverManager { + &BSP_DRIVER_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use driver::interface::DeviceDriver; + +impl driver::interface::DriverManager for BSPDriverManager { + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[..] + } + + fn early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[0..=1] + } + + fn non_early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[2..] + } + + fn post_early_print_device_driver_init(&self) { + // Configure PL011Uart's output pins. + super::GPIO.map_pl011_uart(); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception.rs new file mode 100644 index 000000000..94d0728ff --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception.rs @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! BSP synchronous and asynchronous exception handling. + +pub mod asynchronous; diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception/asynchronous.rs new file mode 100644 index 000000000..a253daa94 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/exception/asynchronous.rs @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! BSP asynchronous exception handling. + +use crate::{bsp, exception}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +#[cfg(feature = "bsp_rpi3")] +pub(in crate::bsp) mod irq_map { + use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; + + pub const PL011_UART: IRQNumber = IRQNumber::Peripheral(PeripheralIRQ::new(57)); +} + +#[cfg(feature = "bsp_rpi4")] +pub(in crate::bsp) mod irq_map { + use super::bsp::device_driver::IRQNumber; + + pub const PL011_UART: IRQNumber = IRQNumber::new(153); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the IRQ manager. +pub fn irq_manager() -> &'static impl exception::asynchronous::interface::IRQManager< + IRQNumberType = bsp::device_driver::IRQNumber, +> { + &super::super::INTERRUPT_CONTROLLER +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld new file mode 100644 index 000000000..0d46f9d97 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld @@ -0,0 +1 @@ +__kernel_virt_addr_space_size = 2 * 1024 * 1024 * 1024 diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld new file mode 100644 index 000000000..7ad75a40c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: MIT OR Apache-2.0 + * + * Copyright (c) 2018-2021 Andre Richter + */ + +/* This file provides __kernel_virt_addr_space_size */ +INCLUDE src/bsp/raspberrypi/kernel_virt_addr_space_size.ld; + +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + +SECTIONS +{ + . = __rpi_load_addr; + + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; + .text : + { + KEEP(*(.text._start)) + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ + } :segment_rx + + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx + + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; + + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + __rw_start = .; + .data : { *(.data*) } :segment_rw + + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ + .bss : ALIGN(8) + { + __bss_start = .; + *(.bss*); + . = ALIGN(8); + + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ + __bss_end_inclusive = . - 8; + } :NONE + + . = ALIGN(64K); /* Align to page boundary */ + __rw_end_exclusive = .; + + /*********************************************************************************************** + * Guard Page between boot core stack and data + ***********************************************************************************************/ + __boot_core_stack_guard_page_start = .; + . += 64K; + __boot_core_stack_guard_page_end_exclusive = .; + + /*********************************************************************************************** + * Boot Core Stack + ***********************************************************************************************/ + __boot_core_stack_start = .; /* ^ */ + /* | stack */ + . += 512K; /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs new file mode 100644 index 000000000..7f571e080 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management. +//! +//! The physical memory layout after the kernel has been loaded by the Raspberry's firmware, which +//! copies the binary to 0x8_0000: +//! +//! +---------------------------------------------+ +//! | | +//! | Unmapped | +//! | | +//! +---------------------------------------------+ +//! | | rx_start @ 0x8_0000 +//! | .text | +//! | .rodata | +//! | .got | +//! | | rx_end_inclusive +//! +---------------------------------------------+ +//! | | rw_start == rx_end +//! | .data | +//! | .bss | +//! | | rw_end_inclusive +//! +---------------------------------------------+ +//! | | rw_end +//! | Unmapped Boot-core Stack Guard Page | +//! | | +//! +---------------------------------------------+ +//! | | boot_core_stack_start ^ +//! | | | stack +//! | Boot-core Stack | | growth +//! | | | direction +//! | | boot_core_stack_end_inclusive | +//! +---------------------------------------------+ + +pub mod mmu; + +use crate::memory::{Address, Physical, Virtual}; +use core::{cell::UnsafeCell, ops::RangeInclusive}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// Symbols from the linker script. +extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + + static __rw_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; + static __bss_end_inclusive: UnsafeCell; + static __rw_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_start: UnsafeCell<()>; + static __boot_core_stack_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_guard_page_start: UnsafeCell<()>; + static __boot_core_stack_guard_page_end_exclusive: UnsafeCell<()>; +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// The board's physical memory map. +#[rustfmt::skip] +pub(super) mod map { + use super::*; + + /// Physical devices. + #[cfg(feature = "bsp_rpi3")] + pub mod mmio { + use super::*; + + pub const PERIPHERAL_IC_START: Address = Address::new(0x3F00_B200); + pub const PERIPHERAL_IC_SIZE: usize = 0x24; + + pub const GPIO_START: Address = Address::new(0x3F20_0000); + pub const GPIO_SIZE: usize = 0xA0; + + pub const PL011_UART_START: Address = Address::new(0x3F20_1000); + pub const PL011_UART_SIZE: usize = 0x48; + + pub const LOCAL_IC_START: Address = Address::new(0x4000_0000); + pub const LOCAL_IC_SIZE: usize = 0x100; + + pub const END: Address = Address::new(0x4001_0000); + } + + /// Physical devices. + #[cfg(feature = "bsp_rpi4")] + pub mod mmio { + use super::*; + + pub const GPIO_START: Address = Address::new(0xFE20_0000); + pub const GPIO_SIZE: usize = 0xA0; + + pub const PL011_UART_START: Address = Address::new(0xFE20_1000); + pub const PL011_UART_SIZE: usize = 0x48; + + pub const GICD_START: Address = Address::new(0xFF84_1000); + pub const GICD_SIZE: usize = 0x824; + + pub const GICC_START: Address = Address::new(0xFF84_2000); + pub const GICC_SIZE: usize = 0x14; + + pub const END: Address = Address::new(0xFF85_0000); + } + + pub const END: Address = mmio::END; +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn virt_rx_start() -> Address { + Address::new(unsafe { __rx_start.get() as usize }) +} + +/// Size of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_size() -> usize { + unsafe { (__rx_end_exclusive.get() as usize) - (__rx_start.get() as usize) } +} + +/// Start address of the Read+Write (RW) range. +#[inline(always)] +fn virt_rw_start() -> Address { + Address::new(unsafe { __rw_start.get() as usize }) +} + +/// Size of the Read+Write (RW) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rw_size() -> usize { + unsafe { (__rw_end_exclusive.get() as usize) - (__rw_start.get() as usize) } +} + +/// Start address of the boot core's stack. +#[inline(always)] +fn virt_boot_core_stack_start() -> Address { + Address::new(unsafe { __boot_core_stack_start.get() as usize }) +} + +/// Size of the boot core's stack. +#[inline(always)] +fn boot_core_stack_size() -> usize { + unsafe { + (__boot_core_stack_end_exclusive.get() as usize) - (__boot_core_stack_start.get() as usize) + } +} + +/// Start address of the boot core's stack guard page. +#[inline(always)] +fn virt_boot_core_stack_guard_page_start() -> Address { + Address::new(unsafe { __boot_core_stack_guard_page_start.get() as usize }) +} + +/// Size of the boot core's stack guard page. +#[inline(always)] +fn boot_core_stack_guard_page_size() -> usize { + unsafe { + (__boot_core_stack_guard_page_end_exclusive.get() as usize) + - (__boot_core_stack_guard_page_start.get() as usize) + } +} + +/// Exclusive end address of the physical address space. +#[inline(always)] +fn phys_addr_space_end() -> Address { + map::END +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the inclusive range spanning the .bss section. +/// +/// # Safety +/// +/// - Values are provided by the linker script and must be trusted as-is. +/// - The linker-provided addresses must be u64 aligned. +pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { + let range; + unsafe { + range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); + } + assert!(!range.is_empty()); + + range +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs new file mode 100644 index 000000000..4ff01e436 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs @@ -0,0 +1,187 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management Unit. + +use crate::{ + common, + memory::{ + mmu as generic_mmu, + mmu::{ + AccessPermissions, AddressSpace, AssociatedTranslationTable, AttributeFields, + MemAttributes, Page, PageSliceDescriptor, TranslationGranule, + }, + Physical, Virtual, + }, + synchronization::InitStateLock, +}; +use core::convert::TryInto; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +type KernelTranslationTable = + ::TableStartFromBottom; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to +/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; + +/// The kernel's virtual address space defined by this BSP. +pub type KernelVirtAddrSpace = AddressSpace<{ get_virt_addr_space_size() }>; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +/// The kernel translation tables. +/// +/// It is mandatory that InitStateLock is transparent. +/// +/// That is, `size_of(InitStateLock) == size_of(KernelTranslationTable)`. +/// There is a unit tests that checks this porperty. +#[link_section = ".data"] +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new_for_precompute()); + +/// This value is needed during early boot for MMU setup. +/// +/// This will be patched to the correct value by the "translation table tool" after linking. This +/// given value here is just a dummy. +#[link_section = ".text._start_arguments"] +#[no_mangle] +static PHYS_KERNEL_TABLES_BASE_ADDR: u64 = 0xCCCCAAAAFFFFEEEE; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// This is a hack for retrieving the value for the kernel's virtual address space size as a +/// constant from a common place, since it is needed as a compile-time/link-time constant in both, +/// the linker script and the Rust sources. +const fn get_virt_addr_space_size() -> usize { + let __kernel_virt_addr_space_size; + + include!("../kernel_virt_addr_space_size.ld"); + + __kernel_virt_addr_space_size +} + +/// Helper function for calculating the number of pages the given parameter spans. +const fn size_to_num_pages(size: usize) -> usize { + assert!(size > 0); + assert!(size % KernelGranule::SIZE == 0); + + size >> KernelGranule::SHIFT +} + +/// The Read+Execute (RX) pages of the kernel binary. +fn virt_rx_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rx_size()); + + PageSliceDescriptor::from_addr(super::virt_rx_start(), num_pages) +} + +/// The Read+Write (RW) pages of the kernel binary. +fn virt_rw_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rw_size()); + + PageSliceDescriptor::from_addr(super::virt_rw_start(), num_pages) +} + +/// The boot core's stack. +fn virt_boot_core_stack_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_size()); + + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) +} + +// There is no reason to expect the following conversions to fail, since they were generated offline +// by the `translation table tool`. If it doesn't work, a panic due to the unwrap is justified. + +/// The Read+Execute (RX) pages of the kernel binary. +fn phys_rx_page_desc() -> PageSliceDescriptor { + virt_rx_page_desc().try_into().unwrap() +} + +/// The Read+Write (RW) pages of the kernel binary. +fn phys_rw_page_desc() -> PageSliceDescriptor { + virt_rw_page_desc().try_into().unwrap() +} + +/// The boot core's stack. +fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { + virt_boot_core_stack_page_desc().try_into().unwrap() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the kernel's translation tables. +pub fn kernel_translation_tables() -> &'static InitStateLock { + &KERNEL_TABLES +} + +/// The boot core's stack guard page. +pub fn virt_boot_core_stack_guard_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_guard_page_size()); + + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_guard_page_start(), num_pages) +} + +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( + super::phys_addr_space_end().into_usize(), + KernelGranule::SIZE, + ) as *const Page<_> +} + +/// Add mapping records for the kernel binary. +/// +/// The actual translation table entries for the kernel binary are generated using the offline +/// `translation table tool` and patched into the kernel binary. This function just adds the mapping +/// record entries. +/// +/// It must be ensured that these entries are in sync with the offline tool. +pub fn kernel_add_mapping_records_for_precomputed() { + generic_mmu::kernel_add_mapping_record( + "Kernel code and RO data", + &virt_rx_page_desc(), + &phys_rx_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadOnly, + execute_never: false, + }, + ); + + generic_mmu::kernel_add_mapping_record( + "Kernel data and bss", + &virt_rw_page_desc(), + &phys_rw_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + ); + + generic_mmu::kernel_add_mapping_record( + "Kernel boot-core stack", + &virt_boot_core_stack_page_desc(), + &phys_boot_core_stack_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + ); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/common.rs b/15_virtual_mem_part3_precomputed_tables/src/common.rs new file mode 100644 index 000000000..71443cfbd --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/common.rs @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! General purpose code. + +/// Check if a value is aligned to a given size. +#[inline(always)] +pub const fn is_aligned(value: usize, alignment: usize) -> bool { + assert!(alignment.is_power_of_two()); + + (value & (alignment - 1)) == 0 +} + +/// Align down. +#[inline(always)] +pub const fn align_down(value: usize, alignment: usize) -> usize { + assert!(alignment.is_power_of_two()); + + value & !(alignment - 1) +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/console.rs b/15_virtual_mem_part3_precomputed_tables/src/console.rs new file mode 100644 index 000000000..c3154ba23 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/console.rs @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! System console. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Console interfaces. +pub mod interface { + use core::fmt; + + /// Console write functions. + pub trait Write { + /// Write a single character. + fn write_char(&self, c: char); + + /// Write a Rust format string. + fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; + + /// Block until the last buffered character has been physically put on the TX wire. + fn flush(&self); + } + + /// Console read functions. + pub trait Read { + /// Read a single character. + fn read_char(&self) -> char { + ' ' + } + + /// Clear RX buffers, if any. + fn clear_rx(&self); + } + + /// Console statistics. + pub trait Statistics { + /// Return the number of characters written. + fn chars_written(&self) -> usize { + 0 + } + + /// Return the number of characters read. + fn chars_read(&self) -> usize { + 0 + } + } + + /// Trait alias for a full-fledged console. + pub trait All = Write + Read + Statistics; +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/cpu.rs b/15_virtual_mem_part3_precomputed_tables/src/cpu.rs new file mode 100644 index 000000000..36b6a9d4c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/cpu.rs @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Processor code. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/cpu.rs"] +mod arch_cpu; + +mod boot; + +pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff --git a/15_virtual_mem_part3_precomputed_tables/src/cpu/boot.rs b/15_virtual_mem_part3_precomputed_tables/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/15_virtual_mem_part3_precomputed_tables/src/cpu/smp.rs b/15_virtual_mem_part3_precomputed_tables/src/cpu/smp.rs new file mode 100644 index 000000000..38230ce1e --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/cpu/smp.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Symmetric multiprocessing. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/smp.rs"] +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/15_virtual_mem_part3_precomputed_tables/src/driver.rs b/15_virtual_mem_part3_precomputed_tables/src/driver.rs new file mode 100644 index 000000000..83fac4a7c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/driver.rs @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Driver support. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Driver interfaces. +pub mod interface { + /// Device Driver functions. + pub trait DeviceDriver { + /// Return a compatibility string for identifying the driver. + fn compatible(&self) -> &'static str; + + /// Called by the kernel to bring up the device. + /// + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + unsafe fn init(&self) -> Result<(), &'static str> { + Ok(()) + } + + /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// + /// Rust's type system will prevent a call to this function unless the calling instance + /// itself has static lifetime. + fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { + Ok(()) + } + + /// After MMIO remapping, returns the new virtual start address. + /// + /// This API assumes a driver has only a single, contiguous MMIO aperture, which will not be + /// the case for more complex devices. This API will likely change in future tutorials. + fn virt_mmio_start_addr(&self) -> Option { + None + } + } + + /// Device driver management functions. + /// + /// The `BSP` is supposed to supply one global instance. + pub trait DriverManager { + /// Return a slice of references to all `BSP`-instantiated drivers. + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Return only those drivers needed for the BSP's early printing functionality. + /// + /// For example, the default UART. + fn early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Return all drivers minus early-print drivers. + fn non_early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Initialization code that runs after the early print driver init. + fn post_early_print_device_driver_init(&self); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/exception.rs b/15_virtual_mem_part3_precomputed_tables/src/exception.rs new file mode 100644 index 000000000..3c5e7bc83 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/exception.rs @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Synchronous and asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/exception.rs"] +mod arch_exception; + +pub mod asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Kernel privilege levels. +#[allow(missing_docs)] +#[derive(PartialEq)] +pub enum PrivilegeLevel { + User, + Kernel, + Hypervisor, + Unknown, +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Libkernel unit tests must execute in kernel mode. + #[kernel_test] + fn test_runner_executes_in_kernel_mode() { + let (level, _) = current_privilege_level(); + + assert!(level == PrivilegeLevel::Kernel) + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/src/exception/asynchronous.rs new file mode 100644 index 000000000..e9aace357 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/exception/asynchronous.rs @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/exception/asynchronous.rs"] +mod arch_asynchronous; + +use core::{fmt, marker::PhantomData}; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::{ + is_local_irq_masked, local_irq_mask, local_irq_mask_save, local_irq_restore, local_irq_unmask, + print_state, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Interrupt descriptor. +#[derive(Copy, Clone)] +pub struct IRQDescriptor { + /// Descriptive name. + pub name: &'static str, + + /// Reference to handler trait object. + pub handler: &'static (dyn interface::IRQHandler + Sync), +} + +/// IRQContext token. +/// +/// An instance of this type indicates that the local core is currently executing in IRQ +/// context, aka executing an interrupt vector or subcalls of it. +/// +/// Concept and implementation derived from the `CriticalSection` introduced in +/// +#[derive(Clone, Copy)] +pub struct IRQContext<'irq_context> { + _0: PhantomData<&'irq_context ()>, +} + +/// Asynchronous exception handling interfaces. +pub mod interface { + + /// Implemented by types that handle IRQs. + pub trait IRQHandler { + /// Called when the corresponding interrupt is asserted. + fn handle(&self) -> Result<(), &'static str>; + } + + /// IRQ management functions. + /// + /// The `BSP` is supposed to supply one global instance. Typically implemented by the + /// platform's interrupt controller. + pub trait IRQManager { + /// The IRQ number type depends on the implementation. + type IRQNumberType; + + /// Register a handler. + fn register_handler( + &self, + irq_number: Self::IRQNumberType, + descriptor: super::IRQDescriptor, + ) -> Result<(), &'static str>; + + /// Enable an interrupt in the controller. + fn enable(&self, irq_number: Self::IRQNumberType); + + /// Handle pending interrupts. + /// + /// This function is called directly from the CPU's IRQ exception vector. On AArch64, + /// this means that the respective CPU core has disabled exception handling. + /// This function can therefore not be preempted and runs start to finish. + /// + /// Takes an IRQContext token to ensure it can only be called from IRQ context. + #[allow(clippy::trivially_copy_pass_by_ref)] + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &super::IRQContext<'irq_context>, + ); + + /// Print list of registered handlers. + fn print_handler(&self); + } +} + +/// A wrapper type for IRQ numbers with integrated range sanity check. +#[derive(Copy, Clone)] +pub struct IRQNumber(usize); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl<'irq_context> IRQContext<'irq_context> { + /// Creates an IRQContext token. + /// + /// # Safety + /// + /// - This must only be called when the current core is in an interrupt context and will not + /// live beyond the end of it. That is, creation is allowed in interrupt vector functions. For + /// example, in the ARMv8-A case, in `extern "C" fn current_elx_irq()`. + /// - Note that the lifetime `'irq_context` of the returned instance is unconstrained. User code + /// must not be able to influence the lifetime picked for this type, since that might cause it + /// to be inferred to `'static`. + #[inline(always)] + pub unsafe fn new() -> Self { + IRQContext { _0: PhantomData } + } +} + +impl IRQNumber<{ MAX_INCLUSIVE }> { + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self { 0: number } + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} + +/// Executes the provided closure while IRQs are masked on the executing core. +/// +/// While the function temporarily changes the HW state of the executing core, it restores it to the +/// previous state before returning, so this is deemed safe. +#[inline(always)] +pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { + let ret: T; + + unsafe { + let saved = local_irq_mask_save(); + ret = f(); + local_irq_restore(saved); + } + + ret +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs new file mode 100644 index 000000000..3346134ad --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +// Rust embedded logo for `make doc`. +#![doc(html_logo_url = "https://git.io/JeGIp")] + +//! The `kernel` library. +//! +//! Used to compose the final kernel binary. +//! +//! # Code organization and architecture +//! +//! The code is divided into different *modules*, each representing a typical **subsystem** of the +//! `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example, +//! `src/memory.rs` contains code that is concerned with all things memory management. +//! +//! ## Visibility of processor architecture code +//! +//! Some of the `kernel`'s subsystems depend on low-level code that is specific to the target +//! processor architecture. For each supported processor architecture, there exists a subfolder in +//! `src/_arch`, for example, `src/_arch/aarch64`. +//! +//! The architecture folders mirror the subsystem modules laid out in `src`. For example, +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. +//! +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. +//! +//! ## BSP code +//! +//! `BSP` stands for Board Support Package. `BSP` code is organized under `src/bsp.rs` and contains +//! target board specific definitions and functions. These are things such as the board's memory map +//! or instances of drivers for devices that are featured on the respective board. +//! +//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. +//! +//! ## Kernel interfaces +//! +//! Both `arch` and `bsp` contain code that is conditionally compiled depending on the actual target +//! and board for which the kernel is compiled. For example, the `interrupt controller` hardware of +//! the `Raspberry Pi 3` and the `Raspberry Pi 4` is different, but we want the rest of the `kernel` +//! code to play nicely with any of the two without much hassle. +//! +//! In order to provide a clean abstraction between `arch`, `bsp` and `generic kernel code`, +//! `interface` traits are provided *whenever possible* and *where it makes sense*. They are defined +//! in the respective subsystem module and help to enforce the idiom of *program to an interface, +//! not an implementation*. For example, there will be a common IRQ handling interface which the two +//! different interrupt controller `drivers` of both Raspberrys will implement, and only export the +//! interface to the rest of the `kernel`. +//! +//! ``` +//! +-------------------+ +//! | Interface (Trait) | +//! | | +//! +--+-------------+--+ +//! ^ ^ +//! | | +//! | | +//! +----------+--+ +--+----------+ +//! | kernel code | | bsp code | +//! | | | arch code | +//! +-------------+ +-------------+ +//! ``` +//! +//! # Summary +//! +//! For a logical `kernel` subsystem, corresponding code can be distributed over several physical +//! locations. Here is an example for the **memory** subsystem: +//! +//! - `src/memory.rs` and `src/memory/**/*` +//! - Common code that is agnostic of target processor architecture and `BSP` characteristics. +//! - Example: A function to zero a chunk of memory. +//! - Interfaces for the memory subsystem that are implemented by `arch` or `BSP` code. +//! - Example: An `MMU` interface that defines `MMU` function prototypes. +//! - `src/bsp/__board_name__/memory.rs` and `src/bsp/__board_name__/memory/**/*` +//! - `BSP` specific code. +//! - Example: The board's memory map (physical addresses of DRAM and MMIO devices). +//! - `src/_arch/__arch_name__/memory.rs` and `src/_arch/__arch_name__/memory/**/*` +//! - Processor architecture specific code. +//! - Example: Implementation of the `MMU` interface for the `__arch_name__` processor +//! architecture. +//! +//! From a namespace perspective, **memory** subsystem code lives in: +//! +//! - `crate::memory::*` +//! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + +#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(incomplete_features)] +#![feature(asm)] +#![feature(const_evaluatable_checked)] +#![feature(const_fn)] +#![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] +#![feature(const_panic)] +#![feature(core_intrinsics)] +#![feature(format_args_nl)] +#![feature(global_asm)] +#![feature(linkage)] +#![feature(panic_info_message)] +#![feature(trait_alias)] +#![no_std] +// Testing +#![cfg_attr(test, no_main)] +#![feature(custom_test_frameworks)] +#![reexport_test_harness_main = "test_main"] +#![test_runner(crate::test_runner)] + +mod panic_wait; +mod runtime_init; +mod synchronization; + +pub mod bsp; +pub mod common; +pub mod console; +pub mod cpu; +pub mod driver; +pub mod exception; +pub mod memory; +pub mod print; +pub mod state; +pub mod time; + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +/// The default runner for unit tests. +pub fn test_runner(tests: &[&test_types::UnitTest]) { + println!("Running {} tests", tests.len()); + println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { + print!("{:>3}. {:.<58}", i + 1, test.name); + + // Run the actual test. + (test.test_func)(); + + // Failed tests call panic!(). Execution reaches here only if the test has passed. + println!("[ok]") + } +} + +/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +#[cfg(test)] +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + test_main(); + + cpu::qemu_exit_success() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/main.rs b/15_virtual_mem_part3_precomputed_tables/src/main.rs new file mode 100644 index 000000000..c317bb96f --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/main.rs @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +// Rust embedded logo for `make doc`. +#![doc(html_logo_url = "https://git.io/JeGIp")] + +//! The `kernel` binary. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; + +/// Early init code. +/// +/// # Safety +/// +/// - Only a single core must be active and running this function. +/// - Printing will not work until the respective driver's MMIO is remapped. +#[no_mangle] +unsafe fn kernel_init() -> ! { + use driver::interface::DriverManager; + + exception::handling_init(); + + // Add the mapping records for the precomputed entries first, so that they appear on the top of + // the list. + bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); + + // Bring up the drivers needed for printing first. + for i in bsp::driver::driver_manager() + .early_print_device_drivers() + .iter() + { + // Any encountered errors cannot be printed yet, obviously, so just safely park the CPU. + i.init().unwrap_or_else(|_| cpu::wait_forever()); + } + bsp::driver::driver_manager().post_early_print_device_driver_init(); + // Printing available from here on. + + // Now bring up the remaining drivers. + for i in bsp::driver::driver_manager() + .non_early_print_device_drivers() + .iter() + { + if let Err(x) = i.init() { + panic!("Error loading driver: {}: {}", i.compatible(), x); + } + } + + // Let device drivers register and enable their handlers with the interrupt controller. + for i in bsp::driver::driver_manager().all_device_drivers() { + if let Err(msg) = i.register_and_enable_irq_handler() { + warn!("Error registering IRQ handler: {}", msg); + } + } + + // Unmask interrupts on the boot CPU core. + exception::asynchronous::local_irq_unmask(); + + // Announce conclusion of the kernel_init() phase. + state::state_manager().transition_to_single_core_main(); + + // Transition from unsafe to safe. + kernel_main() +} + +/// The main function running after the early init. +fn kernel_main() -> ! { + use driver::interface::DriverManager; + use exception::asynchronous::interface::IRQManager; + + info!("Booting on: {}", bsp::board_name()); + + info!("MMU online:"); + memory::mmu::kernel_print_mappings(); + + let (_, privilege_level) = exception::current_privilege_level(); + info!("Current privilege level: {}", privilege_level); + + info!("Exception handling state:"); + exception::asynchronous::print_state(); + + info!( + "Architectural timer resolution: {} ns", + time::time_manager().resolution().as_nanos() + ); + + info!("Drivers loaded:"); + for (i, driver) in bsp::driver::driver_manager() + .all_device_drivers() + .iter() + .enumerate() + { + info!(" {}. {}", i + 1, driver.compatible()); + } + + info!("Registered IRQ handlers:"); + bsp::exception::asynchronous::irq_manager().print_handler(); + + info!("Echoing input now"); + cpu::wait_forever(); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory.rs b/15_virtual_mem_part3_precomputed_tables/src/memory.rs new file mode 100644 index 000000000..87ffa81f5 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/memory.rs @@ -0,0 +1,202 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management. + +pub mod mmu; + +use crate::common; +use core::{ + convert::TryFrom, + fmt, + marker::PhantomData, + ops::{AddAssign, RangeInclusive, SubAssign}, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Metadata trait for marking the type of an address. +pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} + +/// Zero-sized type to mark a physical address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Physical {} + +/// Zero-sized type to mark a virtual address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Virtual {} + +/// Generic address type. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub struct Address { + value: usize, + _address_type: PhantomData ATYPE>, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl AddressType for Physical {} +impl AddressType for Virtual {} + +impl Address { + /// Create an instance. + pub const fn new(value: usize) -> Self { + Self { + value, + _address_type: PhantomData, + } + } + + /// Align down. + pub const fn align_down(self, alignment: usize) -> Self { + let aligned = common::align_down(self.value, alignment); + + Self { + value: aligned, + _address_type: PhantomData, + } + } + + /// Converts `Address` into an usize. + pub const fn into_usize(self) -> usize { + self.value + } +} + +impl TryFrom> for Address { + type Error = mmu::TranslationError; + + fn try_from(virt: Address) -> Result { + mmu::try_virt_to_phys(virt) + } +} + +impl core::ops::Add for Address { + type Output = Self; + + fn add(self, other: usize) -> Self { + Self { + value: self.value + other, + _address_type: PhantomData, + } + } +} + +impl AddAssign for Address { + fn add_assign(&mut self, other: Self) { + *self = Self { + value: self.value + other.into_usize(), + _address_type: PhantomData, + }; + } +} + +impl core::ops::Sub for Address { + type Output = Self; + + fn sub(self, other: usize) -> Self { + Self { + value: self.value - other, + _address_type: PhantomData, + } + } +} + +impl SubAssign for Address { + fn sub_assign(&mut self, other: Self) { + *self = Self { + value: self.value - other.into_usize(), + _address_type: PhantomData, + }; + } +} + +impl fmt::Display for Address { + // Don't expect to see physical addresses greater than 40 bit. + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q3: u8 = ((self.value >> 32) & 0xff) as u8; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:02x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + +impl fmt::Display for Address { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q4: u16 = ((self.value >> 48) & 0xffff) as u16; + let q3: u16 = ((self.value >> 32) & 0xffff) as u16; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:04x}_", q4)?; + write!(f, "{:04x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + +/// Zero out an inclusive memory range. +/// +/// # Safety +/// +/// - `range.start` and `range.end` must be valid. +/// - `range.start` and `range.end` must be `T` aligned. +pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) +where + T: From, +{ + let mut ptr = *range.start(); + let end_inclusive = *range.end(); + + while ptr <= end_inclusive { + core::ptr::write_volatile(ptr, T::from(0)); + ptr = ptr.offset(1); + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check `zero_volatile()`. + #[kernel_test] + fn zero_volatile_works() { + let mut x: [usize; 3] = [10, 11, 12]; + let x_range = x.as_mut_ptr_range(); + let x_range_inclusive = + RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); + + unsafe { zero_volatile(x_range_inclusive) }; + + assert_eq!(x, [0, 0, 0]); + } + + /// Check `bss` section layout. + #[kernel_test] + fn bss_section_is_sane() { + use crate::bsp::memory::bss_range_inclusive; + use core::mem; + + let start = *bss_range_inclusive().start() as usize; + let end = *bss_range_inclusive().end() as usize; + + assert_eq!(start % mem::size_of::(), 0); + assert_eq!(end % mem::size_of::(), 0); + assert!(end >= start); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs new file mode 100644 index 000000000..a7109ec29 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs @@ -0,0 +1,265 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/memory/mmu.rs"] +mod arch_mmu; + +mod mapping_record; +mod translation_table; +mod types; + +use crate::{ + bsp, + memory::{Address, Physical, Virtual}, + synchronization, warn, +}; +use core::fmt; + +pub use types::*; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + +/// Translation error variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum TranslationError { + MMUDisabled, + Aborted, +} + +/// Memory Management interfaces. +pub mod interface { + use super::*; + + /// MMU functions. + pub trait MMU { + /// Turns on the MMU for the first time and enables data and instruction caching. + /// + /// # Safety + /// + /// - Changes the HW's global state. + unsafe fn enable_mmu_and_caching( + &self, + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; + + /// Try to translate a virtual address to a physical address. + /// + /// Will only succeed if there exists a valid mapping for the input VA. + fn try_virt_to_phys( + &self, + virt: Address, + ) -> Result, TranslationError>; + } +} + +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes properties of an address space. +pub struct AddressSpace; + +/// Intended to be implemented for [`AddressSpace`]. +pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// + /// [0, AS_SIZE - 1] + type TableStartFromBottom; +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +use interface::MMU; +use synchronization::interface::ReadWriteEx; +use translation_table::interface::TranslationTable; + +/// Map pages in the kernel's translation tables. +/// +/// No input checks done, input is passed through to the architectural implementation. +/// +/// # Safety +/// +/// - See `map_pages_at()`. +/// - Does not prevent aliasing. +unsafe fn kernel_map_pages_at_unchecked( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; + + kernel_add_mapping_record(name, virt_pages, phys_pages, attr); + + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's mask. + pub const MASK: usize = Self::SIZE - 1; + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpace { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); + + AS_SIZE + } +} + +/// Add an entry to the mapping info record. +pub fn kernel_add_mapping_record( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) { + if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { + warn!("{}", x); + } +} + +/// Raw mapping of virtual to physical pages in the kernel translation tables. +/// +/// Prevents mapping into the MMIO range of the tables. +/// +/// # Safety +/// +/// - See `kernel_map_pages_at_unchecked()`. +/// - Does not prevent aliasing. Currently, the callers must be trusted. +pub unsafe fn kernel_map_pages_at( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + let is_mmio = bsp::memory::mmu::kernel_translation_tables() + .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); + if is_mmio { + return Err("Attempt to manually map into MMIO region"); + } + + kernel_map_pages_at_unchecked(name, virt_pages, phys_pages, attr)?; + + Ok(()) +} + +/// MMIO remapping in the kernel translation tables. +/// +/// Typically used by device drivers. +/// +/// # Safety +/// +/// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. +pub unsafe fn kernel_map_mmio( + name: &'static str, + mmio_descriptor: &MMIODescriptor, +) -> Result, &'static str> { + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let offset_into_start_page = + mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + + // Check if an identical page slice has been mapped for another driver. If so, reuse it. + let virt_addr = if let Some(addr) = + mapping_record::kernel_find_and_insert_mmio_duplicate(mmio_descriptor, name) + { + addr + // Otherwise, allocate a new virtual page slice and map it. + } else { + let virt_pages: PageSliceDescriptor = + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; + + kernel_map_pages_at_unchecked( + name, + &virt_pages, + &phys_pages, + &AttributeFields { + mem_attributes: MemAttributes::Device, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + )?; + + virt_pages.start_addr() + }; + + Ok(virt_addr + offset_into_start_page) +} + +/// Try to translate a virtual address to a physical address. +/// +/// Will only succeed if there exists a valid mapping for the input VA. +pub fn try_virt_to_phys(virt: Address) -> Result, TranslationError> { + arch_mmu::mmu().try_virt_to_phys(virt) +} + +/// Enable the MMU and data + instruction caching. +/// +/// # Safety +/// +/// - Crucial function during kernel init. Changes the the complete memory view of the processor. +#[inline(always)] +pub unsafe fn enable_mmu_and_caching( + phys_tables_base_addr: Address, +) -> Result<(), MMUEnableError> { + arch_mmu::mmu().enable_mmu_and_caching(phys_tables_base_addr) +} + +/// Human-readable print of all recorded kernel mappings. +pub fn kernel_print_mappings() { + mapping_record::kernel_print() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs new file mode 100644 index 000000000..eab62fb39 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! A record of mapped pages. + +use super::{ + AccessPermissions, Address, AttributeFields, MMIODescriptor, MemAttributes, + PageSliceDescriptor, Physical, Virtual, +}; +use crate::{info, synchronization, synchronization::InitStateLock, warn}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Type describing a virtual memory mapping. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +struct MappingRecordEntry { + pub users: [Option<&'static str>; 5], + pub phys_pages: PageSliceDescriptor, + pub virt_start_addr: Address, + pub attribute_fields: AttributeFields, +} + +struct MappingRecord { + inner: [Option; 12], +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static KERNEL_MAPPING_RECORD: InitStateLock = + InitStateLock::new(MappingRecord::new()); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl MappingRecordEntry { + pub fn new( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Self { + Self { + users: [Some(name), None, None, None, None], + phys_pages: *phys_pages, + virt_start_addr: virt_pages.start_addr(), + attribute_fields: *attr, + } + } + + fn find_next_free_user(&mut self) -> Result<&mut Option<&'static str>, &'static str> { + if let Some(x) = self.users.iter_mut().find(|x| x.is_none()) { + return Ok(x); + }; + + Err("Storage for user info exhausted") + } + + pub fn add_user(&mut self, user: &'static str) -> Result<(), &'static str> { + let x = self.find_next_free_user()?; + *x = Some(user); + Ok(()) + } +} + +impl MappingRecord { + pub const fn new() -> Self { + Self { inner: [None; 12] } + } + + fn find_next_free(&mut self) -> Result<&mut Option, &'static str> { + if let Some(x) = self.inner.iter_mut().find(|x| x.is_none()) { + return Ok(x); + } + + Err("Storage for mapping info exhausted") + } + + fn find_duplicate( + &mut self, + phys_pages: &PageSliceDescriptor, + ) -> Option<&mut MappingRecordEntry> { + self.inner + .iter_mut() + .filter(|x| x.is_some()) + .map(|x| x.as_mut().unwrap()) + .filter(|x| x.attribute_fields.mem_attributes == MemAttributes::Device) + .find(|x| x.phys_pages == *phys_pages) + } + + pub fn add( + &mut self, + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + let x = self.find_next_free()?; + + *x = Some(MappingRecordEntry::new(name, virt_pages, phys_pages, attr)); + Ok(()) + } + + pub fn print(&self) { + const KIB_RSHIFT: u32 = 10; // log2(1024). + const MIB_RSHIFT: u32 = 20; // log2(1024 * 1024). + + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + info!( + " {:^44} {:^30} {:^7} {:^9} {:^35}", + "Virtual", "Physical", "Size", "Attr", "Entity" + ); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + + for i in self + .inner + .iter() + .filter(|x| x.is_some()) + .map(|x| x.unwrap()) + { + let virt_start = i.virt_start_addr; + let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; + let phys_start = i.phys_pages.start_addr(); + let phys_end_inclusive = i.phys_pages.end_addr_inclusive(); + let size = i.phys_pages.size(); + + let (size, unit) = if (size >> MIB_RSHIFT) > 0 { + (size >> MIB_RSHIFT, "MiB") + } else if (size >> KIB_RSHIFT) > 0 { + (size >> KIB_RSHIFT, "KiB") + } else { + (size, "Byte") + }; + + let attr = match i.attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => "C", + MemAttributes::Device => "Dev", + }; + + let acc_p = match i.attribute_fields.acc_perms { + AccessPermissions::ReadOnly => "RO", + AccessPermissions::ReadWrite => "RW", + }; + + let xn = if i.attribute_fields.execute_never { + "XN" + } else { + "X" + }; + + info!( + " {}..{} --> {}..{} | \ + {: >3} {} | {: <3} {} {: <2} | {}", + virt_start, + virt_end_inclusive, + phys_start, + phys_end_inclusive, + size, + unit, + attr, + acc_p, + xn, + i.users[0].unwrap() + ); + + for k in i.users[1..].iter() { + if let Some(additional_user) = *k { + info!( + " | {}", + additional_user + ); + } + } + } + + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use synchronization::interface::ReadWriteEx; + +/// Add an entry to the mapping info record. +pub fn kernel_add( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, virt_pages, phys_pages, attr)) +} + +pub fn kernel_find_and_insert_mmio_duplicate( + mmio_descriptor: &MMIODescriptor, + new_user: &'static str, +) -> Option> { + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + + KERNEL_MAPPING_RECORD.write(|mr| { + let dup = mr.find_duplicate(&phys_pages)?; + + if let Err(x) = dup.add_user(new_user) { + warn!("{}", x); + } + + Some(dup.virt_start_addr) + }) +} + +/// Human-readable print of all recorded kernel mappings. +pub fn kernel_print() { + KERNEL_MAPPING_RECORD.read(|mr| mr.print()); +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bf9bed8cb --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/translation_table.rs @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +use crate::memory::{ + mmu::{AttributeFields, PageSliceDescriptor}, + Physical, Virtual, +}; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +#[cfg(target_arch = "aarch64")] +pub use arch_translation_table::FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Translation table interfaces. +pub mod interface { + use super::*; + + /// Translation table operations. + pub trait TranslationTable { + /// Anything that needs to run before any of the other provided functions can be used. + /// + /// # Safety + /// + /// - Implementor must ensure that this function can run only once or is harmless if invoked + /// multiple times. + fn init(&mut self) -> Result<(), &'static str>; + + /// Map the given virtual pages to the given physical pages. + /// + /// # Safety + /// + /// - Using wrong attributes can cause multiple issues of different nature in the system. + /// - It is not required that the architectural implementation prevents aliasing. That is, + /// mapping to the same physical memory using multiple virtual addresses, which would + /// break Rust's ownership assumptions. This should be protected against in the kernel's + /// generic MMU code. + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str>; + + /// Obtain a free virtual page slice in the MMIO region. + /// + /// The "MMIO region" is a distinct region of the implementor's choice, which allows + /// differentiating MMIO addresses from others. This can speed up debugging efforts. + /// Ideally, those MMIO addresses are also standing out visually so that a human eye can + /// identify them. For example, by allocating them from near the end of the virtual address + /// space. + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str>; + + /// Check if a virtual page splice is in the "MMIO region". + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use crate::{bsp, memory::Address}; + use arch_translation_table::MinSizeTranslationTable; + use interface::TranslationTable; + use test_macros::kernel_test; + + /// Sanity checks for the TranslationTable implementation. + #[kernel_test] + fn translationtable_implementation_sanity() { + // This will occupy a lot of space on the stack. + let mut tables = MinSizeTranslationTable::new_for_runtime(); + + assert!(tables.init().is_ok()); + + let x = tables.next_mmio_virt_page_slice(0); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(1_0000_0000); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(2).unwrap(); + assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); + + assert_eq!(tables.is_virt_page_slice_mmio(&x), true); + + assert_eq!( + tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), + false + ); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs new file mode 100644 index 000000000..dc05fd70c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/types.rs @@ -0,0 +1,217 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit types. + +use crate::{ + bsp, common, + memory::{Address, AddressType, Physical, Virtual}, +}; +use core::{ + convert::{From, TryFrom}, + marker::PhantomData, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Generic page type. +#[repr(C)] +pub struct Page { + inner: [u8; bsp::memory::mmu::KernelGranule::SIZE], + _address_type: PhantomData, +} + +/// Type describing a slice of pages. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub struct PageSliceDescriptor { + start: Address, + num_pages: usize, +} + +/// Architecture agnostic memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum MemAttributes { + CacheableDRAM, + Device, +} + +/// Architecture agnostic access permissions. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub enum AccessPermissions { + ReadOnly, + ReadWrite, +} + +/// Collection of memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub struct AttributeFields { + pub mem_attributes: MemAttributes, + pub acc_perms: AccessPermissions, + pub execute_never: bool, +} + +/// An MMIO descriptor for use in device drivers. +#[derive(Copy, Clone)] +pub struct MMIODescriptor { + start_addr: Address, + size: usize, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +//------------------------------------------------------------------------------ +// Page +//------------------------------------------------------------------------------ + +impl Page { + /// Get a pointer to the instance. + pub const fn as_ptr(&self) -> *const Page { + self as *const _ + } +} + +//------------------------------------------------------------------------------ +// PageSliceDescriptor +//------------------------------------------------------------------------------ + +impl PageSliceDescriptor { + /// Create an instance. + pub const fn from_addr(start: Address, num_pages: usize) -> Self { + assert!(common::is_aligned( + start.into_usize(), + bsp::memory::mmu::KernelGranule::SIZE + )); + assert!(num_pages > 0); + + Self { start, num_pages } + } + + /// Return a pointer to the first page of the described slice. + const fn first_page_ptr(&self) -> *const Page { + self.start.into_usize() as *const _ + } + + /// Return the number of Pages the slice describes. + pub const fn num_pages(&self) -> usize { + self.num_pages + } + + /// Return the memory size this descriptor spans. + pub const fn size(&self) -> usize { + self.num_pages * bsp::memory::mmu::KernelGranule::SIZE + } + + /// Return the start address. + pub const fn start_addr(&self) -> Address { + self.start + } + + /// Return the exclusive end address. + pub fn end_addr(&self) -> Address { + self.start + self.size() + } + + /// Return the inclusive end address. + pub fn end_addr_inclusive(&self) -> Address { + self.start + (self.size() - 1) + } + + /// Check if an address is contained within this descriptor. + pub fn contains(&self, addr: Address) -> bool { + (addr >= self.start_addr()) && (addr <= self.end_addr_inclusive()) + } + + /// Return a non-mutable slice of Pages. + /// + /// # Safety + /// + /// - Same as applies for `core::slice::from_raw_parts`. + pub unsafe fn as_slice(&self) -> &[Page] { + core::slice::from_raw_parts(self.first_page_ptr(), self.num_pages) + } +} + +impl TryFrom> for PageSliceDescriptor { + type Error = super::TranslationError; + + fn try_from(desc: PageSliceDescriptor) -> Result { + let phys_start = super::try_virt_to_phys(desc.start)?; + + Ok(Self { + start: phys_start, + num_pages: desc.num_pages, + }) + } +} + +impl From for PageSliceDescriptor { + fn from(desc: MMIODescriptor) -> Self { + let start_page_addr = desc + .start_addr + .align_down(bsp::memory::mmu::KernelGranule::SIZE); + + let len = ((desc.end_addr_inclusive().into_usize() - start_page_addr.into_usize()) + >> bsp::memory::mmu::KernelGranule::SHIFT) + + 1; + + Self { + start: start_page_addr, + num_pages: len, + } + } +} + +//------------------------------------------------------------------------------ +// MMIODescriptor +//------------------------------------------------------------------------------ + +impl MMIODescriptor { + /// Create an instance. + pub const fn new(start_addr: Address, size: usize) -> Self { + assert!(size > 0); + + Self { start_addr, size } + } + + /// Return the start address. + pub const fn start_addr(&self) -> Address { + self.start_addr + } + + /// Return the inclusive end address. + pub fn end_addr_inclusive(&self) -> Address { + self.start_addr + (self.size - 1) + } + + /// Return the size. + pub const fn size(&self) -> usize { + self.size + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct Page` is as expected. + #[kernel_test] + fn size_of_page_equals_granule_size() { + assert_eq!( + core::mem::size_of::>(), + bsp::memory::mmu::KernelGranule::SIZE + ); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs b/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs new file mode 100644 index 000000000..e3a9ed8ab --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! A panic handler that infinitely waits. + +use crate::{bsp, cpu, exception}; +use core::{fmt, panic::PanicInfo}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +fn _panic_print(args: fmt::Arguments) { + use fmt::Write; + + unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; +} + +/// The point of exit for `libkernel`. +/// +/// It is linked weakly, so that the integration tests can overload its standard behavior. +#[linkage = "weak"] +#[no_mangle] +fn _panic_exit() -> ! { + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } +} + +/// Prints with a newline - only use from the panic handler. +/// +/// Carbon copy from +#[macro_export] +macro_rules! panic_println { + ($($arg:tt)*) => ({ + _panic_print(format_args_nl!($($arg)*)); + }) +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + unsafe { exception::asynchronous::local_irq_mask() }; + + if let Some(args) = info.message() { + panic_println!("\nKernel panic: {}", args); + } else { + panic_println!("\nKernel panic!"); + } + + _panic_exit() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/print.rs b/15_virtual_mem_part3_precomputed_tables/src/print.rs new file mode 100644 index 000000000..89fa66943 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/print.rs @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Printing. + +use crate::{bsp, console}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +#[doc(hidden)] +pub fn _print(args: fmt::Arguments) { + use console::interface::Write; + + bsp::console::console().write_fmt(args).unwrap(); +} + +/// Prints without a newline. +/// +/// Carbon copy from +#[macro_export] +macro_rules! print { + ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); +} + +/// Prints with a newline. +/// +/// Carbon copy from +#[macro_export] +macro_rules! println { + () => ($crate::print!("\n")); + ($($arg:tt)*) => ({ + $crate::print::_print(format_args_nl!($($arg)*)); + }) +} + +/// Prints an info, with a newline. +#[macro_export] +macro_rules! info { + ($string:expr) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[ {:>3}.{:03}{:03}] ", $string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000 + )); + }); + ($format_string:expr, $($arg:tt)*) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[ {:>3}.{:03}{:03}] ", $format_string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000, + $($arg)* + )); + }) +} + +/// Prints a warning, with a newline. +#[macro_export] +macro_rules! warn { + ($string:expr) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[W {:>3}.{:03}{:03}] ", $string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000 + )); + }); + ($format_string:expr, $($arg:tt)*) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[W {:>3}.{:03}{:03}] ", $format_string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000, + $($arg)* + )); + }) +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs b/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs new file mode 100644 index 000000000..0a1c685cd --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Rust runtime initialization code. + +use crate::{bsp, memory}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Zero out the .bss section. +/// +/// # Safety +/// +/// - Must only be called pre `kernel_init()`. +#[inline(always)] +unsafe fn zero_bss() { + memory::zero_volatile(bsp::memory::bss_range_inclusive()); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel +/// init code. +/// +/// # Safety +/// +/// - Only a single core must be active and running this function. +pub unsafe fn runtime_init() -> ! { + extern "Rust" { + fn kernel_init() -> !; + } + + zero_bss(); + kernel_init() +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/state.rs b/15_virtual_mem_part3_precomputed_tables/src/state.rs new file mode 100644 index 000000000..c94d04c84 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/state.rs @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! State information about the kernel itself. + +use core::sync::atomic::{AtomicU8, Ordering}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Different stages in the kernel execution. +#[derive(Copy, Clone, Eq, PartialEq)] +enum State { + /// The kernel starts booting in this state. + Init, + + /// The kernel transitions to this state when jumping to `kernel_main()` (at the end of + /// `kernel_init()`, after all init calls are done). + SingleCoreMain, + + /// The kernel transitions to this state when it boots the secondary cores, aka switches + /// exectution mode to symmetric multiprocessing (SMP). + MultiCoreMain, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Maintains the kernel state and state transitions. +pub struct StateManager(AtomicU8); + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static STATE_MANAGER: StateManager = StateManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the global StateManager. +pub fn state_manager() -> &'static StateManager { + &STATE_MANAGER +} + +impl StateManager { + const INIT: u8 = 0; + const SINGLE_CORE_MAIN: u8 = 1; + const MULTI_CORE_MAIN: u8 = 2; + + /// Create a new instance. + pub const fn new() -> Self { + Self(AtomicU8::new(Self::INIT)) + } + + /// Return the current state. + fn state(&self) -> State { + let state = self.0.load(Ordering::Acquire); + + match state { + Self::INIT => State::Init, + Self::SINGLE_CORE_MAIN => State::SingleCoreMain, + Self::MULTI_CORE_MAIN => State::MultiCoreMain, + _ => panic!("Invalid KERNEL_STATE"), + } + } + + /// Return if the kernel is init state. + pub fn is_init(&self) -> bool { + self.state() == State::Init + } + + /// Transition from Init to SingleCoreMain. + pub fn transition_to_single_core_main(&self) { + if self + .0 + .compare_exchange( + Self::INIT, + Self::SINGLE_CORE_MAIN, + Ordering::Acquire, + Ordering::Relaxed, + ) + .is_err() + { + panic!("transition_to_single_core_main() called while state != Init"); + } + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/synchronization.rs b/15_virtual_mem_part3_precomputed_tables/src/synchronization.rs new file mode 100644 index 000000000..94582732c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/synchronization.rs @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Synchronization primitives. +//! +//! # Resources +//! +//! - +//! - +//! - + +use core::cell::UnsafeCell; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Synchronization interfaces. +pub mod interface { + + /// Any object implementing this trait guarantees exclusive access to the data wrapped within + /// the Mutex for the duration of the provided closure. + pub trait Mutex { + /// The type of the data that is wrapped by this mutex. + type Data; + + /// Locks the mutex and grants the closure temporary mutable access to the wrapped data. + fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; + } + + /// A reader-writer exclusion type. + /// + /// The implementing object allows either a number of readers or at most one writer at any point + /// in time. + pub trait ReadWriteEx { + /// The type of encapsulated data. + type Data; + + /// Grants temporary mutable access to the encapsulated data. + fn write(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; + + /// Grants temporary immutable access to the encapsulated data. + fn read(&self, f: impl FnOnce(&Self::Data) -> R) -> R; + } +} + +/// A pseudo-lock for teaching purposes. +/// +/// In contrast to a real Mutex implementation, does not protect against concurrent access from +/// other cores to the contained data. This part is preserved for later lessons. +/// +/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is +/// executing on a single core. +pub struct IRQSafeNullLock +where + T: ?Sized, +{ + data: UnsafeCell, +} + +/// A pseudo-lock that is RW during the single-core kernel init phase and RO afterwards. +/// +/// Intended to encapsulate data that is populated during kernel init when no concurrency exists. +pub struct InitStateLock +where + T: ?Sized, +{ + data: UnsafeCell, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +unsafe impl Send for IRQSafeNullLock where T: ?Sized + Send {} +unsafe impl Sync for IRQSafeNullLock where T: ?Sized + Send {} + +impl IRQSafeNullLock { + /// Create an instance. + pub const fn new(data: T) -> Self { + Self { + data: UnsafeCell::new(data), + } + } +} + +unsafe impl Send for InitStateLock where T: ?Sized + Send {} +unsafe impl Sync for InitStateLock where T: ?Sized + Send {} + +impl InitStateLock { + /// Create an instance. + pub const fn new(data: T) -> Self { + Self { + data: UnsafeCell::new(data), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use crate::{exception, state}; + +impl interface::Mutex for IRQSafeNullLock { + type Data = T; + + fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { + // In a real lock, there would be code encapsulating this line that ensures that this + // mutable reference will ever only be given out once at a time. + let data = unsafe { &mut *self.data.get() }; + + // Execute the closure while IRQs are masked. + exception::asynchronous::exec_with_irq_masked(|| f(data)) + } +} + +impl interface::ReadWriteEx for InitStateLock { + type Data = T; + + fn write(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { + assert!( + state::state_manager().is_init(), + "InitStateLock::write called after kernel init phase" + ); + assert!( + !exception::asynchronous::is_local_irq_masked(), + "InitStateLock::write called with IRQs unmasked" + ); + + let data = unsafe { &mut *self.data.get() }; + + f(data) + } + + fn read(&self, f: impl FnOnce(&Self::Data) -> R) -> R { + let data = unsafe { &*self.data.get() }; + + f(data) + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// InitStateLock must be transparent. + #[kernel_test] + fn init_state_lock_is_transparent() { + use core::mem::size_of; + + assert_eq!(size_of::>(), size_of::()); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/src/time.rs b/15_virtual_mem_part3_precomputed_tables/src/time.rs new file mode 100644 index 000000000..953b6f0df --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/src/time.rs @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Timer primitives. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/time.rs"] +mod arch_time; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Timekeeping interfaces. +pub mod interface { + use core::time::Duration; + + /// Time management functions. + pub trait TimeManager { + /// The timer's resolution. + fn resolution(&self) -> Duration; + + /// The uptime since power-on of the device. + /// + /// This includes time consumed by firmware and bootloaders. + fn uptime(&self) -> Duration; + + /// Spin for a given duration. + fn spin_for(&self, duration: Duration); + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/test-macros/Cargo.toml b/15_virtual_mem_part3_precomputed_tables/test-macros/Cargo.toml new file mode 100644 index 000000000..a570f72b4 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/test-macros/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "test-macros" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" + +[lib] +proc-macro = true + +[dependencies] +proc-macro2 = "1.x" +quote = "1.x" +syn = { version = "1.x", features = ["full"] } +test-types = { path = "../test-types" } diff --git a/15_virtual_mem_part3_precomputed_tables/test-macros/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/test-macros/src/lib.rs new file mode 100644 index 000000000..36c95e8a6 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/test-macros/src/lib.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +use proc_macro::TokenStream; +use proc_macro2::Span; +use quote::quote; +use syn::{parse_macro_input, Ident, ItemFn}; + +#[proc_macro_attribute] +pub fn kernel_test(_attr: TokenStream, input: TokenStream) -> TokenStream { + let f = parse_macro_input!(input as ItemFn); + + let test_name = &format!("{}", f.sig.ident.to_string()); + let test_ident = Ident::new( + &format!("{}_TEST_CONTAINER", f.sig.ident.to_string().to_uppercase()), + Span::call_site(), + ); + let test_code_block = f.block; + + quote!( + #[test_case] + const #test_ident: test_types::UnitTest = test_types::UnitTest { + name: #test_name, + test_func: || #test_code_block, + }; + ) + .into() +} diff --git a/15_virtual_mem_part3_precomputed_tables/test-types/Cargo.toml b/15_virtual_mem_part3_precomputed_tables/test-types/Cargo.toml new file mode 100644 index 000000000..a0be2c57c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/test-types/Cargo.toml @@ -0,0 +1,5 @@ +[package] +name = "test-types" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" diff --git a/15_virtual_mem_part3_precomputed_tables/test-types/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/test-types/src/lib.rs new file mode 100644 index 000000000..fe7d918f5 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/test-types/src/lib.rs @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Types for the `custom_test_frameworks` implementation. + +#![no_std] + +/// Unit test container. +pub struct UnitTest { + /// Name of the test. + pub name: &'static str, + + /// Function pointer to the test. + pub test_func: fn(), +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb new file mode 100644 index 000000000..dfd6b16ea --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb @@ -0,0 +1,50 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'expect' + +TIMEOUT_SECS = 3 + +# Verify sending and receiving works as expected. +class TxRxHandshake + def name + 'Transmit and Receive handshake' + end + + def run(qemu_out, qemu_in) + qemu_in.write_nonblock('ABC') + raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + end +end + +# Check for correct TX statistics implementation. Depends on test 1 being run first. +class TxStatistics + def name + 'Transmit statistics' + end + + def run(qemu_out, _qemu_in) + raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + end +end + +# Check for correct RX statistics implementation. Depends on test 1 being run first. +class RxStatistics + def name + 'Receive statistics' + end + + def run(qemu_out, _qemu_in) + raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + end +end + +##-------------------------------------------------------------------------------------------------- +## Test registration +##-------------------------------------------------------------------------------------------------- +def subtest_collection + [TxRxHandshake.new, TxStatistics.new, RxStatistics.new] +end diff --git a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs new file mode 100644 index 000000000..ad7fd2bf5 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Console sanity tests - RX, TX and statistics. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +use libkernel::{bsp, console, cpu, exception, print}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + use bsp::console::console; + use console::interface::*; + + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + // Handshake + assert_eq!(console().read_char(), 'A'); + assert_eq!(console().read_char(), 'B'); + assert_eq!(console().read_char(), 'C'); + print!("OK1234"); + + // 6 + print!("{}", console().chars_written()); + + // 3 + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. + cpu::wait_forever() +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/01_timer_sanity.rs b/15_virtual_mem_part3_precomputed_tables/tests/01_timer_sanity.rs new file mode 100644 index 000000000..f39d83845 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/01_timer_sanity.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Timer sanity tests. + +#![feature(custom_test_frameworks)] +#![no_main] +#![no_std] +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + +use core::time::Duration; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. + + test_main(); + + cpu::qemu_exit_success() +} + +/// Simple check that the timer is running. +#[kernel_test] +fn timer_is_counting() { + assert!(time::time_manager().uptime().as_nanos() > 0) +} + +/// Timer resolution must be sufficient. +#[kernel_test] +fn timer_resolution_is_sufficient() { + assert!(time::time_manager().resolution().as_nanos() < 100) +} + +/// Sanity check spin_for() implementation. +#[kernel_test] +fn spin_accuracy_check_1_second() { + let t1 = time::time_manager().uptime(); + time::time_manager().spin_for(Duration::from_secs(1)); + let t2 = time::time_manager().uptime(); + + assert_eq!((t2 - t1).as_secs(), 1) +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs new file mode 100644 index 000000000..6a0c11f3c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Page faults must result in synchronous exceptions. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. +/// +/// In this test, teaching the panic is a success, because it is called from the synchronous +/// exception handler, which is what this test wants to achieve. +/// +/// It also means that this integration test can not use any other code that calls panic!() directly +/// or indirectly. +mod panic_exit_success; + +use libkernel::{bsp, cpu, exception, println}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + + println!("Writing beyond mapped area to address 9 GiB..."); + let big_addr: u64 = 9 * 1024 * 1024 * 1024; + core::ptr::read_volatile(big_addr as *mut u64); + + // If execution reaches here, the memory access above did not cause a page fault exception. + cpu::qemu_exit_failure() +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/03_exception_irq_sanity.rs b/15_virtual_mem_part3_precomputed_tables/tests/03_exception_irq_sanity.rs new file mode 100644 index 000000000..ac25d01a8 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/03_exception_irq_sanity.rs @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! IRQ handling sanity tests. + +#![feature(custom_test_frameworks)] +#![no_main] +#![no_std] +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + +use libkernel::{bsp, cpu, exception}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + bsp::console::qemu_bring_up_console(); + + exception::handling_init(); + exception::asynchronous::local_irq_unmask(); + + test_main(); + + cpu::qemu_exit_success() +} + +/// Check that IRQ masking works. +#[kernel_test] +fn local_irq_mask_works() { + // Precondition: IRQs are unmasked. + assert!(exception::asynchronous::is_local_irq_masked()); + + unsafe { exception::asynchronous::local_irq_mask() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + // Restore earlier state. + unsafe { exception::asynchronous::local_irq_unmask() }; +} + +/// Check that IRQ unmasking works. +#[kernel_test] +fn local_irq_unmask_works() { + // Precondition: IRQs are masked. + unsafe { exception::asynchronous::local_irq_mask() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + unsafe { exception::asynchronous::local_irq_unmask() }; + assert!(exception::asynchronous::is_local_irq_masked()); +} + +/// Check that IRQ mask save is saving "something". +#[kernel_test] +fn local_irq_mask_save_works() { + // Precondition: IRQs are unmasked. + assert!(exception::asynchronous::is_local_irq_masked()); + + let first = unsafe { exception::asynchronous::local_irq_mask_save() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + let second = unsafe { exception::asynchronous::local_irq_mask_save() }; + assert_ne!(first, second); + + unsafe { exception::asynchronous::local_irq_restore(first) }; + assert!(exception::asynchronous::is_local_irq_masked()); +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/panic_exit_success/mod.rs b/15_virtual_mem_part3_precomputed_tables/tests/panic_exit_success/mod.rs new file mode 100644 index 000000000..29d1f9750 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/panic_exit_success/mod.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +#[no_mangle] +fn _panic_exit() -> ! { + libkernel::cpu::qemu_exit_success() +} diff --git a/15_virtual_mem_part3_precomputed_tables/tests/runner.rb b/15_virtual_mem_part3_precomputed_tables/tests/runner.rb new file mode 100755 index 000000000..53116e088 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/runner.rb @@ -0,0 +1,143 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'English' +require 'pty' + +# Test base class. +class Test + INDENT = ' ' + + def print_border(status) + puts + puts "#{INDENT}-------------------------------------------------------------------" + puts status + puts "#{INDENT}-------------------------------------------------------------------\n\n\n" + end + + def print_error(error) + puts + print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") + end + + def print_success + print_border("#{INDENT}✅ Success: #{@test_name}") + end + + def print_output + puts "#{INDENT}-------------------------------------------------------------------" + print INDENT + print '🦀 ' + print @output.join.gsub("\n", "\n#{INDENT}") + end + + def finish(error) + print_output + + exit_code = if error + print_error(error) + false + else + print_success + true + end + + exit(exit_code) + end +end + +# Executes tests with console I/O. +class ConsoleTest < Test + def initialize(binary, qemu_cmd, test_name, console_subtests) + super() + + @binary = binary + @qemu_cmd = qemu_cmd + @test_name = test_name + @console_subtests = console_subtests + @cur_subtest = 1 + @output = ["Running #{@console_subtests.length} console-based tests\n", + "-------------------------------------------------------------------\n\n"] + end + + def format_test_name(number, name) + formatted_name = "#{number.to_s.rjust(3)}. #{name}" + formatted_name.ljust(63, '.') + end + + def run_subtest(subtest, qemu_out, qemu_in) + @output << format_test_name(@cur_subtest, subtest.name) + + subtest.run(qemu_out, qemu_in) + + @output << "[ok]\n" + @cur_subtest += 1 + end + + def exec + error = false + + PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| + begin + @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } + rescue StandardError => e + error = e.message + end + + finish(error) + end + end +end + +# A wrapper around the bare QEMU invocation. +class RawTest < Test + MAX_WAIT_SECS = 5 + + def initialize(binary, qemu_cmd, test_name) + super() + + @binary = binary + @qemu_cmd = qemu_cmd + @test_name = test_name + @output = [] + end + + def exec + error = 'Timed out waiting for test' + io = IO.popen(@qemu_cmd) + + while IO.select([io], nil, nil, MAX_WAIT_SECS) + begin + @output << io.read_nonblock(1024) + rescue EOFError + io.close + error = $CHILD_STATUS.to_i != 0 + break + end + end + + finish(error) + end +end + +##-------------------------------------------------------------------------------------------------- +## Script entry point +##-------------------------------------------------------------------------------------------------- +binary = ARGV.last +test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] +console_test_file = "tests/#{test_name}.rb" +qemu_cmd = ARGV.join(' ') + +test_runner = if File.exist?(console_test_file) + load console_test_file + # subtest_collection is provided by console_test_file + ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) + else + RawTest.new(binary, qemu_cmd, test_name) + end + +test_runner.exec diff --git a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb new file mode 100644 index 000000000..72471b113 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb @@ -0,0 +1,323 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +# Bitfield manipulation. +class BitField + def initialize + @value = 0 + end + + def self.attr_bitfield(name, offset, num_bits) + define_method("#{name}=") do |bits| + mask = 2**num_bits - 1 + + raise "Input out of range: #{name} = 0x#{bits.to_s(16)}" if (bits & ~mask).positive? + + # Clear bitfield + @value &= ~(mask << offset) + + # Set it + @value |= (bits << offset) + end + end + + def to_i + @value + end + + def size_in_byte + 8 + end +end + +# An array class that knows its memory location. +class CArray < Array + attr_reader :phys_start_addr + + def initialize(phys_start_addr, size, &block) + @phys_start_addr = phys_start_addr + + super(size, &block) + end + + def size_in_byte + inject(0) { |sum, n| sum + n.size_in_byte } + end +end + +#--------------------------------------------------------------------------------------------------- +# Arch:: +#--------------------------------------------------------------------------------------------------- +module Arch +FALSE = 0b0 +TRUE = 0b1 + +#--------------------------------------------------------------------------------------------------- +# Arch::ARMv8 +#--------------------------------------------------------------------------------------------------- +module ARMv8 +# ARMv8 Table Descriptor. +class Stage1TableDescriptor < BitField + module NextLevelTableAddr + OFFSET = 16 + NUMBITS = 32 + end + + module Type + OFFSET = 1 + NUMBITS = 1 + + BLOCK = 0 + TABLE = 1 + end + + module Valid + OFFSET = 0 + NUMBITS = 1 + end + + attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) + attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) + attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) + + def next_level_table_addr=(addr) + addr = addr >> Granule64KiB::SHIFT + + self.__next_level_table_addr = addr + end + + private :__next_level_table_addr= +end + +# ARMv8 level 3 page descriptor. +class Stage1PageDescriptor < BitField + module UXN + OFFSET = 54 + NUMBITS = 1 + end + + module PXN + OFFSET = 53 + NUMBITS = 1 + end + + module OutputAddr + OFFSET = 16 + NUMBITS = 32 + end + + module AF + OFFSET = 10 + NUMBITS = 1 + end + + module SH + OFFSET = 8 + NUMBITS = 2 + + INNER_SHAREABLE = 0b11 + end + + module AP + OFFSET = 6 + NUMBITS = 2 + + RW_EL1 = 0b00 + RO_EL1 = 0b10 + end + + module AttrIndx + OFFSET = 2 + NUMBITS = 3 + end + + module Type + OFFSET = 1 + NUMBITS = 1 + + RESERVED_INVALID = 0 + PAGE = 1 + end + + module Valid + OFFSET = 0 + NUMBITS = 1 + end + + attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) + attr_bitfield(:pxn, PXN::OFFSET, PXN::NUMBITS) + attr_bitfield(:__output_addr, OutputAddr::OFFSET, OutputAddr::NUMBITS) + attr_bitfield(:af, AF::OFFSET, AF::NUMBITS) + attr_bitfield(:sh, SH::OFFSET, SH::NUMBITS) + attr_bitfield(:ap, AP::OFFSET, AP::NUMBITS) + attr_bitfield(:attr_indx, AttrIndx::OFFSET, AttrIndx::NUMBITS) + attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) + attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) + + def output_addr=(addr) + addr = addr >> Granule64KiB::SHIFT + + self.__output_addr = addr + end + + private :__output_addr= +end + +# Translation table representing the structure defined in translation_table.rs. +class TranslationTable + MMIO_APERTURE_MiB = 256 * 1024 * 1024 + + module MAIR + NORMAL = 1 + end + + def initialize + @virt_mmio_start_addr = (BSP.kernel_virt_addr_space_size - MMIO_APERTURE_MiB) + + BSP.kernel_virt_start_addr + + do_sanity_checks + + num_lvl2_tables = BSP.kernel_virt_addr_space_size >> Granule512MiB::SHIFT + + @lvl3 = new_lvl3(num_lvl2_tables, BSP.phys_table_struct_start_addr) + + @lvl2_phys_start_addr = @lvl3.phys_start_addr + @lvl3.size_in_byte + @lvl2 = new_lvl2(num_lvl2_tables, @lvl2_phys_start_addr) + + populate_lvl2_entries + end + + def map_pages_at(virt_pages, phys_pages, attributes) + return if virt_pages.empty? + + raise if virt_pages.size != phys_pages.size + raise if phys_pages.last > BSP.phys_addr_space_end_page + + virt_pages.zip(phys_pages).each do |virt_page, phys_page| + desc = page_descriptor_from(virt_page) + set_lvl3_entry(desc, phys_page, attributes) + end + end + + def to_binary + data = @lvl3.flatten.map(&:to_i) + @lvl2.map(&:to_i) + data.pack('Q<*') # "Q" == uint64_t, "<" == little endian + end + + def phys_tables_base_addr_binary + [@lvl2_phys_start_addr].pack('Q<*') # "Q" == uint64_t, "<" == little endian + end + + def phys_tables_base_addr + @lvl2_phys_start_addr + end + + private + + def binary_with_mmio_clash? + BSP.rw_end_exclusive >= @virt_mmio_start_addr + end + + def do_sanity_checks + raise unless BSP.kernel_granule::SIZE == Granule64KiB::SIZE + raise unless (BSP.kernel_virt_addr_space_size % Granule512MiB::SIZE).zero? + + # Need to ensure that that the kernel binary does not clash with the upmost 256 MiB of the + # virtual address space, which is reserved for runtime-remapping of MMIO. + return unless binary_with_mmio_clash? + + puts format('__data_end_exclusive: 0x%16x', BSP.data_end_exclusive) + puts format('MMIO start: 0x%16x', @virt_mmio_start_addr) + + raise 'Kernel virtual addresses clash with 256 MiB MMIO window' + end + + def new_lvl3(num_lvl2_tables, start_addr) + CArray.new(start_addr, num_lvl2_tables) do + temp = CArray.new(start_addr, 8192) do + Stage1PageDescriptor.new + end + start_addr += temp.size_in_byte + + temp + end + end + + def new_lvl2(num_lvl2_tables, start_addr) + CArray.new(start_addr, num_lvl2_tables) do + Stage1TableDescriptor.new + end + end + + def populate_lvl2_entries + @lvl2.each_with_index do |descriptor, i| + descriptor.next_level_table_addr = @lvl3[i].phys_start_addr + descriptor.type = Stage1TableDescriptor::Type::TABLE + descriptor.valid = TRUE + end + end + + def lvl2_lvl3_index_from(addr) + addr -= BSP.kernel_virt_start_addr + + lvl2_index = addr >> Granule512MiB::SHIFT + lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT + + raise unless lvl2_index < @lvl2.size + + [lvl2_index, lvl3_index] + end + + def page_descriptor_from(virt_addr) + lvl2_index, lvl3_index = lvl2_lvl3_index_from(virt_addr) + + @lvl3[lvl2_index][lvl3_index] + end + + # rubocop:disable Metrics/MethodLength + def set_attributes(desc, attributes) + case attributes.mem_attributes + when :CacheableDRAM + desc.sh = Stage1PageDescriptor::SH::INNER_SHAREABLE + desc.attr_indx = MAIR::NORMAL + else + raise 'Invalid input' + end + + desc.ap = case attributes.acc_perms + when :ReadOnly + Stage1PageDescriptor::AP::RO_EL1 + when :ReadWrite + Stage1PageDescriptor::AP::RW_EL1 + else + raise 'Invalid input' + + end + + desc.pxn = case attributes.execute_never + when :XN + TRUE + when :X + FALSE + else + raise 'Invalid input' + end + + desc.uxn = TRUE + end + # rubocop:enable Metrics/MethodLength + + def set_lvl3_entry(desc, output_addr, attributes) + desc.output_addr = output_addr + desc.af = TRUE + desc.type = Stage1PageDescriptor::Type::PAGE + desc.valid = TRUE + + set_attributes(desc, attributes) + end +end +end +end diff --git a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb new file mode 100644 index 000000000..6acafe984 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb @@ -0,0 +1,177 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +# Raspberry Pi 3 + 4 +class RaspberryPi + attr_reader :kernel_granule, :kernel_virt_addr_space_size, :kernel_virt_start_addr + + NM_BINARY = 'aarch64-none-elf-nm' + READELF_BINARY = 'aarch64-none-elf-readelf' + MEMORY_SRC = File.read('src/bsp/raspberrypi/memory.rs').split("\n") + + def initialize(kernel_elf) + @kernel_granule = Granule64KiB + + @virt_addresses = { + boot_core_stack_start: /__boot_core_stack_start/, + boot_core_stack_end_exclusive: /__boot_core_stack_end_exclusive/, + + rx_start: /__rx_start/, + rx_end_exclusive: /__rx_end_exclusive/, + + rw_start: /__rw_start/, + rw_end_exclusive: /__rw_end_exclusive/, + + table_struct_start_addr: /bsp::.*::memory::mmu::KERNEL_TABLES/, + phys_tables_base_addr: /PHYS_KERNEL_TABLES_BASE_ADDR/ + } + + symbols = `#{NM_BINARY} --demangle #{kernel_elf}`.split("\n") + @kernel_virt_addr_space_size = parse_from_symbols(symbols, /__kernel_virt_addr_space_size/) + @kernel_virt_start_addr = 0 + @virt_addresses = parse_from_symbols(symbols, @virt_addresses) + @phys_addresses = virt_to_phys(@virt_addresses) + + @descriptors = parse_descriptors + update_max_descriptor_name_length + + @text_section_offset_in_elf = parse_text_section_offset_in_elf(kernel_elf) + end + + def rw_end_exclusive + @virt_addresses[:rw_end_exclusive] + end + + def phys_table_struct_start_addr + @phys_addresses[:table_struct_start_addr] + end + + def table_struct_offset_in_kernel_elf + (@virt_addresses[:table_struct_start_addr] - @virt_addresses[:rx_start]) + + @text_section_offset_in_elf + end + + def phys_tables_base_addr + @phys_addresses[:phys_tables_base_addr] + end + + def phys_tables_base_addr_offset_in_kernel_elf + (@virt_addresses[:phys_tables_base_addr] - @virt_addresses[:rx_start]) + + @text_section_offset_in_elf + end + + def phys_addr_space_end_page + x = MEMORY_SRC.grep(/pub const END/) + x = case BSP_TYPE + when :rpi3 + x[0] + when :rpi4 + x[1] + else + raise + end + + x.scan(/\d+/).join.to_i(16) + end + + def kernel_map_binary + MappingDescriptor.print_header + + @descriptors.each do |i| + print 'Generating'.rjust(12).green.bold + print ' ' + puts i.to_s + + TRANSLATION_TABLES.map_pages_at(i.virt_pages, i.phys_pages, i.attributes) + end + + MappingDescriptor.print_divider + end + + private + + def parse_from_symbols(symbols, input) + case input.class.to_s + when 'Regexp' + symbols.grep(input).first.split.first.to_i(16) + when 'Hash' + input.transform_values do |val| + symbols.grep(val).first.split.first.to_i(16) + end + else + raise + end + end + + def parse_text_section_offset_in_elf(kernel_elf) + `#{READELF_BINARY} --sections #{kernel_elf}`.scan(/.text.*/).first.split.last.to_i(16) + end + + def virt_to_phys(input) + case input.class.to_s + when 'Integer' + input - @kernel_virt_start_addr + when 'Hash' + input.transform_values do |val| + val - @kernel_virt_start_addr + end + else + raise + end + end + + def descriptor_ro + name = 'Code and RO data' + + ro_size = @virt_addresses[:rx_end_exclusive] - + @virt_addresses[:rx_start] + + virt_ro_pages = PageArray.new(@virt_addresses[:rx_start], ro_size, @kernel_granule::SIZE) + phys_ro_pages = PageArray.new(@phys_addresses[:rx_start], ro_size, @kernel_granule::SIZE) + ro_attribues = AttributeFields.new(:CacheableDRAM, :ReadOnly, :X) + + MappingDescriptor.new(name, virt_ro_pages, phys_ro_pages, ro_attribues) + end + + def descriptor_data + name = 'Data and bss' + + data_size = @virt_addresses[:rw_end_exclusive] - + @virt_addresses[:rw_start] + + virt_data_pages = PageArray.new(@virt_addresses[:rw_start], data_size, + @kernel_granule::SIZE) + phys_data_pages = PageArray.new(@phys_addresses[:rw_start], data_size, + @kernel_granule::SIZE) + data_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) + + MappingDescriptor.new(name, virt_data_pages, phys_data_pages, data_attribues) + end + + def descriptor_boot_core_stack + name = 'Boot-core stack' + + boot_core_stack_size = @virt_addresses[:boot_core_stack_end_exclusive] - + @virt_addresses[:boot_core_stack_start] + + virt_boot_core_stack_pages = PageArray.new(@virt_addresses[:boot_core_stack_start], + boot_core_stack_size, @kernel_granule::SIZE) + phys_boot_core_stack_pages = PageArray.new(@phys_addresses[:boot_core_stack_start], + boot_core_stack_size, @kernel_granule::SIZE) + boot_core_stack_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) + + MappingDescriptor.new(name, virt_boot_core_stack_pages, phys_boot_core_stack_pages, + boot_core_stack_attribues) + end + + def parse_descriptors + [descriptor_ro, descriptor_data, descriptor_boot_core_stack] + end + + def update_max_descriptor_name_length + MappingDescriptor.max_descriptor_name_length = @descriptors.map { |i| i.name.size }.max + end +end diff --git a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb new file mode 100644 index 000000000..2ab666afb --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/generic.rb @@ -0,0 +1,125 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +module Granule64KiB + SIZE = 64 * 1024 + SHIFT = Math.log2(SIZE).to_i +end + +module Granule512MiB + SIZE = 512 * 1024 * 1024 + SHIFT = Math.log2(SIZE).to_i + MASK = SIZE - 1 +end + +# Monkey-patch Integer with some helper functions. +class Integer + def power_of_two? + self[0].zero? + end + + def aligned?(alignment) + raise unless alignment.power_of_two? + + (self & (alignment - 1)).zero? + end + + def to_hex_underscore(with_leading_zeros: false) + fmt = with_leading_zeros ? '%016x' : '%x' + value = format(fmt, self).to_s.reverse.scan(/.{4}|.+/).join('_').reverse + + format('0x%s', value) + end +end + +# An array where each value is the start address of a Page. +class PageArray < Array + def initialize(start_addr, size, granule_size) + raise unless start_addr.aligned?(granule_size) + raise unless size.positive? + raise unless (size % granule_size).zero? + + num_pages = size / granule_size + super(num_pages) do |i| + i * granule_size + start_addr + end + end +end + +# Collection of memory attributes. +class AttributeFields + attr_reader :mem_attributes, :acc_perms, :execute_never + + def initialize(mem_attributes, acc_perms, execute_never) + @mem_attributes = mem_attributes + @acc_perms = acc_perms + @execute_never = execute_never + end +end + +# A container that describes a one- or many-page virt-to-phys mapping. +class MappingDescriptor + @max_descriptor_name_length = 0 + + class << self + attr_accessor :max_descriptor_name_length + end + + attr_reader :name, :virt_pages, :phys_pages, :attributes + + def initialize(name, virt_pages, phys_pages, attributes) + @name = name + @virt_pages = virt_pages + @phys_pages = phys_pages + @attributes = attributes + end + + def to_s + name = @name.ljust(self.class.max_descriptor_name_length) + virt_start = @virt_pages.first.to_hex_underscore(with_leading_zeros: true) + size = ((@virt_pages.size * 65_536) / 1024).to_s.rjust(3) + + "#{name} | #{virt_start} | #{size} KiB" + end + + def self.print_divider + print ' ' + print '-' * max_descriptor_name_length + puts '----------------------------------' + end + + def self.print_header + print_divider + print ' ' + print 'Section'.center(max_descriptor_name_length) + print ' ' + print 'Start Virt Addr'.center(21) + print ' ' + print 'Size'.center(7) + puts + print_divider + end +end + +def kernel_patch_tables(kernel_binary) + print 'Patching'.rjust(12).green.bold + print ' Kernel table struct at physical ' + puts BSP.phys_table_struct_start_addr.to_hex_underscore + + IO.binwrite(kernel_binary, TRANSLATION_TABLES.to_binary, + BSP.table_struct_offset_in_kernel_elf) +end + +def kernel_patch_base_addr(kernel_binary) + print 'Patching'.rjust(12).green.bold + print ' Value of kernel table physical base address (' + print TRANSLATION_TABLES.phys_tables_base_addr.to_hex_underscore + print ') at physical ' + puts BSP.phys_tables_base_addr.to_hex_underscore + + IO.binwrite(kernel_binary, TRANSLATION_TABLES.phys_tables_base_addr_binary, + BSP.phys_tables_base_addr_offset_in_kernel_elf) +end diff --git a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb new file mode 100755 index 000000000..5200a4858 --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/main.rb @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +TARGET = ARGV[0].split('-').first.to_sym +BSP_TYPE = ARGV[1].to_sym +kernel_elf = ARGV[2] + +require 'rubygems' +require 'bundler/setup' +require 'colorize' + +require_relative 'generic' +require_relative 'bsp' +require_relative 'arch' + +puts +puts 'Precomputing kernel translation tables and patching kernel ELF'.cyan + +start = Time.now + +BSP = case BSP_TYPE + when :rpi3, :rpi4 + RaspberryPi.new(kernel_elf) + else + raise + end + +TRANSLATION_TABLES = case TARGET + when :aarch64 + Arch::ARMv8::TranslationTable.new + else + raise + end + +BSP.kernel_map_binary + +kernel_patch_tables(kernel_elf) +kernel_patch_base_addr(kernel_elf) + +elapsed = Time.now - start + +print 'Finished'.rjust(12).green.bold +puts " in #{elapsed.round(2)}s" From 7fee5f7114d9450bb7dc15b5bb9d9fa4880f9a25 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 22:55:06 +0200 Subject: [PATCH 052/214] fix typo --- 15_virtual_mem_part3_precomputed_tables/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index 8de51b1bb..f83ff5b4e 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -433,7 +433,7 @@ attractive either, because writing larger pieces of assembly is an error-prone a Fortunately, there is a third way. We are writing an embedded kernel, and therefore the execution environment is way more static and deterministic as compared to a general-purpose kernel that can be deployed on a wide variety of targets. Specifically, for the Raspberrypi, we exactly know the **load -address** of the kernel in advance, and we know about the capabilites of the `MMU`. So there is +address** of the kernel in advance, and we know about the capabilities of the `MMU`. So there is nothing stopping us from precomputing the kernel's translation tables ahead of time. A disadvantage of this approach is an increased binary size, but this is not a deal breaker in our From b4e3ebc606a211ce1a07d2b51aeeb6245cf7091d Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 23:04:37 +0200 Subject: [PATCH 053/214] Add tutorial 16 --- .rubocop.yml | 3 + .../.cargo/config.toml | 2 + .../.vscode/settings.json | 7 + .../Cargo.lock | 93 +++ .../Cargo.toml | 55 ++ .../Makefile | 193 ++++++ .../README.md | 634 ++++++++++++++++++ .../build.rs | 8 + .../src/_arch/aarch64/cpu.rs | 49 ++ .../src/_arch/aarch64/cpu/boot.rs | 97 +++ .../src/_arch/aarch64/cpu/boot.s | 87 +++ .../src/_arch/aarch64/cpu/smp.rs | 29 + .../src/_arch/aarch64/exception.rs | 281 ++++++++ .../src/_arch/aarch64/exception.s | 148 ++++ .../_arch/aarch64/exception/asynchronous.rs | 150 +++++ .../src/_arch/aarch64/memory/mmu.rs | 184 +++++ .../aarch64/memory/mmu/translation_table.rs | 505 ++++++++++++++ .../src/_arch/aarch64/time.rs | 118 ++++ .../src/bsp.rs | 13 + .../src/bsp/device_driver.rs | 16 + .../src/bsp/device_driver/arm.rs | 9 + .../src/bsp/device_driver/arm/gicv2.rs | 248 +++++++ .../src/bsp/device_driver/arm/gicv2/gicc.rs | 152 +++++ .../src/bsp/device_driver/arm/gicv2/gicd.rs | 205 ++++++ .../src/bsp/device_driver/bcm.rs | 15 + .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 255 +++++++ .../bcm/bcm2xxx_interrupt_controller.rs | 138 ++++ .../peripheral_ic.rs | 188 ++++++ .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 537 +++++++++++++++ .../src/bsp/device_driver/common.rs | 38 ++ .../src/bsp/raspberrypi.rs | 62 ++ .../src/bsp/raspberrypi/console.rs | 88 +++ .../src/bsp/raspberrypi/cpu.rs | 14 + .../src/bsp/raspberrypi/driver.rs | 61 ++ .../src/bsp/raspberrypi/exception.rs | 7 + .../bsp/raspberrypi/exception/asynchronous.rs | 36 + .../kernel_virt_addr_space_size.ld | 1 + .../src/bsp/raspberrypi/link.ld | 87 +++ .../src/bsp/raspberrypi/memory.rs | 208 ++++++ .../src/bsp/raspberrypi/memory/mmu.rs | 187 ++++++ .../src/common.rs | 21 + .../src/console.rs | 53 ++ .../src/cpu.rs | 21 + .../src/cpu/boot.rs | 9 + .../src/cpu/smp.rs | 14 + .../src/driver.rs | 62 ++ .../src/exception.rs | 48 ++ .../src/exception/asynchronous.rs | 152 +++++ .../src/lib.rs | 176 +++++ .../src/main.rs | 105 +++ .../src/memory.rs | 202 ++++++ .../src/memory/mmu.rs | 270 ++++++++ .../src/memory/mmu/mapping_record.rs | 221 ++++++ .../src/memory/mmu/translation_table.rs | 109 +++ .../src/memory/mmu/types.rs | 217 ++++++ .../src/panic_wait.rs | 58 ++ .../src/print.rs | 106 +++ .../src/runtime_init.rs | 41 ++ .../src/state.rs | 92 +++ .../src/synchronization.rs | 159 +++++ .../src/time.rs | 37 + .../test-macros/Cargo.toml | 14 + .../test-macros/src/lib.rs | 29 + .../test-types/Cargo.toml | 5 + .../test-types/src/lib.rs | 16 + .../tests/00_console_sanity.rb | 50 ++ .../tests/00_console_sanity.rs | 35 + .../tests/01_timer_sanity.rs | 49 ++ .../tests/02_exception_sync_page_fault.rs | 36 + .../tests/03_exception_irq_sanity.rs | 66 ++ .../tests/panic_exit_success/mod.rs | 9 + .../tests/runner.rb | 143 ++++ .../translation_table_tool/arch.rb | 323 +++++++++ .../translation_table_tool/bsp.rb | 177 +++++ .../translation_table_tool/generic.rb | 125 ++++ .../translation_table_tool/main.rb | 47 ++ 76 files changed, 8505 insertions(+) create mode 100644 16_virtual_mem_part4_higher_half_kernel/.cargo/config.toml create mode 100644 16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json create mode 100644 16_virtual_mem_part4_higher_half_kernel/Cargo.lock create mode 100644 16_virtual_mem_part4_higher_half_kernel/Cargo.toml create mode 100644 16_virtual_mem_part4_higher_half_kernel/Makefile create mode 100644 16_virtual_mem_part4_higher_half_kernel/README.md create mode 100644 16_virtual_mem_part4_higher_half_kernel/build.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.s create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/common.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/console.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/cpu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/driver.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception/asynchronous.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/common.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/console.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/cpu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/cpu/boot.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/cpu/smp.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/driver.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/exception.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/exception/asynchronous.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/lib.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/main.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/memory.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/translation_table.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/types.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/print.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/state.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/synchronization.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/src/time.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/test-macros/Cargo.toml create mode 100644 16_virtual_mem_part4_higher_half_kernel/test-macros/src/lib.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/test-types/Cargo.toml create mode 100644 16_virtual_mem_part4_higher_half_kernel/test-types/src/lib.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/01_timer_sanity.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/03_exception_irq_sanity.rs create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/panic_exit_success/mod.rs create mode 100755 16_virtual_mem_part4_higher_half_kernel/tests/runner.rb create mode 100644 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb create mode 100644 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb create mode 100644 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/generic.rb create mode 100755 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/main.rb diff --git a/.rubocop.yml b/.rubocop.yml index a1687fd16..d61571d60 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -16,6 +16,9 @@ Layout/IndentationWidth: Layout/LineLength: Max: 100 +Lint/DeprecatedConstants: + Enabled: false + Metrics/ClassLength: Enabled: false diff --git a/16_virtual_mem_part4_higher_half_kernel/.cargo/config.toml b/16_virtual_mem_part4_higher_half_kernel/.cargo/config.toml new file mode 100644 index 000000000..e34764853 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/.cargo/config.toml @@ -0,0 +1,2 @@ +[target.'cfg(target_os = "none")'] +runner = "target/kernel_test_runner.sh" diff --git a/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json b/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json new file mode 100644 index 000000000..a0d6a9202 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json @@ -0,0 +1,7 @@ +{ + "editor.formatOnSave": true, + "editor.rulers": [100], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", + "rust-analyzer.cargo.features": ["bsp_rpi3"] +} diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock new file mode 100644 index 000000000..069559203 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock @@ -0,0 +1,93 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "cortex-a" +version = "5.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +dependencies = [ + "register", +] + +[[package]] +name = "kernel" +version = "0.1.0" +dependencies = [ + "cortex-a", + "qemu-exit", + "register", + "test-macros", + "test-types", +] + +[[package]] +name = "proc-macro2" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +dependencies = [ + "unicode-xid", +] + +[[package]] +name = "qemu-exit" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" + +[[package]] +name = "quote" +version = "1.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "register" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" +dependencies = [ + "tock-registers", +] + +[[package]] +name = "syn" +version = "1.0.62" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +dependencies = [ + "proc-macro2", + "quote", + "unicode-xid", +] + +[[package]] +name = "test-macros" +version = "0.1.0" +dependencies = [ + "proc-macro2", + "quote", + "syn", + "test-types", +] + +[[package]] +name = "test-types" +version = "0.1.0" + +[[package]] +name = "tock-registers" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" + +[[package]] +name = "unicode-xid" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f7fe0bb3479651439c9112f72b6c505038574c9fbb575ed1bf3b797fa39dd564" diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.toml b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml new file mode 100644 index 000000000..206dfc878 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml @@ -0,0 +1,55 @@ +[package] +name = "kernel" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" + +[profile.release] +lto = true + +[features] +default = [] +bsp_rpi3 = ["register"] +bsp_rpi4 = ["register"] +test_build = ["qemu-exit"] + +##-------------------------------------------------------------------------------------------------- +## Dependencies +##-------------------------------------------------------------------------------------------------- + +[dependencies] +test-types = { path = "test-types" } + +# Optional dependencies +register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } +qemu-exit = { version = "1.x.x", optional = true } + +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } + +##-------------------------------------------------------------------------------------------------- +## Testing +##-------------------------------------------------------------------------------------------------- + +[dev-dependencies] +test-macros = { path = "test-macros" } + +# Unit tests are done in the library part of the kernel. +[lib] +name = "libkernel" +test = true + +# Disable unit tests for the kernel binary. +[[bin]] +name = "kernel" +test = false + +# List of tests without harness. +[[test]] +name = "00_console_sanity" +harness = false + +[[test]] +name = "02_exception_sync_page_fault" +harness = false diff --git a/16_virtual_mem_part4_higher_half_kernel/Makefile b/16_virtual_mem_part4_higher_half_kernel/Makefile new file mode 100644 index 000000000..feb65cef7 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/Makefile @@ -0,0 +1,193 @@ +## SPDX-License-Identifier: MIT OR Apache-2.0 +## +## Copyright (c) 2018-2021 Andre Richter + +include ../utils/color.mk.in + +# Default to the RPi3 +BSP ?= rpi3 + +# Default to a serial device name that is common in Linux. +DEV_SERIAL ?= /dev/ttyUSB0 + +# Query the host system's kernel name +UNAME_S = $(shell uname -s) + +# BSP-specific arguments +ifeq ($(BSP),rpi3) + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = raspi3 + QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi3.img + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 +else ifeq ($(BSP),rpi4) + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = + QEMU_RELEASE_ARGS = -serial stdio -display none + QEMU_TEST_ARGS = $(QEMU_RELEASE_ARGS) -semihosting + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg + JTAG_BOOT_IMAGE = ../X1_JTAG_boot/jtag_boot_rpi4.img + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 +endif + +# Export for build.rs +export LINKER_FILE + +# Testing-specific arguments +ifdef TEST + ifeq ($(TEST),unit) + TEST_ARG = --lib + else + TEST_ARG = --test $(TEST) + endif +endif + +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) +RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs + +FEATURES = --features bsp_$(BSP) +COMPILER_ARGS = --target=$(TARGET) \ + $(FEATURES) \ + --release + +RUSTC_CMD = cargo rustc $(COMPILER_ARGS) +DOC_CMD = cargo doc $(COMPILER_ARGS) +CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) +CHECK_CMD = cargo check $(COMPILER_ARGS) +TEST_CMD = cargo test $(COMPILER_ARGS) +OBJCOPY_CMD = rust-objcopy \ + --strip-all \ + -O binary + +KERNEL_ELF = target/$(TARGET)/release/kernel + +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host + +DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + +# Dockerize commands that require USB device passthrough only on Linux +ifeq ($(UNAME_S),Linux) + DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) + + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +else + DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# +endif + +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_MINIPUSH = ruby ../utils/minipush.rb + +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ + clippy clean readelf objdump nm check + +all: $(KERNEL_BIN) + +$(KERNEL_ELF): + $(call colorecho, "\nCompiling kernel - $(BSP)") + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) + @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + +$(KERNEL_BIN): $(KERNEL_ELF) + @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) + +doc: + $(call colorecho, "\nGenerating docs") + @$(DOC_CMD) --document-private-items --open + +ifeq ($(QEMU_MACHINE_TYPE),) +qemu test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") +else +qemu: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU") + @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER +test: FEATURES += --features test_build +test: + $(call colorecho, "\nCompiling test(s) - $(BSP)") + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) +endif + +chainboot: $(KERNEL_BIN) + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) + +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + +clippy: + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + +clean: + rm -rf target $(KERNEL_BIN) + +readelf: $(KERNEL_ELF) + $(call colorecho, "\nLaunching readelf") + @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) + +objdump: $(KERNEL_ELF) + $(call colorecho, "\nLaunching objdump") + @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ + --section .text \ + --section .rodata \ + --section .got \ + $(KERNEL_ELF) | rustfilt + +nm: $(KERNEL_ELF) + $(call colorecho, "\nLaunching nm") + @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt + +# For rust-analyzer +check: + @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md new file mode 100644 index 000000000..ddce63fc5 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -0,0 +1,634 @@ +# Tutorial 16 - Virtual Memory Part 4: Higher-Half Kernel + +## tl;dr + +- The time has come: We map and run the kernel from the top of the 64 bit virtual address space! 🥳 + +## Table of Contents + +- [Introduction](#introduction) +- [Implementation](#implementation) + - [Position-Independent Boot Code](#position-independent-boot-code) +- [Test it](#test-it) +- [Diff to previous](#diff-to-previous) + +## Introduction + +A long time in the making, in this tutorial we finally map the kernel to the most significant area +(alternatively: higher-half) of the 64 bit virtual address space. This makes room for future +applications to use the whole of the least significant area of the virtual memory space. + +As has been teased since `tutorial 14`, we will make use of the `AArch64`'s `TTBR1`. Since the +kernel's virtual address space size is `2 GiB` since the last tutorial, `TTBR1` will cover the range +from `0xffff_ffff_ffff_ffff` down to `ffff_ffff_8000_0000` (both inclusive). + +## Implementation + +In `src/memory/mmu.rs`, we extend the `AssociatedTranslationTable` trait with a `TableStartFromTop` +associated type: + +```rust +/// Intended to be implemented for [`AddressSpace`]. +pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// + /// [u64::MAX, (u64::MAX - AS_SIZE) + 1] + type TableStartFromTop; + + /// A translation table whose address range is: + /// + /// [0, AS_SIZE - 1] + type TableStartFromBottom; +} +``` + +Architecture specific code in `_arch/aarch64/memory/mmu/translation_table.rs` populates both types +now by making use of a new generic that is added to `FixedSizeTranslationTable`, which defines +whether the covered address space starts from the top or the bottom: + +```rust +pub struct FixedSizeTranslationTable { + ... +``` + +```rust +impl memory::mmu::AssociatedTranslationTable + for memory::mmu::AddressSpace +where + [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, +{ + type TableStartFromTop = + FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, true>; + + type TableStartFromBottom = + FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, false>; +} +``` + +Thanks to this infrastructure, `BSP` Rust code in `bsp/raspberrypi/memory/mmu.rs` only needs to +change to this newly introduced type in order to switch from lower half to higher half translation +tables for the kernel: + +```rust +type KernelTranslationTable = + ::TableStartFromTop; +``` + +In the `link.ld` linker script, we define a new symbol `__kernel_virt_start_addr` now, which is the +start address of the kernel's virtual address space, calculated as `(u64::MAX - +__kernel_virt_addr_space_size) + 1`. In order to make virtual-to-physical address translation easier +for the human eye (and mind), we link the kernel itself at `__kernel_virt_start_addr + +__rpi_load_addr`. + +Before these tutorials, the first mapped address of the kernel binary was always located at +`__rpi_load_addr == 0x8_0000`. Starting with this tutorial, due to the `2 GiB` virtual address space +size, the new first mapped address is `ffff_ffff_8008_0000`. So by ignoring the upper bits of the +address, you can easily derive the physical address. + +The changes in the `_arch` `MMU` driver are minimal, and mostly concerned with configuring `TCR_EL1` +for use with `TTBR1_EL1` now. And of course, setting `TTBR1_EL1` in `fn +enable_mmu_and_caching(...)`. + +### Position-Independent Boot Code + +Remember all the fuss that we made about `position-independent code` that will be needed until the +`MMU` is enabled. Let's quickly check what it means for us in reality now: + +In `_arch/aarch64/cpu/boot.rs`, we turn on the `MMU` just before returning from `EL2` to `EL1`. So +by the time the CPU enters `EL1`, virtual memory will be active, and the CPU must therefore use the +new higher-half `virtual addresses` for everything it does. + +Specifically, this means the address from which the CPU should execute upon entering `EL1` (function +`runtime_init()`) must be a valid _virtual address_, same as the stack pointer's address. Both of +them are programmed in function `fn prepare_el2_to_el1_transition(...)`, so we must ensure now that +_link-time_ addresses are used here. For this reason, retrieval of these addresses happens in +`assembly` in `boot.s`, where we can explicitly enforce generation of **absolute** addresses: + +```asm +// Load the _absolute_ addresses of the following symbols. Since the kernel is linked at +// the top of the 64 bit address space, these are effectively virtual addresses. +ADR_ABS x1, __boot_core_stack_end_exclusive +ADR_ABS x2, runtime_init +``` + +Both values are forwarded to the Rust entry point function `_start_rust()`, which in turn forwards +them to `fn prepare_el2_to_el1_transition(...)`. + +One more thing to consider is that we keep on programming the boot core's stack address for `EL2` +using an address that is calculated `PC-relative`, because all the `EL2` code will still run while +virtual memory _is disabled_. As such, we need the "physical" address of the stack, so to speak. + +The previous tutorial also explained that it is not easily possible to compile select files using +`-fpic` in `Rust`. Still, we are doing some function calls in `Rust` before virtual memory is +enabled, so _theoretically_, there is room for failure. However, branches to local code in `AArch64` +are usually generated PC-relative. So it is a small risk worth taking. Should it still fail someday, +at least our automated CI pipeline would give notice when the tests start to fail. + +## Test it + +That's it! We are ready for a higher-half kernel now. Power up your Raspberrys and marvel at those +beautiful (virtual) addresses: + +Raspberry Pi 3: + +```console +$ make chainboot +[...] + +Precomputing kernel translation tables and patching kernel ELF + -------------------------------------------------- + Section Start Virt Addr Size + -------------------------------------------------- + Generating Code and RO data | 0xffff_ffff_8008_0000 | 64 KiB + Generating Data and bss | 0xffff_ffff_8009_0000 | 384 KiB + Generating Boot-core stack | 0xffff_ffff_8010_0000 | 512 KiB + -------------------------------------------------- + Patching Kernel table struct at physical 0x9_0000 + Patching Value of kernel table physical base address (0xd_0000) at physical 0x8_0060 + Finished in 0.03s + +Minipush 1.0 + +[MP] ⏳ Waiting for /dev/ttyUSB0 +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now + __ __ _ _ _ _ +| \/ (_)_ _ (_) | ___ __ _ __| | +| |\/| | | ' \| | |__/ _ \/ _` / _` | +|_| |_|_|_||_|_|____\___/\__,_\__,_| + + Raspberry Pi 3 + +[ML] Requesting binary +[MP] ⏩ Pushing 387 KiB =======================================🦀 100% 96 KiB/s Time: 00:00:04 +[ML] Loaded! Executing the payload now + +[ 4.320829] Booting on: Raspberry Pi 3 +[ 4.321102] MMU online: +[ 4.321394] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.323138] Virtual Physical Size Attr Entity +[ 4.324882] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.326629] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 4.328243] 0xffff_ffff_8009_0000..0xffff_ffff_800e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss +[ 4.329813] 0xffff_ffff_8010_0000..0xffff_ffff_8017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 4.331416] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 4.332867] | BCM PL011 UART +[ 4.334385] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 4.336128] ------------------------------------------------------------------------------------------------------------------------------------------- +``` + +Raspberry Pi 4: + +```console +$ BSP=rpi4 make chainboot +[...] + +Precomputing kernel translation tables and patching kernel ELF + -------------------------------------------------- + Section Start Virt Addr Size + -------------------------------------------------- + Generating Code and RO data | 0xffff_ffff_8008_0000 | 64 KiB + Generating Data and bss | 0xffff_ffff_8009_0000 | 448 KiB + Generating Boot-core stack | 0xffff_ffff_8011_0000 | 512 KiB + -------------------------------------------------- + Patching Kernel table struct at physical 0xa_0000 + Patching Value of kernel table physical base address (0xe_0000) at physical 0x8_0068 + Finished in 0.03s + +Minipush 1.0 + +[MP] ⏳ Waiting for /dev/ttyUSB0 +[MP] ✅ Serial connected +[MP] 🔌 Please power the target now + __ __ _ _ _ _ +| \/ (_)_ _ (_) | ___ __ _ __| | +| |\/| | | ' \| | |__/ _ \/ _` / _` | +|_| |_|_|_||_|_|____\___/\__,_\__,_| + + Raspberry Pi 4 + +[ML] Requesting binary +[MP] ⏩ Pushing 449 KiB ======================================🦀 100% 112 KiB/s Time: 00:00:04 +[ML] Loaded! Executing the payload now + +[ 5.011490] Booting on: Raspberry Pi 4 +[ 5.011588] MMU online: +[ 5.011881] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 5.013625] Virtual Physical Size Attr Entity +[ 5.015369] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 5.017115] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 5.018730] 0xffff_ffff_8009_0000..0xffff_ffff_800f_ffff --> 0x00_0009_0000..0x00_000f_ffff | 448 KiB | C RW XN | Kernel data and bss +[ 5.020299] 0xffff_ffff_8011_0000..0xffff_ffff_8018_ffff --> 0x00_0011_0000..0x00_0018_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 5.021903] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 5.023354] | BCM PL011 UART +[ 5.024871] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD +[ 5.026279] | GICC +[ 5.027687] ------------------------------------------------------------------------------------------------------------------------------------------- +``` + +## Diff to previous +```diff + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs +--- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs +@@ -11,7 +11,7 @@ + //! + //! crate::cpu::boot::arch_boot + +-use crate::{cpu, memory, memory::Address, runtime_init}; ++use crate::{cpu, memory, memory::Address}; + use core::intrinsics::unlikely; + use cortex_a::{asm, regs::*}; + +@@ -29,7 +29,10 @@ + /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. + /// - The HW state of EL1 must be prepared in a sound way. + #[inline(always)] +-unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { ++unsafe fn prepare_el2_to_el1_transition( ++ virt_boot_core_stack_end_exclusive_addr: u64, ++ virt_runtime_init_addr: u64, ++) { + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + +@@ -52,11 +55,11 @@ + ); + + // Second, let the link register point to runtime_init(). +- ELR_EL2.set(runtime_init::runtime_init as *const () as u64); ++ ELR_EL2.set(virt_runtime_init_addr); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. +- SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); ++ SP_EL1.set(virt_boot_core_stack_end_exclusive_addr); + } + + //-------------------------------------------------------------------------------------------------- +@@ -74,9 +77,13 @@ + #[no_mangle] + pub unsafe extern "C" fn _start_rust( + phys_kernel_tables_base_addr: u64, +- phys_boot_core_stack_end_exclusive_addr: u64, ++ virt_boot_core_stack_end_exclusive_addr: u64, ++ virt_runtime_init_addr: u64, + ) -> ! { +- prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); ++ prepare_el2_to_el1_transition( ++ virt_boot_core_stack_end_exclusive_addr, ++ virt_runtime_init_addr, ++ ); + + // Turn on the MMU for EL1. + let addr = Address::new(phys_kernel_tables_base_addr as usize); +@@ -84,6 +91,7 @@ + cpu::wait_forever(); + } + +- // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. ++ // Use `eret` to "return" to EL1. Since virtual memory will already be enabled, this results in ++ // execution of runtime_init() in EL1 from its _virtual address_. + asm::eret() + } + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s +--- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s ++++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s +@@ -18,6 +18,18 @@ + add \register, \register, #:lo12:\symbol + .endm + ++// Load the address of a symbol into a register, absolute. ++// ++// # Resources ++// ++// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html ++.macro ADR_ABS register, symbol ++ movz \register, #:abs_g3:\symbol ++ movk \register, #:abs_g2_nc:\symbol ++ movk \register, #:abs_g1_nc:\symbol ++ movk \register, #:abs_g0_nc:\symbol ++.endm ++ + .equ _EL2, 0x8 + .equ _core_id_mask, 0b11 + +@@ -47,11 +59,23 @@ + // Load the base address of the kernel's translation tables. + ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + +- // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. +- ADR_REL x1, __boot_core_stack_end_exclusive +- mov sp, x1 ++ // Load the _absolute_ addresses of the following symbols. Since the kernel is linked at ++ // the top of the 64 bit address space, these are effectively virtual addresses. ++ ADR_ABS x1, __boot_core_stack_end_exclusive ++ ADR_ABS x2, runtime_init ++ ++ // Load the PC-relative address of the stack and set the stack pointer. ++ // ++ // Since _start() is the first function that runs after the firmware has loaded the kernel ++ // into memory, retrieving this symbol PC-relative returns the "physical" address. ++ // ++ // Setting the stack pointer to this value ensures that anything that still runs in EL2, ++ // until the kernel returns to EL1 with the MMU enabled, works as well. After the return to ++ // EL1, the virtual address of the stack retrieved above will be used. ++ ADR_REL x4, __boot_core_stack_end_exclusive ++ mov sp, x4 + +- // Jump to Rust code. x0 and x1 hold the function arguments provided to _start_rust(). ++ // Jump to Rust code. x0, x1 and x2 hold the function arguments provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs +--- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs +@@ -131,7 +131,7 @@ + /// aligned, so the lvl3 is put first. + #[repr(C)] + #[repr(align(65536))] +-pub struct FixedSizeTranslationTable { ++pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + +@@ -258,14 +258,23 @@ + where + [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, + { +- type TableStartFromBottom = FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }>; ++ type TableStartFromTop = ++ FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, true>; ++ ++ type TableStartFromBottom = ++ FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, false>; + } + +-impl FixedSizeTranslationTable { ++impl ++ FixedSizeTranslationTable ++{ + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + ++ const START_FROM_TOP_OFFSET: Address = ++ Address::new((usize::MAX - (Granule512MiB::SIZE * NUM_TABLES)) + 1); ++ + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + const fn _new(for_precompute: bool) -> Self { +@@ -294,20 +303,32 @@ + /// The start address of the table's MMIO range. + #[inline(always)] + fn mmio_start_addr(&self) -> Address { +- Address::new( ++ let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), +- ) ++ ); ++ ++ if START_FROM_TOP { ++ addr += Self::START_FROM_TOP_OFFSET; ++ } ++ ++ addr + } + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] + fn mmio_end_addr_inclusive(&self) -> Address { +- Address::new( ++ let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) + | (Granule64KiB::SIZE - 1), +- ) ++ ); ++ ++ if START_FROM_TOP { ++ addr += Self::START_FROM_TOP_OFFSET; ++ } ++ ++ addr + } + + /// Helper to calculate the lvl2 and lvl3 indices from an address. +@@ -316,7 +337,12 @@ + &self, + addr: *const Page, + ) -> Result<(usize, usize), &'static str> { +- let addr = addr as usize; ++ let mut addr = addr as usize; ++ ++ if START_FROM_TOP { ++ addr -= Self::START_FROM_TOP_OFFSET.into_usize() ++ } ++ + let lvl2_index = addr >> Granule512MiB::SHIFT; + let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + +@@ -343,8 +369,9 @@ + // OS Interface Code + //------------------------------------------------------------------------------ + +-impl memory::mmu::translation_table::interface::TranslationTable +- for FixedSizeTranslationTable ++impl ++ memory::mmu::translation_table::interface::TranslationTable ++ for FixedSizeTranslationTable + { + fn init(&mut self) -> Result<(), &'static str> { + if self.initialized { +@@ -419,12 +446,16 @@ + return Err("Not enough MMIO space left"); + } + +- let addr = Address::new( ++ let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (self.cur_l3_mmio_index << Granule64KiB::SHIFT), + ); + self.cur_l3_mmio_index += num_pages; + ++ if START_FROM_TOP { ++ addr += Self::START_FROM_TOP_OFFSET; ++ } ++ + Ok(PageSliceDescriptor::from_addr(addr, num_pages)) + } + +@@ -447,7 +478,7 @@ + //-------------------------------------------------------------------------------------------------- + + #[cfg(test)] +-pub type MinSizeTranslationTable = FixedSizeTranslationTable<1>; ++pub type MinSizeTranslationTable = FixedSizeTranslationTable<1, false>; + + #[cfg(test)] + mod tests { + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs +--- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs +@@ -65,6 +65,7 @@ + + impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. ++ #[inline(always)] + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( +@@ -78,20 +79,21 @@ + } + + /// Configure various settings of stage 1 of the EL1 translation regime. ++ #[inline(always)] + fn configure_translation_control(&self) { +- let t0sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; ++ let t1sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; + + TCR_EL1.write( +- TCR_EL1::TBI0::Used ++ TCR_EL1::TBI1::Used + + TCR_EL1::IPS::Bits_40 +- + TCR_EL1::TG0::KiB_64 +- + TCR_EL1::SH0::Inner +- + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable +- + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable +- + TCR_EL1::EPD0::EnableTTBR0Walks +- + TCR_EL1::A1::TTBR0 +- + TCR_EL1::T0SZ.val(t0sz) +- + TCR_EL1::EPD1::DisableTTBR1Walks, ++ + TCR_EL1::TG1::KiB_64 ++ + TCR_EL1::SH1::Inner ++ + TCR_EL1::ORGN1::WriteBack_ReadAlloc_WriteAlloc_Cacheable ++ + TCR_EL1::IRGN1::WriteBack_ReadAlloc_WriteAlloc_Cacheable ++ + TCR_EL1::EPD1::EnableTTBR1Walks ++ + TCR_EL1::A1::TTBR1 ++ + TCR_EL1::T1SZ.val(t1sz) ++ + TCR_EL1::EPD0::DisableTTBR0Walks, + ); + } + } +@@ -130,7 +132,7 @@ + self.set_up_mair(); + + // Set the "Translation Table Base Register". +- TTBR0_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); ++ TTBR1_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); + + self.configure_translation_control(); + + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld +--- 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld ++++ 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld +@@ -6,6 +6,15 @@ + /* This file provides __kernel_virt_addr_space_size */ + INCLUDE src/bsp/raspberrypi/kernel_virt_addr_space_size.ld; + ++/* The kernel's virtual address range will be: ++ * ++ * [END_ADDRESS_INCLUSIVE, START_ADDRESS] ++ * [u64::MAX , (u64::MAX - __kernel_virt_addr_space_size) + 1] ++ * ++ * Since the start address is needed to set the linker address below, calculate it now. ++ */ ++__kernel_virt_start_addr = ((0xffffffffffffffff - __kernel_virt_addr_space_size) + 1); ++ + /* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ + __rpi_load_addr = 0x80000; + +@@ -19,13 +28,14 @@ + + SECTIONS + { +- . = __rpi_load_addr; ++ /* Add the load address as an offset. Makes virt-to-phys translation easier for the human eye */ ++ . = __kernel_virt_start_addr + __rpi_load_addr; + + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; +- .text : ++ .text : AT(__rpi_load_addr) + { + KEEP(*(.text._start)) + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs +--- 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs +@@ -23,7 +23,7 @@ + //-------------------------------------------------------------------------------------------------- + + type KernelTranslationTable = +- ::TableStartFromBottom; ++ ::TableStartFromTop; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs +--- 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs +@@ -80,6 +80,11 @@ + pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// ++ /// [u64::MAX, (u64::MAX - AS_SIZE) + 1] ++ type TableStartFromTop; ++ ++ /// A translation table whose address range is: ++ /// + /// [0, AS_SIZE - 1] + type TableStartFromBottom; + } + +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs +--- 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs +@@ -30,6 +30,7 @@ + /// # Safety + /// + /// - Only a single core must be active and running this function. ++#[no_mangle] + pub unsafe fn runtime_init() -> ! { + extern "Rust" { + fn kernel_init() -> !; + +diff -uNr 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs +--- 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs ++++ 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs +@@ -27,8 +27,8 @@ + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + +- println!("Writing beyond mapped area to address 9 GiB..."); +- let big_addr: u64 = 9 * 1024 * 1024 * 1024; ++ println!("Writing to bottom of address space to address 1 GiB..."); ++ let big_addr: u64 = 1 * 1024 * 1024 * 1024; + core::ptr::read_volatile(big_addr as *mut u64); + + // If execution reaches here, the memory access above did not cause a page fault exception. + +diff -uNr 15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb +--- 15_virtual_mem_part3_precomputed_tables/translation_table_tool/bsp.rb ++++ 16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb +@@ -31,7 +31,7 @@ + + symbols = `#{NM_BINARY} --demangle #{kernel_elf}`.split("\n") + @kernel_virt_addr_space_size = parse_from_symbols(symbols, /__kernel_virt_addr_space_size/) +- @kernel_virt_start_addr = 0 ++ @kernel_virt_start_addr = parse_from_symbols(symbols, /__kernel_virt_start_addr/) + @virt_addresses = parse_from_symbols(symbols, @virt_addresses) + @phys_addresses = virt_to_phys(@virt_addresses) + +``` diff --git a/16_virtual_mem_part4_higher_half_kernel/build.rs b/16_virtual_mem_part4_higher_half_kernel/build.rs new file mode 100644 index 000000000..3f0a29110 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/build.rs @@ -0,0 +1,8 @@ +use std::env; + +fn main() { + let linker_file = env::var("LINKER_FILE").unwrap(); + + println!("cargo:rerun-if-changed={}", linker_file); + println!("cargo:rerun-if-changed=build.rs"); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu.rs new file mode 100644 index 000000000..56443865a --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural processor code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::arch_cpu + +use cortex_a::asm; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +pub use asm::nop; + +/// Pause execution on the core. +#[inline(always)] +pub fn wait_forever() -> ! { + loop { + asm::wfe() + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- +#[cfg(feature = "test_build")] +use qemu_exit::QEMUExit; + +#[cfg(feature = "test_build")] +const QEMU_EXIT_HANDLE: qemu_exit::AArch64 = qemu_exit::AArch64::new(); + +/// Make the host QEMU binary execute `exit(1)`. +#[cfg(feature = "test_build")] +pub fn qemu_exit_failure() -> ! { + QEMU_EXIT_HANDLE.exit_failure() +} + +/// Make the host QEMU binary execute `exit(0)`. +#[cfg(feature = "test_build")] +pub fn qemu_exit_success() -> ! { + QEMU_EXIT_HANDLE.exit_success() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs new file mode 100644 index 000000000..4f0ef0b4d --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural boot code. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::boot::arch_boot + +use crate::{cpu, memory, memory::Address}; +use core::intrinsics::unlikely; +use cortex_a::{asm, regs::*}; + +// Assembly counterpart to this file. +global_asm!(include_str!("boot.s")); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Prepares the transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - The HW state of EL1 must be prepared in a sound way. +#[inline(always)] +unsafe fn prepare_el2_to_el1_transition( + virt_boot_core_stack_end_exclusive_addr: u64, + virt_runtime_init_addr: u64, +) { + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(virt_runtime_init_addr); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. + SP_EL1.set(virt_boot_core_stack_end_exclusive_addr); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// The Rust entry of the `kernel` binary. +/// +/// The function is called from the assembly `_start` function. +/// +/// # Safety +/// +/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +#[no_mangle] +pub unsafe extern "C" fn _start_rust( + phys_kernel_tables_base_addr: u64, + virt_boot_core_stack_end_exclusive_addr: u64, + virt_runtime_init_addr: u64, +) -> ! { + prepare_el2_to_el1_transition( + virt_boot_core_stack_end_exclusive_addr, + virt_runtime_init_addr, + ); + + // Turn on the MMU for EL1. + let addr = Address::new(phys_kernel_tables_base_addr as usize); + if unlikely(memory::mmu::enable_mmu_and_caching(addr).is_err()) { + cpu::wait_forever(); + } + + // Use `eret` to "return" to EL1. Since virtual memory will already be enabled, this results in + // execution of runtime_init() in EL1 from its _virtual address_. + asm::eret() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s new file mode 100644 index 000000000..638259d47 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +// Load the address of a symbol into a register, PC-relative. +// +// The symbol must lie within +/- 4 GiB of the Program Counter. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_REL register, symbol + adrp \register, \symbol + add \register, \register, #:lo12:\symbol +.endm + +// Load the address of a symbol into a register, absolute. +// +// # Resources +// +// - https://sourceware.org/binutils/docs-2.36/as/AArch64_002dRelocations.html +.macro ADR_ABS register, symbol + movz \register, #:abs_g3:\symbol + movk \register, #:abs_g2_nc:\symbol + movk \register, #:abs_g1_nc:\symbol + movk \register, #:abs_g0_nc:\symbol +.endm + +.equ _EL2, 0x8 +.equ _core_id_mask, 0b11 + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +.section .text._start + +//------------------------------------------------------------------------------ +// fn _start() +//------------------------------------------------------------------------------ +_start: + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 + b.ne 1f + + // Only proceed on the boot core. Park it otherwise. + mrs x1, MPIDR_EL1 + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 + b.ne 1f + + // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + + // Load the base address of the kernel's translation tables. + ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + + // Load the _absolute_ addresses of the following symbols. Since the kernel is linked at + // the top of the 64 bit address space, these are effectively virtual addresses. + ADR_ABS x1, __boot_core_stack_end_exclusive + ADR_ABS x2, runtime_init + + // Load the PC-relative address of the stack and set the stack pointer. + // + // Since _start() is the first function that runs after the firmware has loaded the kernel + // into memory, retrieving this symbol PC-relative returns the "physical" address. + // + // Setting the stack pointer to this value ensures that anything that still runs in EL2, + // until the kernel returns to EL1 with the MMU enabled, works as well. After the return to + // EL1, the virtual address of the stack retrieved above will be used. + ADR_REL x4, __boot_core_stack_end_exclusive + mov sp, x4 + + // Jump to Rust code. x0, x1 and x2 hold the function arguments provided to _start_rust(). + b _start_rust + + // Infinitely wait for events (aka "park the core"). +1: wfe + b 1b + +.size _start, . - _start +.type _start, function +.global _start diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs new file mode 100644 index 000000000..b9fdd0f73 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural symmetric multiprocessing. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::cpu::smp::arch_smp + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the executing core's id. +#[inline(always)] +pub fn core_id() -> T +where + T: From, +{ + const CORE_MASK: u64 = 0b11; + + T::from((MPIDR_EL1.get() & CORE_MASK) as u8) +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs new file mode 100644 index 000000000..4a2c8de96 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs @@ -0,0 +1,281 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural synchronous and asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::arch_exception + +use crate::{ + bsp::{self}, + exception, + memory::Address, +}; +use core::{cell::UnsafeCell, fmt}; +use cortex_a::{barrier, regs::*}; +use register::InMemoryRegister; + +// Assembly counterpart to this file. +global_asm!(include_str!("exception.s")); + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Wrapper struct for memory copy of SPSR_EL1. +#[repr(transparent)] +struct SpsrEL1(InMemoryRegister); + +/// The exception context as it is stored on the stack on exception entry. +#[repr(C)] +struct ExceptionContext { + /// General Purpose Registers. + gpr: [u64; 30], + + /// The link register, aka x30. + lr: u64, + + /// Exception link register. The program counter at the time the exception happened. + elr_el1: u64, + + /// Saved program status. + spsr_el1: SpsrEL1, +} + +/// Wrapper struct for pretty printing ESR_EL1. +struct EsrEL1; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Check if additional context can be derived from a data abort. +fn inspect_data_abort(f: &mut fmt::Formatter) -> fmt::Result { + let fault_addr = Address::new(FAR_EL1.get() as usize); + + if bsp::memory::mmu::virt_boot_core_stack_guard_page_desc().contains(fault_addr) { + writeln!( + f, + "\n\n >> Attempted to access the guard page of the kernel's boot core stack <<" + )?; + } + + Ok(()) +} + +/// Prints verbose information about the exception and then panics. +fn default_exception_handler(e: &ExceptionContext) { + panic!( + "\n\nCPU Exception!\n\ + FAR_EL1: {:#018x}\n\ + {}\n\ + {}", + FAR_EL1.get(), + EsrEL1 {}, + e + ); +} + +//------------------------------------------------------------------------------ +// Current, EL0 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn current_el0_synchronous(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] +unsafe extern "C" fn current_el0_irq(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +#[no_mangle] +unsafe extern "C" fn current_el0_serror(_e: &mut ExceptionContext) { + panic!("Should not be here. Use of SP_EL0 in EL1 is not supported.") +} + +//------------------------------------------------------------------------------ +// Current, ELx +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn current_elx_irq(_e: &mut ExceptionContext) { + use exception::asynchronous::interface::IRQManager; + + let token = &exception::asynchronous::IRQContext::new(); + bsp::exception::asynchronous::irq_manager().handle_pending_irqs(token); +} + +#[no_mangle] +unsafe extern "C" fn current_elx_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Lower, AArch64 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_irq(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch64_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Lower, AArch32 +//------------------------------------------------------------------------------ + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_synchronous(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_irq(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +#[no_mangle] +unsafe extern "C" fn lower_aarch32_serror(e: &mut ExceptionContext) { + default_exception_handler(e); +} + +//------------------------------------------------------------------------------ +// Pretty printing +//------------------------------------------------------------------------------ + +/// Human readable ESR_EL1. +#[rustfmt::skip] +impl fmt::Display for EsrEL1 { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let esr_el1 = ESR_EL1.extract(); + + // Raw print of whole register. + writeln!(f, "ESR_EL1: {:#010x}", esr_el1.get())?; + + // Raw print of exception class. + write!(f, " Exception Class (EC) : {:#x}", esr_el1.read(ESR_EL1::EC))?; + + // Exception class, translation. + let ec_translation = match esr_el1.read_as_enum(ESR_EL1::EC) { + Some(ESR_EL1::EC::Value::DataAbortCurrentEL) => "Data Abort, current EL", + _ => "N/A", + }; + writeln!(f, " - {}", ec_translation)?; + + // Raw print of instruction specific syndrome. + write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?; + + inspect_data_abort(f) + } +} + +/// Human readable SPSR_EL1. +#[rustfmt::skip] +impl fmt::Display for SpsrEL1 { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + // Raw value. + writeln!(f, "SPSR_EL1: {:#010x}", self.0.get())?; + + let to_flag_str = |x| -> _ { + if x { "Set" } else { "Not set" } + }; + + writeln!(f, " Flags:")?; + writeln!(f, " Negative (N): {}", to_flag_str(self.0.is_set(SPSR_EL1::N)))?; + writeln!(f, " Zero (Z): {}", to_flag_str(self.0.is_set(SPSR_EL1::Z)))?; + writeln!(f, " Carry (C): {}", to_flag_str(self.0.is_set(SPSR_EL1::C)))?; + writeln!(f, " Overflow (V): {}", to_flag_str(self.0.is_set(SPSR_EL1::V)))?; + + let to_mask_str = |x| -> _ { + if x { "Masked" } else { "Unmasked" } + }; + + writeln!(f, " Exception handling state:")?; + writeln!(f, " Debug (D): {}", to_mask_str(self.0.is_set(SPSR_EL1::D)))?; + writeln!(f, " SError (A): {}", to_mask_str(self.0.is_set(SPSR_EL1::A)))?; + writeln!(f, " IRQ (I): {}", to_mask_str(self.0.is_set(SPSR_EL1::I)))?; + writeln!(f, " FIQ (F): {}", to_mask_str(self.0.is_set(SPSR_EL1::F)))?; + + write!(f, " Illegal Execution State (IL): {}", + to_flag_str(self.0.is_set(SPSR_EL1::IL)) + ) + } +} + +/// Human readable print of the exception context. +impl fmt::Display for ExceptionContext { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + writeln!(f, "ELR_EL1: {:#018x}", self.elr_el1)?; + writeln!(f, "{}", self.spsr_el1)?; + writeln!(f)?; + writeln!(f, "General purpose register:")?; + + #[rustfmt::skip] + let alternating = |x| -> _ { + if x % 2 == 0 { " " } else { "\n" } + }; + + // Print two registers per line. + for (i, reg) in self.gpr.iter().enumerate() { + write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?; + } + write!(f, " lr : {:#018x}", self.lr) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::exception::PrivilegeLevel; + +/// The processing element's current privilege level. +pub fn current_privilege_level() -> (PrivilegeLevel, &'static str) { + let el = CurrentEL.read_as_enum(CurrentEL::EL); + match el { + Some(CurrentEL::EL::Value::EL2) => (PrivilegeLevel::Hypervisor, "EL2"), + Some(CurrentEL::EL::Value::EL1) => (PrivilegeLevel::Kernel, "EL1"), + Some(CurrentEL::EL::Value::EL0) => (PrivilegeLevel::User, "EL0"), + _ => (PrivilegeLevel::Unknown, "Unknown"), + } +} + +/// Init exception handling by setting the exception vector base address register. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +/// - The vector table and the symbol `__exception_vector_table_start` from the linker script must +/// adhere to the alignment and size constraints demanded by the ARMv8-A Architecture Reference +/// Manual. +pub unsafe fn handling_init() { + // Provided by exception.S. + extern "Rust" { + static __exception_vector_start: UnsafeCell<()>; + } + + VBAR_EL1.set(__exception_vector_start.get() as u64); + + // Force VBAR update to complete before next instruction. + barrier::isb(barrier::SY); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.s b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.s new file mode 100644 index 000000000..ffb1875ce --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.s @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//-------------------------------------------------------------------------------------------------- +// Definitions +//-------------------------------------------------------------------------------------------------- + +/// Call the function provided by parameter `\handler` after saving the exception context. Provide +/// the context as the first parameter to '\handler'. +.macro CALL_WITH_CONTEXT handler + // Make room on the stack for the exception context. + sub sp, sp, #16 * 17 + + // Store all general purpose registers on the stack. + stp x0, x1, [sp, #16 * 0] + stp x2, x3, [sp, #16 * 1] + stp x4, x5, [sp, #16 * 2] + stp x6, x7, [sp, #16 * 3] + stp x8, x9, [sp, #16 * 4] + stp x10, x11, [sp, #16 * 5] + stp x12, x13, [sp, #16 * 6] + stp x14, x15, [sp, #16 * 7] + stp x16, x17, [sp, #16 * 8] + stp x18, x19, [sp, #16 * 9] + stp x20, x21, [sp, #16 * 10] + stp x22, x23, [sp, #16 * 11] + stp x24, x25, [sp, #16 * 12] + stp x26, x27, [sp, #16 * 13] + stp x28, x29, [sp, #16 * 14] + + // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1). + mrs x1, ELR_EL1 + mrs x2, SPSR_EL1 + + stp lr, x1, [sp, #16 * 15] + str x2, [sp, #16 * 16] + + // x0 is the first argument for the function called through `\handler`. + mov x0, sp + + // Call `\handler`. + bl \handler + + // After returning from exception handling code, replay the saved context and return via + // `eret`. + b __exception_restore_context +.endm + +.macro FIQ_SUSPEND +1: wfe + b 1b +.endm + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +.section .text + +//------------------------------------------------------------------------------ +// The exception vector table. +//------------------------------------------------------------------------------ + +// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script. +.align 11 + +// Export a symbol for the Rust code to use. +__exception_vector_start: + +// Current exception level with SP_EL0. +// +// .org sets the offset relative to section start. +// +// # Safety +// +// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes. +.org 0x000 + CALL_WITH_CONTEXT current_el0_synchronous +.org 0x080 + CALL_WITH_CONTEXT current_el0_irq +.org 0x100 + FIQ_SUSPEND +.org 0x180 + CALL_WITH_CONTEXT current_el0_serror + +// Current exception level with SP_ELx, x > 0. +.org 0x200 + CALL_WITH_CONTEXT current_elx_synchronous +.org 0x280 + CALL_WITH_CONTEXT current_elx_irq +.org 0x300 + FIQ_SUSPEND +.org 0x380 + CALL_WITH_CONTEXT current_elx_serror + +// Lower exception level, AArch64 +.org 0x400 + CALL_WITH_CONTEXT lower_aarch64_synchronous +.org 0x480 + CALL_WITH_CONTEXT lower_aarch64_irq +.org 0x500 + FIQ_SUSPEND +.org 0x580 + CALL_WITH_CONTEXT lower_aarch64_serror + +// Lower exception level, AArch32 +.org 0x600 + CALL_WITH_CONTEXT lower_aarch32_synchronous +.org 0x680 + CALL_WITH_CONTEXT lower_aarch32_irq +.org 0x700 + FIQ_SUSPEND +.org 0x780 + CALL_WITH_CONTEXT lower_aarch32_serror +.org 0x800 + +//------------------------------------------------------------------------------ +// fn __exception_restore_context() +//------------------------------------------------------------------------------ +__exception_restore_context: + ldr w19, [sp, #16 * 16] + ldp lr, x20, [sp, #16 * 15] + + msr SPSR_EL1, x19 + msr ELR_EL1, x20 + + ldp x0, x1, [sp, #16 * 0] + ldp x2, x3, [sp, #16 * 1] + ldp x4, x5, [sp, #16 * 2] + ldp x6, x7, [sp, #16 * 3] + ldp x8, x9, [sp, #16 * 4] + ldp x10, x11, [sp, #16 * 5] + ldp x12, x13, [sp, #16 * 6] + ldp x14, x15, [sp, #16 * 7] + ldp x16, x17, [sp, #16 * 8] + ldp x18, x19, [sp, #16 * 9] + ldp x20, x21, [sp, #16 * 10] + ldp x22, x23, [sp, #16 * 11] + ldp x24, x25, [sp, #16 * 12] + ldp x26, x27, [sp, #16 * 13] + ldp x28, x29, [sp, #16 * 14] + + add sp, sp, #16 * 17 + + eret + +.size __exception_restore_context, . - __exception_restore_context +.type __exception_restore_context, function diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs new file mode 100644 index 000000000..a4b1a548e --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural asynchronous exception handling. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::exception::asynchronous::arch_asynchronous + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +mod daif_bits { + pub const IRQ: u8 = 0b0010; +} + +trait DaifField { + fn daif_field() -> register::Field; +} + +struct Debug; +struct SError; +struct IRQ; +struct FIQ; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DaifField for Debug { + fn daif_field() -> register::Field { + DAIF::D + } +} + +impl DaifField for SError { + fn daif_field() -> register::Field { + DAIF::A + } +} + +impl DaifField for IRQ { + fn daif_field() -> register::Field { + DAIF::I + } +} + +impl DaifField for FIQ { + fn daif_field() -> register::Field { + DAIF::F + } +} + +fn is_masked() -> bool +where + T: DaifField, +{ + DAIF.is_set(T::daif_field()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Returns whether IRQs are masked on the executing core. +pub fn is_local_irq_masked() -> bool { + !is_masked::() +} + +/// Unmask IRQs on the executing core. +/// +/// It is not needed to place an explicit instruction synchronization barrier after the `msr`. +/// Quoting the Architecture Reference Manual for ARMv8-A, section C5.1.3: +/// +/// "Writes to PSTATE.{PAN, D, A, I, F} occur in program order without the need for additional +/// synchronization." +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_unmask() { + #[rustfmt::skip] + asm!( + "msr DAIFClr, {arg}", + arg = const daif_bits::IRQ, + options(nomem, nostack, preserves_flags) + ); +} + +/// Mask IRQs on the executing core. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_mask() { + #[rustfmt::skip] + asm!( + "msr DAIFSet, {arg}", + arg = const daif_bits::IRQ, + options(nomem, nostack, preserves_flags) + ); +} + +/// Mask IRQs on the executing core and return the previously saved interrupt mask bits (DAIF). +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +#[inline(always)] +pub unsafe fn local_irq_mask_save() -> u64 { + let saved = DAIF.get(); + local_irq_mask(); + + saved +} + +/// Restore the interrupt mask bits (DAIF) using the callee's argument. +/// +/// # Safety +/// +/// - Changes the HW state of the executing core. +/// - No sanity checks on the input. +#[inline(always)] +pub unsafe fn local_irq_restore(saved: u64) { + DAIF.set(saved); +} + +/// Print the AArch64 exceptions status. +#[rustfmt::skip] +pub fn print_state() { + use crate::info; + + let to_mask_str = |x| -> _ { + if x { "Masked" } else { "Unmasked" } + }; + + info!(" Debug: {}", to_mask_str(is_masked::())); + info!(" SError: {}", to_mask_str(is_masked::())); + info!(" IRQ: {}", to_mask_str(is_masked::())); + info!(" FIQ: {}", to_mask_str(is_masked::())); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs new file mode 100644 index 000000000..f9e3d59bd --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management Unit Driver. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::arch_mmu + +use crate::{ + bsp, memory, + memory::{mmu::TranslationGranule, Address, Physical, Virtual}, +}; +use core::intrinsics::unlikely; +use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Memory Management Unit type. +struct MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub type Granule512MiB = TranslationGranule<{ 512 * 1024 * 1024 }>; +pub type Granule64KiB = TranslationGranule<{ 64 * 1024 }>; + +/// Constants for indexing the MAIR_EL1. +#[allow(dead_code)] +pub mod mair { + pub const DEVICE: u64 = 0; + pub const NORMAL: u64 = 1; +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static MMU: MemoryManagementUnit = MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl memory::mmu::AddressSpace { + /// Checks for architectural restrictions. + pub const fn arch_address_space_size_sanity_checks() { + // Size must be at least one full 512 MiB table. + assert!((AS_SIZE % Granule512MiB::SIZE) == 0); + + // Check for 48 bit virtual address size as maximum, which is supported by any ARMv8 + // version. + assert!(AS_SIZE <= (1 << 48)); + } +} + +impl MemoryManagementUnit { + /// Setup function for the MAIR_EL1 register. + #[inline(always)] + fn set_up_mair(&self) { + // Define the memory types being mapped. + MAIR_EL1.write( + // Attribute 1 - Cacheable normal DRAM. + MAIR_EL1::Attr1_Normal_Outer::WriteBack_NonTransient_ReadWriteAlloc + + MAIR_EL1::Attr1_Normal_Inner::WriteBack_NonTransient_ReadWriteAlloc + + + // Attribute 0 - Device. + MAIR_EL1::Attr0_Device::nonGathering_nonReordering_EarlyWriteAck, + ); + } + + /// Configure various settings of stage 1 of the EL1 translation regime. + #[inline(always)] + fn configure_translation_control(&self) { + let t1sz = (64 - bsp::memory::mmu::KernelVirtAddrSpace::SIZE_SHIFT) as u64; + + TCR_EL1.write( + TCR_EL1::TBI1::Used + + TCR_EL1::IPS::Bits_40 + + TCR_EL1::TG1::KiB_64 + + TCR_EL1::SH1::Inner + + TCR_EL1::ORGN1::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::IRGN1::WriteBack_ReadAlloc_WriteAlloc_Cacheable + + TCR_EL1::EPD1::EnableTTBR1Walks + + TCR_EL1::A1::TTBR1 + + TCR_EL1::T1SZ.val(t1sz) + + TCR_EL1::EPD0::DisableTTBR0Walks, + ); + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the MMU instance. +pub fn mmu() -> &'static impl memory::mmu::interface::MMU { + &MMU +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use memory::mmu::{MMUEnableError, TranslationError}; + +impl memory::mmu::interface::MMU for MemoryManagementUnit { + unsafe fn enable_mmu_and_caching( + &self, + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError> { + if unlikely(self.is_enabled()) { + return Err(MMUEnableError::AlreadyEnabled); + } + + // Fail early if translation granule is not supported. + if unlikely(!ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported)) { + return Err(MMUEnableError::Other( + "Translation granule not supported in HW", + )); + } + + // Prepare the memory attribute indirection register. + self.set_up_mair(); + + // Set the "Translation Table Base Register". + TTBR1_EL1.set_baddr(phys_tables_base_addr.into_usize() as u64); + + self.configure_translation_control(); + + // Switch the MMU on. + // + // First, force all previous changes to be seen before the MMU is enabled. + barrier::isb(barrier::SY); + + // Enable the MMU and turn on data and instruction caching. + SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); + + // Force MMU init to complete before next instruction. + barrier::isb(barrier::SY); + + Ok(()) + } + + #[inline(always)] + fn is_enabled(&self) -> bool { + SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) + } + + fn try_virt_to_phys( + &self, + virt: Address, + ) -> Result, TranslationError> { + if !self.is_enabled() { + return Err(TranslationError::MMUDisabled); + } + + let addr = virt.into_usize() as u64; + unsafe { + asm!( + "AT S1E1R, {0}", + in(reg) addr, + options(readonly, nostack, preserves_flags) + ); + } + + let par_el1 = PAR_EL1.extract(); + if par_el1.matches_all(PAR_EL1::F::TranslationAborted) { + return Err(TranslationError::Aborted); + } + + let phys_addr = (par_el1.read(PAR_EL1::PA) << 12) | (addr & 0xFFF); + + Ok(Address::new(phys_addr as usize)) + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs new file mode 100644 index 000000000..a58a3bb9c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -0,0 +1,505 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Architectural translation table. +//! +//! Only 64 KiB granule is supported. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::memory::mmu::translation_table::arch_translation_table + +use crate::{ + bsp, memory, + memory::{ + mmu::{ + arch_mmu::{Granule512MiB, Granule64KiB}, + AccessPermissions, AttributeFields, MemAttributes, Page, PageSliceDescriptor, + }, + Address, Physical, Virtual, + }, +}; +use core::convert::{self, TryInto}; +use register::{register_bitfields, InMemoryRegister}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-15. +register_bitfields! {u64, + STAGE1_TABLE_DESCRIPTOR [ + /// Physical address of the next descriptor. + NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + TYPE OFFSET(1) NUMBITS(1) [ + Block = 0, + Table = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D5-17. +register_bitfields! {u64, + STAGE1_PAGE_DESCRIPTOR [ + /// Unprivileged execute-never. + UXN OFFSET(54) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Privileged execute-never. + PXN OFFSET(53) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Physical address of the next table descriptor (lvl2) or the page descriptor (lvl3). + OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] + + /// Access flag. + AF OFFSET(10) NUMBITS(1) [ + False = 0, + True = 1 + ], + + /// Shareability field. + SH OFFSET(8) NUMBITS(2) [ + OuterShareable = 0b10, + InnerShareable = 0b11 + ], + + /// Access Permissions. + AP OFFSET(6) NUMBITS(2) [ + RW_EL1 = 0b00, + RW_EL1_EL0 = 0b01, + RO_EL1 = 0b10, + RO_EL1_EL0 = 0b11 + ], + + /// Memory attributes index into the MAIR_EL1 register. + AttrIndx OFFSET(2) NUMBITS(3) [], + + TYPE OFFSET(1) NUMBITS(1) [ + Reserved_Invalid = 0, + Page = 1 + ], + + VALID OFFSET(0) NUMBITS(1) [ + False = 0, + True = 1 + ] + ] +} + +/// A table descriptor for 64 KiB aperture. +/// +/// The output points to the next table. +#[derive(Copy, Clone)] +#[repr(C)] +struct TableDescriptor { + value: u64, +} + +/// A page descriptor with 64 KiB aperture. +/// +/// The output points to physical memory. +#[derive(Copy, Clone)] +#[repr(C)] +struct PageDescriptor { + value: u64, +} + +trait StartAddr { + fn virt_start_addr(&self) -> Address; +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Big monolithic struct for storing the translation tables. Individual levels must be 64 KiB +/// aligned, so the lvl3 is put first. +#[repr(C)] +#[repr(align(65536))] +pub struct FixedSizeTranslationTable { + /// Page descriptors, covering 64 KiB windows per entry. + lvl3: [[PageDescriptor; 8192]; NUM_TABLES], + + /// Table descriptors, covering 512 MiB windows. + lvl2: [TableDescriptor; NUM_TABLES], + + /// Index of the next free MMIO page. + cur_l3_mmio_index: usize, + + /// Have the tables been initialized? + initialized: bool, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl StartAddr for [T; N] { + fn virt_start_addr(&self) -> Address { + Address::new(self as *const _ as usize) + } +} + +impl TableDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance pointing to the supplied address. + pub fn from_next_lvl_table_addr(phys_next_lvl_table_addr: Address) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = phys_next_lvl_table_addr.into_usize() >> Granule64KiB::SHIFT; + val.write( + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + + STAGE1_TABLE_DESCRIPTOR::TYPE::Table + + STAGE1_TABLE_DESCRIPTOR::VALID::True, + ); + + TableDescriptor { value: val.get() } + } +} + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From + for register::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. + let mut desc = match attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => { + STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::NORMAL) + } + MemAttributes::Device => { + STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable + + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(memory::mmu::arch_mmu::mair::DEVICE) + } + }; + + // Access Permissions. + desc += match attribute_fields.acc_perms { + AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, + AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, + }; + + // The execute-never attribute is mapped to PXN in AArch64. + desc += if attribute_fields.execute_never { + STAGE1_PAGE_DESCRIPTOR::PXN::True + } else { + STAGE1_PAGE_DESCRIPTOR::PXN::False + }; + + // Always set unprivileged exectue-never as long as userspace is not implemented yet. + desc += STAGE1_PAGE_DESCRIPTOR::UXN::True; + + desc + } +} + +impl PageDescriptor { + /// Create an instance. + /// + /// Descriptor is invalid by default. + pub const fn new_zeroed() -> Self { + Self { value: 0 } + } + + /// Create an instance. + pub fn from_output_addr( + phys_output_addr: *const Page, + attribute_fields: &AttributeFields, + ) -> Self { + let val = InMemoryRegister::::new(0); + + let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; + val.write( + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted) + + STAGE1_PAGE_DESCRIPTOR::AF::True + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True + + attribute_fields.clone().into(), + ); + + Self { value: val.get() } + } + + /// Returns the valid bit. + fn is_valid(&self) -> bool { + InMemoryRegister::::new(self.value) + .is_set(STAGE1_PAGE_DESCRIPTOR::VALID) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl memory::mmu::AssociatedTranslationTable + for memory::mmu::AddressSpace +where + [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, +{ + type TableStartFromTop = + FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, true>; + + type TableStartFromBottom = + FixedSizeTranslationTable<{ Self::SIZE >> Granule512MiB::SHIFT }, false>; +} + +impl + FixedSizeTranslationTable +{ + // Reserve the last 256 MiB of the address space for MMIO mappings. + const L2_MMIO_START_INDEX: usize = NUM_TABLES - 1; + const L3_MMIO_START_INDEX: usize = 8192 / 2; + + const START_FROM_TOP_OFFSET: Address = + Address::new((usize::MAX - (Granule512MiB::SIZE * NUM_TABLES)) + 1); + + /// Create an instance. + #[allow(clippy::assertions_on_constants)] + const fn _new(for_precompute: bool) -> Self { + assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); + + // Can't have a zero-sized address space. + assert!(NUM_TABLES > 0); + + Self { + lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], + lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], + cur_l3_mmio_index: Self::L3_MMIO_START_INDEX, + initialized: for_precompute, + } + } + + pub const fn new_for_precompute() -> Self { + Self::_new(true) + } + + #[cfg(test)] + pub fn new_for_runtime() -> Self { + Self::_new(false) + } + + /// The start address of the table's MMIO range. + #[inline(always)] + fn mmio_start_addr(&self) -> Address { + let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (Self::L3_MMIO_START_INDEX << Granule64KiB::SHIFT), + ); + + if START_FROM_TOP { + addr += Self::START_FROM_TOP_OFFSET; + } + + addr + } + + /// The inclusive end address of the table's MMIO range. + #[inline(always)] + fn mmio_end_addr_inclusive(&self) -> Address { + let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (8191 << Granule64KiB::SHIFT) + | (Granule64KiB::SIZE - 1), + ); + + if START_FROM_TOP { + addr += Self::START_FROM_TOP_OFFSET; + } + + addr + } + + /// Helper to calculate the lvl2 and lvl3 indices from an address. + #[inline(always)] + fn lvl2_lvl3_index_from( + &self, + addr: *const Page, + ) -> Result<(usize, usize), &'static str> { + let mut addr = addr as usize; + + if START_FROM_TOP { + addr -= Self::START_FROM_TOP_OFFSET.into_usize() + } + + let lvl2_index = addr >> Granule512MiB::SHIFT; + let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; + + if lvl2_index > (NUM_TABLES - 1) { + return Err("Virtual page is out of bounds of translation table"); + } + + Ok((lvl2_index, lvl3_index)) + } + + /// Returns the PageDescriptor corresponding to the supplied Page. + #[inline(always)] + fn page_descriptor_from( + &mut self, + addr: *const Page, + ) -> Result<&mut PageDescriptor, &'static str> { + let (lvl2_index, lvl3_index) = self.lvl2_lvl3_index_from(addr)?; + + Ok(&mut self.lvl3[lvl2_index][lvl3_index]) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl + memory::mmu::translation_table::interface::TranslationTable + for FixedSizeTranslationTable +{ + fn init(&mut self) -> Result<(), &'static str> { + if self.initialized { + return Ok(()); + } + + // Populate the l2 entries. + for (lvl2_nr, lvl2_entry) in self.lvl2.iter_mut().enumerate() { + let addr = self.lvl3[lvl2_nr] + .virt_start_addr() + .try_into() + .map_err(|_| "Translation error")?; + + let desc = TableDescriptor::from_next_lvl_table_addr(addr); + *lvl2_entry = desc; + } + + self.cur_l3_mmio_index = Self::L3_MMIO_START_INDEX; + self.initialized = true; + + Ok(()) + } + + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + let p = phys_pages.as_slice(); + let v = virt_pages.as_slice(); + + // No work to do for empty slices. + if v.is_empty() { + return Ok(()); + } + + if v.len() != p.len() { + return Err("Tried to map page slices with unequal sizes"); + } + + if p.last().unwrap().as_ptr() >= bsp::memory::mmu::phys_addr_space_end_page() { + return Err("Tried to map outside of physical address space"); + } + + let iter = p.iter().zip(v.iter()); + for (phys_page, virt_page) in iter { + let page_descriptor = self.page_descriptor_from(virt_page.as_ptr())?; + if page_descriptor.is_valid() { + return Err("Virtual page is already mapped"); + } + + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + } + + Ok(()) + } + + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str> { + assert_eq!(self.initialized, true, "Translation tables not initialized"); + + if num_pages == 0 { + return Err("num_pages == 0"); + } + + if (self.cur_l3_mmio_index + num_pages) > 8191 { + return Err("Not enough MMIO space left"); + } + + let mut addr = Address::new( + (Self::L2_MMIO_START_INDEX << Granule512MiB::SHIFT) + | (self.cur_l3_mmio_index << Granule64KiB::SHIFT), + ); + self.cur_l3_mmio_index += num_pages; + + if START_FROM_TOP { + addr += Self::START_FROM_TOP_OFFSET; + } + + Ok(PageSliceDescriptor::from_addr(addr, num_pages)) + } + + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool { + let start_addr = virt_pages.start_addr(); + let end_addr_inclusive = virt_pages.end_addr_inclusive(); + + for i in [start_addr, end_addr_inclusive].iter() { + if (*i >= self.mmio_start_addr()) && (*i <= self.mmio_end_addr_inclusive()) { + return true; + } + } + + false + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +pub type MinSizeTranslationTable = FixedSizeTranslationTable<1, false>; + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct TableDescriptor` is as expected. + #[kernel_test] + fn size_of_tabledescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } + + /// Check if the size of `struct PageDescriptor` is as expected. + #[kernel_test] + fn size_of_pagedescriptor_equals_64_bit() { + assert_eq!( + core::mem::size_of::(), + core::mem::size_of::() + ); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs new file mode 100644 index 000000000..63017305b --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Architectural timer primitives. +//! +//! # Orientation +//! +//! Since arch modules are imported into generic modules using the path attribute, the path of this +//! file is: +//! +//! crate::time::arch_time + +use crate::{time, warn}; +use core::time::Duration; +use cortex_a::{barrier, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NS_PER_S: u64 = 1_000_000_000; + +/// ARMv8 Generic Timer. +struct GenericTimer; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static TIME_MANAGER: GenericTimer = GenericTimer; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl GenericTimer { + #[inline(always)] + fn read_cntpct(&self) -> u64 { + // Prevent that the counter is read ahead of time due to out-of-order execution. + unsafe { barrier::isb(barrier::SY) }; + CNTPCT_EL0.get() + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the time manager. +pub fn time_manager() -> &'static impl time::interface::TimeManager { + &TIME_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl time::interface::TimeManager for GenericTimer { + fn resolution(&self) -> Duration { + Duration::from_nanos(NS_PER_S / (CNTFRQ_EL0.get() as u64)) + } + + fn uptime(&self) -> Duration { + let current_count: u64 = self.read_cntpct() * NS_PER_S; + let frq: u64 = CNTFRQ_EL0.get() as u64; + + Duration::from_nanos(current_count / frq) + } + + fn spin_for(&self, duration: Duration) { + // Instantly return on zero. + if duration.as_nanos() == 0 { + return; + } + + // Calculate the register compare value. + let frq = CNTFRQ_EL0.get(); + let x = match frq.checked_mul(duration.as_nanos() as u64) { + None => { + warn!("Spin duration too long, skipping"); + return; + } + Some(val) => val, + }; + let tval = x / NS_PER_S; + + // Check if it is within supported bounds. + let warn: Option<&str> = if tval == 0 { + Some("smaller") + // The upper 32 bits of CNTP_TVAL_EL0 are reserved. + } else if tval > u32::max_value().into() { + Some("bigger") + } else { + None + }; + + if let Some(w) = warn { + warn!( + "Spin duration {} than architecturally supported, skipping", + w + ); + return; + } + + // Set the compare value register. + CNTP_TVAL_EL0.set(tval); + + // Kick off the counting. // Disable timer interrupt. + CNTP_CTL_EL0.modify(CNTP_CTL_EL0::ENABLE::SET + CNTP_CTL_EL0::IMASK::SET); + + // ISTATUS will be '1' when cval ticks have passed. Busy-check it. + while !CNTP_CTL_EL0.matches_all(CNTP_CTL_EL0::ISTATUS::SET) {} + + // Disable counting again. + CNTP_CTL_EL0.modify(CNTP_CTL_EL0::ENABLE::CLEAR); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp.rs new file mode 100644 index 000000000..c558922f7 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp.rs @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Conditional reexporting of Board Support Packages. + +mod device_driver; + +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +mod raspberrypi; + +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +pub use raspberrypi::*; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver.rs new file mode 100644 index 000000000..eac6bdd16 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver.rs @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Device driver. + +#[cfg(feature = "bsp_rpi4")] +mod arm; +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +mod bcm; +mod common; + +#[cfg(feature = "bsp_rpi4")] +pub use arm::*; +#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] +pub use bcm::*; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm.rs new file mode 100644 index 000000000..0bb1bd4b6 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! ARM driver top level. + +pub mod gicv2; + +pub use gicv2::*; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2.rs new file mode 100644 index 000000000..51c321868 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2.rs @@ -0,0 +1,248 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICv2 Driver - ARM Generic Interrupt Controller v2. +//! +//! The following is a collection of excerpts with useful information from +//! - `Programmer's Guide for ARMv8-A` +//! - `ARM Generic Interrupt Controller Architecture Specification` +//! +//! # Programmer's Guide - 10.6.1 Configuration +//! +//! The GIC is accessed as a memory-mapped peripheral. +//! +//! All cores can access the common Distributor, but the CPU interface is banked, that is, each core +//! uses the same address to access its own private CPU interface. +//! +//! It is not possible for a core to access the CPU interface of another core. +//! +//! # Architecture Specification - 10.6.2 Initialization +//! +//! Both the Distributor and the CPU interfaces are disabled at reset. The GIC must be initialized +//! after reset before it can deliver interrupts to the core. +//! +//! In the Distributor, software must configure the priority, target, security and enable individual +//! interrupts. The Distributor must subsequently be enabled through its control register +//! (GICD_CTLR). For each CPU interface, software must program the priority mask and preemption +//! settings. +//! +//! Each CPU interface block itself must be enabled through its control register (GICD_CTLR). This +//! prepares the GIC to deliver interrupts to the core. +//! +//! Before interrupts are expected in the core, software prepares the core to take interrupts by +//! setting a valid interrupt vector in the vector table, and clearing interrupt mask bits in +//! PSTATE, and setting the routing controls. +//! +//! The entire interrupt mechanism in the system can be disabled by disabling the Distributor. +//! Interrupt delivery to an individual core can be disabled by disabling its CPU interface. +//! Individual interrupts can also be disabled (or enabled) in the distributor. +//! +//! For an interrupt to reach the core, the individual interrupt, Distributor and CPU interface must +//! all be enabled. The interrupt also needs to be of sufficient priority, that is, higher than the +//! core's priority mask. +//! +//! # Architecture Specification - 1.4.2 Interrupt types +//! +//! - Peripheral interrupt +//! - Private Peripheral Interrupt (PPI) +//! - This is a peripheral interrupt that is specific to a single processor. +//! - Shared Peripheral Interrupt (SPI) +//! - This is a peripheral interrupt that the Distributor can route to any of a specified +//! combination of processors. +//! +//! - Software-generated interrupt (SGI) +//! - This is an interrupt generated by software writing to a GICD_SGIR register in the GIC. The +//! system uses SGIs for interprocessor communication. +//! - An SGI has edge-triggered properties. The software triggering of the interrupt is +//! equivalent to the edge transition of the interrupt request signal. +//! - When an SGI occurs in a multiprocessor implementation, the CPUID field in the Interrupt +//! Acknowledge Register, GICC_IAR, or the Aliased Interrupt Acknowledge Register, GICC_AIAR, +//! identifies the processor that requested the interrupt. +//! +//! # Architecture Specification - 2.2.1 Interrupt IDs +//! +//! Interrupts from sources are identified using ID numbers. Each CPU interface can see up to 1020 +//! interrupts. The banking of SPIs and PPIs increases the total number of interrupts supported by +//! the Distributor. +//! +//! The GIC assigns interrupt ID numbers ID0-ID1019 as follows: +//! - Interrupt numbers 32..1019 are used for SPIs. +//! - Interrupt numbers 0..31 are used for interrupts that are private to a CPU interface. These +//! interrupts are banked in the Distributor. +//! - A banked interrupt is one where the Distributor can have multiple interrupts with the +//! same ID. A banked interrupt is identified uniquely by its ID number and its associated +//! CPU interface number. Of the banked interrupt IDs: +//! - 00..15 SGIs +//! - 16..31 PPIs + +mod gicc; +mod gicd; + +use crate::{bsp, cpu, driver, exception, memory, synchronization, synchronization::InitStateLock}; +use core::sync::atomic::{AtomicBool, Ordering}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +type HandlerTable = [Option; GICv2::NUM_IRQS]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; + +/// Representation of the GIC. +pub struct GICv2 { + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + + /// The Distributor. + gicd: gicd::GICD, + + /// The CPU Interface. + gicc: gicc::GICC, + + /// Have the MMIO regions been remapped yet? + is_mmio_remapped: AtomicBool, + + /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. + handler_table: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl GICv2 { + const MAX_IRQ_NUMBER: usize = 300; // Normally 1019, but keep it lower to save some space. + const NUM_IRQS: usize = Self::MAX_IRQ_NUMBER + 1; + + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( + gicd_mmio_descriptor: memory::mmu::MMIODescriptor, + gicc_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { + Self { + gicd_mmio_descriptor, + gicc_mmio_descriptor, + gicd: gicd::GICD::new(gicd_mmio_descriptor.start_addr().into_usize()), + gicc: gicc::GICC::new(gicc_mmio_descriptor.start_addr().into_usize()), + is_mmio_remapped: AtomicBool::new(false), + handler_table: InitStateLock::new([None; Self::NUM_IRQS]), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::ReadWriteEx; + +impl driver::interface::DeviceDriver for GICv2 { + fn compatible(&self) -> &'static str { + "GICv2 (ARM Generic Interrupt Controller v2)" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let remapped = self.is_mmio_remapped.load(Ordering::Relaxed); + if !remapped { + let mut virt_addr; + + // GICD + virt_addr = memory::mmu::kernel_map_mmio("GICD", &self.gicd_mmio_descriptor)?; + self.gicd.set_mmio(virt_addr.into_usize()); + + // GICC + virt_addr = memory::mmu::kernel_map_mmio("GICC", &self.gicc_mmio_descriptor)?; + self.gicc.set_mmio(virt_addr.into_usize()); + + // Conclude remapping. + self.is_mmio_remapped.store(true, Ordering::Relaxed); + } + + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { + self.gicd.boot_core_init(); + } + + self.gicc.priority_accept_all(); + self.gicc.enable(); + + Ok(()) + } +} + +impl exception::asynchronous::interface::IRQManager for GICv2 { + type IRQNumberType = IRQNumber; + + fn register_handler( + &self, + irq_number: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { + let irq_number = irq_number.get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + + table[irq_number] = Some(descriptor); + + Ok(()) + }) + } + + fn enable(&self, irq_number: Self::IRQNumberType) { + self.gicd.enable(irq_number); + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + // Extract the highest priority pending IRQ number from the Interrupt Acknowledge Register + // (IAR). + let irq_number = self.gicc.pending_irq_number(ic); + + // Guard against spurious interrupts. + if irq_number > GICv2::MAX_IRQ_NUMBER { + return; + } + + // Call the IRQ handler. Panic if there is none. + self.handler_table.read(|table| { + match table[irq_number] { + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. + descriptor.handler.handle().expect("Error handling IRQ"); + } + } + }); + + // Signal completion of handling. + self.gicc.mark_comleted(irq_number as u32, ic); + } + + fn print_handler(&self) { + use crate::info; + + info!(" Peripheral handler:"); + + self.handler_table.read(|table| { + for (i, opt) in table.iter().skip(32).enumerate() { + if let Some(handler) = opt { + info!(" {: >3}. {}", i + 32, handler.name); + } + } + }); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs new file mode 100644 index 000000000..a1279f956 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICC Driver - GIC CPU interface. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_bitfields! { + u32, + + /// CPU Interface Control Register + CTLR [ + Enable OFFSET(0) NUMBITS(1) [] + ], + + /// Interrupt Priority Mask Register + PMR [ + Priority OFFSET(0) NUMBITS(8) [] + ], + + /// Interrupt Acknowledge Register + IAR [ + InterruptID OFFSET(0) NUMBITS(10) [] + ], + + /// End of Interrupt Register + EOIR [ + EOIINTID OFFSET(0) NUMBITS(10) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + pub RegisterBlock { + (0x000 => CTLR: ReadWrite), + (0x004 => PMR: ReadWrite), + (0x008 => _reserved1), + (0x00C => IAR: ReadWrite), + (0x010 => EOIR: ReadWrite), + (0x014 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the GIC CPU interface. +pub struct GICC { + registers: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::synchronization::interface::ReadWriteEx; + +impl GICC { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: InitStateLock::new(Registers::new(mmio_start_addr)), + } + } + + pub unsafe fn set_mmio(&self, new_mmio_start_addr: usize) { + self.registers + .write(|regs| *regs = Registers::new(new_mmio_start_addr)); + } + + /// Accept interrupts of any priority. + /// + /// Quoting the GICv2 Architecture Specification: + /// + /// "Writing 255 to the GICC_PMR always sets it to the largest supported priority field + /// value." + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + pub fn priority_accept_all(&self) { + self.registers.read(|regs| { + regs.PMR.write(PMR::Priority.val(255)); // Comment in arch spec. + }); + } + + /// Enable the interface - start accepting IRQs. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + pub fn enable(&self) { + self.registers.read(|regs| { + regs.CTLR.write(CTLR::Enable::SET); + }); + } + + /// Extract the number of the highest-priority pending IRQ. + /// + /// Can only be called from IRQ context, which is ensured by taking an `IRQContext` token. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + #[allow(clippy::trivially_copy_pass_by_ref)] + pub fn pending_irq_number<'irq_context>( + &self, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) -> usize { + self.registers + .read(|regs| regs.IAR.read(IAR::InterruptID) as usize) + } + + /// Complete handling of the currently active IRQ. + /// + /// Can only be called from IRQ context, which is ensured by taking an `IRQContext` token. + /// + /// To be called after `pending_irq_number()`. + /// + /// # Safety + /// + /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead + /// of `&mut self`. + #[allow(clippy::trivially_copy_pass_by_ref)] + pub fn mark_comleted<'irq_context>( + &self, + irq_number: u32, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + self.registers.read(|regs| { + regs.EOIR.write(EOIR::EOIINTID.val(irq_number)); + }); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs new file mode 100644 index 000000000..5f56e54a1 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -0,0 +1,205 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! GICD Driver - GIC Distributor. +//! +//! # Glossary +//! - SPI - Shared Peripheral Interrupt. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, + state, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_bitfields! { + u32, + + /// Distributor Control Register + CTLR [ + Enable OFFSET(0) NUMBITS(1) [] + ], + + /// Interrupt Controller Type Register + TYPER [ + ITLinesNumber OFFSET(0) NUMBITS(5) [] + ], + + /// Interrupt Processor Targets Registers + ITARGETSR [ + Offset3 OFFSET(24) NUMBITS(8) [], + Offset2 OFFSET(16) NUMBITS(8) [], + Offset1 OFFSET(8) NUMBITS(8) [], + Offset0 OFFSET(0) NUMBITS(8) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + SharedRegisterBlock { + (0x000 => CTLR: ReadWrite), + (0x004 => TYPER: ReadOnly), + (0x008 => _reserved1), + (0x104 => ISENABLER: [ReadWrite; 31]), + (0x108 => _reserved2), + (0x820 => ITARGETSR: [ReadWrite; 248]), + (0x824 => @END), + } +} + +register_structs! { + #[allow(non_snake_case)] + BankedRegisterBlock { + (0x000 => _reserved1), + (0x100 => ISENABLER: ReadWrite), + (0x104 => _reserved2), + (0x800 => ITARGETSR: [ReadOnly; 8]), + (0x804 => @END), + } +} + +/// Abstraction for the non-banked parts of the associated MMIO registers. +type SharedRegisters = MMIODerefWrapper; + +/// Abstraction for the banked parts of the associated MMIO registers. +type BankedRegisters = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the GIC Distributor. +pub struct GICD { + /// Access to shared registers is guarded with a lock. + shared_registers: IRQSafeNullLock, + + /// Access to banked registers is unguarded. + banked_registers: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl SharedRegisters { + /// Return the number of IRQs that this HW implements. + #[inline(always)] + fn num_irqs(&mut self) -> usize { + // Query number of implemented IRQs. + // + // Refer to GICv2 Architecture Specification, Section 4.3.2. + ((self.TYPER.read(TYPER::ITLinesNumber) as usize) + 1) * 32 + } + + /// Return a slice of the implemented ITARGETSR. + #[inline(always)] + fn implemented_itargets_slice(&mut self) -> &[ReadWrite] { + assert!(self.num_irqs() >= 36); + + // Calculate the max index of the shared ITARGETSR array. + // + // The first 32 IRQs are private, so not included in `shared_registers`. Each ITARGETS + // register has four entries, so shift right by two. Subtract one because we start + // counting at zero. + let spi_itargetsr_max_index = ((self.num_irqs() - 32) >> 2) - 1; + + // Rust automatically inserts slice range sanity check, i.e. max >= min. + &self.ITARGETSR[0..spi_itargetsr_max_index] + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::synchronization::interface::ReadWriteEx; +use synchronization::interface::Mutex; + +impl GICD { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + shared_registers: IRQSafeNullLock::new(SharedRegisters::new(mmio_start_addr)), + banked_registers: InitStateLock::new(BankedRegisters::new(mmio_start_addr)), + } + } + + pub unsafe fn set_mmio(&self, new_mmio_start_addr: usize) { + self.shared_registers + .lock(|regs| *regs = SharedRegisters::new(new_mmio_start_addr)); + self.banked_registers + .write(|regs| *regs = BankedRegisters::new(new_mmio_start_addr)); + } + + /// Use a banked ITARGETSR to retrieve the executing core's GIC target mask. + /// + /// Quoting the GICv2 Architecture Specification: + /// + /// "GICD_ITARGETSR0 to GICD_ITARGETSR7 are read-only, and each field returns a value that + /// corresponds only to the processor reading the register." + fn local_gic_target_mask(&self) -> u32 { + self.banked_registers + .read(|regs| regs.ITARGETSR[0].read(ITARGETSR::Offset0)) + } + + /// Route all SPIs to the boot core and enable the distributor. + pub fn boot_core_init(&self) { + assert!( + state::state_manager().is_init(), + "Only allowed during kernel init phase" + ); + + // Target all SPIs to the boot core only. + let mask = self.local_gic_target_mask(); + + self.shared_registers.lock(|regs| { + for i in regs.implemented_itargets_slice().iter() { + i.write( + ITARGETSR::Offset3.val(mask) + + ITARGETSR::Offset2.val(mask) + + ITARGETSR::Offset1.val(mask) + + ITARGETSR::Offset0.val(mask), + ); + } + + regs.CTLR.write(CTLR::Enable::SET); + }); + } + + /// Enable an interrupt. + pub fn enable(&self, irq_num: super::IRQNumber) { + let irq_num = irq_num.get(); + + // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 + // (division by 32) and arrive at the index for the respective ISENABLER[i]. + let enable_reg_index = irq_num >> 5; + let enable_bit: u32 = 1u32 << (irq_num % 32); + + // Check if we are handling a private or shared IRQ. + match irq_num { + // Private. + 0..=31 => self.banked_registers.read(|regs| { + let enable_reg = ®s.ISENABLER; + enable_reg.set(enable_reg.get() | enable_bit); + }), + // Shared. + _ => { + let enable_reg_index_shared = enable_reg_index - 1; + + self.shared_registers.lock(|regs| { + let enable_reg = ®s.ISENABLER[enable_reg_index_shared]; + enable_reg.set(enable_reg.get() | enable_bit); + }); + } + } + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm.rs new file mode 100644 index 000000000..01bcaa895 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BCM driver top level. + +mod bcm2xxx_gpio; +#[cfg(feature = "bsp_rpi3")] +mod bcm2xxx_interrupt_controller; +mod bcm2xxx_pl011_uart; + +pub use bcm2xxx_gpio::*; +#[cfg(feature = "bsp_rpi3")] +pub use bcm2xxx_interrupt_controller::*; +pub use bcm2xxx_pl011_uart::*; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs new file mode 100644 index 000000000..e444812ba --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -0,0 +1,255 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! GPIO Driver. + +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, driver, memory, synchronization, + synchronization::IRQSafeNullLock, +}; +use core::sync::atomic::{AtomicUsize, Ordering}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// GPIO registers. +// +// Descriptions taken from +// - https://github.com/raspberrypi/documentation/files/1888662/BCM2837-ARM-Peripherals.-.Revised.-.V2-1.pdf +// - https://datasheets.raspberrypi.org/bcm2711/bcm2711-peripherals.pdf +register_bitfields! { + u32, + + /// GPIO Function Select 1 + GPFSEL1 [ + /// Pin 15 + FSEL15 OFFSET(15) NUMBITS(3) [ + Input = 0b000, + Output = 0b001, + AltFunc0 = 0b100 // PL011 UART RX + + ], + + /// Pin 14 + FSEL14 OFFSET(12) NUMBITS(3) [ + Input = 0b000, + Output = 0b001, + AltFunc0 = 0b100 // PL011 UART TX + ] + ], + + /// GPIO Pull-up/down Register + /// + /// BCM2837 only. + GPPUD [ + /// Controls the actuation of the internal pull-up/down control line to ALL the GPIO pins. + PUD OFFSET(0) NUMBITS(2) [ + Off = 0b00, + PullDown = 0b01, + PullUp = 0b10 + ] + ], + + /// GPIO Pull-up/down Clock Register 0 + /// + /// BCM2837 only. + GPPUDCLK0 [ + /// Pin 15 + PUDCLK15 OFFSET(15) NUMBITS(1) [ + NoEffect = 0, + AssertClock = 1 + ], + + /// Pin 14 + PUDCLK14 OFFSET(14) NUMBITS(1) [ + NoEffect = 0, + AssertClock = 1 + ] + ], + + /// GPIO Pull-up / Pull-down Register 0 + /// + /// BCM2711 only. + GPIO_PUP_PDN_CNTRL_REG0 [ + /// Pin 15 + GPIO_PUP_PDN_CNTRL15 OFFSET(30) NUMBITS(2) [ + NoResistor = 0b00, + PullUp = 0b01 + ], + + /// Pin 14 + GPIO_PUP_PDN_CNTRL14 OFFSET(28) NUMBITS(2) [ + NoResistor = 0b00, + PullUp = 0b01 + ] + ] +} + +register_structs! { + #[allow(non_snake_case)] + RegisterBlock { + (0x00 => _reserved1), + (0x04 => GPFSEL1: ReadWrite), + (0x08 => _reserved2), + (0x94 => GPPUD: ReadWrite), + (0x98 => GPPUDCLK0: ReadWrite), + (0x9C => _reserved3), + (0xE4 => GPIO_PUP_PDN_CNTRL_REG0: ReadWrite), + (0xE8 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct GPIOInner { + registers: Registers, +} + +// Export the inner struct so that BSPs can use it for the panic handler. +pub use GPIOInner as PanicGPIO; + +/// Representation of the GPIO HW. +pub struct GPIO { + mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, + inner: IRQSafeNullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl GPIOInner { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: Registers::new(mmio_start_addr), + } + } + + /// Init code. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub unsafe fn init(&mut self, new_mmio_start_addr: Option) -> Result<(), &'static str> { + if let Some(addr) = new_mmio_start_addr { + self.registers = Registers::new(addr); + } + + Ok(()) + } + + /// Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi3")] + fn disable_pud_14_15_bcm2837(&mut self) { + use crate::{time, time::interface::TimeManager}; + use core::time::Duration; + + // The Linux 2837 GPIO driver waits 1 µs between the steps. + const DELAY: Duration = Duration::from_micros(1); + + self.registers.GPPUD.write(GPPUD::PUD::Off); + time::time_manager().spin_for(DELAY); + + self.registers + .GPPUDCLK0 + .write(GPPUDCLK0::PUDCLK15::AssertClock + GPPUDCLK0::PUDCLK14::AssertClock); + time::time_manager().spin_for(DELAY); + + self.registers.GPPUD.write(GPPUD::PUD::Off); + self.registers.GPPUDCLK0.set(0); + } + + /// Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi4")] + fn disable_pud_14_15_bcm2711(&mut self) { + self.registers.GPIO_PUP_PDN_CNTRL_REG0.write( + GPIO_PUP_PDN_CNTRL_REG0::GPIO_PUP_PDN_CNTRL15::PullUp + + GPIO_PUP_PDN_CNTRL_REG0::GPIO_PUP_PDN_CNTRL14::PullUp, + ); + } + + /// Map PL011 UART as standard output. + /// + /// TX to pin 14 + /// RX to pin 15 + pub fn map_pl011_uart(&mut self) { + // Select the UART on pins 14 and 15. + self.registers + .GPFSEL1 + .modify(GPFSEL1::FSEL15::AltFunc0 + GPFSEL1::FSEL14::AltFunc0); + + // Disable pull-up/down on pins 14 and 15. + #[cfg(feature = "bsp_rpi3")] + self.disable_pud_14_15_bcm2837(); + + #[cfg(feature = "bsp_rpi4")] + self.disable_pud_14_15_bcm2711(); + } +} + +impl GPIO { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + Self { + mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), + inner: IRQSafeNullLock::new(GPIOInner::new(mmio_descriptor.start_addr().into_usize())), + } + } + + /// Concurrency safe version of `GPIOInner.map_pl011_uart()` + pub fn map_pl011_uart(&self) { + self.inner.lock(|inner| inner.map_pl011_uart()) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; + +impl driver::interface::DeviceDriver for GPIO { + fn compatible(&self) -> &'static str { + "BCM GPIO" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; + + self.virt_mmio_start_addr + .store(virt_addr.into_usize(), Ordering::Relaxed); + + Ok(()) + } + + fn virt_mmio_start_addr(&self) -> Option { + let addr = self.virt_mmio_start_addr.load(Ordering::Relaxed); + + if addr == 0 { + return None; + } + + Some(addr) + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs new file mode 100644 index 000000000..0bd84e3cf --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Interrupt Controller Driver. + +mod peripheral_ic; + +use crate::{driver, exception, memory}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Wrapper struct for a bitmask indicating pending IRQ numbers. +struct PendingIRQs { + bitmask: u64, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub type LocalIRQ = + exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = + exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +#[derive(Copy, Clone)] +pub enum IRQNumber { + Local(LocalIRQ), + Peripheral(PeripheralIRQ), +} + +/// Representation of the Interrupt Controller. +pub struct InterruptController { + periph: peripheral_ic::PeripheralIC, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl PendingIRQs { + pub fn new(bitmask: u64) -> Self { + Self { bitmask } + } +} + +impl Iterator for PendingIRQs { + type Item = usize; + + fn next(&mut self) -> Option { + use core::intrinsics::cttz; + + let next = cttz(self.bitmask); + if next == 64 { + return None; + } + + self.bitmask &= !(1 << next); + + Some(next as usize) + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl InterruptController { + const MAX_LOCAL_IRQ_NUMBER: usize = 11; + const MAX_PERIPHERAL_IRQ_NUMBER: usize = 63; + const NUM_PERIPHERAL_IRQS: usize = Self::MAX_PERIPHERAL_IRQ_NUMBER + 1; + + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new( + _local_mmio_descriptor: memory::mmu::MMIODescriptor, + periph_mmio_descriptor: memory::mmu::MMIODescriptor, + ) -> Self { + Self { + periph: peripheral_ic::PeripheralIC::new(periph_mmio_descriptor), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl driver::interface::DeviceDriver for InterruptController { + fn compatible(&self) -> &'static str { + "BCM Interrupt Controller" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + self.periph.init() + } +} + +impl exception::asynchronous::interface::IRQManager for InterruptController { + type IRQNumberType = IRQNumber; + + fn register_handler( + &self, + irq: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + match irq { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), + IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + } + } + + fn enable(&self, irq: Self::IRQNumberType) { + match irq { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), + IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), + } + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + // It can only be a peripheral IRQ pending because enable() does not support local IRQs yet. + self.periph.handle_pending_irqs(ic) + } + + fn print_handler(&self) { + self.periph.print_handler(); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs new file mode 100644 index 000000000..abc3af711 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -0,0 +1,188 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Peripheral Interrupt Controller Driver. + +use super::{InterruptController, PendingIRQs, PeripheralIRQ}; +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, + driver, exception, memory, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, +}; +use register::{mmio::*, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +register_structs! { + #[allow(non_snake_case)] + WORegisterBlock { + (0x00 => _reserved1), + (0x10 => ENABLE_1: WriteOnly), + (0x14 => ENABLE_2: WriteOnly), + (0x24 => @END), + } +} + +register_structs! { + #[allow(non_snake_case)] + RORegisterBlock { + (0x00 => _reserved1), + (0x04 => PENDING_1: ReadOnly), + (0x08 => PENDING_2: ReadOnly), + (0x0c => @END), + } +} + +/// Abstraction for the WriteOnly parts of the associated MMIO registers. +type WriteOnlyRegisters = MMIODerefWrapper; + +/// Abstraction for the ReadOnly parts of the associated MMIO registers. +type ReadOnlyRegisters = MMIODerefWrapper; + +type HandlerTable = + [Option; InterruptController::NUM_PERIPHERAL_IRQS]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the peripheral interrupt controller. +pub struct PeripheralIC { + mmio_descriptor: memory::mmu::MMIODescriptor, + + /// Access to write registers is guarded with a lock. + wo_registers: IRQSafeNullLock, + + /// Register read access is unguarded. + ro_registers: InitStateLock, + + /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. + handler_table: InitStateLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl PeripheralIC { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + pub const unsafe fn new(mmio_descriptor: memory::mmu::MMIODescriptor) -> Self { + let addr = mmio_descriptor.start_addr().into_usize(); + + Self { + mmio_descriptor, + wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(addr)), + ro_registers: InitStateLock::new(ReadOnlyRegisters::new(addr)), + handler_table: InitStateLock::new([None; InterruptController::NUM_PERIPHERAL_IRQS]), + } + } + + /// Query the list of pending IRQs. + fn pending_irqs(&self) -> PendingIRQs { + self.ro_registers.read(|regs| { + let pending_mask: u64 = + (u64::from(regs.PENDING_2.get()) << 32) | u64::from(regs.PENDING_1.get()); + + PendingIRQs::new(pending_mask) + }) + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::{Mutex, ReadWriteEx}; + +impl driver::interface::DeviceDriver for PeripheralIC { + fn compatible(&self) -> &'static str { + "BCM Peripheral Interrupt Controller" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = + memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?.into_usize(); + + self.wo_registers + .lock(|regs| *regs = WriteOnlyRegisters::new(virt_addr)); + self.ro_registers + .write(|regs| *regs = ReadOnlyRegisters::new(virt_addr)); + + Ok(()) + } +} + +impl exception::asynchronous::interface::IRQManager for PeripheralIC { + type IRQNumberType = PeripheralIRQ; + + fn register_handler( + &self, + irq: Self::IRQNumberType, + descriptor: exception::asynchronous::IRQDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { + let irq_number = irq.get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + + table[irq_number] = Some(descriptor); + + Ok(()) + }) + } + + fn enable(&self, irq: Self::IRQNumberType) { + self.wo_registers.lock(|regs| { + let enable_reg = if irq.get() <= 31 { + ®s.ENABLE_1 + } else { + ®s.ENABLE_2 + }; + + let enable_bit: u32 = 1 << (irq.get() % 32); + + // Writing a 1 to a bit will set the corresponding IRQ enable bit. All other IRQ enable + // bits are unaffected. So we don't need read and OR'ing here. + enable_reg.set(enable_bit); + }); + } + + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + _ic: &exception::asynchronous::IRQContext<'irq_context>, + ) { + self.handler_table.read(|table| { + for irq_number in self.pending_irqs() { + match table[irq_number] { + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. + descriptor.handler.handle().expect("Error handling IRQ"); + } + } + } + }) + } + + fn print_handler(&self) { + use crate::info; + + info!(" Peripheral handler:"); + + self.handler_table.read(|table| { + for (i, opt) in table.iter().enumerate() { + if let Some(handler) = opt { + info!(" {: >3}. {}", i, handler.name); + } + } + }); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs new file mode 100644 index 000000000..3adb9137d --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -0,0 +1,537 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! PL011 UART driver. +//! +//! # Resources +//! +//! - +//! - + +use crate::{ + bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, memory, + synchronization, synchronization::IRQSafeNullLock, +}; +use core::{ + fmt, + sync::atomic::{AtomicUsize, Ordering}, +}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// PL011 UART registers. +// +// Descriptions taken from "PrimeCell UART (PL011) Technical Reference Manual" r1p5. +register_bitfields! { + u32, + + /// Flag Register. + FR [ + /// Transmit FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// Line Control Register, LCR_H. + /// + /// - If the FIFO is disabled, this bit is set when the transmit holding register is empty. + /// - If the FIFO is enabled, the TXFE bit is set when the transmit FIFO is empty. + /// - This bit does not indicate if there is data in the transmit shift register. + TXFE OFFSET(7) NUMBITS(1) [], + + /// Transmit FIFO full. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the transmit holding register is full. + /// - If the FIFO is enabled, the TXFF bit is set when the transmit FIFO is full. + TXFF OFFSET(5) NUMBITS(1) [], + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// If the FIFO is disabled, this bit is set when the receive holding register is empty. If + /// the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + + /// Receive FIFO empty. The meaning of this bit depends on the state of the FEN bit in the + /// LCR_H Register. + /// + /// - If the FIFO is disabled, this bit is set when the receive holding register is empty. + /// - If the FIFO is enabled, the RXFE bit is set when the receive FIFO is empty. + RXFE OFFSET(4) NUMBITS(1) [], + + /// UART busy. If this bit is set to 1, the UART is busy transmitting data. This bit remains + /// set until the complete byte, including all the stop bits, has been sent from the shift + /// register. + /// + /// This bit is set as soon as the transmit FIFO becomes non-empty, regardless of whether + /// the UART is enabled or not. + BUSY OFFSET(3) NUMBITS(1) [] + ], + + /// Integer Baud Rate Divisor. + IBRD [ + /// The integer baud rate divisor. + BAUD_DIVINT OFFSET(0) NUMBITS(16) [] + ], + + /// Fractional Baud Rate Divisor. + FBRD [ + /// The fractional baud rate divisor. + BAUD_DIVFRAC OFFSET(0) NUMBITS(6) [] + ], + + /// Line Control Register. + LCR_H [ + /// Word length. These bits indicate the number of data bits transmitted or received in a + /// frame. + WLEN OFFSET(5) NUMBITS(2) [ + FiveBit = 0b00, + SixBit = 0b01, + SevenBit = 0b10, + EightBit = 0b11 + ], + + /// Enable FIFOs: + /// + /// 0 = FIFOs are disabled (character mode) that is, the FIFOs become 1-byte-deep holding + /// registers. + /// + /// 1 = Transmit and receive FIFO buffers are enabled (FIFO mode). + FEN OFFSET(4) NUMBITS(1) [ + FifosDisabled = 0, + FifosEnabled = 1 + ] + ], + + /// Control Register. + CR [ + /// Receive enable. If this bit is set to 1, the receive section of the UART is enabled. + /// Data reception occurs for either UART signals or SIR signals depending on the setting of + /// the SIREN bit. When the UART is disabled in the middle of reception, it completes the + /// current character before stopping. + RXE OFFSET(9) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// Transmit enable. If this bit is set to 1, the transmit section of the UART is enabled. + /// Data transmission occurs for either UART signals, or SIR signals depending on the + /// setting of the SIREN bit. When the UART is disabled in the middle of transmission, it + /// completes the current character before stopping. + TXE OFFSET(8) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// UART enable: + /// + /// 0 = UART is disabled. If the UART is disabled in the middle of transmission or + /// reception, it completes the current character before stopping. + /// + /// 1 = The UART is enabled. Data transmission and reception occurs for either UART signals + /// or SIR signals depending on the setting of the SIREN bit + UARTEN OFFSET(0) NUMBITS(1) [ + /// If the UART is disabled in the middle of transmission or reception, it completes the + /// current character before stopping. + Disabled = 0, + Enabled = 1 + ] + ], + + /// Interrupt FIFO Level Select Register. + IFLS [ + /// Receive interrupt FIFO level select. The trigger points for the receive interrupt are as + /// follows. + RXIFLSEL OFFSET(3) NUMBITS(5) [ + OneEigth = 0b000, + OneQuarter = 0b001, + OneHalf = 0b010, + ThreeQuarters = 0b011, + SevenEights = 0b100 + ] + ], + + /// Interrupt Mask Set/Clear Register. + IMSC [ + /// Receive timeout interrupt mask. A read returns the current mask for the UARTRTINTR + /// interrupt. + /// + /// - On a write of 1, the mask of the UARTRTINTR interrupt is set. + /// - A write of 0 clears the mask. + RTIM OFFSET(6) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ], + + /// Receive interrupt mask. A read returns the current mask for the UARTRXINTR interrupt. + /// + /// - On a write of 1, the mask of the UARTRXINTR interrupt is set. + /// - A write of 0 clears the mask. + RXIM OFFSET(4) NUMBITS(1) [ + Disabled = 0, + Enabled = 1 + ] + ], + + /// Masked Interrupt Status Register. + MIS [ + /// Receive timeout masked interrupt status. Returns the masked interrupt state of the + /// UARTRTINTR interrupt. + RTMIS OFFSET(6) NUMBITS(1) [], + + /// Receive masked interrupt status. Returns the masked interrupt state of the UARTRXINTR + /// interrupt. + RXMIS OFFSET(4) NUMBITS(1) [] + ], + + /// Interrupt Clear Register. + ICR [ + /// Meta field for all pending interrupts. + ALL OFFSET(0) NUMBITS(11) [] + ] +} + +register_structs! { + #[allow(non_snake_case)] + pub RegisterBlock { + (0x00 => DR: ReadWrite), + (0x04 => _reserved1), + (0x18 => FR: ReadOnly), + (0x1c => _reserved2), + (0x24 => IBRD: WriteOnly), + (0x28 => FBRD: WriteOnly), + (0x2c => LCR_H: WriteOnly), + (0x30 => CR: WriteOnly), + (0x34 => IFLS: ReadWrite), + (0x38 => IMSC: ReadWrite), + (0x3C => _reserved3), + (0x40 => MIS: ReadOnly), + (0x44 => ICR: WriteOnly), + (0x48 => @END), + } +} + +/// Abstraction for the associated MMIO registers. +type Registers = MMIODerefWrapper; + +#[derive(PartialEq)] +enum BlockingMode { + Blocking, + NonBlocking, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct PL011UartInner { + registers: Registers, + chars_written: usize, + chars_read: usize, +} + +// Export the inner struct so that BSPs can use it for the panic handler. +pub use PL011UartInner as PanicUart; + +/// Representation of the UART. +pub struct PL011Uart { + mmio_descriptor: memory::mmu::MMIODescriptor, + virt_mmio_start_addr: AtomicUsize, + inner: IRQSafeNullLock, + irq_number: bsp::device_driver::IRQNumber, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl PL011UartInner { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub const unsafe fn new(mmio_start_addr: usize) -> Self { + Self { + registers: Registers::new(mmio_start_addr), + chars_written: 0, + chars_read: 0, + } + } + + /// Set up baud rate and characteristics. + /// + /// This results in 8N1 and 921_600 baud. + /// + /// The calculation for the BRD is (we set the clock to 48 MHz in config.txt): + /// `(48_000_000 / 16) / 921_600 = 3.2552083`. + /// + /// This means the integer part is `3` and goes into the `IBRD`. + /// The fractional part is `0.2552083`. + /// + /// `FBRD` calculation according to the PL011 Technical Reference Manual: + /// `INTEGER((0.2552083 * 64) + 0.5) = 16`. + /// + /// Therefore, the generated baud rate divider is: `3 + 16/64 = 3.25`. Which results in a + /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. + /// + /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16%`. + /// + /// # Safety + /// + /// - The user must ensure to provide a correct MMIO start address. + pub unsafe fn init(&mut self, new_mmio_start_addr: Option) -> Result<(), &'static str> { + if let Some(addr) = new_mmio_start_addr { + self.registers = Registers::new(addr); + } + + // Execution can arrive here while there are still characters queued in the TX FIFO and + // actively being sent out by the UART hardware. If the UART is turned off in this case, + // those queued characters would be lost. + // + // For example, this can happen during runtime on a call to panic!(), because panic!() + // initializes its own UART instance and calls init(). + // + // Hence, flush first to ensure all pending characters are transmitted. + self.flush(); + + // Turn the UART off temporarily. + self.registers.CR.set(0); + + // Clear all pending interrupts. + self.registers.ICR.write(ICR::ALL::CLEAR); + + // From the PL011 Technical Reference Manual: + // + // The LCR_H, IBRD, and FBRD registers form the single 30-bit wide LCR Register that is + // updated on a single write strobe generated by a LCR_H write. So, to internally update the + // contents of IBRD or FBRD, a LCR_H write must always be performed at the end. + // + // Set the baud rate, 8N1 and FIFO enabled. + self.registers.IBRD.write(IBRD::BAUD_DIVINT.val(3)); + self.registers.FBRD.write(FBRD::BAUD_DIVFRAC.val(16)); + self.registers + .LCR_H + .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); + + // Set RX FIFO fill level at 1/8. + self.registers.IFLS.write(IFLS::RXIFLSEL::OneEigth); + + // Enable RX IRQ + RX timeout IRQ. + self.registers + .IMSC + .write(IMSC::RXIM::Enabled + IMSC::RTIM::Enabled); + + // Turn the UART on. + self.registers + .CR + .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); + + Ok(()) + } + + /// Send a character. + fn write_char(&mut self, c: char) { + // Spin while TX FIFO full is set, waiting for an empty slot. + while self.registers.FR.matches_all(FR::TXFF::SET) { + cpu::nop(); + } + + // Write the character to the buffer. + self.registers.DR.set(c as u32); + + self.chars_written += 1; + } + + /// Block execution until the last buffered character has been physically put on the TX wire. + fn flush(&self) { + // Spin until the busy bit is cleared. + while self.registers.FR.matches_all(FR::BUSY::SET) { + cpu::nop(); + } + } + + /// Retrieve a character. + fn read_char_converting(&mut self, blocking_mode: BlockingMode) -> Option { + // If RX FIFO is empty, + if self.registers.FR.matches_all(FR::RXFE::SET) { + // immediately return in non-blocking mode. + if blocking_mode == BlockingMode::NonBlocking { + return None; + } + + // Otherwise, wait until a char was received. + while self.registers.FR.matches_all(FR::RXFE::SET) { + cpu::nop(); + } + } + + // Read one character. + let mut ret = self.registers.DR.get() as u8 as char; + + // Convert carrige return to newline. + if ret == '\r' { + ret = '\n' + } + + // Update statistics. + self.chars_read += 1; + + Some(ret) + } +} + +/// Implementing `core::fmt::Write` enables usage of the `format_args!` macros, which in turn are +/// used to implement the `kernel`'s `print!` and `println!` macros. By implementing `write_str()`, +/// we get `write_fmt()` automatically. +/// +/// The function takes an `&mut self`, so it must be implemented for the inner struct. +/// +/// See [`src/print.rs`]. +/// +/// [`src/print.rs`]: ../../print/index.html +impl fmt::Write for PL011UartInner { + fn write_str(&mut self, s: &str) -> fmt::Result { + for c in s.chars() { + self.write_char(c); + } + + Ok(()) + } +} + +impl PL011Uart { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide correct MMIO descriptors. + /// - The user must ensure to provide correct IRQ numbers. + pub const unsafe fn new( + mmio_descriptor: memory::mmu::MMIODescriptor, + irq_number: bsp::device_driver::IRQNumber, + ) -> Self { + Self { + mmio_descriptor, + virt_mmio_start_addr: AtomicUsize::new(0), + inner: IRQSafeNullLock::new(PL011UartInner::new( + mmio_descriptor.start_addr().into_usize(), + )), + irq_number, + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; + +impl driver::interface::DeviceDriver for PL011Uart { + fn compatible(&self) -> &'static str { + "BCM PL011 UART" + } + + unsafe fn init(&self) -> Result<(), &'static str> { + let virt_addr = memory::mmu::kernel_map_mmio(self.compatible(), &self.mmio_descriptor)?; + + self.inner + .lock(|inner| inner.init(Some(virt_addr.into_usize())))?; + + self.virt_mmio_start_addr + .store(virt_addr.into_usize(), Ordering::Relaxed); + + Ok(()) + } + + fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { + use bsp::exception::asynchronous::irq_manager; + use exception::asynchronous::{interface::IRQManager, IRQDescriptor}; + + let descriptor = IRQDescriptor { + name: "BCM PL011 UART", + handler: self, + }; + + irq_manager().register_handler(self.irq_number, descriptor)?; + irq_manager().enable(self.irq_number); + + Ok(()) + } + + fn virt_mmio_start_addr(&self) -> Option { + let addr = self.virt_mmio_start_addr.load(Ordering::Relaxed); + + if addr == 0 { + return None; + } + + Some(addr) + } +} + +impl console::interface::Write for PL011Uart { + /// Passthrough of `args` to the `core::fmt::Write` implementation, but guarded by a Mutex to + /// serialize access. + fn write_char(&self, c: char) { + self.inner.lock(|inner| inner.write_char(c)); + } + + fn write_fmt(&self, args: core::fmt::Arguments) -> fmt::Result { + // Fully qualified syntax for the call to `core::fmt::Write::write:fmt()` to increase + // readability. + self.inner.lock(|inner| fmt::Write::write_fmt(inner, args)) + } + + fn flush(&self) { + // Spin until TX FIFO empty is set. + self.inner.lock(|inner| inner.flush()); + } +} + +impl console::interface::Read for PL011Uart { + fn read_char(&self) -> char { + self.inner + .lock(|inner| inner.read_char_converting(BlockingMode::Blocking).unwrap()) + } + + fn clear_rx(&self) { + // Read from the RX FIFO until it is indicating empty. + while self + .inner + .lock(|inner| inner.read_char_converting(BlockingMode::NonBlocking)) + .is_some() + {} + } +} + +impl console::interface::Statistics for PL011Uart { + fn chars_written(&self) -> usize { + self.inner.lock(|inner| inner.chars_written) + } + + fn chars_read(&self) -> usize { + self.inner.lock(|inner| inner.chars_read) + } +} + +impl exception::asynchronous::interface::IRQHandler for PL011Uart { + fn handle(&self) -> Result<(), &'static str> { + self.inner.lock(|inner| { + let pending = inner.registers.MIS.extract(); + + // Clear all pending IRQs. + inner.registers.ICR.write(ICR::ALL::CLEAR); + + // Check for any kind of RX interrupt. + if pending.matches_any(MIS::RXMIS::SET + MIS::RTMIS::SET) { + // Echo any received characters. + while let Some(c) = inner.read_char_converting(BlockingMode::NonBlocking) { + inner.write_char(c) + } + } + }); + + Ok(()) + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/common.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/common.rs new file mode 100644 index 000000000..41553b7a5 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/common.rs @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Common device driver code. + +use core::{marker::PhantomData, ops}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct MMIODerefWrapper { + start_addr: usize, + phantom: PhantomData T>, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl MMIODerefWrapper { + /// Create an instance. + pub const unsafe fn new(start_addr: usize) -> Self { + Self { + start_addr, + phantom: PhantomData, + } + } +} + +impl ops::Deref for MMIODerefWrapper { + type Target = T; + + fn deref(&self) -> &Self::Target { + unsafe { &*(self.start_addr as *const _) } + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi.rs new file mode 100644 index 000000000..10d482781 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi.rs @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Top-level BSP file for the Raspberry Pi 3 and 4. + +pub mod console; +pub mod cpu; +pub mod driver; +pub mod exception; +pub mod memory; + +use super::device_driver; +use crate::memory::mmu::MMIODescriptor; +use memory::map::mmio; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static GPIO: device_driver::GPIO = + unsafe { device_driver::GPIO::new(MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE)) }; + +static PL011_UART: device_driver::PL011Uart = unsafe { + device_driver::PL011Uart::new( + MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE), + exception::asynchronous::irq_map::PL011_UART, + ) +}; + +#[cfg(feature = "bsp_rpi3")] +static INTERRUPT_CONTROLLER: device_driver::InterruptController = unsafe { + device_driver::InterruptController::new( + MMIODescriptor::new(mmio::LOCAL_IC_START, mmio::LOCAL_IC_SIZE), + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE), + ) +}; + +#[cfg(feature = "bsp_rpi4")] +static INTERRUPT_CONTROLLER: device_driver::GICv2 = unsafe { + device_driver::GICv2::new( + MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE), + MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE), + ) +}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Board identification. +pub fn board_name() -> &'static str { + #[cfg(feature = "bsp_rpi3")] + { + "Raspberry Pi 3" + } + + #[cfg(feature = "bsp_rpi4")] + { + "Raspberry Pi 4" + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/console.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/console.rs new file mode 100644 index 000000000..e16b3a3d6 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/console.rs @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP console facilities. + +use super::memory; +use crate::{bsp::device_driver, console, cpu, driver}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// In case of a panic, the panic handler uses this function to take a last shot at printing +/// something before the system is halted. +/// +/// We try to init panic-versions of the GPIO and the UART. The panic versions are not protected +/// with synchronization primitives, which increases chances that we get to print something, even +/// when the kernel's default GPIO or UART instances happen to be locked at the time of the panic. +/// +/// # Safety +/// +/// - Use only for printing during a panic. +#[cfg(not(feature = "test_build"))] +pub unsafe fn panic_console_out() -> impl fmt::Write { + use driver::interface::DeviceDriver; + + let mut panic_gpio = device_driver::PanicGPIO::new(memory::map::mmio::GPIO_START.into_usize()); + let mut panic_uart = + device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START.into_usize()); + + // If remapping of the driver's MMIO already happened, take the remapped start address. + // Otherwise, take a chance with the default physical address. + let maybe_gpio_mmio_start_addr = super::GPIO.virt_mmio_start_addr(); + let maybe_uart_mmio_start_addr = super::PL011_UART.virt_mmio_start_addr(); + + panic_gpio + .init(maybe_gpio_mmio_start_addr) + .unwrap_or_else(|_| cpu::wait_forever()); + panic_gpio.map_pl011_uart(); + panic_uart + .init(maybe_uart_mmio_start_addr) + .unwrap_or_else(|_| cpu::wait_forever()); + + panic_uart +} + +/// Reduced version for test builds. +#[cfg(feature = "test_build")] +pub unsafe fn panic_console_out() -> impl fmt::Write { + use driver::interface::DeviceDriver; + + let mut panic_uart = + device_driver::PanicUart::new(memory::map::mmio::PL011_UART_START.into_usize()); + + let maybe_uart_mmio_start_addr = super::PL011_UART.virt_mmio_start_addr(); + + panic_uart + .init(maybe_uart_mmio_start_addr) + .unwrap_or_else(|_| cpu::qemu_exit_failure()); + + panic_uart +} + +/// Return a reference to the console. +pub fn console() -> &'static impl console::interface::All { + &super::PL011_UART +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use driver::interface::DeviceDriver; + + // Calling the UART's init ensures that the BSP's instance of the UART does remap the MMIO + // addresses. + unsafe { + super::PL011_UART + .init() + .unwrap_or_else(|_| cpu::qemu_exit_failure()); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/cpu.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/cpu.rs new file mode 100644 index 000000000..d09ff9568 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/cpu.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Processor code. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used by `arch` code to find the early boot core. +#[no_mangle] +#[link_section = ".text._start_arguments"] +pub static BOOT_CORE_ID: u64 = 0; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/driver.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/driver.rs new file mode 100644 index 000000000..4d5007e4e --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/driver.rs @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP driver support. + +use crate::driver; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Device Driver Manager type. +struct BSPDriverManager { + device_drivers: [&'static (dyn DeviceDriver + Sync); 3], +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { + device_drivers: [ + &super::GPIO, + &super::PL011_UART, + &super::INTERRUPT_CONTROLLER, + ], +}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the driver manager. +pub fn driver_manager() -> &'static impl driver::interface::DriverManager { + &BSP_DRIVER_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use driver::interface::DeviceDriver; + +impl driver::interface::DriverManager for BSPDriverManager { + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[..] + } + + fn early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[0..=1] + } + + fn non_early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[2..] + } + + fn post_early_print_device_driver_init(&self) { + // Configure PL011Uart's output pins. + super::GPIO.map_pl011_uart(); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception.rs new file mode 100644 index 000000000..94d0728ff --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception.rs @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! BSP synchronous and asynchronous exception handling. + +pub mod asynchronous; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception/asynchronous.rs new file mode 100644 index 000000000..a253daa94 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! BSP asynchronous exception handling. + +use crate::{bsp, exception}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +#[cfg(feature = "bsp_rpi3")] +pub(in crate::bsp) mod irq_map { + use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; + + pub const PL011_UART: IRQNumber = IRQNumber::Peripheral(PeripheralIRQ::new(57)); +} + +#[cfg(feature = "bsp_rpi4")] +pub(in crate::bsp) mod irq_map { + use super::bsp::device_driver::IRQNumber; + + pub const PL011_UART: IRQNumber = IRQNumber::new(153); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the IRQ manager. +pub fn irq_manager() -> &'static impl exception::asynchronous::interface::IRQManager< + IRQNumberType = bsp::device_driver::IRQNumber, +> { + &super::super::INTERRUPT_CONTROLLER +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld new file mode 100644 index 000000000..0d46f9d97 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/kernel_virt_addr_space_size.ld @@ -0,0 +1 @@ +__kernel_virt_addr_space_size = 2 * 1024 * 1024 * 1024 diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld new file mode 100644 index 000000000..962638d00 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: MIT OR Apache-2.0 + * + * Copyright (c) 2018-2021 Andre Richter + */ + +/* This file provides __kernel_virt_addr_space_size */ +INCLUDE src/bsp/raspberrypi/kernel_virt_addr_space_size.ld; + +/* The kernel's virtual address range will be: + * + * [END_ADDRESS_INCLUSIVE, START_ADDRESS] + * [u64::MAX , (u64::MAX - __kernel_virt_addr_space_size) + 1] + * + * Since the start address is needed to set the linker address below, calculate it now. + */ +__kernel_virt_start_addr = ((0xffffffffffffffff - __kernel_virt_addr_space_size) + 1); + +/* The address at which the the kernel binary will be loaded by the Raspberry's firmware */ +__rpi_load_addr = 0x80000; + +ENTRY(__rpi_load_addr) + +PHDRS +{ + segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ + segment_rw PT_LOAD FLAGS(6); /* 6 == RW */ +} + +SECTIONS +{ + /* Add the load address as an offset. Makes virt-to-phys translation easier for the human eye */ + . = __kernel_virt_start_addr + __rpi_load_addr; + + /*********************************************************************************************** + * Code + RO Data + Global Offset Table + ***********************************************************************************************/ + __rx_start = .; + .text : AT(__rpi_load_addr) + { + KEEP(*(.text._start)) + *(.text._start_arguments) /* Constants (or statics in Rust speak) read by _start(). */ + *(.text._start_rust) /* The Rust entry point */ + *(.text*) /* Everything else */ + } :segment_rx + + .rodata : ALIGN(8) { *(.rodata*) } :segment_rx + .got : ALIGN(8) { *(.got) } :segment_rx + + . = ALIGN(64K); /* Align to page boundary */ + __rx_end_exclusive = .; + + /*********************************************************************************************** + * Data + BSS + ***********************************************************************************************/ + __rw_start = .; + .data : { *(.data*) } :segment_rw + + /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ + .bss : ALIGN(8) + { + __bss_start = .; + *(.bss*); + . = ALIGN(8); + + . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ + __bss_end_inclusive = . - 8; + } :NONE + + . = ALIGN(64K); /* Align to page boundary */ + __rw_end_exclusive = .; + + /*********************************************************************************************** + * Guard Page between boot core stack and data + ***********************************************************************************************/ + __boot_core_stack_guard_page_start = .; + . += 64K; + __boot_core_stack_guard_page_end_exclusive = .; + + /*********************************************************************************************** + * Boot Core Stack + ***********************************************************************************************/ + __boot_core_stack_start = .; /* ^ */ + /* | stack */ + . += 512K; /* | growth */ + /* | direction */ + __boot_core_stack_end_exclusive = .; /* | */ +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs new file mode 100644 index 000000000..7f571e080 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management. +//! +//! The physical memory layout after the kernel has been loaded by the Raspberry's firmware, which +//! copies the binary to 0x8_0000: +//! +//! +---------------------------------------------+ +//! | | +//! | Unmapped | +//! | | +//! +---------------------------------------------+ +//! | | rx_start @ 0x8_0000 +//! | .text | +//! | .rodata | +//! | .got | +//! | | rx_end_inclusive +//! +---------------------------------------------+ +//! | | rw_start == rx_end +//! | .data | +//! | .bss | +//! | | rw_end_inclusive +//! +---------------------------------------------+ +//! | | rw_end +//! | Unmapped Boot-core Stack Guard Page | +//! | | +//! +---------------------------------------------+ +//! | | boot_core_stack_start ^ +//! | | | stack +//! | Boot-core Stack | | growth +//! | | | direction +//! | | boot_core_stack_end_inclusive | +//! +---------------------------------------------+ + +pub mod mmu; + +use crate::memory::{Address, Physical, Virtual}; +use core::{cell::UnsafeCell, ops::RangeInclusive}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// Symbols from the linker script. +extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; + + static __rw_start: UnsafeCell<()>; + static __bss_start: UnsafeCell; + static __bss_end_inclusive: UnsafeCell; + static __rw_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_start: UnsafeCell<()>; + static __boot_core_stack_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_guard_page_start: UnsafeCell<()>; + static __boot_core_stack_guard_page_end_exclusive: UnsafeCell<()>; +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// The board's physical memory map. +#[rustfmt::skip] +pub(super) mod map { + use super::*; + + /// Physical devices. + #[cfg(feature = "bsp_rpi3")] + pub mod mmio { + use super::*; + + pub const PERIPHERAL_IC_START: Address = Address::new(0x3F00_B200); + pub const PERIPHERAL_IC_SIZE: usize = 0x24; + + pub const GPIO_START: Address = Address::new(0x3F20_0000); + pub const GPIO_SIZE: usize = 0xA0; + + pub const PL011_UART_START: Address = Address::new(0x3F20_1000); + pub const PL011_UART_SIZE: usize = 0x48; + + pub const LOCAL_IC_START: Address = Address::new(0x4000_0000); + pub const LOCAL_IC_SIZE: usize = 0x100; + + pub const END: Address = Address::new(0x4001_0000); + } + + /// Physical devices. + #[cfg(feature = "bsp_rpi4")] + pub mod mmio { + use super::*; + + pub const GPIO_START: Address = Address::new(0xFE20_0000); + pub const GPIO_SIZE: usize = 0xA0; + + pub const PL011_UART_START: Address = Address::new(0xFE20_1000); + pub const PL011_UART_SIZE: usize = 0x48; + + pub const GICD_START: Address = Address::new(0xFF84_1000); + pub const GICD_SIZE: usize = 0x824; + + pub const GICC_START: Address = Address::new(0xFF84_2000); + pub const GICC_SIZE: usize = 0x14; + + pub const END: Address = Address::new(0xFF85_0000); + } + + pub const END: Address = mmio::END; +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Start address of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn virt_rx_start() -> Address { + Address::new(unsafe { __rx_start.get() as usize }) +} + +/// Size of the Read+Execute (RX) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rx_size() -> usize { + unsafe { (__rx_end_exclusive.get() as usize) - (__rx_start.get() as usize) } +} + +/// Start address of the Read+Write (RW) range. +#[inline(always)] +fn virt_rw_start() -> Address { + Address::new(unsafe { __rw_start.get() as usize }) +} + +/// Size of the Read+Write (RW) range. +/// +/// # Safety +/// +/// - Value is provided by the linker script and must be trusted as-is. +#[inline(always)] +fn rw_size() -> usize { + unsafe { (__rw_end_exclusive.get() as usize) - (__rw_start.get() as usize) } +} + +/// Start address of the boot core's stack. +#[inline(always)] +fn virt_boot_core_stack_start() -> Address { + Address::new(unsafe { __boot_core_stack_start.get() as usize }) +} + +/// Size of the boot core's stack. +#[inline(always)] +fn boot_core_stack_size() -> usize { + unsafe { + (__boot_core_stack_end_exclusive.get() as usize) - (__boot_core_stack_start.get() as usize) + } +} + +/// Start address of the boot core's stack guard page. +#[inline(always)] +fn virt_boot_core_stack_guard_page_start() -> Address { + Address::new(unsafe { __boot_core_stack_guard_page_start.get() as usize }) +} + +/// Size of the boot core's stack guard page. +#[inline(always)] +fn boot_core_stack_guard_page_size() -> usize { + unsafe { + (__boot_core_stack_guard_page_end_exclusive.get() as usize) + - (__boot_core_stack_guard_page_start.get() as usize) + } +} + +/// Exclusive end address of the physical address space. +#[inline(always)] +fn phys_addr_space_end() -> Address { + map::END +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the inclusive range spanning the .bss section. +/// +/// # Safety +/// +/// - Values are provided by the linker script and must be trusted as-is. +/// - The linker-provided addresses must be u64 aligned. +pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { + let range; + unsafe { + range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); + } + assert!(!range.is_empty()); + + range +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs new file mode 100644 index 000000000..dbb970208 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory/mmu.rs @@ -0,0 +1,187 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! BSP Memory Management Unit. + +use crate::{ + common, + memory::{ + mmu as generic_mmu, + mmu::{ + AccessPermissions, AddressSpace, AssociatedTranslationTable, AttributeFields, + MemAttributes, Page, PageSliceDescriptor, TranslationGranule, + }, + Physical, Virtual, + }, + synchronization::InitStateLock, +}; +use core::convert::TryInto; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +type KernelTranslationTable = + ::TableStartFromTop; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to +/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; + +/// The kernel's virtual address space defined by this BSP. +pub type KernelVirtAddrSpace = AddressSpace<{ get_virt_addr_space_size() }>; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +/// The kernel translation tables. +/// +/// It is mandatory that InitStateLock is transparent. +/// +/// That is, `size_of(InitStateLock) == size_of(KernelTranslationTable)`. +/// There is a unit tests that checks this porperty. +#[link_section = ".data"] +static KERNEL_TABLES: InitStateLock = + InitStateLock::new(KernelTranslationTable::new_for_precompute()); + +/// This value is needed during early boot for MMU setup. +/// +/// This will be patched to the correct value by the "translation table tool" after linking. This +/// given value here is just a dummy. +#[link_section = ".text._start_arguments"] +#[no_mangle] +static PHYS_KERNEL_TABLES_BASE_ADDR: u64 = 0xCCCCAAAAFFFFEEEE; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// This is a hack for retrieving the value for the kernel's virtual address space size as a +/// constant from a common place, since it is needed as a compile-time/link-time constant in both, +/// the linker script and the Rust sources. +const fn get_virt_addr_space_size() -> usize { + let __kernel_virt_addr_space_size; + + include!("../kernel_virt_addr_space_size.ld"); + + __kernel_virt_addr_space_size +} + +/// Helper function for calculating the number of pages the given parameter spans. +const fn size_to_num_pages(size: usize) -> usize { + assert!(size > 0); + assert!(size % KernelGranule::SIZE == 0); + + size >> KernelGranule::SHIFT +} + +/// The Read+Execute (RX) pages of the kernel binary. +fn virt_rx_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rx_size()); + + PageSliceDescriptor::from_addr(super::virt_rx_start(), num_pages) +} + +/// The Read+Write (RW) pages of the kernel binary. +fn virt_rw_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::rw_size()); + + PageSliceDescriptor::from_addr(super::virt_rw_start(), num_pages) +} + +/// The boot core's stack. +fn virt_boot_core_stack_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_size()); + + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_start(), num_pages) +} + +// There is no reason to expect the following conversions to fail, since they were generated offline +// by the `translation table tool`. If it doesn't work, a panic due to the unwrap is justified. + +/// The Read+Execute (RX) pages of the kernel binary. +fn phys_rx_page_desc() -> PageSliceDescriptor { + virt_rx_page_desc().try_into().unwrap() +} + +/// The Read+Write (RW) pages of the kernel binary. +fn phys_rw_page_desc() -> PageSliceDescriptor { + virt_rw_page_desc().try_into().unwrap() +} + +/// The boot core's stack. +fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { + virt_boot_core_stack_page_desc().try_into().unwrap() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the kernel's translation tables. +pub fn kernel_translation_tables() -> &'static InitStateLock { + &KERNEL_TABLES +} + +/// The boot core's stack guard page. +pub fn virt_boot_core_stack_guard_page_desc() -> PageSliceDescriptor { + let num_pages = size_to_num_pages(super::boot_core_stack_guard_page_size()); + + PageSliceDescriptor::from_addr(super::virt_boot_core_stack_guard_page_start(), num_pages) +} + +/// Pointer to the last page of the physical address space. +pub fn phys_addr_space_end_page() -> *const Page { + common::align_down( + super::phys_addr_space_end().into_usize(), + KernelGranule::SIZE, + ) as *const Page<_> +} + +/// Add mapping records for the kernel binary. +/// +/// The actual translation table entries for the kernel binary are generated using the offline +/// `translation table tool` and patched into the kernel binary. This function just adds the mapping +/// record entries. +/// +/// It must be ensured that these entries are in sync with the offline tool. +pub fn kernel_add_mapping_records_for_precomputed() { + generic_mmu::kernel_add_mapping_record( + "Kernel code and RO data", + &virt_rx_page_desc(), + &phys_rx_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadOnly, + execute_never: false, + }, + ); + + generic_mmu::kernel_add_mapping_record( + "Kernel data and bss", + &virt_rw_page_desc(), + &phys_rw_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + ); + + generic_mmu::kernel_add_mapping_record( + "Kernel boot-core stack", + &virt_boot_core_stack_page_desc(), + &phys_boot_core_stack_page_desc(), + &AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + ); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/common.rs b/16_virtual_mem_part4_higher_half_kernel/src/common.rs new file mode 100644 index 000000000..71443cfbd --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/common.rs @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! General purpose code. + +/// Check if a value is aligned to a given size. +#[inline(always)] +pub const fn is_aligned(value: usize, alignment: usize) -> bool { + assert!(alignment.is_power_of_two()); + + (value & (alignment - 1)) == 0 +} + +/// Align down. +#[inline(always)] +pub const fn align_down(value: usize, alignment: usize) -> usize { + assert!(alignment.is_power_of_two()); + + value & !(alignment - 1) +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/console.rs b/16_virtual_mem_part4_higher_half_kernel/src/console.rs new file mode 100644 index 000000000..c3154ba23 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/console.rs @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! System console. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Console interfaces. +pub mod interface { + use core::fmt; + + /// Console write functions. + pub trait Write { + /// Write a single character. + fn write_char(&self, c: char); + + /// Write a Rust format string. + fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; + + /// Block until the last buffered character has been physically put on the TX wire. + fn flush(&self); + } + + /// Console read functions. + pub trait Read { + /// Read a single character. + fn read_char(&self) -> char { + ' ' + } + + /// Clear RX buffers, if any. + fn clear_rx(&self); + } + + /// Console statistics. + pub trait Statistics { + /// Return the number of characters written. + fn chars_written(&self) -> usize { + 0 + } + + /// Return the number of characters read. + fn chars_read(&self) -> usize { + 0 + } + } + + /// Trait alias for a full-fledged console. + pub trait All = Write + Read + Statistics; +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/cpu.rs b/16_virtual_mem_part4_higher_half_kernel/src/cpu.rs new file mode 100644 index 000000000..36b6a9d4c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/cpu.rs @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Processor code. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/cpu.rs"] +mod arch_cpu; + +mod boot; + +pub mod smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_cpu::{nop, wait_forever}; + +#[cfg(feature = "test_build")] +pub use arch_cpu::{qemu_exit_failure, qemu_exit_success}; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/cpu/boot.rs b/16_virtual_mem_part4_higher_half_kernel/src/cpu/boot.rs new file mode 100644 index 000000000..1dc5c1808 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/cpu/boot.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Boot code. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/boot.rs"] +mod arch_boot; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/cpu/smp.rs b/16_virtual_mem_part4_higher_half_kernel/src/cpu/smp.rs new file mode 100644 index 000000000..38230ce1e --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/cpu/smp.rs @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Symmetric multiprocessing. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/smp.rs"] +mod arch_smp; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_smp::core_id; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/driver.rs b/16_virtual_mem_part4_higher_half_kernel/src/driver.rs new file mode 100644 index 000000000..83fac4a7c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/driver.rs @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Driver support. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Driver interfaces. +pub mod interface { + /// Device Driver functions. + pub trait DeviceDriver { + /// Return a compatibility string for identifying the driver. + fn compatible(&self) -> &'static str; + + /// Called by the kernel to bring up the device. + /// + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + unsafe fn init(&self) -> Result<(), &'static str> { + Ok(()) + } + + /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// + /// Rust's type system will prevent a call to this function unless the calling instance + /// itself has static lifetime. + fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { + Ok(()) + } + + /// After MMIO remapping, returns the new virtual start address. + /// + /// This API assumes a driver has only a single, contiguous MMIO aperture, which will not be + /// the case for more complex devices. This API will likely change in future tutorials. + fn virt_mmio_start_addr(&self) -> Option { + None + } + } + + /// Device driver management functions. + /// + /// The `BSP` is supposed to supply one global instance. + pub trait DriverManager { + /// Return a slice of references to all `BSP`-instantiated drivers. + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Return only those drivers needed for the BSP's early printing functionality. + /// + /// For example, the default UART. + fn early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Return all drivers minus early-print drivers. + fn non_early_print_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Initialization code that runs after the early print driver init. + fn post_early_print_device_driver_init(&self); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/exception.rs b/16_virtual_mem_part4_higher_half_kernel/src/exception.rs new file mode 100644 index 000000000..3c5e7bc83 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/exception.rs @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Synchronous and asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/exception.rs"] +mod arch_exception; + +pub mod asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_exception::{current_privilege_level, handling_init}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Kernel privilege levels. +#[allow(missing_docs)] +#[derive(PartialEq)] +pub enum PrivilegeLevel { + User, + Kernel, + Hypervisor, + Unknown, +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Libkernel unit tests must execute in kernel mode. + #[kernel_test] + fn test_runner_executes_in_kernel_mode() { + let (level, _) = current_privilege_level(); + + assert!(level == PrivilegeLevel::Kernel) + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/src/exception/asynchronous.rs new file mode 100644 index 000000000..e9aace357 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/exception/asynchronous.rs @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/exception/asynchronous.rs"] +mod arch_asynchronous; + +use core::{fmt, marker::PhantomData}; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_asynchronous::{ + is_local_irq_masked, local_irq_mask, local_irq_mask_save, local_irq_restore, local_irq_unmask, + print_state, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Interrupt descriptor. +#[derive(Copy, Clone)] +pub struct IRQDescriptor { + /// Descriptive name. + pub name: &'static str, + + /// Reference to handler trait object. + pub handler: &'static (dyn interface::IRQHandler + Sync), +} + +/// IRQContext token. +/// +/// An instance of this type indicates that the local core is currently executing in IRQ +/// context, aka executing an interrupt vector or subcalls of it. +/// +/// Concept and implementation derived from the `CriticalSection` introduced in +/// +#[derive(Clone, Copy)] +pub struct IRQContext<'irq_context> { + _0: PhantomData<&'irq_context ()>, +} + +/// Asynchronous exception handling interfaces. +pub mod interface { + + /// Implemented by types that handle IRQs. + pub trait IRQHandler { + /// Called when the corresponding interrupt is asserted. + fn handle(&self) -> Result<(), &'static str>; + } + + /// IRQ management functions. + /// + /// The `BSP` is supposed to supply one global instance. Typically implemented by the + /// platform's interrupt controller. + pub trait IRQManager { + /// The IRQ number type depends on the implementation. + type IRQNumberType; + + /// Register a handler. + fn register_handler( + &self, + irq_number: Self::IRQNumberType, + descriptor: super::IRQDescriptor, + ) -> Result<(), &'static str>; + + /// Enable an interrupt in the controller. + fn enable(&self, irq_number: Self::IRQNumberType); + + /// Handle pending interrupts. + /// + /// This function is called directly from the CPU's IRQ exception vector. On AArch64, + /// this means that the respective CPU core has disabled exception handling. + /// This function can therefore not be preempted and runs start to finish. + /// + /// Takes an IRQContext token to ensure it can only be called from IRQ context. + #[allow(clippy::trivially_copy_pass_by_ref)] + fn handle_pending_irqs<'irq_context>( + &'irq_context self, + ic: &super::IRQContext<'irq_context>, + ); + + /// Print list of registered handlers. + fn print_handler(&self); + } +} + +/// A wrapper type for IRQ numbers with integrated range sanity check. +#[derive(Copy, Clone)] +pub struct IRQNumber(usize); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl<'irq_context> IRQContext<'irq_context> { + /// Creates an IRQContext token. + /// + /// # Safety + /// + /// - This must only be called when the current core is in an interrupt context and will not + /// live beyond the end of it. That is, creation is allowed in interrupt vector functions. For + /// example, in the ARMv8-A case, in `extern "C" fn current_elx_irq()`. + /// - Note that the lifetime `'irq_context` of the returned instance is unconstrained. User code + /// must not be able to influence the lifetime picked for this type, since that might cause it + /// to be inferred to `'static`. + #[inline(always)] + pub unsafe fn new() -> Self { + IRQContext { _0: PhantomData } + } +} + +impl IRQNumber<{ MAX_INCLUSIVE }> { + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self { 0: number } + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} + +/// Executes the provided closure while IRQs are masked on the executing core. +/// +/// While the function temporarily changes the HW state of the executing core, it restores it to the +/// previous state before returning, so this is deemed safe. +#[inline(always)] +pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { + let ret: T; + + unsafe { + let saved = local_irq_mask_save(); + ret = f(); + local_irq_restore(saved); + } + + ret +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs new file mode 100644 index 000000000..3346134ad --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +// Rust embedded logo for `make doc`. +#![doc(html_logo_url = "https://git.io/JeGIp")] + +//! The `kernel` library. +//! +//! Used to compose the final kernel binary. +//! +//! # Code organization and architecture +//! +//! The code is divided into different *modules*, each representing a typical **subsystem** of the +//! `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example, +//! `src/memory.rs` contains code that is concerned with all things memory management. +//! +//! ## Visibility of processor architecture code +//! +//! Some of the `kernel`'s subsystems depend on low-level code that is specific to the target +//! processor architecture. For each supported processor architecture, there exists a subfolder in +//! `src/_arch`, for example, `src/_arch/aarch64`. +//! +//! The architecture folders mirror the subsystem modules laid out in `src`. For example, +//! architectural code that belongs to the `kernel`'s MMU subsystem (`src/memory/mmu.rs`) would go +//! into `src/_arch/aarch64/memory/mmu.rs`. The latter file is loaded as a module in +//! `src/memory/mmu.rs` using the `path attribute`. Usually, the chosen module name is the generic +//! module's name prefixed with `arch_`. +//! +//! For example, this is the top of `src/memory/mmu.rs`: +//! +//! ``` +//! #[cfg(target_arch = "aarch64")] +//! #[path = "../_arch/aarch64/memory/mmu.rs"] +//! mod arch_mmu; +//! ``` +//! +//! Often times, items from the `arch_ module` will be publicly reexported by the parent module. +//! This way, each architecture specific module can provide its implementation of an item, while the +//! caller must not be concerned which architecture has been conditionally compiled. +//! +//! ## BSP code +//! +//! `BSP` stands for Board Support Package. `BSP` code is organized under `src/bsp.rs` and contains +//! target board specific definitions and functions. These are things such as the board's memory map +//! or instances of drivers for devices that are featured on the respective board. +//! +//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the +//! `kernel`'s subsystem modules, but there is no reexporting this time. That means whatever is +//! provided must be called starting from the `bsp` namespace, e.g. `bsp::driver::driver_manager()`. +//! +//! ## Kernel interfaces +//! +//! Both `arch` and `bsp` contain code that is conditionally compiled depending on the actual target +//! and board for which the kernel is compiled. For example, the `interrupt controller` hardware of +//! the `Raspberry Pi 3` and the `Raspberry Pi 4` is different, but we want the rest of the `kernel` +//! code to play nicely with any of the two without much hassle. +//! +//! In order to provide a clean abstraction between `arch`, `bsp` and `generic kernel code`, +//! `interface` traits are provided *whenever possible* and *where it makes sense*. They are defined +//! in the respective subsystem module and help to enforce the idiom of *program to an interface, +//! not an implementation*. For example, there will be a common IRQ handling interface which the two +//! different interrupt controller `drivers` of both Raspberrys will implement, and only export the +//! interface to the rest of the `kernel`. +//! +//! ``` +//! +-------------------+ +//! | Interface (Trait) | +//! | | +//! +--+-------------+--+ +//! ^ ^ +//! | | +//! | | +//! +----------+--+ +--+----------+ +//! | kernel code | | bsp code | +//! | | | arch code | +//! +-------------+ +-------------+ +//! ``` +//! +//! # Summary +//! +//! For a logical `kernel` subsystem, corresponding code can be distributed over several physical +//! locations. Here is an example for the **memory** subsystem: +//! +//! - `src/memory.rs` and `src/memory/**/*` +//! - Common code that is agnostic of target processor architecture and `BSP` characteristics. +//! - Example: A function to zero a chunk of memory. +//! - Interfaces for the memory subsystem that are implemented by `arch` or `BSP` code. +//! - Example: An `MMU` interface that defines `MMU` function prototypes. +//! - `src/bsp/__board_name__/memory.rs` and `src/bsp/__board_name__/memory/**/*` +//! - `BSP` specific code. +//! - Example: The board's memory map (physical addresses of DRAM and MMIO devices). +//! - `src/_arch/__arch_name__/memory.rs` and `src/_arch/__arch_name__/memory/**/*` +//! - Processor architecture specific code. +//! - Example: Implementation of the `MMU` interface for the `__arch_name__` processor +//! architecture. +//! +//! From a namespace perspective, **memory** subsystem code lives in: +//! +//! - `crate::memory::*` +//! - `crate::bsp::memory::*` +//! +//! # Boot flow +//! +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. +//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. +//! +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + +#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(incomplete_features)] +#![feature(asm)] +#![feature(const_evaluatable_checked)] +#![feature(const_fn)] +#![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] +#![feature(const_panic)] +#![feature(core_intrinsics)] +#![feature(format_args_nl)] +#![feature(global_asm)] +#![feature(linkage)] +#![feature(panic_info_message)] +#![feature(trait_alias)] +#![no_std] +// Testing +#![cfg_attr(test, no_main)] +#![feature(custom_test_frameworks)] +#![reexport_test_harness_main = "test_main"] +#![test_runner(crate::test_runner)] + +mod panic_wait; +mod runtime_init; +mod synchronization; + +pub mod bsp; +pub mod common; +pub mod console; +pub mod cpu; +pub mod driver; +pub mod exception; +pub mod memory; +pub mod print; +pub mod state; +pub mod time; + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +/// The default runner for unit tests. +pub fn test_runner(tests: &[&test_types::UnitTest]) { + println!("Running {} tests", tests.len()); + println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { + print!("{:>3}. {:.<58}", i + 1, test.name); + + // Run the actual test. + (test.test_func)(); + + // Failed tests call panic!(). Execution reaches here only if the test has passed. + println!("[ok]") + } +} + +/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +#[cfg(test)] +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + test_main(); + + cpu::qemu_exit_success() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/main.rs b/16_virtual_mem_part4_higher_half_kernel/src/main.rs new file mode 100644 index 000000000..c317bb96f --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/main.rs @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +// Rust embedded logo for `make doc`. +#![doc(html_logo_url = "https://git.io/JeGIp")] + +//! The `kernel` binary. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; + +/// Early init code. +/// +/// # Safety +/// +/// - Only a single core must be active and running this function. +/// - Printing will not work until the respective driver's MMIO is remapped. +#[no_mangle] +unsafe fn kernel_init() -> ! { + use driver::interface::DriverManager; + + exception::handling_init(); + + // Add the mapping records for the precomputed entries first, so that they appear on the top of + // the list. + bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); + + // Bring up the drivers needed for printing first. + for i in bsp::driver::driver_manager() + .early_print_device_drivers() + .iter() + { + // Any encountered errors cannot be printed yet, obviously, so just safely park the CPU. + i.init().unwrap_or_else(|_| cpu::wait_forever()); + } + bsp::driver::driver_manager().post_early_print_device_driver_init(); + // Printing available from here on. + + // Now bring up the remaining drivers. + for i in bsp::driver::driver_manager() + .non_early_print_device_drivers() + .iter() + { + if let Err(x) = i.init() { + panic!("Error loading driver: {}: {}", i.compatible(), x); + } + } + + // Let device drivers register and enable their handlers with the interrupt controller. + for i in bsp::driver::driver_manager().all_device_drivers() { + if let Err(msg) = i.register_and_enable_irq_handler() { + warn!("Error registering IRQ handler: {}", msg); + } + } + + // Unmask interrupts on the boot CPU core. + exception::asynchronous::local_irq_unmask(); + + // Announce conclusion of the kernel_init() phase. + state::state_manager().transition_to_single_core_main(); + + // Transition from unsafe to safe. + kernel_main() +} + +/// The main function running after the early init. +fn kernel_main() -> ! { + use driver::interface::DriverManager; + use exception::asynchronous::interface::IRQManager; + + info!("Booting on: {}", bsp::board_name()); + + info!("MMU online:"); + memory::mmu::kernel_print_mappings(); + + let (_, privilege_level) = exception::current_privilege_level(); + info!("Current privilege level: {}", privilege_level); + + info!("Exception handling state:"); + exception::asynchronous::print_state(); + + info!( + "Architectural timer resolution: {} ns", + time::time_manager().resolution().as_nanos() + ); + + info!("Drivers loaded:"); + for (i, driver) in bsp::driver::driver_manager() + .all_device_drivers() + .iter() + .enumerate() + { + info!(" {}. {}", i + 1, driver.compatible()); + } + + info!("Registered IRQ handlers:"); + bsp::exception::asynchronous::irq_manager().print_handler(); + + info!("Echoing input now"); + cpu::wait_forever(); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory.rs new file mode 100644 index 000000000..87ffa81f5 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory.rs @@ -0,0 +1,202 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Memory Management. + +pub mod mmu; + +use crate::common; +use core::{ + convert::TryFrom, + fmt, + marker::PhantomData, + ops::{AddAssign, RangeInclusive, SubAssign}, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Metadata trait for marking the type of an address. +pub trait AddressType: Copy + Clone + PartialOrd + PartialEq {} + +/// Zero-sized type to mark a physical address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Physical {} + +/// Zero-sized type to mark a virtual address. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum Virtual {} + +/// Generic address type. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub struct Address { + value: usize, + _address_type: PhantomData ATYPE>, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl AddressType for Physical {} +impl AddressType for Virtual {} + +impl Address { + /// Create an instance. + pub const fn new(value: usize) -> Self { + Self { + value, + _address_type: PhantomData, + } + } + + /// Align down. + pub const fn align_down(self, alignment: usize) -> Self { + let aligned = common::align_down(self.value, alignment); + + Self { + value: aligned, + _address_type: PhantomData, + } + } + + /// Converts `Address` into an usize. + pub const fn into_usize(self) -> usize { + self.value + } +} + +impl TryFrom> for Address { + type Error = mmu::TranslationError; + + fn try_from(virt: Address) -> Result { + mmu::try_virt_to_phys(virt) + } +} + +impl core::ops::Add for Address { + type Output = Self; + + fn add(self, other: usize) -> Self { + Self { + value: self.value + other, + _address_type: PhantomData, + } + } +} + +impl AddAssign for Address { + fn add_assign(&mut self, other: Self) { + *self = Self { + value: self.value + other.into_usize(), + _address_type: PhantomData, + }; + } +} + +impl core::ops::Sub for Address { + type Output = Self; + + fn sub(self, other: usize) -> Self { + Self { + value: self.value - other, + _address_type: PhantomData, + } + } +} + +impl SubAssign for Address { + fn sub_assign(&mut self, other: Self) { + *self = Self { + value: self.value - other.into_usize(), + _address_type: PhantomData, + }; + } +} + +impl fmt::Display for Address { + // Don't expect to see physical addresses greater than 40 bit. + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q3: u8 = ((self.value >> 32) & 0xff) as u8; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:02x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + +impl fmt::Display for Address { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + let q4: u16 = ((self.value >> 48) & 0xffff) as u16; + let q3: u16 = ((self.value >> 32) & 0xffff) as u16; + let q2: u16 = ((self.value >> 16) & 0xffff) as u16; + let q1: u16 = (self.value & 0xffff) as u16; + + write!(f, "0x")?; + write!(f, "{:04x}_", q4)?; + write!(f, "{:04x}_", q3)?; + write!(f, "{:04x}_", q2)?; + write!(f, "{:04x}", q1) + } +} + +/// Zero out an inclusive memory range. +/// +/// # Safety +/// +/// - `range.start` and `range.end` must be valid. +/// - `range.start` and `range.end` must be `T` aligned. +pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) +where + T: From, +{ + let mut ptr = *range.start(); + let end_inclusive = *range.end(); + + while ptr <= end_inclusive { + core::ptr::write_volatile(ptr, T::from(0)); + ptr = ptr.offset(1); + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check `zero_volatile()`. + #[kernel_test] + fn zero_volatile_works() { + let mut x: [usize; 3] = [10, 11, 12]; + let x_range = x.as_mut_ptr_range(); + let x_range_inclusive = + RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); + + unsafe { zero_volatile(x_range_inclusive) }; + + assert_eq!(x, [0, 0, 0]); + } + + /// Check `bss` section layout. + #[kernel_test] + fn bss_section_is_sane() { + use crate::bsp::memory::bss_range_inclusive; + use core::mem; + + let start = *bss_range_inclusive().start() as usize; + let end = *bss_range_inclusive().end() as usize; + + assert_eq!(start % mem::size_of::(), 0); + assert_eq!(end % mem::size_of::(), 0); + assert!(end >= start); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs new file mode 100644 index 000000000..255093193 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs @@ -0,0 +1,270 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/memory/mmu.rs"] +mod arch_mmu; + +mod mapping_record; +mod translation_table; +mod types; + +use crate::{ + bsp, + memory::{Address, Physical, Virtual}, + synchronization, warn, +}; +use core::fmt; + +pub use types::*; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// MMU enable errors variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum MMUEnableError { + AlreadyEnabled, + Other(&'static str), +} + +/// Translation error variants. +#[allow(missing_docs)] +#[derive(Debug)] +pub enum TranslationError { + MMUDisabled, + Aborted, +} + +/// Memory Management interfaces. +pub mod interface { + use super::*; + + /// MMU functions. + pub trait MMU { + /// Turns on the MMU for the first time and enables data and instruction caching. + /// + /// # Safety + /// + /// - Changes the HW's global state. + unsafe fn enable_mmu_and_caching( + &self, + phys_tables_base_addr: Address, + ) -> Result<(), MMUEnableError>; + + /// Returns true if the MMU is enabled, false otherwise. + fn is_enabled(&self) -> bool; + + /// Try to translate a virtual address to a physical address. + /// + /// Will only succeed if there exists a valid mapping for the input VA. + fn try_virt_to_phys( + &self, + virt: Address, + ) -> Result, TranslationError>; + } +} + +/// Describes the characteristics of a translation granule. +pub struct TranslationGranule; + +/// Describes properties of an address space. +pub struct AddressSpace; + +/// Intended to be implemented for [`AddressSpace`]. +pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// + /// [u64::MAX, (u64::MAX - AS_SIZE) + 1] + type TableStartFromTop; + + /// A translation table whose address range is: + /// + /// [0, AS_SIZE - 1] + type TableStartFromBottom; +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- +use interface::MMU; +use synchronization::interface::ReadWriteEx; +use translation_table::interface::TranslationTable; + +/// Map pages in the kernel's translation tables. +/// +/// No input checks done, input is passed through to the architectural implementation. +/// +/// # Safety +/// +/// - See `map_pages_at()`. +/// - Does not prevent aliasing. +unsafe fn kernel_map_pages_at_unchecked( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.map_pages_at(virt_pages, phys_pages, attr))?; + + kernel_add_mapping_record(name, virt_pages, phys_pages, attr); + + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl fmt::Display for MMUEnableError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + MMUEnableError::AlreadyEnabled => write!(f, "MMU is already enabled"), + MMUEnableError::Other(x) => write!(f, "{}", x), + } + } +} + +impl TranslationGranule { + /// The granule's size. + pub const SIZE: usize = Self::size_checked(); + + /// The granule's mask. + pub const MASK: usize = Self::SIZE - 1; + + /// The granule's shift, aka log2(size). + pub const SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(GRANULE_SIZE.is_power_of_two()); + + GRANULE_SIZE + } +} + +impl AddressSpace { + /// The address space size. + pub const SIZE: usize = Self::size_checked(); + + /// The address space shift, aka log2(size). + pub const SIZE_SHIFT: usize = Self::SIZE.trailing_zeros() as usize; + + const fn size_checked() -> usize { + assert!(AS_SIZE.is_power_of_two()); + + // Check for architectural restrictions as well. + Self::arch_address_space_size_sanity_checks(); + + AS_SIZE + } +} + +/// Add an entry to the mapping info record. +pub fn kernel_add_mapping_record( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) { + if let Err(x) = mapping_record::kernel_add(name, virt_pages, phys_pages, attr) { + warn!("{}", x); + } +} + +/// Raw mapping of virtual to physical pages in the kernel translation tables. +/// +/// Prevents mapping into the MMIO range of the tables. +/// +/// # Safety +/// +/// - See `kernel_map_pages_at_unchecked()`. +/// - Does not prevent aliasing. Currently, the callers must be trusted. +pub unsafe fn kernel_map_pages_at( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + let is_mmio = bsp::memory::mmu::kernel_translation_tables() + .read(|tables| tables.is_virt_page_slice_mmio(virt_pages)); + if is_mmio { + return Err("Attempt to manually map into MMIO region"); + } + + kernel_map_pages_at_unchecked(name, virt_pages, phys_pages, attr)?; + + Ok(()) +} + +/// MMIO remapping in the kernel translation tables. +/// +/// Typically used by device drivers. +/// +/// # Safety +/// +/// - Same as `kernel_map_pages_at_unchecked()`, minus the aliasing part. +pub unsafe fn kernel_map_mmio( + name: &'static str, + mmio_descriptor: &MMIODescriptor, +) -> Result, &'static str> { + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let offset_into_start_page = + mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + + // Check if an identical page slice has been mapped for another driver. If so, reuse it. + let virt_addr = if let Some(addr) = + mapping_record::kernel_find_and_insert_mmio_duplicate(mmio_descriptor, name) + { + addr + // Otherwise, allocate a new virtual page slice and map it. + } else { + let virt_pages: PageSliceDescriptor = + bsp::memory::mmu::kernel_translation_tables() + .write(|tables| tables.next_mmio_virt_page_slice(phys_pages.num_pages()))?; + + kernel_map_pages_at_unchecked( + name, + &virt_pages, + &phys_pages, + &AttributeFields { + mem_attributes: MemAttributes::Device, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + }, + )?; + + virt_pages.start_addr() + }; + + Ok(virt_addr + offset_into_start_page) +} + +/// Try to translate a virtual address to a physical address. +/// +/// Will only succeed if there exists a valid mapping for the input VA. +pub fn try_virt_to_phys(virt: Address) -> Result, TranslationError> { + arch_mmu::mmu().try_virt_to_phys(virt) +} + +/// Enable the MMU and data + instruction caching. +/// +/// # Safety +/// +/// - Crucial function during kernel init. Changes the the complete memory view of the processor. +#[inline(always)] +pub unsafe fn enable_mmu_and_caching( + phys_tables_base_addr: Address, +) -> Result<(), MMUEnableError> { + arch_mmu::mmu().enable_mmu_and_caching(phys_tables_base_addr) +} + +/// Human-readable print of all recorded kernel mappings. +pub fn kernel_print_mappings() { + mapping_record::kernel_print() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs new file mode 100644 index 000000000..eab62fb39 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! A record of mapped pages. + +use super::{ + AccessPermissions, Address, AttributeFields, MMIODescriptor, MemAttributes, + PageSliceDescriptor, Physical, Virtual, +}; +use crate::{info, synchronization, synchronization::InitStateLock, warn}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Type describing a virtual memory mapping. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +struct MappingRecordEntry { + pub users: [Option<&'static str>; 5], + pub phys_pages: PageSliceDescriptor, + pub virt_start_addr: Address, + pub attribute_fields: AttributeFields, +} + +struct MappingRecord { + inner: [Option; 12], +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static KERNEL_MAPPING_RECORD: InitStateLock = + InitStateLock::new(MappingRecord::new()); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl MappingRecordEntry { + pub fn new( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Self { + Self { + users: [Some(name), None, None, None, None], + phys_pages: *phys_pages, + virt_start_addr: virt_pages.start_addr(), + attribute_fields: *attr, + } + } + + fn find_next_free_user(&mut self) -> Result<&mut Option<&'static str>, &'static str> { + if let Some(x) = self.users.iter_mut().find(|x| x.is_none()) { + return Ok(x); + }; + + Err("Storage for user info exhausted") + } + + pub fn add_user(&mut self, user: &'static str) -> Result<(), &'static str> { + let x = self.find_next_free_user()?; + *x = Some(user); + Ok(()) + } +} + +impl MappingRecord { + pub const fn new() -> Self { + Self { inner: [None; 12] } + } + + fn find_next_free(&mut self) -> Result<&mut Option, &'static str> { + if let Some(x) = self.inner.iter_mut().find(|x| x.is_none()) { + return Ok(x); + } + + Err("Storage for mapping info exhausted") + } + + fn find_duplicate( + &mut self, + phys_pages: &PageSliceDescriptor, + ) -> Option<&mut MappingRecordEntry> { + self.inner + .iter_mut() + .filter(|x| x.is_some()) + .map(|x| x.as_mut().unwrap()) + .filter(|x| x.attribute_fields.mem_attributes == MemAttributes::Device) + .find(|x| x.phys_pages == *phys_pages) + } + + pub fn add( + &mut self, + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { + let x = self.find_next_free()?; + + *x = Some(MappingRecordEntry::new(name, virt_pages, phys_pages, attr)); + Ok(()) + } + + pub fn print(&self) { + const KIB_RSHIFT: u32 = 10; // log2(1024). + const MIB_RSHIFT: u32 = 20; // log2(1024 * 1024). + + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + info!( + " {:^44} {:^30} {:^7} {:^9} {:^35}", + "Virtual", "Physical", "Size", "Attr", "Entity" + ); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + + for i in self + .inner + .iter() + .filter(|x| x.is_some()) + .map(|x| x.unwrap()) + { + let virt_start = i.virt_start_addr; + let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; + let phys_start = i.phys_pages.start_addr(); + let phys_end_inclusive = i.phys_pages.end_addr_inclusive(); + let size = i.phys_pages.size(); + + let (size, unit) = if (size >> MIB_RSHIFT) > 0 { + (size >> MIB_RSHIFT, "MiB") + } else if (size >> KIB_RSHIFT) > 0 { + (size >> KIB_RSHIFT, "KiB") + } else { + (size, "Byte") + }; + + let attr = match i.attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => "C", + MemAttributes::Device => "Dev", + }; + + let acc_p = match i.attribute_fields.acc_perms { + AccessPermissions::ReadOnly => "RO", + AccessPermissions::ReadWrite => "RW", + }; + + let xn = if i.attribute_fields.execute_never { + "XN" + } else { + "X" + }; + + info!( + " {}..{} --> {}..{} | \ + {: >3} {} | {: <3} {} {: <2} | {}", + virt_start, + virt_end_inclusive, + phys_start, + phys_end_inclusive, + size, + unit, + attr, + acc_p, + xn, + i.users[0].unwrap() + ); + + for k in i.users[1..].iter() { + if let Some(additional_user) = *k { + info!( + " | {}", + additional_user + ); + } + } + } + + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use synchronization::interface::ReadWriteEx; + +/// Add an entry to the mapping info record. +pub fn kernel_add( + name: &'static str, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, +) -> Result<(), &'static str> { + KERNEL_MAPPING_RECORD.write(|mr| mr.add(name, virt_pages, phys_pages, attr)) +} + +pub fn kernel_find_and_insert_mmio_duplicate( + mmio_descriptor: &MMIODescriptor, + new_user: &'static str, +) -> Option> { + let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + + KERNEL_MAPPING_RECORD.write(|mr| { + let dup = mr.find_duplicate(&phys_pages)?; + + if let Err(x) = dup.add_user(new_user) { + warn!("{}", x); + } + + Some(dup.virt_start_addr) + }) +} + +/// Human-readable print of all recorded kernel mappings. +pub fn kernel_print() { + KERNEL_MAPPING_RECORD.read(|mr| mr.print()); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/translation_table.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/translation_table.rs new file mode 100644 index 000000000..bf9bed8cb --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/translation_table.rs @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter + +//! Translation table. + +#[cfg(target_arch = "aarch64")] +#[path = "../../_arch/aarch64/memory/mmu/translation_table.rs"] +mod arch_translation_table; + +use crate::memory::{ + mmu::{AttributeFields, PageSliceDescriptor}, + Physical, Virtual, +}; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +#[cfg(target_arch = "aarch64")] +pub use arch_translation_table::FixedSizeTranslationTable; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Translation table interfaces. +pub mod interface { + use super::*; + + /// Translation table operations. + pub trait TranslationTable { + /// Anything that needs to run before any of the other provided functions can be used. + /// + /// # Safety + /// + /// - Implementor must ensure that this function can run only once or is harmless if invoked + /// multiple times. + fn init(&mut self) -> Result<(), &'static str>; + + /// Map the given virtual pages to the given physical pages. + /// + /// # Safety + /// + /// - Using wrong attributes can cause multiple issues of different nature in the system. + /// - It is not required that the architectural implementation prevents aliasing. That is, + /// mapping to the same physical memory using multiple virtual addresses, which would + /// break Rust's ownership assumptions. This should be protected against in the kernel's + /// generic MMU code. + unsafe fn map_pages_at( + &mut self, + virt_pages: &PageSliceDescriptor, + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str>; + + /// Obtain a free virtual page slice in the MMIO region. + /// + /// The "MMIO region" is a distinct region of the implementor's choice, which allows + /// differentiating MMIO addresses from others. This can speed up debugging efforts. + /// Ideally, those MMIO addresses are also standing out visually so that a human eye can + /// identify them. For example, by allocating them from near the end of the virtual address + /// space. + fn next_mmio_virt_page_slice( + &mut self, + num_pages: usize, + ) -> Result, &'static str>; + + /// Check if a virtual page splice is in the "MMIO region". + fn is_virt_page_slice_mmio(&self, virt_pages: &PageSliceDescriptor) -> bool; + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use crate::{bsp, memory::Address}; + use arch_translation_table::MinSizeTranslationTable; + use interface::TranslationTable; + use test_macros::kernel_test; + + /// Sanity checks for the TranslationTable implementation. + #[kernel_test] + fn translationtable_implementation_sanity() { + // This will occupy a lot of space on the stack. + let mut tables = MinSizeTranslationTable::new_for_runtime(); + + assert!(tables.init().is_ok()); + + let x = tables.next_mmio_virt_page_slice(0); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(1_0000_0000); + assert!(x.is_err()); + + let x = tables.next_mmio_virt_page_slice(2).unwrap(); + assert_eq!(x.size(), bsp::memory::mmu::KernelGranule::SIZE * 2); + + assert_eq!(tables.is_virt_page_slice_mmio(&x), true); + + assert_eq!( + tables.is_virt_page_slice_mmio(&PageSliceDescriptor::from_addr(Address::new(0), 1)), + false + ); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/types.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/types.rs new file mode 100644 index 000000000..dc05fd70c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/types.rs @@ -0,0 +1,217 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Memory Management Unit types. + +use crate::{ + bsp, common, + memory::{Address, AddressType, Physical, Virtual}, +}; +use core::{ + convert::{From, TryFrom}, + marker::PhantomData, +}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Generic page type. +#[repr(C)] +pub struct Page { + inner: [u8; bsp::memory::mmu::KernelGranule::SIZE], + _address_type: PhantomData, +} + +/// Type describing a slice of pages. +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub struct PageSliceDescriptor { + start: Address, + num_pages: usize, +} + +/// Architecture agnostic memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone, PartialOrd, PartialEq)] +pub enum MemAttributes { + CacheableDRAM, + Device, +} + +/// Architecture agnostic access permissions. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub enum AccessPermissions { + ReadOnly, + ReadWrite, +} + +/// Collection of memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub struct AttributeFields { + pub mem_attributes: MemAttributes, + pub acc_perms: AccessPermissions, + pub execute_never: bool, +} + +/// An MMIO descriptor for use in device drivers. +#[derive(Copy, Clone)] +pub struct MMIODescriptor { + start_addr: Address, + size: usize, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +//------------------------------------------------------------------------------ +// Page +//------------------------------------------------------------------------------ + +impl Page { + /// Get a pointer to the instance. + pub const fn as_ptr(&self) -> *const Page { + self as *const _ + } +} + +//------------------------------------------------------------------------------ +// PageSliceDescriptor +//------------------------------------------------------------------------------ + +impl PageSliceDescriptor { + /// Create an instance. + pub const fn from_addr(start: Address, num_pages: usize) -> Self { + assert!(common::is_aligned( + start.into_usize(), + bsp::memory::mmu::KernelGranule::SIZE + )); + assert!(num_pages > 0); + + Self { start, num_pages } + } + + /// Return a pointer to the first page of the described slice. + const fn first_page_ptr(&self) -> *const Page { + self.start.into_usize() as *const _ + } + + /// Return the number of Pages the slice describes. + pub const fn num_pages(&self) -> usize { + self.num_pages + } + + /// Return the memory size this descriptor spans. + pub const fn size(&self) -> usize { + self.num_pages * bsp::memory::mmu::KernelGranule::SIZE + } + + /// Return the start address. + pub const fn start_addr(&self) -> Address { + self.start + } + + /// Return the exclusive end address. + pub fn end_addr(&self) -> Address { + self.start + self.size() + } + + /// Return the inclusive end address. + pub fn end_addr_inclusive(&self) -> Address { + self.start + (self.size() - 1) + } + + /// Check if an address is contained within this descriptor. + pub fn contains(&self, addr: Address) -> bool { + (addr >= self.start_addr()) && (addr <= self.end_addr_inclusive()) + } + + /// Return a non-mutable slice of Pages. + /// + /// # Safety + /// + /// - Same as applies for `core::slice::from_raw_parts`. + pub unsafe fn as_slice(&self) -> &[Page] { + core::slice::from_raw_parts(self.first_page_ptr(), self.num_pages) + } +} + +impl TryFrom> for PageSliceDescriptor { + type Error = super::TranslationError; + + fn try_from(desc: PageSliceDescriptor) -> Result { + let phys_start = super::try_virt_to_phys(desc.start)?; + + Ok(Self { + start: phys_start, + num_pages: desc.num_pages, + }) + } +} + +impl From for PageSliceDescriptor { + fn from(desc: MMIODescriptor) -> Self { + let start_page_addr = desc + .start_addr + .align_down(bsp::memory::mmu::KernelGranule::SIZE); + + let len = ((desc.end_addr_inclusive().into_usize() - start_page_addr.into_usize()) + >> bsp::memory::mmu::KernelGranule::SHIFT) + + 1; + + Self { + start: start_page_addr, + num_pages: len, + } + } +} + +//------------------------------------------------------------------------------ +// MMIODescriptor +//------------------------------------------------------------------------------ + +impl MMIODescriptor { + /// Create an instance. + pub const fn new(start_addr: Address, size: usize) -> Self { + assert!(size > 0); + + Self { start_addr, size } + } + + /// Return the start address. + pub const fn start_addr(&self) -> Address { + self.start_addr + } + + /// Return the inclusive end address. + pub fn end_addr_inclusive(&self) -> Address { + self.start_addr + (self.size - 1) + } + + /// Return the size. + pub const fn size(&self) -> usize { + self.size + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// Check if the size of `struct Page` is as expected. + #[kernel_test] + fn size_of_page_equals_granule_size() { + assert_eq!( + core::mem::size_of::>(), + bsp::memory::mmu::KernelGranule::SIZE + ); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs b/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs new file mode 100644 index 000000000..e3a9ed8ab --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! A panic handler that infinitely waits. + +use crate::{bsp, cpu, exception}; +use core::{fmt, panic::PanicInfo}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +fn _panic_print(args: fmt::Arguments) { + use fmt::Write; + + unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; +} + +/// The point of exit for `libkernel`. +/// +/// It is linked weakly, so that the integration tests can overload its standard behavior. +#[linkage = "weak"] +#[no_mangle] +fn _panic_exit() -> ! { + #[cfg(not(test_build))] + { + cpu::wait_forever() + } + + #[cfg(test_build)] + { + cpu::qemu_exit_failure() + } +} + +/// Prints with a newline - only use from the panic handler. +/// +/// Carbon copy from +#[macro_export] +macro_rules! panic_println { + ($($arg:tt)*) => ({ + _panic_print(format_args_nl!($($arg)*)); + }) +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + unsafe { exception::asynchronous::local_irq_mask() }; + + if let Some(args) = info.message() { + panic_println!("\nKernel panic: {}", args); + } else { + panic_println!("\nKernel panic!"); + } + + _panic_exit() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/print.rs b/16_virtual_mem_part4_higher_half_kernel/src/print.rs new file mode 100644 index 000000000..89fa66943 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/print.rs @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Printing. + +use crate::{bsp, console}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +#[doc(hidden)] +pub fn _print(args: fmt::Arguments) { + use console::interface::Write; + + bsp::console::console().write_fmt(args).unwrap(); +} + +/// Prints without a newline. +/// +/// Carbon copy from +#[macro_export] +macro_rules! print { + ($($arg:tt)*) => ($crate::print::_print(format_args!($($arg)*))); +} + +/// Prints with a newline. +/// +/// Carbon copy from +#[macro_export] +macro_rules! println { + () => ($crate::print!("\n")); + ($($arg:tt)*) => ({ + $crate::print::_print(format_args_nl!($($arg)*)); + }) +} + +/// Prints an info, with a newline. +#[macro_export] +macro_rules! info { + ($string:expr) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[ {:>3}.{:03}{:03}] ", $string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000 + )); + }); + ($format_string:expr, $($arg:tt)*) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[ {:>3}.{:03}{:03}] ", $format_string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000, + $($arg)* + )); + }) +} + +/// Prints a warning, with a newline. +#[macro_export] +macro_rules! warn { + ($string:expr) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[W {:>3}.{:03}{:03}] ", $string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000 + )); + }); + ($format_string:expr, $($arg:tt)*) => ({ + #[allow(unused_imports)] + use crate::time::interface::TimeManager; + + let timestamp = $crate::time::time_manager().uptime(); + let timestamp_subsec_us = timestamp.subsec_micros(); + + $crate::print::_print(format_args_nl!( + concat!("[W {:>3}.{:03}{:03}] ", $format_string), + timestamp.as_secs(), + timestamp_subsec_us / 1_000, + timestamp_subsec_us % 1_000, + $($arg)* + )); + }) +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs b/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs new file mode 100644 index 000000000..2c0a62e67 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter + +//! Rust runtime initialization code. + +use crate::{bsp, memory}; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +/// Zero out the .bss section. +/// +/// # Safety +/// +/// - Must only be called pre `kernel_init()`. +#[inline(always)] +unsafe fn zero_bss() { + memory::zero_volatile(bsp::memory::bss_range_inclusive()); +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel +/// init code. +/// +/// # Safety +/// +/// - Only a single core must be active and running this function. +#[no_mangle] +pub unsafe fn runtime_init() -> ! { + extern "Rust" { + fn kernel_init() -> !; + } + + zero_bss(); + kernel_init() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/state.rs b/16_virtual_mem_part4_higher_half_kernel/src/state.rs new file mode 100644 index 000000000..c94d04c84 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/state.rs @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! State information about the kernel itself. + +use core::sync::atomic::{AtomicU8, Ordering}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +/// Different stages in the kernel execution. +#[derive(Copy, Clone, Eq, PartialEq)] +enum State { + /// The kernel starts booting in this state. + Init, + + /// The kernel transitions to this state when jumping to `kernel_main()` (at the end of + /// `kernel_init()`, after all init calls are done). + SingleCoreMain, + + /// The kernel transitions to this state when it boots the secondary cores, aka switches + /// exectution mode to symmetric multiprocessing (SMP). + MultiCoreMain, +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Maintains the kernel state and state transitions. +pub struct StateManager(AtomicU8); + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static STATE_MANAGER: StateManager = StateManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the global StateManager. +pub fn state_manager() -> &'static StateManager { + &STATE_MANAGER +} + +impl StateManager { + const INIT: u8 = 0; + const SINGLE_CORE_MAIN: u8 = 1; + const MULTI_CORE_MAIN: u8 = 2; + + /// Create a new instance. + pub const fn new() -> Self { + Self(AtomicU8::new(Self::INIT)) + } + + /// Return the current state. + fn state(&self) -> State { + let state = self.0.load(Ordering::Acquire); + + match state { + Self::INIT => State::Init, + Self::SINGLE_CORE_MAIN => State::SingleCoreMain, + Self::MULTI_CORE_MAIN => State::MultiCoreMain, + _ => panic!("Invalid KERNEL_STATE"), + } + } + + /// Return if the kernel is init state. + pub fn is_init(&self) -> bool { + self.state() == State::Init + } + + /// Transition from Init to SingleCoreMain. + pub fn transition_to_single_core_main(&self) { + if self + .0 + .compare_exchange( + Self::INIT, + Self::SINGLE_CORE_MAIN, + Ordering::Acquire, + Ordering::Relaxed, + ) + .is_err() + { + panic!("transition_to_single_core_main() called while state != Init"); + } + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/synchronization.rs b/16_virtual_mem_part4_higher_half_kernel/src/synchronization.rs new file mode 100644 index 000000000..94582732c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/synchronization.rs @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Synchronization primitives. +//! +//! # Resources +//! +//! - +//! - +//! - + +use core::cell::UnsafeCell; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Synchronization interfaces. +pub mod interface { + + /// Any object implementing this trait guarantees exclusive access to the data wrapped within + /// the Mutex for the duration of the provided closure. + pub trait Mutex { + /// The type of the data that is wrapped by this mutex. + type Data; + + /// Locks the mutex and grants the closure temporary mutable access to the wrapped data. + fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; + } + + /// A reader-writer exclusion type. + /// + /// The implementing object allows either a number of readers or at most one writer at any point + /// in time. + pub trait ReadWriteEx { + /// The type of encapsulated data. + type Data; + + /// Grants temporary mutable access to the encapsulated data. + fn write(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R; + + /// Grants temporary immutable access to the encapsulated data. + fn read(&self, f: impl FnOnce(&Self::Data) -> R) -> R; + } +} + +/// A pseudo-lock for teaching purposes. +/// +/// In contrast to a real Mutex implementation, does not protect against concurrent access from +/// other cores to the contained data. This part is preserved for later lessons. +/// +/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is +/// executing on a single core. +pub struct IRQSafeNullLock +where + T: ?Sized, +{ + data: UnsafeCell, +} + +/// A pseudo-lock that is RW during the single-core kernel init phase and RO afterwards. +/// +/// Intended to encapsulate data that is populated during kernel init when no concurrency exists. +pub struct InitStateLock +where + T: ?Sized, +{ + data: UnsafeCell, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +unsafe impl Send for IRQSafeNullLock where T: ?Sized + Send {} +unsafe impl Sync for IRQSafeNullLock where T: ?Sized + Send {} + +impl IRQSafeNullLock { + /// Create an instance. + pub const fn new(data: T) -> Self { + Self { + data: UnsafeCell::new(data), + } + } +} + +unsafe impl Send for InitStateLock where T: ?Sized + Send {} +unsafe impl Sync for InitStateLock where T: ?Sized + Send {} + +impl InitStateLock { + /// Create an instance. + pub const fn new(data: T) -> Self { + Self { + data: UnsafeCell::new(data), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use crate::{exception, state}; + +impl interface::Mutex for IRQSafeNullLock { + type Data = T; + + fn lock(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { + // In a real lock, there would be code encapsulating this line that ensures that this + // mutable reference will ever only be given out once at a time. + let data = unsafe { &mut *self.data.get() }; + + // Execute the closure while IRQs are masked. + exception::asynchronous::exec_with_irq_masked(|| f(data)) + } +} + +impl interface::ReadWriteEx for InitStateLock { + type Data = T; + + fn write(&self, f: impl FnOnce(&mut Self::Data) -> R) -> R { + assert!( + state::state_manager().is_init(), + "InitStateLock::write called after kernel init phase" + ); + assert!( + !exception::asynchronous::is_local_irq_masked(), + "InitStateLock::write called with IRQs unmasked" + ); + + let data = unsafe { &mut *self.data.get() }; + + f(data) + } + + fn read(&self, f: impl FnOnce(&Self::Data) -> R) -> R { + let data = unsafe { &*self.data.get() }; + + f(data) + } +} + +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use test_macros::kernel_test; + + /// InitStateLock must be transparent. + #[kernel_test] + fn init_state_lock_is_transparent() { + use core::mem::size_of; + + assert_eq!(size_of::>(), size_of::()); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/time.rs b/16_virtual_mem_part4_higher_half_kernel/src/time.rs new file mode 100644 index 000000000..953b6f0df --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/src/time.rs @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! Timer primitives. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/time.rs"] +mod arch_time; + +//-------------------------------------------------------------------------------------------------- +// Architectural Public Reexports +//-------------------------------------------------------------------------------------------------- +pub use arch_time::time_manager; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Timekeeping interfaces. +pub mod interface { + use core::time::Duration; + + /// Time management functions. + pub trait TimeManager { + /// The timer's resolution. + fn resolution(&self) -> Duration; + + /// The uptime since power-on of the device. + /// + /// This includes time consumed by firmware and bootloaders. + fn uptime(&self) -> Duration; + + /// Spin for a given duration. + fn spin_for(&self, duration: Duration); + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/test-macros/Cargo.toml b/16_virtual_mem_part4_higher_half_kernel/test-macros/Cargo.toml new file mode 100644 index 000000000..a570f72b4 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/test-macros/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "test-macros" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" + +[lib] +proc-macro = true + +[dependencies] +proc-macro2 = "1.x" +quote = "1.x" +syn = { version = "1.x", features = ["full"] } +test-types = { path = "../test-types" } diff --git a/16_virtual_mem_part4_higher_half_kernel/test-macros/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/test-macros/src/lib.rs new file mode 100644 index 000000000..36c95e8a6 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/test-macros/src/lib.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +use proc_macro::TokenStream; +use proc_macro2::Span; +use quote::quote; +use syn::{parse_macro_input, Ident, ItemFn}; + +#[proc_macro_attribute] +pub fn kernel_test(_attr: TokenStream, input: TokenStream) -> TokenStream { + let f = parse_macro_input!(input as ItemFn); + + let test_name = &format!("{}", f.sig.ident.to_string()); + let test_ident = Ident::new( + &format!("{}_TEST_CONTAINER", f.sig.ident.to_string().to_uppercase()), + Span::call_site(), + ); + let test_code_block = f.block; + + quote!( + #[test_case] + const #test_ident: test_types::UnitTest = test_types::UnitTest { + name: #test_name, + test_func: || #test_code_block, + }; + ) + .into() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/test-types/Cargo.toml b/16_virtual_mem_part4_higher_half_kernel/test-types/Cargo.toml new file mode 100644 index 000000000..a0be2c57c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/test-types/Cargo.toml @@ -0,0 +1,5 @@ +[package] +name = "test-types" +version = "0.1.0" +authors = ["Andre Richter "] +edition = "2018" diff --git a/16_virtual_mem_part4_higher_half_kernel/test-types/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/test-types/src/lib.rs new file mode 100644 index 000000000..fe7d918f5 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/test-types/src/lib.rs @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Types for the `custom_test_frameworks` implementation. + +#![no_std] + +/// Unit test container. +pub struct UnitTest { + /// Name of the test. + pub name: &'static str, + + /// Function pointer to the test. + pub test_func: fn(), +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb new file mode 100644 index 000000000..dfd6b16ea --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb @@ -0,0 +1,50 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'expect' + +TIMEOUT_SECS = 3 + +# Verify sending and receiving works as expected. +class TxRxHandshake + def name + 'Transmit and Receive handshake' + end + + def run(qemu_out, qemu_in) + qemu_in.write_nonblock('ABC') + raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + end +end + +# Check for correct TX statistics implementation. Depends on test 1 being run first. +class TxStatistics + def name + 'Transmit statistics' + end + + def run(qemu_out, _qemu_in) + raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + end +end + +# Check for correct RX statistics implementation. Depends on test 1 being run first. +class RxStatistics + def name + 'Receive statistics' + end + + def run(qemu_out, _qemu_in) + raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + end +end + +##-------------------------------------------------------------------------------------------------- +## Test registration +##-------------------------------------------------------------------------------------------------- +def subtest_collection + [TxRxHandshake.new, TxStatistics.new, RxStatistics.new] +end diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs new file mode 100644 index 000000000..ad7fd2bf5 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Console sanity tests - RX, TX and statistics. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +use libkernel::{bsp, console, cpu, exception, print}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + use bsp::console::console; + use console::interface::*; + + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + // Handshake + assert_eq!(console().read_char(), 'A'); + assert_eq!(console().read_char(), 'B'); + assert_eq!(console().read_char(), 'C'); + print!("OK1234"); + + // 6 + print!("{}", console().chars_written()); + + // 3 + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. + cpu::wait_forever() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/01_timer_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/tests/01_timer_sanity.rs new file mode 100644 index 000000000..f39d83845 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/01_timer_sanity.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Timer sanity tests. + +#![feature(custom_test_frameworks)] +#![no_main] +#![no_std] +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + +use core::time::Duration; +use libkernel::{bsp, cpu, exception, time, time::interface::TimeManager}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. + + test_main(); + + cpu::qemu_exit_success() +} + +/// Simple check that the timer is running. +#[kernel_test] +fn timer_is_counting() { + assert!(time::time_manager().uptime().as_nanos() > 0) +} + +/// Timer resolution must be sufficient. +#[kernel_test] +fn timer_resolution_is_sufficient() { + assert!(time::time_manager().resolution().as_nanos() < 100) +} + +/// Sanity check spin_for() implementation. +#[kernel_test] +fn spin_accuracy_check_1_second() { + let t1 = time::time_manager().uptime(); + time::time_manager().spin_for(Duration::from_secs(1)); + let t2 = time::time_manager().uptime(); + + assert_eq!((t2 - t1).as_secs(), 1) +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs b/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs new file mode 100644 index 000000000..30d420a75 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +//! Page faults must result in synchronous exceptions. + +#![feature(format_args_nl)] +#![no_main] +#![no_std] + +/// Overwrites libkernel's `panic_wait::_panic_exit()` so that it returns a "success" code. +/// +/// In this test, teaching the panic is a success, because it is called from the synchronous +/// exception handler, which is what this test wants to achieve. +/// +/// It also means that this integration test can not use any other code that calls panic!() directly +/// or indirectly. +mod panic_exit_success; + +use libkernel::{bsp, cpu, exception, println}; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + + println!("Testing synchronous exception handling by causing a page fault"); + println!("-------------------------------------------------------------------\n"); + + println!("Writing to bottom of address space to address 1 GiB..."); + let big_addr: u64 = 1 * 1024 * 1024 * 1024; + core::ptr::read_volatile(big_addr as *mut u64); + + // If execution reaches here, the memory access above did not cause a page fault exception. + cpu::qemu_exit_failure() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/03_exception_irq_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/tests/03_exception_irq_sanity.rs new file mode 100644 index 000000000..ac25d01a8 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/03_exception_irq_sanity.rs @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + +//! IRQ handling sanity tests. + +#![feature(custom_test_frameworks)] +#![no_main] +#![no_std] +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + +use libkernel::{bsp, cpu, exception}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { + bsp::console::qemu_bring_up_console(); + + exception::handling_init(); + exception::asynchronous::local_irq_unmask(); + + test_main(); + + cpu::qemu_exit_success() +} + +/// Check that IRQ masking works. +#[kernel_test] +fn local_irq_mask_works() { + // Precondition: IRQs are unmasked. + assert!(exception::asynchronous::is_local_irq_masked()); + + unsafe { exception::asynchronous::local_irq_mask() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + // Restore earlier state. + unsafe { exception::asynchronous::local_irq_unmask() }; +} + +/// Check that IRQ unmasking works. +#[kernel_test] +fn local_irq_unmask_works() { + // Precondition: IRQs are masked. + unsafe { exception::asynchronous::local_irq_mask() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + unsafe { exception::asynchronous::local_irq_unmask() }; + assert!(exception::asynchronous::is_local_irq_masked()); +} + +/// Check that IRQ mask save is saving "something". +#[kernel_test] +fn local_irq_mask_save_works() { + // Precondition: IRQs are unmasked. + assert!(exception::asynchronous::is_local_irq_masked()); + + let first = unsafe { exception::asynchronous::local_irq_mask_save() }; + assert!(!exception::asynchronous::is_local_irq_masked()); + + let second = unsafe { exception::asynchronous::local_irq_mask_save() }; + assert_ne!(first, second); + + unsafe { exception::asynchronous::local_irq_restore(first) }; + assert!(exception::asynchronous::is_local_irq_masked()); +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/panic_exit_success/mod.rs b/16_virtual_mem_part4_higher_half_kernel/tests/panic_exit_success/mod.rs new file mode 100644 index 000000000..29d1f9750 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/panic_exit_success/mod.rs @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter + +/// Overwrites libkernel's `panic_wait::_panic_exit()` with the QEMU-exit version. +#[no_mangle] +fn _panic_exit() -> ! { + libkernel::cpu::qemu_exit_success() +} diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb b/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb new file mode 100755 index 000000000..53116e088 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb @@ -0,0 +1,143 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'English' +require 'pty' + +# Test base class. +class Test + INDENT = ' ' + + def print_border(status) + puts + puts "#{INDENT}-------------------------------------------------------------------" + puts status + puts "#{INDENT}-------------------------------------------------------------------\n\n\n" + end + + def print_error(error) + puts + print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") + end + + def print_success + print_border("#{INDENT}✅ Success: #{@test_name}") + end + + def print_output + puts "#{INDENT}-------------------------------------------------------------------" + print INDENT + print '🦀 ' + print @output.join.gsub("\n", "\n#{INDENT}") + end + + def finish(error) + print_output + + exit_code = if error + print_error(error) + false + else + print_success + true + end + + exit(exit_code) + end +end + +# Executes tests with console I/O. +class ConsoleTest < Test + def initialize(binary, qemu_cmd, test_name, console_subtests) + super() + + @binary = binary + @qemu_cmd = qemu_cmd + @test_name = test_name + @console_subtests = console_subtests + @cur_subtest = 1 + @output = ["Running #{@console_subtests.length} console-based tests\n", + "-------------------------------------------------------------------\n\n"] + end + + def format_test_name(number, name) + formatted_name = "#{number.to_s.rjust(3)}. #{name}" + formatted_name.ljust(63, '.') + end + + def run_subtest(subtest, qemu_out, qemu_in) + @output << format_test_name(@cur_subtest, subtest.name) + + subtest.run(qemu_out, qemu_in) + + @output << "[ok]\n" + @cur_subtest += 1 + end + + def exec + error = false + + PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| + begin + @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } + rescue StandardError => e + error = e.message + end + + finish(error) + end + end +end + +# A wrapper around the bare QEMU invocation. +class RawTest < Test + MAX_WAIT_SECS = 5 + + def initialize(binary, qemu_cmd, test_name) + super() + + @binary = binary + @qemu_cmd = qemu_cmd + @test_name = test_name + @output = [] + end + + def exec + error = 'Timed out waiting for test' + io = IO.popen(@qemu_cmd) + + while IO.select([io], nil, nil, MAX_WAIT_SECS) + begin + @output << io.read_nonblock(1024) + rescue EOFError + io.close + error = $CHILD_STATUS.to_i != 0 + break + end + end + + finish(error) + end +end + +##-------------------------------------------------------------------------------------------------- +## Script entry point +##-------------------------------------------------------------------------------------------------- +binary = ARGV.last +test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] +console_test_file = "tests/#{test_name}.rb" +qemu_cmd = ARGV.join(' ') + +test_runner = if File.exist?(console_test_file) + load console_test_file + # subtest_collection is provided by console_test_file + ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) + else + RawTest.new(binary, qemu_cmd, test_name) + end + +test_runner.exec diff --git a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb new file mode 100644 index 000000000..72471b113 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb @@ -0,0 +1,323 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +# Bitfield manipulation. +class BitField + def initialize + @value = 0 + end + + def self.attr_bitfield(name, offset, num_bits) + define_method("#{name}=") do |bits| + mask = 2**num_bits - 1 + + raise "Input out of range: #{name} = 0x#{bits.to_s(16)}" if (bits & ~mask).positive? + + # Clear bitfield + @value &= ~(mask << offset) + + # Set it + @value |= (bits << offset) + end + end + + def to_i + @value + end + + def size_in_byte + 8 + end +end + +# An array class that knows its memory location. +class CArray < Array + attr_reader :phys_start_addr + + def initialize(phys_start_addr, size, &block) + @phys_start_addr = phys_start_addr + + super(size, &block) + end + + def size_in_byte + inject(0) { |sum, n| sum + n.size_in_byte } + end +end + +#--------------------------------------------------------------------------------------------------- +# Arch:: +#--------------------------------------------------------------------------------------------------- +module Arch +FALSE = 0b0 +TRUE = 0b1 + +#--------------------------------------------------------------------------------------------------- +# Arch::ARMv8 +#--------------------------------------------------------------------------------------------------- +module ARMv8 +# ARMv8 Table Descriptor. +class Stage1TableDescriptor < BitField + module NextLevelTableAddr + OFFSET = 16 + NUMBITS = 32 + end + + module Type + OFFSET = 1 + NUMBITS = 1 + + BLOCK = 0 + TABLE = 1 + end + + module Valid + OFFSET = 0 + NUMBITS = 1 + end + + attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) + attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) + attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) + + def next_level_table_addr=(addr) + addr = addr >> Granule64KiB::SHIFT + + self.__next_level_table_addr = addr + end + + private :__next_level_table_addr= +end + +# ARMv8 level 3 page descriptor. +class Stage1PageDescriptor < BitField + module UXN + OFFSET = 54 + NUMBITS = 1 + end + + module PXN + OFFSET = 53 + NUMBITS = 1 + end + + module OutputAddr + OFFSET = 16 + NUMBITS = 32 + end + + module AF + OFFSET = 10 + NUMBITS = 1 + end + + module SH + OFFSET = 8 + NUMBITS = 2 + + INNER_SHAREABLE = 0b11 + end + + module AP + OFFSET = 6 + NUMBITS = 2 + + RW_EL1 = 0b00 + RO_EL1 = 0b10 + end + + module AttrIndx + OFFSET = 2 + NUMBITS = 3 + end + + module Type + OFFSET = 1 + NUMBITS = 1 + + RESERVED_INVALID = 0 + PAGE = 1 + end + + module Valid + OFFSET = 0 + NUMBITS = 1 + end + + attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) + attr_bitfield(:pxn, PXN::OFFSET, PXN::NUMBITS) + attr_bitfield(:__output_addr, OutputAddr::OFFSET, OutputAddr::NUMBITS) + attr_bitfield(:af, AF::OFFSET, AF::NUMBITS) + attr_bitfield(:sh, SH::OFFSET, SH::NUMBITS) + attr_bitfield(:ap, AP::OFFSET, AP::NUMBITS) + attr_bitfield(:attr_indx, AttrIndx::OFFSET, AttrIndx::NUMBITS) + attr_bitfield(:type, Type::OFFSET, Type::NUMBITS) + attr_bitfield(:valid, Valid::OFFSET, Valid::NUMBITS) + + def output_addr=(addr) + addr = addr >> Granule64KiB::SHIFT + + self.__output_addr = addr + end + + private :__output_addr= +end + +# Translation table representing the structure defined in translation_table.rs. +class TranslationTable + MMIO_APERTURE_MiB = 256 * 1024 * 1024 + + module MAIR + NORMAL = 1 + end + + def initialize + @virt_mmio_start_addr = (BSP.kernel_virt_addr_space_size - MMIO_APERTURE_MiB) + + BSP.kernel_virt_start_addr + + do_sanity_checks + + num_lvl2_tables = BSP.kernel_virt_addr_space_size >> Granule512MiB::SHIFT + + @lvl3 = new_lvl3(num_lvl2_tables, BSP.phys_table_struct_start_addr) + + @lvl2_phys_start_addr = @lvl3.phys_start_addr + @lvl3.size_in_byte + @lvl2 = new_lvl2(num_lvl2_tables, @lvl2_phys_start_addr) + + populate_lvl2_entries + end + + def map_pages_at(virt_pages, phys_pages, attributes) + return if virt_pages.empty? + + raise if virt_pages.size != phys_pages.size + raise if phys_pages.last > BSP.phys_addr_space_end_page + + virt_pages.zip(phys_pages).each do |virt_page, phys_page| + desc = page_descriptor_from(virt_page) + set_lvl3_entry(desc, phys_page, attributes) + end + end + + def to_binary + data = @lvl3.flatten.map(&:to_i) + @lvl2.map(&:to_i) + data.pack('Q<*') # "Q" == uint64_t, "<" == little endian + end + + def phys_tables_base_addr_binary + [@lvl2_phys_start_addr].pack('Q<*') # "Q" == uint64_t, "<" == little endian + end + + def phys_tables_base_addr + @lvl2_phys_start_addr + end + + private + + def binary_with_mmio_clash? + BSP.rw_end_exclusive >= @virt_mmio_start_addr + end + + def do_sanity_checks + raise unless BSP.kernel_granule::SIZE == Granule64KiB::SIZE + raise unless (BSP.kernel_virt_addr_space_size % Granule512MiB::SIZE).zero? + + # Need to ensure that that the kernel binary does not clash with the upmost 256 MiB of the + # virtual address space, which is reserved for runtime-remapping of MMIO. + return unless binary_with_mmio_clash? + + puts format('__data_end_exclusive: 0x%16x', BSP.data_end_exclusive) + puts format('MMIO start: 0x%16x', @virt_mmio_start_addr) + + raise 'Kernel virtual addresses clash with 256 MiB MMIO window' + end + + def new_lvl3(num_lvl2_tables, start_addr) + CArray.new(start_addr, num_lvl2_tables) do + temp = CArray.new(start_addr, 8192) do + Stage1PageDescriptor.new + end + start_addr += temp.size_in_byte + + temp + end + end + + def new_lvl2(num_lvl2_tables, start_addr) + CArray.new(start_addr, num_lvl2_tables) do + Stage1TableDescriptor.new + end + end + + def populate_lvl2_entries + @lvl2.each_with_index do |descriptor, i| + descriptor.next_level_table_addr = @lvl3[i].phys_start_addr + descriptor.type = Stage1TableDescriptor::Type::TABLE + descriptor.valid = TRUE + end + end + + def lvl2_lvl3_index_from(addr) + addr -= BSP.kernel_virt_start_addr + + lvl2_index = addr >> Granule512MiB::SHIFT + lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT + + raise unless lvl2_index < @lvl2.size + + [lvl2_index, lvl3_index] + end + + def page_descriptor_from(virt_addr) + lvl2_index, lvl3_index = lvl2_lvl3_index_from(virt_addr) + + @lvl3[lvl2_index][lvl3_index] + end + + # rubocop:disable Metrics/MethodLength + def set_attributes(desc, attributes) + case attributes.mem_attributes + when :CacheableDRAM + desc.sh = Stage1PageDescriptor::SH::INNER_SHAREABLE + desc.attr_indx = MAIR::NORMAL + else + raise 'Invalid input' + end + + desc.ap = case attributes.acc_perms + when :ReadOnly + Stage1PageDescriptor::AP::RO_EL1 + when :ReadWrite + Stage1PageDescriptor::AP::RW_EL1 + else + raise 'Invalid input' + + end + + desc.pxn = case attributes.execute_never + when :XN + TRUE + when :X + FALSE + else + raise 'Invalid input' + end + + desc.uxn = TRUE + end + # rubocop:enable Metrics/MethodLength + + def set_lvl3_entry(desc, output_addr, attributes) + desc.output_addr = output_addr + desc.af = TRUE + desc.type = Stage1PageDescriptor::Type::PAGE + desc.valid = TRUE + + set_attributes(desc, attributes) + end +end +end +end diff --git a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb new file mode 100644 index 000000000..ddc0f3ba6 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/bsp.rb @@ -0,0 +1,177 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +# Raspberry Pi 3 + 4 +class RaspberryPi + attr_reader :kernel_granule, :kernel_virt_addr_space_size, :kernel_virt_start_addr + + NM_BINARY = 'aarch64-none-elf-nm' + READELF_BINARY = 'aarch64-none-elf-readelf' + MEMORY_SRC = File.read('src/bsp/raspberrypi/memory.rs').split("\n") + + def initialize(kernel_elf) + @kernel_granule = Granule64KiB + + @virt_addresses = { + boot_core_stack_start: /__boot_core_stack_start/, + boot_core_stack_end_exclusive: /__boot_core_stack_end_exclusive/, + + rx_start: /__rx_start/, + rx_end_exclusive: /__rx_end_exclusive/, + + rw_start: /__rw_start/, + rw_end_exclusive: /__rw_end_exclusive/, + + table_struct_start_addr: /bsp::.*::memory::mmu::KERNEL_TABLES/, + phys_tables_base_addr: /PHYS_KERNEL_TABLES_BASE_ADDR/ + } + + symbols = `#{NM_BINARY} --demangle #{kernel_elf}`.split("\n") + @kernel_virt_addr_space_size = parse_from_symbols(symbols, /__kernel_virt_addr_space_size/) + @kernel_virt_start_addr = parse_from_symbols(symbols, /__kernel_virt_start_addr/) + @virt_addresses = parse_from_symbols(symbols, @virt_addresses) + @phys_addresses = virt_to_phys(@virt_addresses) + + @descriptors = parse_descriptors + update_max_descriptor_name_length + + @text_section_offset_in_elf = parse_text_section_offset_in_elf(kernel_elf) + end + + def rw_end_exclusive + @virt_addresses[:rw_end_exclusive] + end + + def phys_table_struct_start_addr + @phys_addresses[:table_struct_start_addr] + end + + def table_struct_offset_in_kernel_elf + (@virt_addresses[:table_struct_start_addr] - @virt_addresses[:rx_start]) + + @text_section_offset_in_elf + end + + def phys_tables_base_addr + @phys_addresses[:phys_tables_base_addr] + end + + def phys_tables_base_addr_offset_in_kernel_elf + (@virt_addresses[:phys_tables_base_addr] - @virt_addresses[:rx_start]) + + @text_section_offset_in_elf + end + + def phys_addr_space_end_page + x = MEMORY_SRC.grep(/pub const END/) + x = case BSP_TYPE + when :rpi3 + x[0] + when :rpi4 + x[1] + else + raise + end + + x.scan(/\d+/).join.to_i(16) + end + + def kernel_map_binary + MappingDescriptor.print_header + + @descriptors.each do |i| + print 'Generating'.rjust(12).green.bold + print ' ' + puts i.to_s + + TRANSLATION_TABLES.map_pages_at(i.virt_pages, i.phys_pages, i.attributes) + end + + MappingDescriptor.print_divider + end + + private + + def parse_from_symbols(symbols, input) + case input.class.to_s + when 'Regexp' + symbols.grep(input).first.split.first.to_i(16) + when 'Hash' + input.transform_values do |val| + symbols.grep(val).first.split.first.to_i(16) + end + else + raise + end + end + + def parse_text_section_offset_in_elf(kernel_elf) + `#{READELF_BINARY} --sections #{kernel_elf}`.scan(/.text.*/).first.split.last.to_i(16) + end + + def virt_to_phys(input) + case input.class.to_s + when 'Integer' + input - @kernel_virt_start_addr + when 'Hash' + input.transform_values do |val| + val - @kernel_virt_start_addr + end + else + raise + end + end + + def descriptor_ro + name = 'Code and RO data' + + ro_size = @virt_addresses[:rx_end_exclusive] - + @virt_addresses[:rx_start] + + virt_ro_pages = PageArray.new(@virt_addresses[:rx_start], ro_size, @kernel_granule::SIZE) + phys_ro_pages = PageArray.new(@phys_addresses[:rx_start], ro_size, @kernel_granule::SIZE) + ro_attribues = AttributeFields.new(:CacheableDRAM, :ReadOnly, :X) + + MappingDescriptor.new(name, virt_ro_pages, phys_ro_pages, ro_attribues) + end + + def descriptor_data + name = 'Data and bss' + + data_size = @virt_addresses[:rw_end_exclusive] - + @virt_addresses[:rw_start] + + virt_data_pages = PageArray.new(@virt_addresses[:rw_start], data_size, + @kernel_granule::SIZE) + phys_data_pages = PageArray.new(@phys_addresses[:rw_start], data_size, + @kernel_granule::SIZE) + data_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) + + MappingDescriptor.new(name, virt_data_pages, phys_data_pages, data_attribues) + end + + def descriptor_boot_core_stack + name = 'Boot-core stack' + + boot_core_stack_size = @virt_addresses[:boot_core_stack_end_exclusive] - + @virt_addresses[:boot_core_stack_start] + + virt_boot_core_stack_pages = PageArray.new(@virt_addresses[:boot_core_stack_start], + boot_core_stack_size, @kernel_granule::SIZE) + phys_boot_core_stack_pages = PageArray.new(@phys_addresses[:boot_core_stack_start], + boot_core_stack_size, @kernel_granule::SIZE) + boot_core_stack_attribues = AttributeFields.new(:CacheableDRAM, :ReadWrite, :XN) + + MappingDescriptor.new(name, virt_boot_core_stack_pages, phys_boot_core_stack_pages, + boot_core_stack_attribues) + end + + def parse_descriptors + [descriptor_ro, descriptor_data, descriptor_boot_core_stack] + end + + def update_max_descriptor_name_length + MappingDescriptor.max_descriptor_name_length = @descriptors.map { |i| i.name.size }.max + end +end diff --git a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/generic.rb b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/generic.rb new file mode 100644 index 000000000..2ab666afb --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/generic.rb @@ -0,0 +1,125 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +module Granule64KiB + SIZE = 64 * 1024 + SHIFT = Math.log2(SIZE).to_i +end + +module Granule512MiB + SIZE = 512 * 1024 * 1024 + SHIFT = Math.log2(SIZE).to_i + MASK = SIZE - 1 +end + +# Monkey-patch Integer with some helper functions. +class Integer + def power_of_two? + self[0].zero? + end + + def aligned?(alignment) + raise unless alignment.power_of_two? + + (self & (alignment - 1)).zero? + end + + def to_hex_underscore(with_leading_zeros: false) + fmt = with_leading_zeros ? '%016x' : '%x' + value = format(fmt, self).to_s.reverse.scan(/.{4}|.+/).join('_').reverse + + format('0x%s', value) + end +end + +# An array where each value is the start address of a Page. +class PageArray < Array + def initialize(start_addr, size, granule_size) + raise unless start_addr.aligned?(granule_size) + raise unless size.positive? + raise unless (size % granule_size).zero? + + num_pages = size / granule_size + super(num_pages) do |i| + i * granule_size + start_addr + end + end +end + +# Collection of memory attributes. +class AttributeFields + attr_reader :mem_attributes, :acc_perms, :execute_never + + def initialize(mem_attributes, acc_perms, execute_never) + @mem_attributes = mem_attributes + @acc_perms = acc_perms + @execute_never = execute_never + end +end + +# A container that describes a one- or many-page virt-to-phys mapping. +class MappingDescriptor + @max_descriptor_name_length = 0 + + class << self + attr_accessor :max_descriptor_name_length + end + + attr_reader :name, :virt_pages, :phys_pages, :attributes + + def initialize(name, virt_pages, phys_pages, attributes) + @name = name + @virt_pages = virt_pages + @phys_pages = phys_pages + @attributes = attributes + end + + def to_s + name = @name.ljust(self.class.max_descriptor_name_length) + virt_start = @virt_pages.first.to_hex_underscore(with_leading_zeros: true) + size = ((@virt_pages.size * 65_536) / 1024).to_s.rjust(3) + + "#{name} | #{virt_start} | #{size} KiB" + end + + def self.print_divider + print ' ' + print '-' * max_descriptor_name_length + puts '----------------------------------' + end + + def self.print_header + print_divider + print ' ' + print 'Section'.center(max_descriptor_name_length) + print ' ' + print 'Start Virt Addr'.center(21) + print ' ' + print 'Size'.center(7) + puts + print_divider + end +end + +def kernel_patch_tables(kernel_binary) + print 'Patching'.rjust(12).green.bold + print ' Kernel table struct at physical ' + puts BSP.phys_table_struct_start_addr.to_hex_underscore + + IO.binwrite(kernel_binary, TRANSLATION_TABLES.to_binary, + BSP.table_struct_offset_in_kernel_elf) +end + +def kernel_patch_base_addr(kernel_binary) + print 'Patching'.rjust(12).green.bold + print ' Value of kernel table physical base address (' + print TRANSLATION_TABLES.phys_tables_base_addr.to_hex_underscore + print ') at physical ' + puts BSP.phys_tables_base_addr.to_hex_underscore + + IO.binwrite(kernel_binary, TRANSLATION_TABLES.phys_tables_base_addr_binary, + BSP.phys_tables_base_addr_offset_in_kernel_elf) +end diff --git a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/main.rb b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/main.rb new file mode 100755 index 000000000..5200a4858 --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/main.rb @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +TARGET = ARGV[0].split('-').first.to_sym +BSP_TYPE = ARGV[1].to_sym +kernel_elf = ARGV[2] + +require 'rubygems' +require 'bundler/setup' +require 'colorize' + +require_relative 'generic' +require_relative 'bsp' +require_relative 'arch' + +puts +puts 'Precomputing kernel translation tables and patching kernel ELF'.cyan + +start = Time.now + +BSP = case BSP_TYPE + when :rpi3, :rpi4 + RaspberryPi.new(kernel_elf) + else + raise + end + +TRANSLATION_TABLES = case TARGET + when :aarch64 + Arch::ARMv8::TranslationTable.new + else + raise + end + +BSP.kernel_map_binary + +kernel_patch_tables(kernel_elf) +kernel_patch_base_addr(kernel_elf) + +elapsed = Time.now - start + +print 'Finished'.rjust(12).green.bold +puts " in #{elapsed.round(2)}s" From d6e4a0318946a961ba71846d5fc3ba28383f5a5b Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 30 Mar 2021 23:14:09 +0200 Subject: [PATCH 054/214] Add missing cfg for test --- 12_integrated_testing/README.md | 3 ++- 12_integrated_testing/src/bsp/raspberrypi/console.rs | 1 + .../src/bsp/raspberrypi/console.rs | 1 + .../src/bsp/raspberrypi/console.rs | 1 + 15_virtual_mem_part3_precomputed_tables/README.md | 4 ++-- 5 files changed, 7 insertions(+), 3 deletions(-) diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index eefbfa4da..600efa2a8 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -1098,7 +1098,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 12_inte diff -uNr 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 12_integrated_testing/src/bsp/raspberrypi/console.rs --- 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs +++ 12_integrated_testing/src/bsp/raspberrypi/console.rs -@@ -35,3 +35,13 @@ +@@ -35,3 +35,14 @@ pub fn console() -> &'static impl console::interface::All { &super::PL011_UART } @@ -1111,6 +1111,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs 12_integ +/// than on real hardware due to QEMU's abstractions. +/// +/// For the RPi, nothing needs to be done. ++#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() {} diff -uNr 11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory/mmu.rs 12_integrated_testing/src/bsp/raspberrypi/memory/mmu.rs diff --git a/12_integrated_testing/src/bsp/raspberrypi/console.rs b/12_integrated_testing/src/bsp/raspberrypi/console.rs index d6f95e4e9..75c0540bd 100644 --- a/12_integrated_testing/src/bsp/raspberrypi/console.rs +++ b/12_integrated_testing/src/bsp/raspberrypi/console.rs @@ -44,4 +44,5 @@ pub fn console() -> &'static impl console::interface::All { /// than on real hardware due to QEMU's abstractions. /// /// For the RPi, nothing needs to be done. +#[cfg(feature = "test_build")] pub fn qemu_bring_up_console() {} diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs index d6f95e4e9..75c0540bd 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/console.rs @@ -44,4 +44,5 @@ pub fn console() -> &'static impl console::interface::All { /// than on real hardware due to QEMU's abstractions. /// /// For the RPi, nothing needs to be done. +#[cfg(feature = "test_build")] pub fn qemu_bring_up_console() {} diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs index abf8f89cc..0d0f8e40b 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs @@ -58,4 +58,5 @@ pub fn console() -> &'static impl console::interface::All { /// than on real hardware due to QEMU's abstractions. /// /// For the RPi, nothing needs to be done. +#[cfg(feature = "test_build")] pub fn qemu_bring_up_console() {} diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index f83ff5b4e..25d9cc360 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -1024,14 +1024,14 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/console.rs 15_virt /// Return a reference to the console. pub fn console() -> &'static impl console::interface::All { &super::PL011_UART -@@ -56,6 +74,15 @@ +@@ -56,7 +74,15 @@ /// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps /// than on real hardware due to QEMU's abstractions. -/// -/// For the RPi, nothing needs to be done. + #[cfg(feature = "test_build")] -pub fn qemu_bring_up_console() {} -+#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use driver::interface::DeviceDriver; + From 37cb58a944085d7100f44795fddbb7d67917971f Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 31 Mar 2021 22:45:17 +0200 Subject: [PATCH 055/214] rubocop: Fix some errors --- .rubocop.yml | 3 -- .../README.md | 32 +++++++++++++------ .../translation_table_tool/arch.rb | 30 +++++++++++------ .../translation_table_tool/arch.rb | 30 +++++++++++------ utils/devtool.rb | 4 +-- 5 files changed, 66 insertions(+), 33 deletions(-) diff --git a/.rubocop.yml b/.rubocop.yml index d61571d60..a1687fd16 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -16,9 +16,6 @@ Layout/IndentationWidth: Layout/LineLength: Max: 100 -Lint/DeprecatedConstants: - Enabled: false - Metrics/ClassLength: Enabled: false diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index 25d9cc360..5329426a0 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -1562,7 +1562,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb --- 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb +++ 15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb -@@ -0,0 +1,323 @@ +@@ -0,0 +1,335 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 @@ -1617,9 +1617,6 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt +# Arch:: +#--------------------------------------------------------------------------------------------------- +module Arch -+FALSE = 0b0 -+TRUE = 0b1 -+ +#--------------------------------------------------------------------------------------------------- +# Arch::ARMv8 +#--------------------------------------------------------------------------------------------------- @@ -1642,6 +1639,9 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + module Valid + OFFSET = 0 + NUMBITS = 1 ++ ++ FALSE = 0 ++ TRUE = 1 + end + + attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) @@ -1662,11 +1662,17 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + module UXN + OFFSET = 54 + NUMBITS = 1 ++ ++ FALSE = 0 ++ TRUE = 1 + end + + module PXN + OFFSET = 53 + NUMBITS = 1 ++ ++ FALSE = 0 ++ TRUE = 1 + end + + module OutputAddr @@ -1677,6 +1683,9 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + module AF + OFFSET = 10 + NUMBITS = 1 ++ ++ FALSE = 0 ++ TRUE = 1 + end + + module SH @@ -1710,6 +1719,9 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + module Valid + OFFSET = 0 + NUMBITS = 1 ++ ++ FALSE = 0 ++ TRUE = 1 + end + + attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) @@ -1821,7 +1833,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + @lvl2.each_with_index do |descriptor, i| + descriptor.next_level_table_addr = @lvl3[i].phys_start_addr + descriptor.type = Stage1TableDescriptor::Type::TABLE -+ descriptor.valid = TRUE ++ descriptor.valid = Stage1TableDescriptor::Valid::TRUE + end + end + @@ -1864,22 +1876,22 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/translation_table_tool/arch.rb 15_virt + + desc.pxn = case attributes.execute_never + when :XN -+ TRUE ++ Stage1PageDescriptor::PXN::TRUE + when :X -+ FALSE ++ Stage1PageDescriptor::PXN::FALSE + else + raise 'Invalid input' + end + -+ desc.uxn = TRUE ++ desc.uxn = Stage1PageDescriptor::UXN::TRUE + end + # rubocop:enable Metrics/MethodLength + + def set_lvl3_entry(desc, output_addr, attributes) + desc.output_addr = output_addr -+ desc.af = TRUE ++ desc.af = Stage1PageDescriptor::AF::TRUE + desc.type = Stage1PageDescriptor::Type::PAGE -+ desc.valid = TRUE ++ desc.valid = Stage1PageDescriptor::Valid::TRUE + + set_attributes(desc, attributes) + end diff --git a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb index 72471b113..a6e5cf68c 100644 --- a/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb +++ b/15_virtual_mem_part3_precomputed_tables/translation_table_tool/arch.rb @@ -52,9 +52,6 @@ def size_in_byte # Arch:: #--------------------------------------------------------------------------------------------------- module Arch -FALSE = 0b0 -TRUE = 0b1 - #--------------------------------------------------------------------------------------------------- # Arch::ARMv8 #--------------------------------------------------------------------------------------------------- @@ -77,6 +74,9 @@ module Type module Valid OFFSET = 0 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) @@ -97,11 +97,17 @@ class Stage1PageDescriptor < BitField module UXN OFFSET = 54 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module PXN OFFSET = 53 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module OutputAddr @@ -112,6 +118,9 @@ module OutputAddr module AF OFFSET = 10 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module SH @@ -145,6 +154,9 @@ module Type module Valid OFFSET = 0 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) @@ -256,7 +268,7 @@ def populate_lvl2_entries @lvl2.each_with_index do |descriptor, i| descriptor.next_level_table_addr = @lvl3[i].phys_start_addr descriptor.type = Stage1TableDescriptor::Type::TABLE - descriptor.valid = TRUE + descriptor.valid = Stage1TableDescriptor::Valid::TRUE end end @@ -299,22 +311,22 @@ def set_attributes(desc, attributes) desc.pxn = case attributes.execute_never when :XN - TRUE + Stage1PageDescriptor::PXN::TRUE when :X - FALSE + Stage1PageDescriptor::PXN::FALSE else raise 'Invalid input' end - desc.uxn = TRUE + desc.uxn = Stage1PageDescriptor::UXN::TRUE end # rubocop:enable Metrics/MethodLength def set_lvl3_entry(desc, output_addr, attributes) desc.output_addr = output_addr - desc.af = TRUE + desc.af = Stage1PageDescriptor::AF::TRUE desc.type = Stage1PageDescriptor::Type::PAGE - desc.valid = TRUE + desc.valid = Stage1PageDescriptor::Valid::TRUE set_attributes(desc, attributes) end diff --git a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb index 72471b113..a6e5cf68c 100644 --- a/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb +++ b/16_virtual_mem_part4_higher_half_kernel/translation_table_tool/arch.rb @@ -52,9 +52,6 @@ def size_in_byte # Arch:: #--------------------------------------------------------------------------------------------------- module Arch -FALSE = 0b0 -TRUE = 0b1 - #--------------------------------------------------------------------------------------------------- # Arch::ARMv8 #--------------------------------------------------------------------------------------------------- @@ -77,6 +74,9 @@ module Type module Valid OFFSET = 0 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end attr_bitfield(:__next_level_table_addr, NextLevelTableAddr::OFFSET, NextLevelTableAddr::NUMBITS) @@ -97,11 +97,17 @@ class Stage1PageDescriptor < BitField module UXN OFFSET = 54 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module PXN OFFSET = 53 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module OutputAddr @@ -112,6 +118,9 @@ module OutputAddr module AF OFFSET = 10 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end module SH @@ -145,6 +154,9 @@ module Type module Valid OFFSET = 0 NUMBITS = 1 + + FALSE = 0 + TRUE = 1 end attr_bitfield(:uxn, UXN::OFFSET, UXN::NUMBITS) @@ -256,7 +268,7 @@ def populate_lvl2_entries @lvl2.each_with_index do |descriptor, i| descriptor.next_level_table_addr = @lvl3[i].phys_start_addr descriptor.type = Stage1TableDescriptor::Type::TABLE - descriptor.valid = TRUE + descriptor.valid = Stage1TableDescriptor::Valid::TRUE end end @@ -299,22 +311,22 @@ def set_attributes(desc, attributes) desc.pxn = case attributes.execute_never when :XN - TRUE + Stage1PageDescriptor::PXN::TRUE when :X - FALSE + Stage1PageDescriptor::PXN::FALSE else raise 'Invalid input' end - desc.uxn = TRUE + desc.uxn = Stage1PageDescriptor::UXN::TRUE end # rubocop:enable Metrics/MethodLength def set_lvl3_entry(desc, output_addr, attributes) desc.output_addr = output_addr - desc.af = TRUE + desc.af = Stage1PageDescriptor::AF::TRUE desc.type = Stage1PageDescriptor::Type::PAGE - desc.valid = TRUE + desc.valid = Stage1PageDescriptor::Valid::TRUE set_attributes(desc, attributes) end diff --git a/utils/devtool.rb b/utils/devtool.rb index c29987bba..a3d5e3d7b 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -166,6 +166,8 @@ def misspell def rubocop puts 'Rubocop'.light_blue + system('which bundle') + system('bundle --version') exit(1) unless system('bundle exec rubocop') end @@ -180,8 +182,6 @@ def ready_for_publish diff clean - make('rpi4') - make('rpi3') make_xtra test_unit test_integration From b025f5577cb210d08002f5038010bc35a567698c Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Wed, 31 Mar 2021 22:48:02 +0200 Subject: [PATCH 056/214] Use raw string literal for logo --- 06_uart_chainloader/README.md | 19 +++++++++++++------ 06_uart_chainloader/src/main.rs | 13 ++++++++----- 07_timestamps/README.md | 23 ++++++++++++++--------- 3 files changed, 35 insertions(+), 20 deletions(-) diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 040c498d3..131dc19bb 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -373,18 +373,25 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] -@@ -150,29 +151,49 @@ +@@ -146,33 +147,56 @@ + kernel_main() + } + ++const MINILOAD_LOGO: &str = r#" ++ __ __ _ _ _ _ ++| \/ (_)_ _ (_) | ___ __ _ __| | ++| |\/| | | ' \| | |__/ _ \/ _` / _` | ++|_| |_|_|_||_|_|____\___/\__,_\__,_| ++"#; ++ + /// The main function running after the early init. fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; - use driver::interface::DriverManager; - println!("[0] Booting on: {}", bsp::board_name()); -+ println!(" __ __ _ _ _ _ "); -+ println!("| \\/ (_)_ _ (_) | ___ __ _ __| |"); -+ println!("| |\\/| | | ' \\| | |__/ _ \\/ _` / _` |"); -+ println!("|_| |_|_|_||_|_|____\\___/\\__,_\\__,_|"); -+ println!(); ++ println!("{}", MINILOAD_LOGO); + println!("{:^37}", bsp::board_name()); + println!(); + println!("[ML] Requesting binary"); diff --git a/06_uart_chainloader/src/main.rs b/06_uart_chainloader/src/main.rs index 1ac8140cc..a94b97e42 100644 --- a/06_uart_chainloader/src/main.rs +++ b/06_uart_chainloader/src/main.rs @@ -147,16 +147,19 @@ unsafe fn kernel_init() -> ! { kernel_main() } +const MINILOAD_LOGO: &str = r#" + __ __ _ _ _ _ +| \/ (_)_ _ (_) | ___ __ _ __| | +| |\/| | | ' \| | |__/ _ \/ _` / _` | +|_| |_|_|_||_|_|____\___/\__,_\__,_| +"#; + /// The main function running after the early init. fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; - println!(" __ __ _ _ _ _ "); - println!("| \\/ (_)_ _ (_) | ___ __ _ __| |"); - println!("| |\\/| | | ' \\| | |__/ _ \\/ _` / _` |"); - println!("|_| |_|_|_||_|_|____\\___/\\__,_\\__,_|"); - println!(); + println!("{}", MINILOAD_LOGO); println!("{:^37}", bsp::board_name()); println!(); println!("[ML] Requesting binary"); diff --git a/07_timestamps/README.md b/07_timestamps/README.md index dc40f8048..e221a12c3 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -475,8 +475,17 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs /// Early init code. /// -@@ -149,51 +149,31 @@ +@@ -147,56 +147,33 @@ + kernel_main() + } +-const MINILOAD_LOGO: &str = r#" +- __ __ _ _ _ _ +-| \/ (_)_ _ (_) | ___ __ _ __| | +-| |\/| | | ' \| | |__/ _ \/ _` / _` | +-|_| |_|_|_||_|_|____\___/\__,_\__,_| +-"#; +- /// The main function running after the early init. fn kernel_main() -> ! { - use bsp::console::console; @@ -485,11 +494,7 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs + use driver::interface::DriverManager; + use time::interface::TimeManager; -- println!(" __ __ _ _ _ _ "); -- println!("| \\/ (_)_ _ (_) | ___ __ _ __| |"); -- println!("| |\\/| | | ' \\| | |__/ _ \\/ _` / _` |"); -- println!("|_| |_|_|_||_|_|____\\___/\\__,_\\__,_|"); -- println!(); +- println!("{}", MINILOAD_LOGO); - println!("{:^37}", bsp::board_name()); - println!(); - println!("[ML] Requesting binary"); @@ -536,12 +541,12 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - println!("[ML] Loaded! Executing the payload now\n"); - console().flush(); +- +- // Use black magic to create a function pointer. +- let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; + // Test a failing timer case. + time::time_manager().spin_for(Duration::from_nanos(1)); -- // Use black magic to create a function pointer. -- let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; -- - // Jump to loaded kernel! - kernel() + loop { From 9f1920c2c6221c324b048c0cb4b899fbf3d33cf1 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 22:30:40 +0200 Subject: [PATCH 057/214] Give the project a name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 🦩 --- 01_wait_forever/Cargo.lock | 5 +- 01_wait_forever/Cargo.toml | 6 +- 02_runtime_init/Cargo.lock | 4 +- 02_runtime_init/Cargo.toml | 9 +- 02_runtime_init/README.md | 11 ++- 03_hacky_hello_world/Cargo.lock | 4 +- 03_hacky_hello_world/Cargo.toml | 8 +- 03_hacky_hello_world/README.md | 13 ++- 04_safe_globals/Cargo.lock | 4 +- 04_safe_globals/Cargo.toml | 8 +- 04_safe_globals/README.md | 22 ++++- 04_safe_globals/src/main.rs | 2 +- 05_drivers_gpio_uart/Cargo.lock | 4 +- 05_drivers_gpio_uart/Cargo.toml | 8 +- 05_drivers_gpio_uart/README.md | 40 +++++--- 05_drivers_gpio_uart/src/main.rs | 13 ++- 06_uart_chainloader/Cargo.lock | 4 +- 06_uart_chainloader/Cargo.toml | 8 +- 06_uart_chainloader/README.md | 67 ++++++++----- 06_uart_chainloader/demo_payload_rpi3.img | Bin 6840 -> 7024 bytes 06_uart_chainloader/demo_payload_rpi4.img | Bin 6680 -> 6872 bytes 07_timestamps/Cargo.lock | 4 +- 07_timestamps/Cargo.toml | 8 +- 07_timestamps/README.md | 75 ++++++++------ 07_timestamps/src/main.rs | 5 + 08_hw_debug_JTAG/Cargo.lock | 4 +- 08_hw_debug_JTAG/Cargo.toml | 8 +- 08_hw_debug_JTAG/README.md | 12 +++ 08_hw_debug_JTAG/src/main.rs | 5 + 09_privilege_level/Cargo.lock | 4 +- 09_privilege_level/Cargo.toml | 8 +- 09_privilege_level/README.md | 48 +++++---- 09_privilege_level/src/main.rs | 5 + .../Cargo.lock | 4 +- .../Cargo.toml | 8 +- .../README.md | 55 +++++++---- .../src/main.rs | 5 + 11_exceptions_part1_groundwork/Cargo.lock | 4 +- 11_exceptions_part1_groundwork/Cargo.toml | 8 +- 11_exceptions_part1_groundwork/README.md | 93 ++++++++++-------- 11_exceptions_part1_groundwork/src/main.rs | 5 + 12_integrated_testing/Cargo.lock | 4 +- 12_integrated_testing/Cargo.toml | 5 +- 12_integrated_testing/README.md | 54 +++++++--- 12_integrated_testing/src/lib.rs | 13 +++ 12_integrated_testing/src/main.rs | 1 + .../Cargo.lock | 4 +- .../Cargo.toml | 5 +- 13_exceptions_part2_peripheral_IRQs/README.md | 92 +++++++++-------- .../src/lib.rs | 13 +++ .../src/main.rs | 1 + 14_virtual_mem_part2_mmio_remap/Cargo.lock | 4 +- 14_virtual_mem_part2_mmio_remap/Cargo.toml | 5 +- 14_virtual_mem_part2_mmio_remap/README.md | 68 ++++++++----- 14_virtual_mem_part2_mmio_remap/src/lib.rs | 13 +++ 14_virtual_mem_part2_mmio_remap/src/main.rs | 1 + .../Cargo.lock | 4 +- .../Cargo.toml | 5 +- .../README.md | 37 ++++--- .../src/lib.rs | 13 +++ .../src/main.rs | 1 + .../Cargo.lock | 4 +- .../Cargo.toml | 5 +- .../README.md | 64 +++++++----- .../src/lib.rs | 13 +++ .../src/main.rs | 1 + X1_JTAG_boot/Cargo.lock | 4 +- X1_JTAG_boot/Cargo.toml | 8 +- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8176 -> 8160 bytes 69 files changed, 704 insertions(+), 346 deletions(-) diff --git a/01_wait_forever/Cargo.lock b/01_wait_forever/Cargo.lock index 3e9c31110..aaed18257 100644 --- a/01_wait_forever/Cargo.lock +++ b/01_wait_forever/Cargo.lock @@ -1,6 +1,7 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. +version = 3 + [[package]] -name = "kernel" +name = "mingo" version = "0.1.0" - diff --git a/01_wait_forever/Cargo.toml b/01_wait_forever/Cargo.toml index a75809963..2b46585e9 100644 --- a/01_wait_forever/Cargo.toml +++ b/01_wait_forever/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "kernel" +name = "mingo" version = "0.1.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = [] bsp_rpi4 = [] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/02_runtime_init/Cargo.lock b/02_runtime_init/Cargo.lock index 3b926a347..090fdaee4 100644 --- a/02_runtime_init/Cargo.lock +++ b/02_runtime_init/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.2.0" dependencies = [ "cortex-a", ] diff --git a/02_runtime_init/Cargo.toml b/02_runtime_init/Cargo.toml index 4a03b9623..be6091a4d 100644 --- a/02_runtime_init/Cargo.toml +++ b/02_runtime_init/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.2.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = [] bsp_rpi4 = [] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- @@ -21,3 +25,4 @@ bsp_rpi4 = [] # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } + diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index b294e2e1f..dfa43562a 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -31,7 +31,15 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml --- 01_wait_forever/Cargo.toml +++ 02_runtime_init/Cargo.toml -@@ -17,3 +17,7 @@ +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.1.0" ++version = "0.2.0" + authors = ["Andre Richter "] + edition = "2018" + +@@ -21,3 +21,8 @@ ##-------------------------------------------------------------------------------------------------- [dependencies] @@ -39,6 +47,7 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } ++ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile --- 01_wait_forever/Makefile diff --git a/03_hacky_hello_world/Cargo.lock b/03_hacky_hello_world/Cargo.lock index 3b926a347..09dfcc720 100644 --- a/03_hacky_hello_world/Cargo.lock +++ b/03_hacky_hello_world/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.3.0" dependencies = [ "cortex-a", ] diff --git a/03_hacky_hello_world/Cargo.toml b/03_hacky_hello_world/Cargo.toml index 5deb70108..6ed23060e 100644 --- a/03_hacky_hello_world/Cargo.toml +++ b/03_hacky_hello_world/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.3.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = [] bsp_rpi4 = [] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index a002a6198..6958b8bc5 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -31,11 +31,14 @@ Kernel panic: Stopping here. diff -uNr 02_runtime_init/Cargo.toml 03_hacky_hello_world/Cargo.toml --- 02_runtime_init/Cargo.toml +++ 03_hacky_hello_world/Cargo.toml -@@ -21,3 +21,4 @@ - # Platform specific dependencies - [target.'cfg(target_arch = "aarch64")'.dependencies] - cortex-a = { version = "5.x.x" } -+ +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.2.0" ++version = "0.3.0" + authors = ["Andre Richter "] + edition = "2018" + diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile --- 02_runtime_init/Makefile diff --git a/04_safe_globals/Cargo.lock b/04_safe_globals/Cargo.lock index 3b926a347..ccfbefbf3 100644 --- a/04_safe_globals/Cargo.lock +++ b/04_safe_globals/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.4.0" dependencies = [ "cortex-a", ] diff --git a/04_safe_globals/Cargo.toml b/04_safe_globals/Cargo.toml index 5deb70108..16e4741df 100644 --- a/04_safe_globals/Cargo.toml +++ b/04_safe_globals/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.4.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = [] bsp_rpi4 = [] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/04_safe_globals/README.md b/04_safe_globals/README.md index 13b9e298a..272edfce6 100644 --- a/04_safe_globals/README.md +++ b/04_safe_globals/README.md @@ -46,14 +46,27 @@ implemntations in the [spin crate] or the [parking lot crate]. ```console $ make qemu [...] -[0] Hello from pure Rust! -[1] Chars written: 27 + +[0] Hello from Rust! +[1] Chars written: 22 [2] Stopping here. ``` ## Diff to previous ```diff +diff -uNr 03_hacky_hello_world/Cargo.toml 04_safe_globals/Cargo.toml +--- 03_hacky_hello_world/Cargo.toml ++++ 04_safe_globals/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.3.0" ++version = "0.4.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 03_hacky_hello_world/src/bsp/raspberrypi/console.rs 04_safe_globals/src/bsp/raspberrypi/console.rs --- 03_hacky_hello_world/src/bsp/raspberrypi/console.rs +++ 04_safe_globals/src/bsp/raspberrypi/console.rs @@ -231,12 +244,11 @@ diff -uNr 03_hacky_hello_world/src/main.rs 04_safe_globals/src/main.rs /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { -- println!("[0] Hello from Rust!"); + use console::interface::Statistics; ++ + println!("[0] Hello from Rust!"); - panic!("Stopping here.") -+ println!("[0] Hello from pure Rust!"); -+ + println!( + "[1] Chars written: {}", + bsp::console::console().chars_written() diff --git a/04_safe_globals/src/main.rs b/04_safe_globals/src/main.rs index 579f90ddc..48687ace2 100644 --- a/04_safe_globals/src/main.rs +++ b/04_safe_globals/src/main.rs @@ -130,7 +130,7 @@ mod synchronization; unsafe fn kernel_init() -> ! { use console::interface::Statistics; - println!("[0] Hello from pure Rust!"); + println!("[0] Hello from Rust!"); println!( "[1] Chars written: {}", diff --git a/05_drivers_gpio_uart/Cargo.lock b/05_drivers_gpio_uart/Cargo.lock index a3d8bb9ae..87044397a 100644 --- a/05_drivers_gpio_uart/Cargo.lock +++ b/05_drivers_gpio_uart/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.5.0" dependencies = [ "cortex-a", "register", diff --git a/05_drivers_gpio_uart/Cargo.toml b/05_drivers_gpio_uart/Cargo.toml index df5f7f974..03ec9cdb9 100644 --- a/05_drivers_gpio_uart/Cargo.toml +++ b/05_drivers_gpio_uart/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.5.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index 4ecdd41ad..9f4a8749b 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -95,12 +95,13 @@ Miniterm 1.0 [MT] ⏳ Waiting for /dev/ttyUSB0 [MT] ✅ Serial connected -[0] Booting on: Raspberry Pi 3 -[1] Drivers loaded: +[0] mingo version 0.5.0 +[1] Booting on: Raspberry Pi 3 +[2] Drivers loaded: 1. BCM GPIO 2. BCM PL011 UART -[2] Chars written: 93 -[3] Echoing input now +[3] Chars written: 117 +[4] Echoing input now ``` 8. Exit by pressing ctrl-c. @@ -111,6 +112,14 @@ Miniterm 1.0 diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml --- 04_safe_globals/Cargo.toml +++ 05_drivers_gpio_uart/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.4.0" ++version = "0.5.0" + authors = ["Andre Richter "] + edition = "2018" + @@ -9,8 +9,8 @@ [features] @@ -120,9 +129,9 @@ diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml +bsp_rpi3 = ["register"] +bsp_rpi4 = ["register"] - ##-------------------------------------------------------------------------------------------------- - ## Dependencies -@@ -18,6 +18,9 @@ + [[bin]] + name = "kernel" +@@ -22,6 +22,9 @@ [dependencies] @@ -1329,7 +1338,7 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs mod memory; mod panic_wait; mod print; -@@ -127,16 +130,49 @@ +@@ -127,16 +130,54 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1338,7 +1347,7 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs - use console::interface::Statistics; + use driver::interface::DriverManager; -- println!("[0] Hello from pure Rust!"); +- println!("[0] Hello from Rust!"); + for i in bsp::driver::driver_manager().all_device_drivers().iter() { + if let Err(x) = i.init() { + panic!("Error loading driver: {}: {}", i.compatible(), x); @@ -1357,9 +1366,14 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs + use console::interface::All; + use driver::interface::DriverManager; + -+ println!("[0] Booting on: {}", bsp::board_name()); ++ println!( ++ "[0] {} version {}", ++ env!("CARGO_PKG_NAME"), ++ env!("CARGO_PKG_VERSION") ++ ); ++ println!("[1] Booting on: {}", bsp::board_name()); + -+ println!("[1] Drivers loaded:"); ++ println!("[2] Drivers loaded:"); + for (i, driver) in bsp::driver::driver_manager() + .all_device_drivers() + .iter() @@ -1370,10 +1384,10 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs println!( - "[1] Chars written: {}", -+ "[2] Chars written: {}", ++ "[3] Chars written: {}", bsp::console::console().chars_written() ); -+ println!("[3] Echoing input now"); ++ println!("[4] Echoing input now"); - println!("[2] Stopping here."); - cpu::wait_forever() diff --git a/05_drivers_gpio_uart/src/main.rs b/05_drivers_gpio_uart/src/main.rs index 0f3c493df..93d6ffef5 100644 --- a/05_drivers_gpio_uart/src/main.rs +++ b/05_drivers_gpio_uart/src/main.rs @@ -152,9 +152,14 @@ fn kernel_main() -> ! { use console::interface::All; use driver::interface::DriverManager; - println!("[0] Booting on: {}", bsp::board_name()); + println!( + "[0] {} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); + println!("[1] Booting on: {}", bsp::board_name()); - println!("[1] Drivers loaded:"); + println!("[2] Drivers loaded:"); for (i, driver) in bsp::driver::driver_manager() .all_device_drivers() .iter() @@ -164,10 +169,10 @@ fn kernel_main() -> ! { } println!( - "[2] Chars written: {}", + "[3] Chars written: {}", bsp::console::console().chars_written() ); - println!("[3] Echoing input now"); + println!("[4] Echoing input now"); // Discard any spurious received characters before going into echo mode. console().clear_rx(); diff --git a/06_uart_chainloader/Cargo.lock b/06_uart_chainloader/Cargo.lock index a3d8bb9ae..5d2602a8e 100644 --- a/06_uart_chainloader/Cargo.lock +++ b/06_uart_chainloader/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.6.0" dependencies = [ "cortex-a", "register", diff --git a/06_uart_chainloader/Cargo.toml b/06_uart_chainloader/Cargo.toml index df5f7f974..0e1c5c28f 100644 --- a/06_uart_chainloader/Cargo.toml +++ b/06_uart_chainloader/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.6.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 131dc19bb..80aa21332 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -75,12 +75,13 @@ Minipush 1.0 [MP] ⏩ Pushing 6 KiB ==========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[0] Booting on: Raspberry Pi 3 -[1] Drivers loaded: +[0] mingo version 0.5.0 +[1] Booting on: Raspberry Pi 3 +[2] Drivers loaded: 1. BCM GPIO 2. BCM PL011 UART -[2] Chars written: 93 -[3] Echoing input now +[3] Chars written: 117 +[4] Echoing input now ``` In this tutorial, a version of the kernel from the previous tutorial is loaded for demo purposes. In @@ -118,6 +119,18 @@ IN: ## Diff to previous ```diff + +diff -uNr 05_drivers_gpio_uart/Cargo.toml 06_uart_chainloader/Cargo.toml +--- 05_drivers_gpio_uart/Cargo.toml ++++ 06_uart_chainloader/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.5.0" ++version = "0.6.0" + authors = ["Andre Richter "] + edition = "2018" + Binary files 05_drivers_gpio_uart/demo_payload_rpi3.img and 06_uart_chainloader/demo_payload_rpi3.img differ Binary files 05_drivers_gpio_uart/demo_payload_rpi4.img and 06_uart_chainloader/demo_payload_rpi4.img differ @@ -373,7 +386,7 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] -@@ -146,33 +147,56 @@ +@@ -146,38 +147,56 @@ kernel_main() } @@ -389,15 +402,15 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs use bsp::console::console; use console::interface::All; - use driver::interface::DriverManager; - -- println!("[0] Booting on: {}", bsp::board_name()); -+ println!("{}", MINILOAD_LOGO); -+ println!("{:^37}", bsp::board_name()); -+ println!(); -+ println!("[ML] Requesting binary"); -+ console().flush(); - -- println!("[1] Drivers loaded:"); +- +- println!( +- "[0] {} version {}", +- env!("CARGO_PKG_NAME"), +- env!("CARGO_PKG_VERSION") +- ); +- println!("[1] Booting on: {}", bsp::board_name()); +- +- println!("[2] Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() @@ -405,24 +418,30 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs - { - println!(" {}. {}", i + 1, driver.compatible()); - } -+ // Discard any spurious received characters before starting with the loader protocol. -+ console().clear_rx(); - println!( -- "[2] Chars written: {}", +- "[3] Chars written: {}", - bsp::console::console().chars_written() - ); -- println!("[3] Echoing input now"); -+ // Notify `Minipush` to send the binary. -+ for _ in 0..3 { -+ console().write_char(3 as char); -+ } +- println!("[4] Echoing input now"); ++ println!("{}", MINILOAD_LOGO); ++ println!("{:^37}", bsp::board_name()); ++ println!(); ++ println!("[ML] Requesting binary"); ++ console().flush(); - // Discard any spurious received characters before going into echo mode. -- console().clear_rx(); ++ // Discard any spurious received characters before starting with the loader protocol. + console().clear_rx(); - loop { - let c = bsp::console::console().read_char(); - bsp::console::console().write_char(c); ++ ++ // Notify `Minipush` to send the binary. ++ for _ in 0..3 { ++ console().write_char(3 as char); + } ++ + // Read the binary's size. + let mut size: u32 = u32::from(console().read_char() as u8); + size |= u32::from(console().read_char() as u8) << 8; @@ -439,7 +458,7 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs + for i in 0..size { + core::ptr::write_volatile(kernel_addr.offset(i as isize), console().read_char() as u8) + } - } ++ } + + println!("[ML] Loaded! Executing the payload now\n"); + console().flush(); diff --git a/06_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img index 203ebb7652369e33837c1da09e9b7b7998e4bcf4..9b1cf35bce83d123cb099774e0377f9ac1271a2b 100755 GIT binary patch delta 1645 zcmZuxUrbw77(e&kwzQpDfdc)5z_~+5TLfXOtK)?WTXn%?SqKqLL!fSG>z@`G%Hu7I zW)GOi@hq9~!HA2&OR_{m$i@R^h6lrwe`cl&iMgmwCJzF+(ML_6#{T?DR4$LORh#=|#dRe+8j)&+WTemnBob9un z6+oSxw!bV^jE6{A4w8JskD6X0Xu5>R&aB{k;1GVH8;4JeKh@Bf=QUBOrJs3fo%neo zH=Picd>gK8B2SD3&WlQ5I7LhXG{%@RUsMTnZI|0YC=qk%SBeWE*q_Y71S>8_JDh35eAt z@65Ir8R>t)c3HHTPch68Ux40EtMwQLhk1=UbzCcC9hwI9)IleTr5uXEh6}IA7E9fG zV6Jhtfe1G6V*te%Uk8F1S&W=7poro?F_!;7G07uOC4~{f$e4hfuWi=t)J?js#|?H( zk$^J4j&Q6)Q0CopaiiOgh|`!P&g(}> z(w^)2c(B2`fdUj~OU7zW*m6BCEp9~Y%KST(6>Skans+^cb?4!*Ya-mQE^D8oTuZ=Yvo|aRfA_!z0&vN=4K!iM^En|+gmOJQFP;;olk5pkuaJ+< zzveki(4C+4z92-VlQgwdPb;lI>P8jO4H4+_Jdsv?7J= zw!gYnyG}}dQcaSEG3oTVG_AvbPQb}3t)Tnyy9Y{rsp*~cW~0m}p?soS$hI~s>M6Xd zSRTDH19as#tds%Mn(^uz8`i~AHFWU&T9>np^CaT*@)t>mM}%DtkqG8;lShAuMJ>&% zKS1)oHJ|<`k_>HwjtCO)FnTBAaFP2Shme#}A8*K;`O4N=AbzudROczlX0{vOD*9$bz&%O&iQ;XhP4G{VWBU%TXdYMed zp6qAqq0HWLk{*L*n)jCa-Y+%V#U#tw3C=m`DfRgeWzDo#T{u}AX-|7$!M~1Ma)_M_ zAABX7HB&ET%`PI{@e?=#m?7YSavRNK<^eTD8&F@qVYWOR`vE-Q`0JK+1l&s2RkFCz z4(2L*6s{MD%;d;tA>6IDC{{zh9xloLR(36t5Tp&Z5}DPu+}eWWlhkO!t*gR~(>%E2 zqR#u5@D){1v|Cq!F5<4Mru7`}QpDrI1AP9W7H58=g>u~MC@$}q;|RV8D7Dd_#KD_} z7@u3h!@r{ds@c6^$x^ebDmZQAuw zGuujRLGIGd);p+bH)^?DxE!{PxcB%FY&+`u_-bu?9Iv|fn^ikq-{<0=cYU`D4IU>@ zYRB6Zy85 zeHcAVI@zz$45?u?u|WdtR7?)tbtMbBBU(}RX>3#|#8_bap1>8i?v)tp-QMyq>K(u2 diff --git a/06_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img index b06166dcbae527c6f1cc42c11d9200ed60f9dabf..2357ef5e93fd8fe7d0716198726efb7c85810edc 100755 GIT binary patch delta 2516 zcmcIlUrbwN6hGg+ZFg_S289+140-{9w#>=s7_cRy{Bs*7VxVpkf*Wp&bWX4g-Gjti zV9SVG44)S>&3stg!)*0o6HUYi2b+o6UX-YFCO+J;Ea6Xdnz@Y**X#LiZ&8=H#2CNi z^7Z^b=XcKeczeUu%F$~1vI)i{kiMPdcU{qjVLIMIA{pS1X{j(#3tVUjAaKnGHEsyE z76aI8K?)TT2)G96 z=74n32I;OrKGc8$<|K@_)$XruM%jKxj?Xp4hlUN=B;Eqb1eDp);R5gSsyvA%eQlES`(K=ZS6@Z#f^C#j4mm6aaTj<%MY5zlQjs z2+RLJ#1BPy`+tFW>U{<;AdC>sQV#J)Nr|HI&0L*x{0UPFAtmD{&0D#Q66%@CqO+x+ zN7NNW@4(IR+bhH;BA<;Lpr%)tt}5@0k1z0EfOeSvJI$GdWsveX@%nyU<_ z7Lh{bOlp04$bqg**cYHUFlU>|8f7UpKP|k5s%ZA}eeDEW=ZlIb%`Fe+Q3v|`qz`R& zLPWsd!c-pqW;+g}2e~qOD+^*epXAdZt`qaE$>zu%^v`zrPLU;ogEGqIsOeCt+tOlsi7;D;-1(j};^b5D)@eSno$aVHZL`bElSb%IaqUDn4JvQ(M)W!t z^`K%VM!Hg1qF1PAA^#bD2{T&6$ZUjJ20x;Y-K(L}&9csb81CWd*o+tD_Naxxn0l0p z+|{&S?hd8Vd&5AVySED>F-$*(<&P8s)eY(;I{OVD>1IweEd8cA!pg>{ES4-gu^zY* z7*MrBRR_wKFrdn3L=XIqJPf4!-l@E1b9u4M+t(2&{8*YFvNQ##>c&mIjTtb&MNHK* z=u*D41JR4t=IBpsFIp8N%kBnoe!%_&o&4`Nr7-b?mC%!5=_OZ*eo zU{kCa9|&tuc`|*oh}3bUQi95x>E&_>rra#m=Uf z*X*A-W(v3Nci4SiGfW7`6Cv6v-hc_V1o{{}&3IT@dsD2I2n!;RIoQN6oEpACFAAv)?#8W@6oP>Qcr6Yzs?H}YJLyn8YjttKl418Sl^Pt0RhCEq^ z-=Hmq{^JIntm{F0Nk8kPjt!ks)1L0$V9)bXu;-&jn5 zQ7_BsQv;E&jl?DmdeNZ$Mr(E4lsxB3vEPxv@UXJp=@R?-B-QS8SqD-R)HB)2kaHuq wK1cb?xuYVlD0}OWi@k=xKsRrMbqts{ja3eX&N7)<2tA_Nd@@sbKA3le|O#lD@ delta 2375 zcmZ8he@v8R9REJ=dx!TP7;xSJccQ#-Cf$LFo(k9&F9>U*PCLo|U=CQiM3IxrvX$*U zGe-q(yn8YIs}%k^vRPAF{8Oa;p&4GoL0JNa!0OYMY8seTWj#kdww^MBWCH z`cL!E;T)LL(Hv)S_r14!Gt0;XVz)HYzY}Mhs?cwU2NC+`q z-HUmQY6abVit4)&CKd6+&skuz4eKtMW^oF|v0}|ywaBbht8+mynG#78 zauU(cp#?_tae1dtOWh#tI_&QWG(5K$nh=>4SmB}zTq7Y824+My*5jM3dv^(W25Gi- zO3WS0MT0eHPz8gXXt>Iu)R((R+YgpD6_Td|3~zwFfuxy)Kx$jHSE(C>^DHRt02`cs z?wrIKhOZUAEOk?$H=9J5Gu!rq+B8_>MHivyElkKU*<76d9x#jjTNuu48kl`xb^y%o zWi9S-2dGUzr94TN+8$8*HK^SSYTcl=6*Rg*EpM!A)@B`TOsRhg;`ls{8f@&u_dVcxBi;SJnZ$4I2)VEs8 zB^t^@wlo&B@2~aRAPeX#^LGe|fZm@UWM%q?`AwceFH$*@)7T3}`MC8=KsWQ37RZaW z6vF;^V4mbp`JBF@;0aZ$(5|pVbh?r(d=c{h;6eP)LK0LBy(B!Lxp2D5OEvyxZ;y(} zn=1HEw3PQl`t738wCcxxInR+DoH;Vop`@fkx zH^^2u6g_92>~vA>o*c!vgQB;K$&gSiG*zlE_AFI-G4#QEJ_o3hL$wbF?^$i$MXg0!5Wn#dQi0<`yBVi{^ zxL}iLEGqp<6I{F2_Wt2E%xg3o4a_wo;T=HJLnJTP@D_%eCYNhn$WaZBd2VTa7JQ6m zdB7FZY|FO`o0r&ka}hG;Fttm3GwS;4i@2Oa7kGiHXTAE>MQYVDOKj7XA~=KCf*G+* zS2`bRB{j;l>z7EY(BJSLQ!8wa#iZ44c~G!@$cJryIqy$ zCy=f0Nx40;Q*L=-PprH5MY+58z{>~a-q;~``|6$YhCN;Jm*hk7?t=$+_pX=QSM8LW z&9>*;_71-;1bT(b Temx#oCzV(o5S8ek1S"] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/07_timestamps/README.md b/07_timestamps/README.md index e221a12c3..a5063cb64 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -27,22 +27,35 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 11 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 12 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.543941] Booting on: Raspberry Pi 3 -[ 0.545059] Architectural timer resolution: 52 ns -[ 0.547358] Drivers loaded: -[ 0.548703] 1. BCM GPIO -[ 0.550135] 2. BCM PL011 UART -[W 0.551828] Spin duration smaller than architecturally supported, skipping -[ 0.555212] Spinning for 1 second -[ 1.556818] Spinning for 1 second -[ 2.557690] Spinning for 1 second +[ 0.140431] mingo version 0.7.0 +[ 0.140630] Booting on: Raspberry Pi 3 +[ 0.141085] Architectural timer resolution: 52 ns +[ 0.141660] Drivers loaded: +[ 0.141995] 1. BCM GPIO +[ 0.142353] 2. BCM PL011 UART +[W 0.142777] Spin duration smaller than architecturally supported, skipping +[ 0.143621] Spinning for 1 second +[ 1.144023] Spinning for 1 second +[ 2.144245] Spinning for 1 second ``` ## Diff to previous ```diff + +diff -uNr 06_uart_chainloader/Cargo.toml 07_timestamps/Cargo.toml +--- 06_uart_chainloader/Cargo.toml ++++ 07_timestamps/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.6.0" ++version = "0.7.0" + authors = ["Andre Richter "] + edition = "2018" + Binary files 06_uart_chainloader/demo_payload_rpi3.img and 07_timestamps/demo_payload_rpi3.img differ Binary files 06_uart_chainloader/demo_payload_rpi4.img and 07_timestamps/demo_payload_rpi4.img differ @@ -475,7 +488,7 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs /// Early init code. /// -@@ -147,56 +147,33 @@ +@@ -147,56 +147,38 @@ kernel_main() } @@ -506,8 +519,26 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - // Notify `Minipush` to send the binary. - for _ in 0..3 { - console().write_char(3 as char); -- } ++ info!( ++ "{} version {}", ++ env!("CARGO_PKG_NAME"), ++ env!("CARGO_PKG_VERSION") ++ ); + info!("Booting on: {}", bsp::board_name()); ++ ++ info!( ++ "Architectural timer resolution: {} ns", ++ time::time_manager().resolution().as_nanos() ++ ); ++ ++ info!("Drivers loaded:"); ++ for (i, driver) in bsp::driver::driver_manager() ++ .all_device_drivers() ++ .iter() ++ .enumerate() ++ { ++ info!(" {}. {}", i + 1, driver.compatible()); + } - // Read the binary's size. - let mut size: u32 = u32::from(console().read_char() as u8); @@ -525,28 +556,16 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - for i in 0..size { - core::ptr::write_volatile(kernel_addr.offset(i as isize), console().read_char() as u8) - } -+ info!( -+ "Architectural timer resolution: {} ns", -+ time::time_manager().resolution().as_nanos() -+ ); -+ -+ info!("Drivers loaded:"); -+ for (i, driver) in bsp::driver::driver_manager() -+ .all_device_drivers() -+ .iter() -+ .enumerate() -+ { -+ info!(" {}. {}", i + 1, driver.compatible()); - } +- } ++ // Test a failing timer case. ++ time::time_manager().spin_for(Duration::from_nanos(1)); - println!("[ML] Loaded! Executing the payload now\n"); - console().flush(); - - // Use black magic to create a function pointer. - let kernel: fn() -> ! = unsafe { core::mem::transmute(kernel_addr) }; -+ // Test a failing timer case. -+ time::time_manager().spin_for(Duration::from_nanos(1)); - +- - // Jump to loaded kernel! - kernel() + loop { diff --git a/07_timestamps/src/main.rs b/07_timestamps/src/main.rs index b7a74ea6b..5f465cbfe 100644 --- a/07_timestamps/src/main.rs +++ b/07_timestamps/src/main.rs @@ -153,6 +153,11 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use time::interface::TimeManager; + info!( + "{} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); info!("Booting on: {}", bsp::board_name()); info!( diff --git a/08_hw_debug_JTAG/Cargo.lock b/08_hw_debug_JTAG/Cargo.lock index a3d8bb9ae..011adc0f3 100644 --- a/08_hw_debug_JTAG/Cargo.lock +++ b/08_hw_debug_JTAG/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.8.0" dependencies = [ "cortex-a", "register", diff --git a/08_hw_debug_JTAG/Cargo.toml b/08_hw_debug_JTAG/Cargo.toml index df5f7f974..e84540cac 100644 --- a/08_hw_debug_JTAG/Cargo.toml +++ b/08_hw_debug_JTAG/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.8.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/08_hw_debug_JTAG/README.md b/08_hw_debug_JTAG/README.md index 03d16a4f0..a30a296ed 100644 --- a/08_hw_debug_JTAG/README.md +++ b/08_hw_debug_JTAG/README.md @@ -305,6 +305,18 @@ Thanks to [@naotaco](https://github.com/naotaco) for laying the groundwork for t ## Diff to previous ```diff +diff -uNr 07_timestamps/Cargo.toml 08_hw_debug_JTAG/Cargo.toml +--- 07_timestamps/Cargo.toml ++++ 08_hw_debug_JTAG/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.7.0" ++version = "0.8.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile --- 07_timestamps/Makefile +++ 08_hw_debug_JTAG/Makefile diff --git a/08_hw_debug_JTAG/src/main.rs b/08_hw_debug_JTAG/src/main.rs index b7a74ea6b..5f465cbfe 100644 --- a/08_hw_debug_JTAG/src/main.rs +++ b/08_hw_debug_JTAG/src/main.rs @@ -153,6 +153,11 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use time::interface::TimeManager; + info!( + "{} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); info!("Booting on: {}", bsp::board_name()); info!( diff --git a/09_privilege_level/Cargo.lock b/09_privilege_level/Cargo.lock index a3d8bb9ae..753961f7f 100644 --- a/09_privilege_level/Cargo.lock +++ b/09_privilege_level/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.9.0" dependencies = [ "cortex-a", "register", diff --git a/09_privilege_level/Cargo.toml b/09_privilege_level/Cargo.toml index df5f7f974..f6750ed49 100644 --- a/09_privilege_level/Cargo.toml +++ b/09_privilege_level/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.9.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index b786bf33c..869c39600 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -177,27 +177,40 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 13 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 +[MP] ⏩ Pushing 14 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.637617] Booting on: Raspberry Pi 3 -[ 0.638737] Current privilege level: EL1 -[ 0.640645] Exception handling state: -[ 0.642424] Debug: Masked -[ 0.643986] SError: Masked -[ 0.645548] IRQ: Masked -[ 0.647110] FIQ: Masked -[ 0.648672] Architectural timer resolution: 52 ns -[ 0.650971] Drivers loaded: -[ 0.652316] 1. BCM GPIO -[ 0.653748] 2. BCM PL011 UART -[ 0.655440] Timer test, spinning for 1 second -[ 1.657567] Echoing input now +[ 0.165757] mingo version 0.9.0 +[ 0.165957] Booting on: Raspberry Pi 3 +[ 0.166412] Current privilege level: EL1 +[ 0.166888] Exception handling state: +[ 0.167333] Debug: Masked +[ 0.167723] SError: Masked +[ 0.168112] IRQ: Masked +[ 0.168502] FIQ: Masked +[ 0.168893] Architectural timer resolution: 52 ns +[ 0.169467] Drivers loaded: +[ 0.169803] 1. BCM GPIO +[ 0.170160] 2. BCM PL011 UART +[ 0.170583] Timer test, spinning for 1 second +[ 1.171115] Echoing input now ``` ## Diff to previous ```diff +diff -uNr 08_hw_debug_JTAG/Cargo.toml 09_privilege_level/Cargo.toml +--- 08_hw_debug_JTAG/Cargo.toml ++++ 09_privilege_level/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.8.0" ++version = "0.9.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/_arch/aarch64/cpu/boot.rs --- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ 09_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -494,7 +507,7 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs mod memory; mod panic_wait; mod print; -@@ -149,12 +150,20 @@ +@@ -149,6 +150,8 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -503,7 +516,8 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs use core::time::Duration; use driver::interface::DriverManager; use time::interface::TimeManager; - +@@ -160,6 +163,12 @@ + ); info!("Booting on: {}", bsp::board_name()); + let (_, privilege_level) = exception::current_privilege_level(); @@ -515,7 +529,7 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -169,11 +178,15 @@ +@@ -174,11 +183,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/09_privilege_level/src/main.rs b/09_privilege_level/src/main.rs index 0c3a06489..83d96ca8a 100644 --- a/09_privilege_level/src/main.rs +++ b/09_privilege_level/src/main.rs @@ -156,6 +156,11 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use time::interface::TimeManager; + info!( + "{} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); info!("Booting on: {}", bsp::board_name()); let (_, privilege_level) = exception::current_privilege_level(); diff --git a/10_virtual_mem_part1_identity_mapping/Cargo.lock b/10_virtual_mem_part1_identity_mapping/Cargo.lock index a3d8bb9ae..5022ffadb 100644 --- a/10_virtual_mem_part1_identity_mapping/Cargo.lock +++ b/10_virtual_mem_part1_identity_mapping/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.10.0" dependencies = [ "cortex-a", "register", diff --git a/10_virtual_mem_part1_identity_mapping/Cargo.toml b/10_virtual_mem_part1_identity_mapping/Cargo.toml index df5f7f974..8e54f2cdb 100644 --- a/10_virtual_mem_part1_identity_mapping/Cargo.toml +++ b/10_virtual_mem_part1_identity_mapping/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.10.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index 3161e6b01..ba2932a44 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -322,32 +322,45 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 +[MP] ⏩ Pushing 64 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.175017] Booting on: Raspberry Pi 3 -[ 3.176100] MMU online. Special regions: -[ 3.178009] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.182088] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO -[ 3.186036] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.189594] Current privilege level: EL1 -[ 3.191502] Exception handling state: -[ 3.193281] Debug: Masked -[ 3.194843] SError: Masked -[ 3.196405] IRQ: Masked -[ 3.197967] FIQ: Masked -[ 3.199529] Architectural timer resolution: 52 ns -[ 3.201828] Drivers loaded: -[ 3.203173] 1. BCM GPIO -[ 3.204605] 2. BCM PL011 UART -[ 3.206297] Timer test, spinning for 1 second +[ 1.034062] mingo version 0.10.0 +[ 1.034270] Booting on: Raspberry Pi 3 +[ 1.034725] MMU online. Special regions: +[ 1.035201] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 1.036220] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO +[ 1.037205] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 1.038094] Current privilege level: EL1 +[ 1.038570] Exception handling state: +[ 1.039015] Debug: Masked +[ 1.039405] SError: Masked +[ 1.039794] IRQ: Masked +[ 1.040184] FIQ: Masked +[ 1.040575] Architectural timer resolution: 52 ns +[ 1.041148] Drivers loaded: +[ 1.041484] 1. BCM GPIO +[ 1.041842] 2. BCM PL011 UART +[ 1.042264] Timer test, spinning for 1 second [ !!! ] Writing through the remapped UART at 0x1FFF_1000 -[ 4.210458] Echoing input now +[ 2.043305] Echoing input now ``` ## Diff to previous ```diff +diff -uNr 09_privilege_level/Cargo.toml 10_virtual_mem_part1_identity_mapping/Cargo.toml +--- 09_privilege_level/Cargo.toml ++++ 10_virtual_mem_part1_identity_mapping/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.9.0" ++version = "0.10.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs --- 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -1058,8 +1071,8 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -158,6 +170,9 @@ - +@@ -163,6 +175,9 @@ + ); info!("Booting on: {}", bsp::board_name()); + info!("MMU online. Special regions:"); @@ -1068,7 +1081,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -181,6 +196,13 @@ +@@ -186,6 +201,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/10_virtual_mem_part1_identity_mapping/src/main.rs b/10_virtual_mem_part1_identity_mapping/src/main.rs index 8ce5d3d92..74d49754a 100644 --- a/10_virtual_mem_part1_identity_mapping/src/main.rs +++ b/10_virtual_mem_part1_identity_mapping/src/main.rs @@ -168,6 +168,11 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use time::interface::TimeManager; + info!( + "{} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); diff --git a/11_exceptions_part1_groundwork/Cargo.lock b/11_exceptions_part1_groundwork/Cargo.lock index a3d8bb9ae..c956c58cd 100644 --- a/11_exceptions_part1_groundwork/Cargo.lock +++ b/11_exceptions_part1_groundwork/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.11.0" dependencies = [ "cortex-a", "register", diff --git a/11_exceptions_part1_groundwork/Cargo.toml b/11_exceptions_part1_groundwork/Cargo.toml index df5f7f974..764939bf2 100644 --- a/11_exceptions_part1_groundwork/Cargo.toml +++ b/11_exceptions_part1_groundwork/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.11.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/11_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md index 878ea4362..f9c70824b 100644 --- a/11_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -406,32 +406,33 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 +[MP] ⏩ Pushing 64 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.091032] Booting on: Raspberry Pi 3 -[ 3.092116] MMU online. Special regions: -[ 3.094025] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.098103] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.101661] Current privilege level: EL1 -[ 3.103570] Exception handling state: -[ 3.105348] Debug: Masked -[ 3.106910] SError: Masked -[ 3.108472] IRQ: Masked -[ 3.110034] FIQ: Masked -[ 3.111596] Architectural timer resolution: 52 ns -[ 3.113895] Drivers loaded: -[ 3.115240] 1. BCM GPIO -[ 3.116672] 2. BCM PL011 UART -[ 3.118364] Timer test, spinning for 1 second -[ 4.120490] -[ 4.120494] Trying to read from address 8 GiB... -[ 4.122700] ************************************************ -[ 4.125476] Whoa! We recovered from a synchronous exception! -[ 4.128253] ************************************************ -[ 4.131030] -[ 4.131724] Let's try again -[ 4.133069] Trying to read from address 9 GiB... +[ 0.980247] mingo version 0.11.0 +[ 0.980454] Booting on: Raspberry Pi 3 +[ 0.980909] MMU online. Special regions: +[ 0.981386] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 0.982404] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 0.983293] Current privilege level: EL1 +[ 0.983769] Exception handling state: +[ 0.984213] Debug: Masked +[ 0.984604] SError: Masked +[ 0.984993] IRQ: Masked +[ 0.985383] FIQ: Masked +[ 0.985773] Architectural timer resolution: 52 ns +[ 0.986347] Drivers loaded: +[ 0.986684] 1. BCM GPIO +[ 0.987041] 2. BCM PL011 UART +[ 0.987463] Timer test, spinning for 1 second +[ 1.987994] +[ 1.987998] Trying to read from address 8 GiB... +[ 1.988547] ************************************************ +[ 1.989240] Whoa! We recovered from a synchronous exception! +[ 1.989933] ************************************************ +[ 1.990627] +[ 1.990800] Let's try again +[ 1.991136] Trying to read from address 9 GiB... Kernel panic: @@ -440,7 +441,7 @@ FAR_EL1: 0x0000000240000000 ESR_EL1: 0x96000004 Exception Class (EC) : 0x25 - Data Abort, current EL Instr Specific Syndrome (ISS): 0x4 -ELR_EL1: 0x0000000000081458 +ELR_EL1: 0x0000000000082578 SPSR_EL1: 0x600003c5 Flags: Negative (N): Not set @@ -455,27 +456,39 @@ SPSR_EL1: 0x600003c5 Illegal Execution State (IL): Not set General purpose register: - x0 : 0x0000000000000000 x1 : 0x0000000000085727 - x2 : 0x0000000000000027 x3 : 0x0000000000000000 - x4 : 0x0000000000000002 x5 : 0x3f27329c00000000 - x6 : 0x0000000000000000 x7 : 0xdbd1b90800000000 + x0 : 0x0000000000000000 x1 : 0x00000000000858c7 + x2 : 0x0000000000000027 x3 : 0x0000000000084cc4 + x4 : 0x0000000000000003 x5 : 0x3f27329400000000 + x6 : 0x0000000000000000 x7 : 0xd3d0b80800000000 x8 : 0x0000000240000000 x9 : 0x000000003f201000 - x10: 0x0000000000000019 x11: 0x00000000000819d0 - x12: 0x0000000000000000 x13: 0x0000000000000033 + x10: 0x0000000000000019 x11: 0x0000000000000000 + x12: 0x0000000000000001 x13: 0x0000000000000036 x14: 0x000000000007fc2d x15: 0x0000000000000000 - x16: 0x0000000000000040 x17: 0xfd7f702255f847d0 - x18: 0x0000000000000003 x19: 0x0000000000090008 - x20: 0x0000000000085510 x21: 0x000000003b9aca00 - x22: 0x00000000000003e8 x23: 0x0000000000081610 - x24: 0x0000000000082268 x25: 0x00000000000f4240 - x26: 0xffffffffc4653600 x27: 0x00000000000855f0 - x28: 0x0000000000083f80 x29: 0x0000000000086810 - lr : 0x000000000008144c + x16: 0x0000000000000040 x17: 0xfd39702255e846c0 + x18: 0x9cd47880832f1200 x19: 0x0000000000090008 + x20: 0x00000000000856b0 x21: 0x000000003b9aca00 + x22: 0x000000000008274c x23: 0x00000000000833d8 + x24: 0x00000000000003e8 x25: 0xffffffffc4653600 + x26: 0x00000000000f4240 x27: 0x0000000000085790 + x28: 0x0000000000086a50 x29: 0x00000000000867ed + lr : 0x000000000008256c ``` ## Diff to previous ```diff +diff -uNr 10_virtual_mem_part1_identity_mapping/Cargo.toml 11_exceptions_part1_groundwork/Cargo.toml +--- 10_virtual_mem_part1_identity_mapping/Cargo.toml ++++ 11_exceptions_part1_groundwork/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.10.0" ++version = "0.11.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs --- 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -963,7 +976,7 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } -@@ -196,13 +198,28 @@ +@@ -201,13 +203,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_exceptions_part1_groundwork/src/main.rs b/11_exceptions_part1_groundwork/src/main.rs index 2af372ab9..a1c5bc150 100644 --- a/11_exceptions_part1_groundwork/src/main.rs +++ b/11_exceptions_part1_groundwork/src/main.rs @@ -170,6 +170,11 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use time::interface::TimeManager; + info!( + "{} version {}", + env!("CARGO_PKG_NAME"), + env!("CARGO_PKG_VERSION") + ); info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); diff --git a/12_integrated_testing/Cargo.lock b/12_integrated_testing/Cargo.lock index 069559203..9dac661b5 100644 --- a/12_integrated_testing/Cargo.lock +++ b/12_integrated_testing/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.12.0" dependencies = [ "cortex-a", "qemu-exit", diff --git a/12_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml index 06c10cf07..4bb92ff47 100644 --- a/12_integrated_testing/Cargo.toml +++ b/12_integrated_testing/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.12.0" authors = ["Andre Richter "] edition = "2018" @@ -45,6 +45,7 @@ test = true # Disable unit tests for the kernel binary. [[bin]] name = "kernel" +path = "src/main.rs" test = false # List of tests without harness. diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index 600efa2a8..4a5940d1a 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -738,9 +738,8 @@ Believe it or not, that is all. There are three ways you can run tests: ```console $ make test [...] -RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D warnings -D missing_docs" cargo test --target=aarch64-unknown-none-softfloat --features bsp_rpi3 --release - Finished release [optimized] target(s) in 0.01s - Running target/aarch64-unknown-none-softfloat/release/deps/libkernel-4cc6412ddf631982 + + Running unittests (target/aarch64-unknown-none-softfloat/release/deps/libkernel-836110ac5dd535ba) ------------------------------------------------------------------- 🦀 Running 8 tests ------------------------------------------------------------------- @@ -748,9 +747,9 @@ RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D 1. virt_mem_layout_sections_are_64KiB_aligned................[ok] 2. virt_mem_layout_has_no_overlaps...........................[ok] 3. test_runner_executes_in_kernel_mode.......................[ok] - 4. size_of_tabledescriptor_equals_64_bit.....................[ok] - 5. size_of_pagedescriptor_equals_64_bit......................[ok] - 6. kernel_tables_in_bss......................................[ok] + 4. kernel_tables_in_bss......................................[ok] + 5. size_of_tabledescriptor_equals_64_bit.....................[ok] + 6. size_of_pagedescriptor_equals_64_bit......................[ok] 7. zero_volatile_works.......................................[ok] 8. bss_section_is_sane.......................................[ok] @@ -759,7 +758,7 @@ RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D ------------------------------------------------------------------- - Running target/aarch64-unknown-none-softfloat/release/deps/00_console_sanity-557819b436f15a18 + Running tests/00_console_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/00_console_sanity-78c12c5472d40df7) ------------------------------------------------------------------- 🦀 Running 3 console-based tests ------------------------------------------------------------------- @@ -773,7 +772,7 @@ RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D ------------------------------------------------------------------- - Running target/aarch64-unknown-none-softfloat/release/deps/01_timer_sanity-1e25e7d559a9009f + Running tests/01_timer_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/01_timer_sanity-4866734b14c83c9b) ------------------------------------------------------------------- 🦀 Running 3 tests ------------------------------------------------------------------- @@ -787,7 +786,7 @@ RUSTFLAGS="-C link-arg=-Tsrc/bsp/raspberrypi/link.ld -C target-cpu=cortex-a53 -D ------------------------------------------------------------------- - Running target/aarch64-unknown-none-softfloat/release/deps/02_exception_sync_page_fault-14172ce39b3fae1c + Running tests/02_exception_sync_page_fault.rs (target/aarch64-unknown-none-softfloat/release/deps/02_exception_sync_page_fault-f2d0885cada1105b) ------------------------------------------------------------------- 🦀 Testing synchronous exception handling by causing a page fault ------------------------------------------------------------------- @@ -819,7 +818,11 @@ diff -uNr 11_exceptions_part1_groundwork/.cargo/config.toml 12_integrated_testin diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo.toml --- 11_exceptions_part1_groundwork/Cargo.toml +++ 12_integrated_testing/Cargo.toml -@@ -4,24 +4,54 @@ +@@ -1,31 +1,58 @@ + [package] + name = "mingo" +-version = "0.11.0" ++version = "0.12.0" authors = ["Andre Richter "] edition = "2018" @@ -833,6 +836,10 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +- +-[[bin]] +-name = "kernel" +-path = "src/main.rs" +test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -866,6 +873,7 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. +# Disable unit tests for the kernel binary. +[[bin]] +name = "kernel" ++path = "src/main.rs" +test = false + +# List of tests without harness. @@ -1205,7 +1213,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/exception.rs 12_integrated_testing/ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/lib.rs --- 11_exceptions_part1_groundwork/src/lib.rs +++ 12_integrated_testing/src/lib.rs -@@ -0,0 +1,171 @@ +@@ -0,0 +1,184 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1348,6 +1356,19 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li +pub mod time; + +//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Version string. ++pub fn version() -> &'static str { ++ concat!( ++ env!("CARGO_PKG_NAME"), ++ " version ", ++ env!("CARGO_PKG_VERSION") ++ ) ++} ++ ++//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- + @@ -1522,7 +1543,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -166,9 +49,7 @@ +@@ -166,15 +49,9 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1530,9 +1551,16 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m use driver::interface::DriverManager; - use time::interface::TimeManager; +- info!( +- "{} version {}", +- env!("CARGO_PKG_NAME"), +- env!("CARGO_PKG_VERSION") +- ); ++ info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); -@@ -195,31 +76,6 @@ + info!("MMU online. Special regions:"); +@@ -200,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/12_integrated_testing/src/lib.rs b/12_integrated_testing/src/lib.rs index 745122f85..dc991fd12 100644 --- a/12_integrated_testing/src/lib.rs +++ b/12_integrated_testing/src/lib.rs @@ -139,6 +139,19 @@ pub mod memory; pub mod print; pub mod time; +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Version string. +pub fn version() -> &'static str { + concat!( + env!("CARGO_PKG_NAME"), + " version ", + env!("CARGO_PKG_VERSION") + ) +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- diff --git a/12_integrated_testing/src/main.rs b/12_integrated_testing/src/main.rs index 77c9d494b..1daeb4038 100644 --- a/12_integrated_testing/src/main.rs +++ b/12_integrated_testing/src/main.rs @@ -51,6 +51,7 @@ fn kernel_main() -> ! { use console::interface::All; use driver::interface::DriverManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); diff --git a/13_exceptions_part2_peripheral_IRQs/Cargo.lock b/13_exceptions_part2_peripheral_IRQs/Cargo.lock index 069559203..922341af2 100644 --- a/13_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/13_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.13.0" dependencies = [ "cortex-a", "qemu-exit", diff --git a/13_exceptions_part2_peripheral_IRQs/Cargo.toml b/13_exceptions_part2_peripheral_IRQs/Cargo.toml index 206dfc878..343bdfda3 100644 --- a/13_exceptions_part2_peripheral_IRQs/Cargo.toml +++ b/13_exceptions_part2_peripheral_IRQs/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.13.0" authors = ["Andre Richter "] edition = "2018" @@ -43,6 +43,7 @@ test = true # Disable unit tests for the kernel binary. [[bin]] name = "kernel" +path = "src/main.rs" test = false # List of tests without harness. diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 18b3cf812..4cb82c6b4 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -675,28 +675,29 @@ Minipush 1.0 Raspberry Pi 3 [ML] Requesting binary -[MP] ⏩ Pushing 66 KiB ========================================🦀 100% 33 KiB/s Time: 00:00:02 +[MP] ⏩ Pushing 66 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.203172] Booting on: Raspberry Pi 3 -[ 3.204255] MMU online. Special regions: -[ 3.206164] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.210242] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO -[ 3.213800] Current privilege level: EL1 -[ 3.215709] Exception handling state: -[ 3.217487] Debug: Masked -[ 3.219049] SError: Masked -[ 3.220611] IRQ: Unmasked -[ 3.222260] FIQ: Masked -[ 3.223822] Architectural timer resolution: 52 ns -[ 3.226121] Drivers loaded: -[ 3.227466] 1. BCM GPIO -[ 3.228898] 2. BCM PL011 UART -[ 3.230590] 3. BCM Interrupt Controller -[ 3.232716] Registered IRQ handlers: -[ 3.234451] Peripheral handler: -[ 3.236232] 57. BCM PL011 UART -[ 3.238269] Echoing input now +[ 1.010579] mingo version 0.13.0 +[ 1.010787] Booting on: Raspberry Pi 3 +[ 1.011242] MMU online. Special regions: +[ 1.011718] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 1.012737] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 1.013625] Current privilege level: EL1 +[ 1.014102] Exception handling state: +[ 1.014546] Debug: Masked +[ 1.014936] SError: Masked +[ 1.015326] IRQ: Unmasked +[ 1.015738] FIQ: Masked +[ 1.016127] Architectural timer resolution: 52 ns +[ 1.016702] Drivers loaded: +[ 1.017038] 1. BCM GPIO +[ 1.017395] 2. BCM PL011 UART +[ 1.017817] 3. BCM Interrupt Controller +[ 1.018348] Registered IRQ handlers: +[ 1.018782] Peripheral handler: +[ 1.019228] 57. BCM PL011 UART +[ 1.019735] Echoing input now ``` Raspberry Pi 4: @@ -717,28 +718,29 @@ Minipush 1.0 Raspberry Pi 4 [ML] Requesting binary -[MP] ⏩ Pushing 73 KiB ========================================🦀 100% 24 KiB/s Time: 00:00:03 +[MP] ⏩ Pushing 73 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 3.486234] Booting on: Raspberry Pi 4 -[ 3.486623] MMU online. Special regions: -[ 3.488532] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 3.492610] 0xfe000000 - 0xff84ffff | 24 MiB | Dev RW PXN | Device MMIO -[ 3.496167] Current privilege level: EL1 -[ 3.498076] Exception handling state: -[ 3.499855] Debug: Masked -[ 3.501417] SError: Masked -[ 3.502979] IRQ: Unmasked -[ 3.504628] FIQ: Masked -[ 3.506189] Architectural timer resolution: 18 ns -[ 3.508489] Drivers loaded: -[ 3.509834] 1. BCM GPIO -[ 3.511266] 2. BCM PL011 UART -[ 3.512958] 3. GICv2 (ARM Generic Interrupt Controller v2) -[ 3.515908] Registered IRQ handlers: -[ 3.517643] Peripheral handler: -[ 3.519425] 153. BCM PL011 UART -[ 3.521463] Echoing input now +[ 1.030536] mingo version 0.13.0 +[ 1.030569] Booting on: Raspberry Pi 4 +[ 1.031024] MMU online. Special regions: +[ 1.031501] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 1.032519] 0xfe000000 - 0xff84ffff | 24 MiB | Dev RW PXN | Device MMIO +[ 1.033408] Current privilege level: EL1 +[ 1.033884] Exception handling state: +[ 1.034328] Debug: Masked +[ 1.034718] SError: Masked +[ 1.035108] IRQ: Unmasked +[ 1.035520] FIQ: Masked +[ 1.035910] Architectural timer resolution: 18 ns +[ 1.036484] Drivers loaded: +[ 1.036820] 1. BCM GPIO +[ 1.037178] 2. BCM PL011 UART +[ 1.037600] 3. GICv2 (ARM Generic Interrupt Controller v2) +[ 1.038337] Registered IRQ handlers: +[ 1.038770] Peripheral handler: +[ 1.039217] 153. BCM PL011 UART +[ 1.039725] Echoing input now ``` ## Diff to previous @@ -747,7 +749,11 @@ Minipush 1.0 diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/Cargo.toml --- 12_integrated_testing/Cargo.toml +++ 13_exceptions_part2_peripheral_IRQs/Cargo.toml -@@ -4,10 +4,8 @@ +@@ -1,13 +1,11 @@ + [package] + name = "mingo" +-version = "0.12.0" ++version = "0.13.0" authors = ["Andre Richter "] edition = "2018" @@ -2444,9 +2450,9 @@ diff -uNr 12_integrated_testing/src/main.rs 13_exceptions_part2_peripheral_IRQs/ use driver::interface::DriverManager; + use exception::asynchronous::interface::IRQManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); - -@@ -76,12 +88,9 @@ +@@ -77,12 +89,9 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/13_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs index c4e744608..3cc973299 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -141,6 +141,19 @@ pub mod print; pub mod state; pub mod time; +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Version string. +pub fn version() -> &'static str { + concat!( + env!("CARGO_PKG_NAME"), + " version ", + env!("CARGO_PKG_VERSION") + ) +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- diff --git a/13_exceptions_part2_peripheral_IRQs/src/main.rs b/13_exceptions_part2_peripheral_IRQs/src/main.rs index 6ec5a670c..1dde38845 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/main.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/main.rs @@ -63,6 +63,7 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use exception::asynchronous::interface::IRQManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); diff --git a/14_virtual_mem_part2_mmio_remap/Cargo.lock b/14_virtual_mem_part2_mmio_remap/Cargo.lock index 069559203..e1591cd42 100644 --- a/14_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/14_virtual_mem_part2_mmio_remap/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.14.0" dependencies = [ "cortex-a", "qemu-exit", diff --git a/14_virtual_mem_part2_mmio_remap/Cargo.toml b/14_virtual_mem_part2_mmio_remap/Cargo.toml index 206dfc878..ad9a78151 100644 --- a/14_virtual_mem_part2_mmio_remap/Cargo.toml +++ b/14_virtual_mem_part2_mmio_remap/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.14.0" authors = ["Andre Richter "] edition = "2018" @@ -43,6 +43,7 @@ test = true # Disable unit tests for the kernel binary. [[bin]] name = "kernel" +path = "src/main.rs" test = false # List of tests without harness. diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 4c1c940c3..95042ed3a 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -316,18 +316,19 @@ Minipush 1.0 [MP] ⏩ Pushing 67 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.789721] Booting on: Raspberry Pi 3 -[ 0.789994] MMU online: -[ 0.790286] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.792030] Virtual Physical Size Attr Entity -[ 0.793774] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.795520] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 0.797134] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss -[ 0.798704] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack -[ 0.800307] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 0.801759] | BCM PL011 UART -[ 0.803276] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 0.805020] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.785711] mingo version 0.14.0 +[ 0.785919] Booting on: Raspberry Pi 3 +[ 0.786374] MMU online: +[ 0.786666] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.788410] Virtual Physical Size Attr Entity +[ 0.790154] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.791902] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.793515] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.795085] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 0.796688] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.798139] | BCM PL011 UART +[ 0.799657] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 0.801400] ------------------------------------------------------------------------------------------------------------------------------------------- ``` Raspberry Pi 4: @@ -351,24 +352,37 @@ Minipush 1.0 [MP] ⏩ Pushing 74 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00 [ML] Loaded! Executing the payload now -[ 0.870371] Booting on: Raspberry Pi 4 -[ 0.870470] MMU online: -[ 0.870763] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.872507] Virtual Physical Size Attr Entity -[ 0.874251] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 0.875996] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 0.877611] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss -[ 0.879181] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack -[ 0.880784] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 0.882235] | BCM PL011 UART -[ 0.883752] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD -[ 0.885160] | GICC -[ 0.886569] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.869392] mingo version 0.14.0 +[ 0.869425] Booting on: Raspberry Pi 4 +[ 0.869880] MMU online: +[ 0.870173] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.871917] Virtual Physical Size Attr Entity +[ 0.873661] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 0.875407] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 0.877021] 0x0000_0000_0009_0000..0x0000_0000_001b_ffff --> 0x00_0009_0000..0x00_001b_ffff | 1 MiB | C RW XN | Kernel data and bss +[ 0.878591] 0x0000_0000_001d_0000..0x0000_0000_0024_ffff --> 0x00_001d_0000..0x00_0024_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 0.880195] 0x0000_0001_f000_0000..0x0000_0001_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 0.881645] | BCM PL011 UART +[ 0.883163] 0x0000_0001_f001_0000..0x0000_0001_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD +[ 0.884570] | GICC +[ 0.885979] ------------------------------------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous ```diff +diff -uNr 13_exceptions_part2_peripheral_IRQs/Cargo.toml 14_virtual_mem_part2_mmio_remap/Cargo.toml +--- 13_exceptions_part2_peripheral_IRQs/Cargo.toml ++++ 14_virtual_mem_part2_mmio_remap/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.13.0" ++version = "0.14.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs --- 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -2184,8 +2198,8 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/main.rs 14_virtual_mem_part2_m // Let device drivers register and enable their handlers with the interrupt controller. for i in bsp::driver::driver_manager().all_device_drivers() { -@@ -65,8 +83,8 @@ - +@@ -66,8 +84,8 @@ + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); - info!("MMU online. Special regions:"); diff --git a/14_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs index 3346134ad..afa495060 100644 --- a/14_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -144,6 +144,19 @@ pub mod print; pub mod state; pub mod time; +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Version string. +pub fn version() -> &'static str { + concat!( + env!("CARGO_PKG_NAME"), + " version ", + env!("CARGO_PKG_VERSION") + ) +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- diff --git a/14_virtual_mem_part2_mmio_remap/src/main.rs b/14_virtual_mem_part2_mmio_remap/src/main.rs index 0dd1c28d2..3052db666 100644 --- a/14_virtual_mem_part2_mmio_remap/src/main.rs +++ b/14_virtual_mem_part2_mmio_remap/src/main.rs @@ -81,6 +81,7 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use exception::asynchronous::interface::IRQManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); info!("MMU online:"); diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.lock b/15_virtual_mem_part3_precomputed_tables/Cargo.lock index 069559203..62c78322e 100644 --- a/15_virtual_mem_part3_precomputed_tables/Cargo.lock +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.15.0" dependencies = [ "cortex-a", "qemu-exit", diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.toml b/15_virtual_mem_part3_precomputed_tables/Cargo.toml index 206dfc878..4449da657 100644 --- a/15_virtual_mem_part3_precomputed_tables/Cargo.toml +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.15.0" authors = ["Andre Richter "] edition = "2018" @@ -43,6 +43,7 @@ test = true # Disable unit tests for the kernel binary. [[bin]] name = "kernel" +path = "src/main.rs" test = false # List of tests without harness. diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index 5329426a0..459b65d25 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -743,23 +743,36 @@ Minipush 1.0 [MP] ⏩ Pushing 387 KiB =======================================🦀 100% 96 KiB/s Time: 00:00:04 [ML] Loaded! Executing the payload now -[ 4.319874] Booting on: Raspberry Pi 3 -[ 4.320147] MMU online: -[ 4.320439] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 4.322183] Virtual Physical Size Attr Entity -[ 4.323927] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 4.325674] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 4.327288] 0x0000_0000_0009_0000..0x0000_0000_000e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss -[ 4.328858] 0x0000_0000_0010_0000..0x0000_0000_0017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack -[ 4.330461] 0x0000_0000_7000_0000..0x0000_0000_7000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 4.331912] | BCM PL011 UART -[ 4.333430] 0x0000_0000_7001_0000..0x0000_0000_7001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 4.335173] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.324361] mingo version 0.15.0 +[ 4.324568] Booting on: Raspberry Pi 3 +[ 4.325023] MMU online: +[ 4.325316] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.327060] Virtual Physical Size Attr Entity +[ 4.328804] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.330551] 0x0000_0000_0008_0000..0x0000_0000_0008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 4.332164] 0x0000_0000_0009_0000..0x0000_0000_000e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss +[ 4.333735] 0x0000_0000_0010_0000..0x0000_0000_0017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 4.335338] 0x0000_0000_7000_0000..0x0000_0000_7000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 4.336788] | BCM PL011 UART +[ 4.338306] 0x0000_0000_7001_0000..0x0000_0000_7001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 4.340050] ------------------------------------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous ```diff +diff -uNr 14_virtual_mem_part2_mmio_remap/Cargo.toml 15_virtual_mem_part3_precomputed_tables/Cargo.toml +--- 14_virtual_mem_part2_mmio_remap/Cargo.toml ++++ 15_virtual_mem_part3_precomputed_tables/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.14.0" ++version = "0.15.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 14_virtual_mem_part2_mmio_remap/Makefile 15_virtual_mem_part3_precomputed_tables/Makefile --- 14_virtual_mem_part2_mmio_remap/Makefile +++ 15_virtual_mem_part3_precomputed_tables/Makefile diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs index 3346134ad..afa495060 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -144,6 +144,19 @@ pub mod print; pub mod state; pub mod time; +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Version string. +pub fn version() -> &'static str { + concat!( + env!("CARGO_PKG_NAME"), + " version ", + env!("CARGO_PKG_VERSION") + ) +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- diff --git a/15_virtual_mem_part3_precomputed_tables/src/main.rs b/15_virtual_mem_part3_precomputed_tables/src/main.rs index c317bb96f..421dcbec2 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/main.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/main.rs @@ -72,6 +72,7 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use exception::asynchronous::interface::IRQManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); info!("MMU online:"); diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock index 069559203..ba2e9eb5b 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.16.0" dependencies = [ "cortex-a", "qemu-exit", diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.toml b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml index 206dfc878..f7147ecd3 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Cargo.toml +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.16.0" authors = ["Andre Richter "] edition = "2018" @@ -43,6 +43,7 @@ test = true # Disable unit tests for the kernel binary. [[bin]] name = "kernel" +path = "src/main.rs" test = false # List of tests without harness. diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index ddce63fc5..4f7014423 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -163,18 +163,19 @@ Minipush 1.0 [MP] ⏩ Pushing 387 KiB =======================================🦀 100% 96 KiB/s Time: 00:00:04 [ML] Loaded! Executing the payload now -[ 4.320829] Booting on: Raspberry Pi 3 -[ 4.321102] MMU online: -[ 4.321394] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 4.323138] Virtual Physical Size Attr Entity -[ 4.324882] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 4.326629] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 4.328243] 0xffff_ffff_8009_0000..0xffff_ffff_800e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss -[ 4.329813] 0xffff_ffff_8010_0000..0xffff_ffff_8017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack -[ 4.331416] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 4.332867] | BCM PL011 UART -[ 4.334385] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller -[ 4.336128] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.316420] mingo version 0.16.0 +[ 4.316627] Booting on: Raspberry Pi 3 +[ 4.317082] MMU online: +[ 4.317375] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.319119] Virtual Physical Size Attr Entity +[ 4.320863] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 4.322610] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 4.324223] 0xffff_ffff_8009_0000..0xffff_ffff_800e_ffff --> 0x00_0009_0000..0x00_000e_ffff | 384 KiB | C RW XN | Kernel data and bss +[ 4.325793] 0xffff_ffff_8010_0000..0xffff_ffff_8017_ffff --> 0x00_0010_0000..0x00_0017_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 4.327397] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_3f20_0000..0x00_3f20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 4.328847] | BCM PL011 UART +[ 4.330365] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_3f00_0000..0x00_3f00_ffff | 64 KiB | Dev RW XN | BCM Peripheral Interrupt Controller +[ 4.332108] ------------------------------------------------------------------------------------------------------------------------------------------- ``` Raspberry Pi 4: @@ -211,24 +212,37 @@ Minipush 1.0 [MP] ⏩ Pushing 449 KiB ======================================🦀 100% 112 KiB/s Time: 00:00:04 [ML] Loaded! Executing the payload now -[ 5.011490] Booting on: Raspberry Pi 4 -[ 5.011588] MMU online: -[ 5.011881] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 5.013625] Virtual Physical Size Attr Entity -[ 5.015369] ------------------------------------------------------------------------------------------------------------------------------------------- -[ 5.017115] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data -[ 5.018730] 0xffff_ffff_8009_0000..0xffff_ffff_800f_ffff --> 0x00_0009_0000..0x00_000f_ffff | 448 KiB | C RW XN | Kernel data and bss -[ 5.020299] 0xffff_ffff_8011_0000..0xffff_ffff_8018_ffff --> 0x00_0011_0000..0x00_0018_ffff | 512 KiB | C RW XN | Kernel boot-core stack -[ 5.021903] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO -[ 5.023354] | BCM PL011 UART -[ 5.024871] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD -[ 5.026279] | GICC -[ 5.027687] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 5.009551] mingo version 0.16.0 +[ 5.009585] Booting on: Raspberry Pi 4 +[ 5.010040] MMU online: +[ 5.010332] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 5.012077] Virtual Physical Size Attr Entity +[ 5.013821] ------------------------------------------------------------------------------------------------------------------------------------------- +[ 5.015566] 0xffff_ffff_8008_0000..0xffff_ffff_8008_ffff --> 0x00_0008_0000..0x00_0008_ffff | 64 KiB | C RO X | Kernel code and RO data +[ 5.017181] 0xffff_ffff_8009_0000..0xffff_ffff_800f_ffff --> 0x00_0009_0000..0x00_000f_ffff | 448 KiB | C RW XN | Kernel data and bss +[ 5.018751] 0xffff_ffff_8011_0000..0xffff_ffff_8018_ffff --> 0x00_0011_0000..0x00_0018_ffff | 512 KiB | C RW XN | Kernel boot-core stack +[ 5.020354] 0xffff_ffff_f000_0000..0xffff_ffff_f000_ffff --> 0x00_fe20_0000..0x00_fe20_ffff | 64 KiB | Dev RW XN | BCM GPIO +[ 5.021805] | BCM PL011 UART +[ 5.023322] 0xffff_ffff_f001_0000..0xffff_ffff_f001_ffff --> 0x00_ff84_0000..0x00_ff84_ffff | 64 KiB | Dev RW XN | GICD +[ 5.024730] | GICC +[ 5.026138] ------------------------------------------------------------------------------------------------------------------------------------------- ``` ## Diff to previous ```diff +diff -uNr 15_virtual_mem_part3_precomputed_tables/Cargo.toml 16_virtual_mem_part4_higher_half_kernel/Cargo.toml +--- 15_virtual_mem_part3_precomputed_tables/Cargo.toml ++++ 16_virtual_mem_part4_higher_half_kernel/Cargo.toml +@@ -1,6 +1,6 @@ + [package] + name = "mingo" +-version = "0.15.0" ++version = "0.16.0" + authors = ["Andre Richter "] + edition = "2018" + + diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs --- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs index 3346134ad..afa495060 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -144,6 +144,19 @@ pub mod print; pub mod state; pub mod time; +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Version string. +pub fn version() -> &'static str { + concat!( + env!("CARGO_PKG_NAME"), + " version ", + env!("CARGO_PKG_VERSION") + ) +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- diff --git a/16_virtual_mem_part4_higher_half_kernel/src/main.rs b/16_virtual_mem_part4_higher_half_kernel/src/main.rs index c317bb96f..421dcbec2 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/main.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/main.rs @@ -72,6 +72,7 @@ fn kernel_main() -> ! { use driver::interface::DriverManager; use exception::asynchronous::interface::IRQManager; + info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); info!("MMU online:"); diff --git a/X1_JTAG_boot/Cargo.lock b/X1_JTAG_boot/Cargo.lock index a3d8bb9ae..011adc0f3 100644 --- a/X1_JTAG_boot/Cargo.lock +++ b/X1_JTAG_boot/Cargo.lock @@ -12,8 +12,8 @@ dependencies = [ ] [[package]] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.8.0" dependencies = [ "cortex-a", "register", diff --git a/X1_JTAG_boot/Cargo.toml b/X1_JTAG_boot/Cargo.toml index df5f7f974..e84540cac 100644 --- a/X1_JTAG_boot/Cargo.toml +++ b/X1_JTAG_boot/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "kernel" -version = "0.1.0" +name = "mingo" +version = "0.8.0" authors = ["Andre Richter "] edition = "2018" @@ -12,6 +12,10 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] +[[bin]] +name = "kernel" +path = "src/main.rs" + ##-------------------------------------------------------------------------------------------------- ## Dependencies ##-------------------------------------------------------------------------------------------------- diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 53be6bb3a5207dad2a75a526053ad11b367272c0..30ce719cf6080dcd104be8b5465aa8aa91e66e95 100755 GIT binary patch delta 2281 zcmcIlZ){Ul6hF7G-QF7;Tf26D+Ktz3%(Wc|rRdm}(YK9+Z43t83_(c-Vq}gFX44Ov zK-vuwVG!)~P7wT);1_%gixALQ5^=*sfe)GmCDBOohJNU}$Upr+gX{C$_D%eg5Hay4 zudnx>-|w7zf9Ia_ZU?nsbXC}5fl)WOznKtvujzYXD%C~QLQs-HFb(MW5E};Q@jxUr z9f1030DBPJ1C<2Vc4u`<7Uznbf=H+h?LIpMW)PF8gL)UoDaycb8kgHGNv^br*Uus$(FpYmk>uHJ>y{8C=9T?H@UF=X%#-36U_U1!XBIh$Ak?E1s^L>X z#Di027?pv{YT3{;Bfyx?F|n$x9BdLb=h&519cb%!I0N#eFtFFKP2gKZn}{|$I-C_`pC*v8c^{^f-cxIJ z26(`@u)tS{n1q3u_jPej@9~ALb_0~-%Rx{aL0)NLcl%Ojz(Yiz+`MKWf--VPCbY|L z;agv+61$==tMd?u@s0LnDM`@+v@Z*i+wbW;sjg;KLqC{Knog@grbJAsCIB^MMM9}6 zL46m~P5@4awI5MBsG>K}c+v!;DVLJe1(GJFx;|6mcq)D$D!_D56U4DsA6epuDjfd* zC4Q*F>;Id?T;5dI@{JSG_xFYBN<}n^N+9}%%9qu8>p8ortz8! zs+LVKGe8^IxMd@0KAW;UlTFy!r}HinvV)c9FL(F@bOAs#7kwsLf$lofNDrN0&*pC? zLub$D?B^l7(Idif#ukGNsRT| z`1**ghVxDx`ed+Qx^jG$a~Pq!HnA^kwGMwJz4@V_&MbqZgkIO}8GpN=l4TXj-ijsk z)ZFt|^Kfb`!IHyeJjuq2`*=18dT#@eK0QDW^5dqA>4jL+paA=aYnN7FWV zD{eYY|)zNbt4i#i}yM^Y_AMs zoko8VuRY?K~UVEtps9V?1@pWt>s#qOXy?MNJ#4$J8 zxx4sYLcJ(g0u!laXQ^C>ipWZ;VZP!uW|eotM2$sFhbAkj9yX~nRm7&wU|G#z?G^o& zQQ-`0cXU=8De*{`j;9%z&YPH;pKC$wKYt+9k^BDOSDV;*hm1-6-Vt;RJxMz-9WyRS zc6_wZg>4S&mpxwO46AecQH8)0z1SmkT`zv1%YzI&XXwIh0zPJZ;4e7~U@IVWX)3cv zT`*0h0+9T*D_d>CHOPi@qj*5RRr^mT2jdvtYptXscZ2N9B5&Rsh!HVk`;S2Ebv9Sj zS%kf%&j~TE3byU^E9vfkoyvX$*A4(9d#iL#Yb!Un~|DcJ^vG*GPyP)sw5DX1dG z7qPP9VT*d>bKC=-eIlu^{KvVVF>@Cqjg;d)MjXF^97lN3x|`Ci4hwp08XsQJrGWkJ zojJW&T&$=l#u(53G=4rx*my~!ccTe|<5+_q4n9hZc}weh_r}iUe!u&v<`7=WQ_Nag zOinXz>DSiiq6{`oz@IHla@LVH*6ghF$6Xm4rXxMp8Cbkm>G5}NHB+5(%cHVpjVJ|% z*ZZ!sIj#+)bVANV?<>o^FB$VF8z_^dzLL!9E9@A~UKTR5eQdH!&MB~FX1B{&qU^DX z*7D4>$Oy2-m}6z8$13J85Bl4T!~NUCcCr)Yw@5vsi#AL10X^)_qGD@_vC(|Pe+?^@ SUnEg>K(3QC!=|yza?S6OE0>!9 delta 2349 zcmb7EZ){Ul6hF7GrG4uLYiT#uZL}TeXgh>iU>g(qHi)nS0^JA%G-D7H7z=iO(FD>? zjR7{2UN3?`j00ovH4ro#O8o#EBu0s#1SQ7A*rLR(_ z=bZaH=bmKXP9RnvF&bc>3tZm}3GG*v1#sB}QZ@j(urdJ>M*-SA5DlgL(6HM8r@8@b z0dR#k5d6MlT&s7J<4JRIG}M9^pAGyep}?C)Y^DwxL_M5(p5rAcd``s|wtBaetN23X z_-e-aYitnlbHWy$1j(FuqY)ZBIyki&$sz*1JFKmqg@yiLSH<7Rqlm=-{@of^O^nl4Sf9-r>;_Y$)=(zGxPn2-n3q2K8e zzTI0UB`zv6>Us$zc)@0alB#}ul{ zriE2bZzoFIi_&f-1>Sa)Hj2~AK@X(ehSGkC(r!g*c`*ejZ4jj_KxsSfDQ#Rzt_Gl5 z!z3C4n93r;tE7kN^o?YAV7|VU*m~wtPt#O7Qhi2F8i~x6eu}+sSV4LQemAT$l7WFx z(K$jkG5ds>_LY8W28b16!1Mwf%FB(kFU8hQSWSl6eEoj5xcD+z&m5K)>_>`ei}=MM zoRV0uBS3E}%AMO`w%_6;DfY3&=N_Izw{QHQ`^I@9UmAP<$~L?!7k3sN#>U1xJ5L^V zOH5~NcW1rSgo>GmK6`V?OTXN0C*};^pf+B^1!Q!l*sIoRp6htu>F%xKg+m#hhY{Kt zW@oHEd$@|;Iw{DbGaxG>XUWbJhl@9`oYm>+sG-L(ZxMdob;4909P7Sv!tJ9xfBFST z?7`@Fj~QjG!!4@CMDT5?tV`Itz ziNNf3+$jp~iHG_;?8F95AMO+Ir^g@st#^K07lit9`5F~LcRA;W>~G7BawC2NBMk1t z1M*UBKOgHxv8ZQd72R_;z&=Q zFRdhNSyySG=($HIYdIyNQKoT_b^) z*1ygrHRfk>sF8BqAmaEWavb3oe*HXk9pWO0r|`#@K^fp)JT#`1^65Y^a2ibL6yqiA zok@+J7j>8%pM{Wr*K2n3f@Q81OJ~;Bx>hxXa5x7UDJvzXSy|avmj20ktV+bcEi7AB zN7k@fM_ui`^1Kh*p+4(0EY6ks{F6JI9dtOG|1aylmZeyWYOWag!|@C$``Ve${&GeB z{hem_vyKX<`0}Lu*IjHkT04Zi)yF=na28Cm87yiHnI&*atOgM{@gHM!I>O)GSQ7Qc>(&h4b1z2dAB?`Xa|Ea9yF E3mZPY^#A|> From d296ad4aa4eacfe13c68e13bbf28e6d602459db8 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 22:41:55 +0200 Subject: [PATCH 058/214] Minor fixes --- 01_wait_forever/README.md | 2 +- 02_runtime_init/Cargo.toml | 1 - 02_runtime_init/README.md | 3 +-- 03_hacky_hello_world/Cargo.toml | 1 - 04_safe_globals/Cargo.toml | 1 - 05_drivers_gpio_uart/Cargo.toml | 1 - 06_uart_chainloader/Cargo.toml | 1 - 07_timestamps/Cargo.toml | 1 - 08_hw_debug_JTAG/Cargo.toml | 1 - 09_privilege_level/Cargo.toml | 1 - 10_virtual_mem_part1_identity_mapping/Cargo.toml | 1 - 11_exceptions_part1_groundwork/Cargo.toml | 1 - 12_integrated_testing/Cargo.toml | 2 +- 12_integrated_testing/README.md | 6 +++--- 13_exceptions_part2_peripheral_IRQs/README.md | 6 +++--- 14_virtual_mem_part2_mmio_remap/README.md | 2 +- 15_virtual_mem_part3_precomputed_tables/README.md | 9 +++++++-- 15_virtual_mem_part3_precomputed_tables/src/main.rs | 2 ++ 16_virtual_mem_part4_higher_half_kernel/src/main.rs | 2 ++ X1_JTAG_boot/Cargo.toml | 1 - 20 files changed, 21 insertions(+), 24 deletions(-) diff --git a/01_wait_forever/README.md b/01_wait_forever/README.md index bf214b68f..9c6eaf762 100644 --- a/01_wait_forever/README.md +++ b/01_wait_forever/README.md @@ -18,7 +18,7 @@ ## Code to look at -- Custom `link.ld` linker script. +- `BSP`-specific `link.ld` linker script. - Load address at `0x8_0000` - Only `.text` section. - `main.rs`: Important [inner attributes]: diff --git a/02_runtime_init/Cargo.toml b/02_runtime_init/Cargo.toml index be6091a4d..69f9f06d6 100644 --- a/02_runtime_init/Cargo.toml +++ b/02_runtime_init/Cargo.toml @@ -25,4 +25,3 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index dfa43562a..5cba10fb3 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -39,7 +39,7 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml authors = ["Andre Richter "] edition = "2018" -@@ -21,3 +21,8 @@ +@@ -21,3 +21,7 @@ ##-------------------------------------------------------------------------------------------------- [dependencies] @@ -47,7 +47,6 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] +cortex-a = { version = "5.x.x" } -+ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile --- 01_wait_forever/Makefile diff --git a/03_hacky_hello_world/Cargo.toml b/03_hacky_hello_world/Cargo.toml index 6ed23060e..ffe9206f2 100644 --- a/03_hacky_hello_world/Cargo.toml +++ b/03_hacky_hello_world/Cargo.toml @@ -25,4 +25,3 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/04_safe_globals/Cargo.toml b/04_safe_globals/Cargo.toml index 16e4741df..fc077af93 100644 --- a/04_safe_globals/Cargo.toml +++ b/04_safe_globals/Cargo.toml @@ -25,4 +25,3 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/05_drivers_gpio_uart/Cargo.toml b/05_drivers_gpio_uart/Cargo.toml index 03ec9cdb9..05abef4d4 100644 --- a/05_drivers_gpio_uart/Cargo.toml +++ b/05_drivers_gpio_uart/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/06_uart_chainloader/Cargo.toml b/06_uart_chainloader/Cargo.toml index 0e1c5c28f..f3b9ba2f9 100644 --- a/06_uart_chainloader/Cargo.toml +++ b/06_uart_chainloader/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/07_timestamps/Cargo.toml b/07_timestamps/Cargo.toml index 07aa05171..8d97be1f1 100644 --- a/07_timestamps/Cargo.toml +++ b/07_timestamps/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/08_hw_debug_JTAG/Cargo.toml b/08_hw_debug_JTAG/Cargo.toml index e84540cac..7b1adb5b3 100644 --- a/08_hw_debug_JTAG/Cargo.toml +++ b/08_hw_debug_JTAG/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/09_privilege_level/Cargo.toml b/09_privilege_level/Cargo.toml index f6750ed49..7af3572f6 100644 --- a/09_privilege_level/Cargo.toml +++ b/09_privilege_level/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/10_virtual_mem_part1_identity_mapping/Cargo.toml b/10_virtual_mem_part1_identity_mapping/Cargo.toml index 8e54f2cdb..9d47f28da 100644 --- a/10_virtual_mem_part1_identity_mapping/Cargo.toml +++ b/10_virtual_mem_part1_identity_mapping/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/11_exceptions_part1_groundwork/Cargo.toml b/11_exceptions_part1_groundwork/Cargo.toml index 764939bf2..2f6683e5a 100644 --- a/11_exceptions_part1_groundwork/Cargo.toml +++ b/11_exceptions_part1_groundwork/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - diff --git a/12_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml index 4bb92ff47..d59d58176 100644 --- a/12_integrated_testing/Cargo.toml +++ b/12_integrated_testing/Cargo.toml @@ -4,7 +4,7 @@ version = "0.12.0" authors = ["Andre Richter "] edition = "2018" -# TODO: Fixme +# TODO: FIXME # LTO seems to kill the console integration test (empty text section). Disable until a fix is found. [profile.release] lto = false diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index 4a5940d1a..c49f83614 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -818,7 +818,7 @@ diff -uNr 11_exceptions_part1_groundwork/.cargo/config.toml 12_integrated_testin diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo.toml --- 11_exceptions_part1_groundwork/Cargo.toml +++ 12_integrated_testing/Cargo.toml -@@ -1,31 +1,58 @@ +@@ -1,30 +1,58 @@ [package] name = "mingo" -version = "0.11.0" @@ -826,7 +826,7 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. authors = ["Andre Richter "] edition = "2018" -+# TODO: Fixme ++# TODO: FIXME +# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. [profile.release] -lto = true @@ -857,7 +857,7 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - ++ +##-------------------------------------------------------------------------------------------------- +## Testing +##-------------------------------------------------------------------------------------------------- diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 4cb82c6b4..6da5a0750 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -2,8 +2,8 @@ ## tl;dr -- We write `device drivers` for the two interrupt controllers on the **Raspberry Pi 3** (Broadcom - custom controller) and **Pi 4** (ARM Generic Interrupt Controller v2, `GICv2`). +- We write `device drivers` for the two interrupt controllers on the **Raspberry Pi 3** (`Broadcom` + custom controller) and **Pi 4** (`ARM` Generic Interrupt Controller v2, `GICv2`). - Modularity is ensured by interfacing everything through a trait named `IRQManager`. - Handling for our first peripheral IRQs is implemented: The `UART`'s receive IRQs. @@ -757,7 +757,7 @@ diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/C authors = ["Andre Richter "] edition = "2018" --# TODO: Fixme +-# TODO: FIXME -# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. [profile.release] -lto = false diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 95042ed3a..3f0651022 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -2,7 +2,7 @@ ## tl;dr -- We introduce a first set of changes which we eventually need for separating `kernel` and `user` +- We introduce a first set of changes which is eventually needed for separating `kernel` and `user` address spaces. - The memory mapping strategy gets more sophisticated as we do away with `identity mapping` the whole of the board's address space. diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index 459b65d25..fab278160 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -1274,7 +1274,12 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs 15_v diff -uNr 14_virtual_mem_part2_mmio_remap/src/main.rs 15_virtual_mem_part3_precomputed_tables/src/main.rs --- 14_virtual_mem_part2_mmio_remap/src/main.rs +++ 15_virtual_mem_part3_precomputed_tables/src/main.rs -@@ -18,25 +18,16 @@ +@@ -15,28 +15,21 @@ + + /// Early init code. + /// ++/// When this code runs, virtual memory is already enabled. ++/// /// # Safety /// /// - Only a single core must be active and running this function. @@ -1304,7 +1309,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/main.rs 15_virtual_mem_part3_preco // Bring up the drivers needed for printing first. for i in bsp::driver::driver_manager() -@@ -47,7 +38,7 @@ +@@ -47,7 +40,7 @@ i.init().unwrap_or_else(|_| cpu::wait_forever()); } bsp::driver::driver_manager().post_early_print_device_driver_init(); diff --git a/15_virtual_mem_part3_precomputed_tables/src/main.rs b/15_virtual_mem_part3_precomputed_tables/src/main.rs index 421dcbec2..9ecba1e7e 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/main.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/main.rs @@ -15,6 +15,8 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// Early init code. /// +/// When this code runs, virtual memory is already enabled. +/// /// # Safety /// /// - Only a single core must be active and running this function. diff --git a/16_virtual_mem_part4_higher_half_kernel/src/main.rs b/16_virtual_mem_part4_higher_half_kernel/src/main.rs index 421dcbec2..9ecba1e7e 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/main.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/main.rs @@ -15,6 +15,8 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// Early init code. /// +/// When this code runs, virtual memory is already enabled. +/// /// # Safety /// /// - Only a single core must be active and running this function. diff --git a/X1_JTAG_boot/Cargo.toml b/X1_JTAG_boot/Cargo.toml index e84540cac..7b1adb5b3 100644 --- a/X1_JTAG_boot/Cargo.toml +++ b/X1_JTAG_boot/Cargo.toml @@ -28,4 +28,3 @@ register = { version = "1.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] cortex-a = { version = "5.x.x" } - From ee76b2226e733a25567cd0d72e76edba8c2d3f02 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 22:56:03 +0200 Subject: [PATCH 059/214] typo --- 15_virtual_mem_part3_precomputed_tables/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index fab278160..bb9a50bb6 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -144,9 +144,9 @@ executing from. Without going in too much detail, what the instruction basically does is: It retrieves the `4 KiB` page address that belongs to the program counter's (PC) current position (PC is at `0x8_0010`, so -the page address is `0x8_0000`), and adds adds `0x1_0000`. So after the `ADRP` instruction, register -`x0` holds the value `0x9_0000`. To this value, `8` is added in the next instruction, resulting in -the overall address of `0x9_0008`, which is exactly where `global_data_word` is located. This works, +the page address is `0x8_0000`), and adds `0x1_0000`. So after the `ADRP` instruction, register `x0` +holds the value `0x9_0000`. To this value, `8` is added in the next instruction, resulting in the +overall address of `0x9_0008`, which is exactly where `global_data_word` is located. This works, because after linking a `static executable binary` like we do since `tutorial 01`, relative positions of code and data are fixed, and not supposed to change during runtime. @@ -155,9 +155,6 @@ positions of code and data are fixed, and not supposed to change during runtime. If the Raspberry's firmware now loads this binary at address `0x8_0000`, as always, we can be sure that our function returns the correct address of our global data word. -> So far, all looks good, doesn't it? However, this was a lot to digest already, and we're far from -> finished. So take a minute or two and clear your mind before we continue. 🧘 - Now lets link this to the most significant area of memory: ```lds @@ -288,6 +285,9 @@ What we need is called [position-independent code]. [position-independent code]: https://en.wikipedia.org/wiki/Position-independent_code +> Much low-level stuff in this tutorial, isn't it? This was a lot to digest already, but we're far +> from finished. So take a minute or two and clear your mind before we continue. 🧘 + ## Position-Independent Code (PIC) As describend by Wikipedia, position-independent code From b41865179868fd7544b1aafec57135bf22b0577c Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 23:02:48 +0200 Subject: [PATCH 060/214] Update dependencies --- 12_integrated_testing/Cargo.lock | 8 ++++---- 13_exceptions_part2_peripheral_IRQs/Cargo.lock | 8 ++++---- 14_virtual_mem_part2_mmio_remap/Cargo.lock | 8 ++++---- 15_virtual_mem_part3_precomputed_tables/Cargo.lock | 8 ++++---- 16_virtual_mem_part4_higher_half_kernel/Cargo.lock | 8 ++++---- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/12_integrated_testing/Cargo.lock b/12_integrated_testing/Cargo.lock index 9dac661b5..009654700 100644 --- a/12_integrated_testing/Cargo.lock +++ b/12_integrated_testing/Cargo.lock @@ -24,9 +24,9 @@ dependencies = [ [[package]] name = "proc-macro2" -version = "1.0.24" +version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec" dependencies = [ "unicode-xid", ] @@ -57,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.62" +version = "1.0.68" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +checksum = "3ce15dd3ed8aa2f8eeac4716d6ef5ab58b6b9256db41d7e1a0224c2788e8fd87" dependencies = [ "proc-macro2", "quote", diff --git a/13_exceptions_part2_peripheral_IRQs/Cargo.lock b/13_exceptions_part2_peripheral_IRQs/Cargo.lock index 922341af2..ed9501003 100644 --- a/13_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/13_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -24,9 +24,9 @@ dependencies = [ [[package]] name = "proc-macro2" -version = "1.0.24" +version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec" dependencies = [ "unicode-xid", ] @@ -57,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.62" +version = "1.0.68" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +checksum = "3ce15dd3ed8aa2f8eeac4716d6ef5ab58b6b9256db41d7e1a0224c2788e8fd87" dependencies = [ "proc-macro2", "quote", diff --git a/14_virtual_mem_part2_mmio_remap/Cargo.lock b/14_virtual_mem_part2_mmio_remap/Cargo.lock index e1591cd42..24a35e1d8 100644 --- a/14_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/14_virtual_mem_part2_mmio_remap/Cargo.lock @@ -24,9 +24,9 @@ dependencies = [ [[package]] name = "proc-macro2" -version = "1.0.24" +version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec" dependencies = [ "unicode-xid", ] @@ -57,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.62" +version = "1.0.68" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +checksum = "3ce15dd3ed8aa2f8eeac4716d6ef5ab58b6b9256db41d7e1a0224c2788e8fd87" dependencies = [ "proc-macro2", "quote", diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.lock b/15_virtual_mem_part3_precomputed_tables/Cargo.lock index 62c78322e..cc4041447 100644 --- a/15_virtual_mem_part3_precomputed_tables/Cargo.lock +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.lock @@ -24,9 +24,9 @@ dependencies = [ [[package]] name = "proc-macro2" -version = "1.0.24" +version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec" dependencies = [ "unicode-xid", ] @@ -57,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.62" +version = "1.0.68" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +checksum = "3ce15dd3ed8aa2f8eeac4716d6ef5ab58b6b9256db41d7e1a0224c2788e8fd87" dependencies = [ "proc-macro2", "quote", diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock index ba2e9eb5b..5703573b5 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock @@ -24,9 +24,9 @@ dependencies = [ [[package]] name = "proc-macro2" -version = "1.0.24" +version = "1.0.26" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71" +checksum = "a152013215dca273577e18d2bf00fa862b89b24169fb78c4c95aeb07992c9cec" dependencies = [ "unicode-xid", ] @@ -57,9 +57,9 @@ dependencies = [ [[package]] name = "syn" -version = "1.0.62" +version = "1.0.68" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "123a78a3596b24fee53a6464ce52d8ecbf62241e6294c7e7fe12086cd161f512" +checksum = "3ce15dd3ed8aa2f8eeac4716d6ef5ab58b6b9256db41d7e1a0224c2788e8fd87" dependencies = [ "proc-macro2", "quote", From 86c041a41f7be447aee3e540ebe2c6eeaafd4a09 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 23:24:44 +0200 Subject: [PATCH 061/214] Fix integration test fail --- 12_integrated_testing/Cargo.toml | 4 +-- 12_integrated_testing/README.md | 21 +++++++------- .../tests/00_console_sanity.rs | 9 +++++- 13_exceptions_part2_peripheral_IRQs/README.md | 28 +++++++++++++------ utils/devtool.rb | 1 + 5 files changed, 40 insertions(+), 23 deletions(-) diff --git a/12_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml index d59d58176..686df7640 100644 --- a/12_integrated_testing/Cargo.toml +++ b/12_integrated_testing/Cargo.toml @@ -4,10 +4,8 @@ version = "0.12.0" authors = ["Andre Richter "] edition = "2018" -# TODO: FIXME -# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. [profile.release] -lto = false +lto = true [features] default = [] diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index c49f83614..113e09d2d 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -818,7 +818,7 @@ diff -uNr 11_exceptions_part1_groundwork/.cargo/config.toml 12_integrated_testin diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo.toml --- 11_exceptions_part1_groundwork/Cargo.toml +++ 12_integrated_testing/Cargo.toml -@@ -1,30 +1,58 @@ +@@ -1,6 +1,6 @@ [package] name = "mingo" -version = "0.11.0" @@ -826,13 +826,7 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. authors = ["Andre Richter "] edition = "2018" -+# TODO: FIXME -+# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. - [profile.release] --lto = true -+lto = false - - [features] +@@ -11,20 +11,46 @@ default = [] bsp_rpi3 = ["register"] bsp_rpi4 = ["register"] @@ -1823,7 +1817,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrated_testing/tests/00_console_sanity.rs --- 11_exceptions_part1_groundwork/tests/00_console_sanity.rs +++ 12_integrated_testing/tests/00_console_sanity.rs -@@ -0,0 +1,35 @@ +@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter @@ -1857,7 +1851,14 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrate + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. -+ cpu::wait_forever() ++ // cpu::wait_forever(); ++ ++ // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary ++ // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time ++ // being, the following lines are just a workaround to fix this compiler/linker weirdness. ++ use libkernel::time::interface::TimeManager; ++ libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); ++ cpu::qemu_exit_success() +} diff -uNr 11_exceptions_part1_groundwork/tests/01_timer_sanity.rs 12_integrated_testing/tests/01_timer_sanity.rs diff --git a/12_integrated_testing/tests/00_console_sanity.rs b/12_integrated_testing/tests/00_console_sanity.rs index ad7fd2bf5..40dc4b41e 100644 --- a/12_integrated_testing/tests/00_console_sanity.rs +++ b/12_integrated_testing/tests/00_console_sanity.rs @@ -31,5 +31,12 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - cpu::wait_forever() + // cpu::wait_forever(); + + // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary + // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); + cpu::qemu_exit_success() } diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 6da5a0750..b970cc253 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -749,7 +749,7 @@ Minipush 1.0 diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/Cargo.toml --- 12_integrated_testing/Cargo.toml +++ 13_exceptions_part2_peripheral_IRQs/Cargo.toml -@@ -1,13 +1,11 @@ +@@ -1,6 +1,6 @@ [package] name = "mingo" -version = "0.12.0" @@ -757,14 +757,6 @@ diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/C authors = ["Andre Richter "] edition = "2018" --# TODO: FIXME --# LTO seems to kill the console integration test (empty text section). Disable until a fix is found. - [profile.release] --lto = false -+lto = true - - [features] - default = [] diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs --- 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs @@ -2724,6 +2716,24 @@ diff -uNr 12_integrated_testing/src/synchronization.rs 13_exceptions_part2_perip + } } +diff -uNr 12_integrated_testing/tests/00_console_sanity.rs 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +--- 12_integrated_testing/tests/00_console_sanity.rs ++++ 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +@@ -31,12 +31,5 @@ + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. +- // cpu::wait_forever(); +- +- // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary +- // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time +- // being, the following lines are just a workaround to fix this compiler/linker weirdness. +- use libkernel::time::interface::TimeManager; +- libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); +- cpu::qemu_exit_success() ++ cpu::wait_forever() + } + diff -uNr 12_integrated_testing/tests/03_exception_irq_sanity.rs 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs --- 12_integrated_testing/tests/03_exception_irq_sanity.rs +++ 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs diff --git a/utils/devtool.rb b/utils/devtool.rb index a3d5e3d7b..a62af3802 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -190,6 +190,7 @@ def ready_for_publish def ready_for_publish_no_rust clean + fmt misspell rubocop copyright From 5fec597ae0e0adefb498189dd724c7af02dc5f93 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 23:26:53 +0200 Subject: [PATCH 062/214] Fix filename --- 00_before_we_start/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/00_before_we_start/README.md b/00_before_we_start/README.md index 60e060bcc..9e79caf4b 100644 --- a/00_before_we_start/README.md +++ b/00_before_we_start/README.md @@ -106,6 +106,6 @@ From a namespace perspective, **memory** subsystem code lives in: # Boot flow 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. - - It is implemented in `src/_arch/__arch_name__/cpu/boot.rs`. + - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. From 29a58d7d51ac1fa1e95b1993ab0f670f01e9af78 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Apr 2021 23:35:35 +0200 Subject: [PATCH 063/214] Next attempt at integration test fix --- 12_integrated_testing/README.md | 4 ++-- .../tests/00_console_sanity.rs | 4 ++-- 13_exceptions_part2_peripheral_IRQs/README.md | 18 ------------------ .../tests/00_console_sanity.rs | 9 ++++++++- .../tests/00_console_sanity.rs | 9 ++++++++- .../tests/00_console_sanity.rs | 9 ++++++++- .../tests/00_console_sanity.rs | 9 ++++++++- 7 files changed, 36 insertions(+), 26 deletions(-) diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index 113e09d2d..192728cfd 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -1853,8 +1853,8 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrate + // The QEMU process running this test will be closed by the I/O test harness. + // cpu::wait_forever(); + -+ // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary -+ // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time ++ // For some reason, in this test, rustc or the linker produces an empty binary when ++ // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); diff --git a/12_integrated_testing/tests/00_console_sanity.rs b/12_integrated_testing/tests/00_console_sanity.rs index 40dc4b41e..84b744798 100644 --- a/12_integrated_testing/tests/00_console_sanity.rs +++ b/12_integrated_testing/tests/00_console_sanity.rs @@ -33,8 +33,8 @@ unsafe fn kernel_init() -> ! { // The QEMU process running this test will be closed by the I/O test harness. // cpu::wait_forever(); - // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary - // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // For some reason, in this test, rustc or the linker produces an empty binary when + // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time // being, the following lines are just a workaround to fix this compiler/linker weirdness. use libkernel::time::interface::TimeManager; libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index b970cc253..0c7188b9a 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -2716,24 +2716,6 @@ diff -uNr 12_integrated_testing/src/synchronization.rs 13_exceptions_part2_perip + } } -diff -uNr 12_integrated_testing/tests/00_console_sanity.rs 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs ---- 12_integrated_testing/tests/00_console_sanity.rs -+++ 13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs -@@ -31,12 +31,5 @@ - print!("{}", console().chars_read()); - - // The QEMU process running this test will be closed by the I/O test harness. -- // cpu::wait_forever(); -- -- // For some reason, in this test in this tutorial, rustc or the linker produces an empty binary -- // when wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time -- // being, the following lines are just a workaround to fix this compiler/linker weirdness. -- use libkernel::time::interface::TimeManager; -- libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); -- cpu::qemu_exit_success() -+ cpu::wait_forever() - } - diff -uNr 12_integrated_testing/tests/03_exception_irq_sanity.rs 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs --- 12_integrated_testing/tests/03_exception_irq_sanity.rs +++ 13_exceptions_part2_peripheral_IRQs/tests/03_exception_irq_sanity.rs diff --git a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs index ad7fd2bf5..84b744798 100644 --- a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs @@ -31,5 +31,12 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - cpu::wait_forever() + // cpu::wait_forever(); + + // For some reason, in this test, rustc or the linker produces an empty binary when + // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); + cpu::qemu_exit_success() } diff --git a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs index ad7fd2bf5..84b744798 100644 --- a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs @@ -31,5 +31,12 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - cpu::wait_forever() + // cpu::wait_forever(); + + // For some reason, in this test, rustc or the linker produces an empty binary when + // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); + cpu::qemu_exit_success() } diff --git a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs index ad7fd2bf5..84b744798 100644 --- a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs @@ -31,5 +31,12 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - cpu::wait_forever() + // cpu::wait_forever(); + + // For some reason, in this test, rustc or the linker produces an empty binary when + // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); + cpu::qemu_exit_success() } diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs index ad7fd2bf5..84b744798 100644 --- a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs @@ -31,5 +31,12 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - cpu::wait_forever() + // cpu::wait_forever(); + + // For some reason, in this test, rustc or the linker produces an empty binary when + // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time + // being, the following lines are just a workaround to fix this compiler/linker weirdness. + use libkernel::time::interface::TimeManager; + libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); + cpu::qemu_exit_success() } From 07fb63ae5f4a90dc91f7832b26cc215a35ddd7af Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 5 Apr 2021 15:16:06 +0200 Subject: [PATCH 064/214] Fix wrong order of range in comment --- 14_virtual_mem_part2_mmio_remap/README.md | 2 +- 14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs | 2 +- 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs | 2 +- 16_virtual_mem_part4_higher_half_kernel/README.md | 4 ++-- 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 3f0651022..4ddb7d1a5 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -2862,7 +2862,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 14_virtual_mem_p +pub trait AssociatedTranslationTable { + /// A translation table whose address range is: + /// -+ /// [0, AS_SIZE - 1] ++ /// [AS_SIZE - 1, 0] + type TableStartFromBottom; } diff --git a/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index e942f7f3e..360e149ae 100644 --- a/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -64,7 +64,7 @@ pub struct AddressSpace; pub trait AssociatedTranslationTable { /// A translation table whose address range is: /// - /// [0, AS_SIZE - 1] + /// [AS_SIZE - 1, 0] type TableStartFromBottom; } diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs index a7109ec29..969ae6dd0 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs @@ -80,7 +80,7 @@ pub struct AddressSpace; pub trait AssociatedTranslationTable { /// A translation table whose address range is: /// - /// [0, AS_SIZE - 1] + /// [AS_SIZE - 1, 0] type TableStartFromBottom; } diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index 4f7014423..17d71e2b9 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -37,7 +37,7 @@ pub trait AssociatedTranslationTable { /// A translation table whose address range is: /// - /// [0, AS_SIZE - 1] + /// [AS_SIZE - 1, 0] type TableStartFromBottom; } ``` @@ -602,7 +602,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs 16_virtual_m + + /// A translation table whose address range is: + /// - /// [0, AS_SIZE - 1] + /// [AS_SIZE - 1, 0] type TableStartFromBottom; } diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs index 255093193..d1a5a3e65 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs @@ -85,7 +85,7 @@ pub trait AssociatedTranslationTable { /// A translation table whose address range is: /// - /// [0, AS_SIZE - 1] + /// [AS_SIZE - 1, 0] type TableStartFromBottom; } From 8982682d47c5ec2bad1edac74b3ad34e220f1809 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 29 Apr 2021 22:54:57 +0200 Subject: [PATCH 065/214] Add chainloader test closes #101 --- 06_uart_chainloader/Makefile | 14 ++- 06_uart_chainloader/README.md | 117 +++++++++++++++++++- 06_uart_chainloader/tests/qemu_minipush.rb | 82 ++++++++++++++ 07_timestamps/README.md | 122 ++++++++++++++++++++- devtool_completion.bash | 2 +- utils/devtool.rb | 17 ++- utils/minipush.rb | 29 +++-- utils/miniterm.rb | 9 +- 8 files changed, 359 insertions(+), 33 deletions(-) create mode 100644 06_uart_chainloader/tests/qemu_minipush.rb diff --git a/06_uart_chainloader/Makefile b/06_uart_chainloader/Makefile index ce9d5a640..6805ed1cf 100644 --- a/06_uart_chainloader/Makefile +++ b/06_uart_chainloader/Makefile @@ -70,6 +70,7 @@ DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux @@ -79,8 +80,9 @@ ifeq ($(UNAME_S),Linux) DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_MINIPUSH = ruby ../utils/minipush.rb +EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ check @@ -99,7 +101,7 @@ doc: @$(DOC_CMD) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) -qemu: +qemu test: $(call colorecho, "\n$(QEMU_MISSING_STRING)") else qemu: $(KERNEL_BIN) @@ -109,6 +111,12 @@ qemu: $(KERNEL_BIN) qemuasm: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU with ASM output") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm + +test: $(KERNEL_BIN) + $(call colorecho, "\nTesting chainloading - $(BSP)") + @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ + -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) + endif chainboot: diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 80aa21332..2674bf63f 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -153,7 +153,14 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile endif # Export for build.rs -@@ -74,13 +76,14 @@ +@@ -68,19 +70,22 @@ + DOCKER_ARG_DEV = --privileged -v /dev:/dev + + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) ++DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + + # Dockerize commands that require USB device passthrough only on Linux ifeq ($(UNAME_S),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) @@ -161,9 +168,11 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) endif - EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +-EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINITERM = ruby ../utils/miniterm.rb -+EXEC_MINIPUSH = ruby ../utils/minipush.rb ++EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) ++EXEC_MINIPUSH = ruby ../utils/minipush.rb ++EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu miniterm clippy clean readelf objdump nm check +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ @@ -171,7 +180,14 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile all: $(KERNEL_BIN) -@@ -102,10 +105,14 @@ +@@ -96,16 +101,26 @@ + @$(DOC_CMD) --document-private-items --open + + ifeq ($(QEMU_MACHINE_TYPE),) +-qemu: ++qemu test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + else qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) @@ -179,6 +195,12 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile +qemuasm: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU with ASM output") + @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm ++ ++test: $(KERNEL_BIN) ++ $(call colorecho, "\nTesting chainloading - $(BSP)") ++ @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ ++ -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) ++ endif -miniterm: @@ -470,6 +492,93 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs + kernel() } +diff -uNr 05_drivers_gpio_uart/tests/qemu_minipush.rb 06_uart_chainloader/tests/qemu_minipush.rb +--- 05_drivers_gpio_uart/tests/qemu_minipush.rb ++++ 06_uart_chainloader/tests/qemu_minipush.rb +@@ -0,0 +1,82 @@ ++# frozen_string_literal: true ++ ++# SPDX-License-Identifier: MIT OR Apache-2.0 ++# ++# Copyright (c) 2020-2021 Andre Richter ++ ++require_relative '../../utils/minipush' ++require 'expect' ++require 'timeout' ++ ++# Match for the last print that 'demo_payload_rpiX.img' produces. ++EXPECTED_PRINT = 'Echoing input now' ++ ++# The main class ++class QEMUMiniPush < MiniPush ++ TIMEOUT_SECS = 3 ++ ++ # override ++ def initialize(qemu_cmd, binary_image_path) ++ super(nil, binary_image_path) ++ ++ @qemu_cmd = qemu_cmd ++ end ++ ++ private ++ ++ def quit_qemu_graceful ++ Timeout.timeout(5) do ++ pid = @target_serial.pid ++ Process.kill('TERM', pid) ++ Process.wait(pid) ++ end ++ end ++ ++ # override ++ def open_serial ++ @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') ++ ++ # Ensure all output is immediately flushed to the device. ++ @target_serial.sync = true ++ ++ puts "[#{@name_short}] ✅ Serial connected" ++ end ++ ++ # override ++ def terminal ++ result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) ++ exit(1) if result.nil? ++ ++ puts result ++ ++ quit_qemu_graceful ++ end ++ ++ public ++ ++ # override ++ def connetion_reset; end ++ ++ # override ++ def handle_reconnect(error) ++ handle_unexpected(error) ++ end ++end ++ ++##-------------------------------------------------------------------------------------------------- ++## Execution starts here ++##-------------------------------------------------------------------------------------------------- ++puts ++puts 'QEMUMiniPush 1.0'.cyan ++puts ++ ++# CTRL + C handler. Only here to suppress Ruby's default exception print. ++trap('INT') do ++ # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. ++ exit ++end ++ ++binary_image_path = ARGV.pop ++qemu_cmd = ARGV.join(' ') ++ ++QEMUMiniPush.new(qemu_cmd, binary_image_path).run + diff -uNr 05_drivers_gpio_uart/update.sh 06_uart_chainloader/update.sh --- 05_drivers_gpio_uart/update.sh +++ 06_uart_chainloader/update.sh diff --git a/06_uart_chainloader/tests/qemu_minipush.rb b/06_uart_chainloader/tests/qemu_minipush.rb new file mode 100644 index 000000000..36a8abda7 --- /dev/null +++ b/06_uart_chainloader/tests/qemu_minipush.rb @@ -0,0 +1,82 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2020-2021 Andre Richter + +require_relative '../../utils/minipush' +require 'expect' +require 'timeout' + +# Match for the last print that 'demo_payload_rpiX.img' produces. +EXPECTED_PRINT = 'Echoing input now' + +# The main class +class QEMUMiniPush < MiniPush + TIMEOUT_SECS = 3 + + # override + def initialize(qemu_cmd, binary_image_path) + super(nil, binary_image_path) + + @qemu_cmd = qemu_cmd + end + + private + + def quit_qemu_graceful + Timeout.timeout(5) do + pid = @target_serial.pid + Process.kill('TERM', pid) + Process.wait(pid) + end + end + + # override + def open_serial + @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') + + # Ensure all output is immediately flushed to the device. + @target_serial.sync = true + + puts "[#{@name_short}] ✅ Serial connected" + end + + # override + def terminal + result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) + exit(1) if result.nil? + + puts result + + quit_qemu_graceful + end + + public + + # override + def connetion_reset; end + + # override + def handle_reconnect(error) + handle_unexpected(error) + end +end + +##-------------------------------------------------------------------------------------------------- +## Execution starts here +##-------------------------------------------------------------------------------------------------- +puts +puts 'QEMUMiniPush 1.0'.cyan +puts + +# CTRL + C handler. Only here to suppress Ruby's default exception print. +trap('INT') do + # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. + exit +end + +binary_image_path = ARGV.pop +qemu_cmd = ARGV.join(' ') + +QEMUMiniPush.new(qemu_cmd, binary_image_path).run diff --git a/07_timestamps/README.md b/07_timestamps/README.md index a5063cb64..1701ebf63 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -78,9 +78,23 @@ diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile endif # Export for build.rs -@@ -82,8 +80,7 @@ - EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) - EXEC_MINIPUSH = ruby ../utils/minipush.rb +@@ -70,7 +68,6 @@ + DOCKER_ARG_DEV = --privileged -v /dev:/dev + + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) +-DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + + # Dockerize commands that require USB device passthrough only on Linux +@@ -80,12 +77,10 @@ + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + endif + +-EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +-EXEC_MINIPUSH = ruby ../utils/minipush.rb +-EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb ++EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) ++EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ - check @@ -88,7 +102,14 @@ diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile all: $(KERNEL_BIN) -@@ -105,14 +102,10 @@ +@@ -101,26 +96,16 @@ + @$(DOC_CMD) --document-private-items --open + + ifeq ($(QEMU_MACHINE_TYPE),) +-qemu test: ++qemu: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + else qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) @@ -96,6 +117,12 @@ diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile -qemuasm: $(KERNEL_BIN) - $(call colorecho, "\nLaunching QEMU with ASM output") - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm +- +-test: $(KERNEL_BIN) +- $(call colorecho, "\nTesting chainloading - $(BSP)") +- @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ +- -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) +- endif -chainboot: @@ -692,6 +719,93 @@ diff -uNr 06_uart_chainloader/src/time.rs 07_timestamps/src/time.rs + } +} +diff -uNr 06_uart_chainloader/tests/qemu_minipush.rb 07_timestamps/tests/qemu_minipush.rb +--- 06_uart_chainloader/tests/qemu_minipush.rb ++++ 07_timestamps/tests/qemu_minipush.rb +@@ -1,82 +0,0 @@ +-# frozen_string_literal: true +- +-# SPDX-License-Identifier: MIT OR Apache-2.0 +-# +-# Copyright (c) 2020-2021 Andre Richter +- +-require_relative '../../utils/minipush' +-require 'expect' +-require 'timeout' +- +-# Match for the last print that 'demo_payload_rpiX.img' produces. +-EXPECTED_PRINT = 'Echoing input now' +- +-# The main class +-class QEMUMiniPush < MiniPush +- TIMEOUT_SECS = 3 +- +- # override +- def initialize(qemu_cmd, binary_image_path) +- super(nil, binary_image_path) +- +- @qemu_cmd = qemu_cmd +- end +- +- private +- +- def quit_qemu_graceful +- Timeout.timeout(5) do +- pid = @target_serial.pid +- Process.kill('TERM', pid) +- Process.wait(pid) +- end +- end +- +- # override +- def open_serial +- @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') +- +- # Ensure all output is immediately flushed to the device. +- @target_serial.sync = true +- +- puts "[#{@name_short}] ✅ Serial connected" +- end +- +- # override +- def terminal +- result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) +- exit(1) if result.nil? +- +- puts result +- +- quit_qemu_graceful +- end +- +- public +- +- # override +- def connetion_reset; end +- +- # override +- def handle_reconnect(error) +- handle_unexpected(error) +- end +-end +- +-##-------------------------------------------------------------------------------------------------- +-## Execution starts here +-##-------------------------------------------------------------------------------------------------- +-puts +-puts 'QEMUMiniPush 1.0'.cyan +-puts +- +-# CTRL + C handler. Only here to suppress Ruby's default exception print. +-trap('INT') do +- # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. +- exit +-end +- +-binary_image_path = ARGV.pop +-qemu_cmd = ARGV.join(' ') +- +-QEMUMiniPush.new(qemu_cmd, binary_image_path).run + diff -uNr 06_uart_chainloader/update.sh 07_timestamps/update.sh --- 06_uart_chainloader/update.sh +++ 07_timestamps/update.sh diff --git a/devtool_completion.bash b/devtool_completion.bash index e351aa2ee..3831425cd 100755 --- a/devtool_completion.bash +++ b/devtool_completion.bash @@ -1,3 +1,3 @@ #!/usr/bin/env bash -complete -W "clean clippy copyright diff fmt fmt_check make make_xtra misspell ready_for_publish ready_for_publish_no_rust rubocop test_integration test_unit update" devtool +complete -W "clean clippy copyright diff fmt fmt_check make make_xtra misspell ready_for_publish ready_for_publish_no_rust rubocop test_integration test_unit test_xtra update" devtool diff --git a/utils/devtool.rb b/utils/devtool.rb index a62af3802..494de032b 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -52,7 +52,7 @@ def make(bsp) end def test_unit - return unless testable? + return unless kernel_tests? puts "Unit Tests #{@folder}".light_blue @@ -60,7 +60,7 @@ def test_unit end def test_integration - return unless testable? + return unless kernel_tests? puts "Integration Tests #{@folder}".light_blue @@ -74,8 +74,8 @@ def test_integration private - def testable? - Dir.exist?("#{@folder}/tests") + def kernel_tests? + File.exist?("#{@folder}/tests/runner.rb") end end @@ -147,6 +147,13 @@ def make_xtra system('cd X1_JTAG_boot && bash update.sh') end + def test_xtra + return if @user_has_supplied_crates + + puts 'Test Xtra stuff'.light_blue + exit(1) unless system('cd *_uart_chainloader && make test') + end + def test_unit @crates.each(&:test_unit) end @@ -166,8 +173,6 @@ def misspell def rubocop puts 'Rubocop'.light_blue - system('which bundle') - system('bundle --version') exit(1) unless system('bundle exec rubocop') end diff --git a/utils/minipush.rb b/utils/minipush.rb index 14c1e6a5f..a9ee9bb2a 100755 --- a/utils/minipush.rb +++ b/utils/minipush.rb @@ -80,7 +80,7 @@ def send_binary end # override - def handle_reconnect + def handle_reconnect(_error) connetion_reset puts @@ -100,8 +100,8 @@ def run send_size send_binary terminal - rescue ConnectionError, EOFError, Errno::EIO, ProtocolError, Timeout::Error - handle_reconnect + rescue ConnectionError, EOFError, Errno::EIO, ProtocolError, Timeout::Error => e + handle_reconnect(e) retry rescue StandardError => e handle_unexpected(e) @@ -112,14 +112,19 @@ def run end end -puts -puts 'Minipush 1.0'.cyan -puts +##-------------------------------------------------------------------------------------------------- +## Execution starts here +##-------------------------------------------------------------------------------------------------- +if __FILE__ == $PROGRAM_NAME + puts + puts 'Minipush 1.0'.cyan + puts + + # CTRL + C handler. Only here to suppress Ruby's default exception print. + trap('INT') do + # The `ensure` block from `MiniPush::run` will run after exit, restoring console state. + exit + end -# CTRL + C handler. Only here to suppress Ruby's default exception print. -trap('INT') do - # The `ensure` block from `MiniPush::run` will run after exit, restoring console state. - exit + MiniPush.new(ARGV[0], ARGV[1]).run end - -MiniPush.new(ARGV[0], ARGV[1]).run diff --git a/utils/miniterm.rb b/utils/miniterm.rb index ff0c64fb6..10cc4bc5d 100755 --- a/utils/miniterm.rb +++ b/utils/miniterm.rb @@ -94,7 +94,7 @@ def connetion_reset end # When the serial lost power or was removed during R/W operation. - def handle_reconnect + def handle_reconnect(_error) connetion_reset puts @@ -113,8 +113,8 @@ def handle_unexpected(error) def run open_serial terminal - rescue ConnectionError, EOFError, Errno::EIO - handle_reconnect + rescue ConnectionError, EOFError, Errno::EIO => e + handle_reconnect(e) retry rescue StandardError => e handle_unexpected(e) @@ -125,6 +125,9 @@ def run end end +##-------------------------------------------------------------------------------------------------- +## Execution starts here +##-------------------------------------------------------------------------------------------------- if __FILE__ == $PROGRAM_NAME puts puts 'Miniterm 1.0'.cyan From 389fde2aa59536af0ba355d6aa9064118474ab15 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 29 Apr 2021 23:01:22 +0200 Subject: [PATCH 066/214] Remove wrong public access control --- 06_uart_chainloader/README.md | 4 +--- 06_uart_chainloader/tests/qemu_minipush.rb | 2 -- 07_timestamps/README.md | 4 +--- 3 files changed, 2 insertions(+), 8 deletions(-) diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 2674bf63f..1d3a6d4e4 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -495,7 +495,7 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs diff -uNr 05_drivers_gpio_uart/tests/qemu_minipush.rb 06_uart_chainloader/tests/qemu_minipush.rb --- 05_drivers_gpio_uart/tests/qemu_minipush.rb +++ 06_uart_chainloader/tests/qemu_minipush.rb -@@ -0,0 +1,82 @@ +@@ -0,0 +1,80 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 @@ -550,8 +550,6 @@ diff -uNr 05_drivers_gpio_uart/tests/qemu_minipush.rb 06_uart_chainloader/tests/ + quit_qemu_graceful + end + -+ public -+ + # override + def connetion_reset; end + diff --git a/06_uart_chainloader/tests/qemu_minipush.rb b/06_uart_chainloader/tests/qemu_minipush.rb index 36a8abda7..73857cd6b 100644 --- a/06_uart_chainloader/tests/qemu_minipush.rb +++ b/06_uart_chainloader/tests/qemu_minipush.rb @@ -52,8 +52,6 @@ def terminal quit_qemu_graceful end - public - # override def connetion_reset; end diff --git a/07_timestamps/README.md b/07_timestamps/README.md index 1701ebf63..de94f00d9 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -722,7 +722,7 @@ diff -uNr 06_uart_chainloader/src/time.rs 07_timestamps/src/time.rs diff -uNr 06_uart_chainloader/tests/qemu_minipush.rb 07_timestamps/tests/qemu_minipush.rb --- 06_uart_chainloader/tests/qemu_minipush.rb +++ 07_timestamps/tests/qemu_minipush.rb -@@ -1,82 +0,0 @@ +@@ -1,80 +0,0 @@ -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 @@ -777,8 +777,6 @@ diff -uNr 06_uart_chainloader/tests/qemu_minipush.rb 07_timestamps/tests/qemu_mi - quit_qemu_graceful - end - -- public -- - # override - def connetion_reset; end - From f6f668c7810d018fb9bf7b501754c07f15120dc1 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 29 Apr 2021 23:08:49 +0200 Subject: [PATCH 067/214] Fix misspelled clippy lint --- 05_drivers_gpio_uart/README.md | 2 +- 05_drivers_gpio_uart/src/main.rs | 2 +- 06_uart_chainloader/README.md | 2 +- 06_uart_chainloader/src/main.rs | 2 +- 07_timestamps/README.md | 2 +- 07_timestamps/src/main.rs | 2 +- 08_hw_debug_JTAG/src/main.rs | 2 +- 09_privilege_level/src/main.rs | 2 +- 10_virtual_mem_part1_identity_mapping/README.md | 2 +- 10_virtual_mem_part1_identity_mapping/src/main.rs | 2 +- 11_exceptions_part1_groundwork/src/main.rs | 2 +- 12_integrated_testing/README.md | 4 ++-- 12_integrated_testing/src/lib.rs | 2 +- 13_exceptions_part2_peripheral_IRQs/README.md | 2 +- 13_exceptions_part2_peripheral_IRQs/src/lib.rs | 2 +- 14_virtual_mem_part2_mmio_remap/README.md | 2 +- 14_virtual_mem_part2_mmio_remap/src/lib.rs | 2 +- 15_virtual_mem_part3_precomputed_tables/src/lib.rs | 2 +- 16_virtual_mem_part4_higher_half_kernel/src/lib.rs | 2 +- X1_JTAG_boot/src/main.rs | 2 +- 20 files changed, 21 insertions(+), 21 deletions(-) diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index 9f4a8749b..b5895cd77 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -1325,7 +1325,7 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -+#![allow(clippy::clippy::upper_case_acronyms)] ++#![allow(clippy::upper_case_acronyms)] +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] diff --git a/05_drivers_gpio_uart/src/main.rs b/05_drivers_gpio_uart/src/main.rs index 93d6ffef5..d72919118 100644 --- a/05_drivers_gpio_uart/src/main.rs +++ b/05_drivers_gpio_uart/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 1d3a6d4e4..ba6fc0c13 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -403,7 +403,7 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs @@ -107,6 +107,7 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - #![allow(clippy::clippy::upper_case_acronyms)] + #![allow(clippy::upper_case_acronyms)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/06_uart_chainloader/src/main.rs b/06_uart_chainloader/src/main.rs index a94b97e42..aed10f7a8 100644 --- a/06_uart_chainloader/src/main.rs +++ b/06_uart_chainloader/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/07_timestamps/README.md b/07_timestamps/README.md index de94f00d9..aa6d237c8 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -502,7 +502,7 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs @@ -107,7 +107,6 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - #![allow(clippy::clippy::upper_case_acronyms)] + #![allow(clippy::upper_case_acronyms)] -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] diff --git a/07_timestamps/src/main.rs b/07_timestamps/src/main.rs index 5f465cbfe..54b69275c 100644 --- a/07_timestamps/src/main.rs +++ b/07_timestamps/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] diff --git a/08_hw_debug_JTAG/src/main.rs b/08_hw_debug_JTAG/src/main.rs index 5f465cbfe..54b69275c 100644 --- a/08_hw_debug_JTAG/src/main.rs +++ b/08_hw_debug_JTAG/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] diff --git a/09_privilege_level/src/main.rs b/09_privilege_level/src/main.rs index 83d96ca8a..5cede7f98 100644 --- a/09_privilege_level/src/main.rs +++ b/09_privilege_level/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index ba2932a44..e33329fd8 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -1043,7 +1043,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s @@ -107,7 +107,11 @@ //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - #![allow(clippy::clippy::upper_case_acronyms)] + #![allow(clippy::upper_case_acronyms)] +#![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] diff --git a/10_virtual_mem_part1_identity_mapping/src/main.rs b/10_virtual_mem_part1_identity_mapping/src/main.rs index 74d49754a..35f41c0e7 100644 --- a/10_virtual_mem_part1_identity_mapping/src/main.rs +++ b/10_virtual_mem_part1_identity_mapping/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/11_exceptions_part1_groundwork/src/main.rs b/11_exceptions_part1_groundwork/src/main.rs index a1c5bc150..5d00393c4 100644 --- a/11_exceptions_part1_groundwork/src/main.rs +++ b/11_exceptions_part1_groundwork/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index 192728cfd..cff8587b9 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -1318,7 +1318,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li +//! +//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html + -+#![allow(clippy::clippy::upper_case_acronyms)] ++#![allow(clippy::upper_case_acronyms)] +#![allow(incomplete_features)] +#![feature(const_fn_fn_ptr_basics)] +#![feature(const_generics)] @@ -1500,7 +1500,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html - --#![allow(clippy::clippy::upper_case_acronyms)] +-#![allow(clippy::upper_case_acronyms)] -#![allow(incomplete_features)] -#![feature(const_fn_fn_ptr_basics)] -#![feature(const_generics)] diff --git a/12_integrated_testing/src/lib.rs b/12_integrated_testing/src/lib.rs index dc991fd12..1e315b4da 100644 --- a/12_integrated_testing/src/lib.rs +++ b/12_integrated_testing/src/lib.rs @@ -108,7 +108,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 0c7188b9a..2a44ecdac 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -2378,7 +2378,7 @@ diff -uNr 12_integrated_testing/src/lib.rs 13_exceptions_part2_peripheral_IRQs/s +++ 13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -110,6 +110,7 @@ - #![allow(clippy::clippy::upper_case_acronyms)] + #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] diff --git a/13_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs index 3cc973299..9a787e60e 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -108,7 +108,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 4ddb7d1a5..6d35086fb 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -2132,7 +2132,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/lib.rs 14_virtual_mem_part2_mm --- 13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -111,6 +111,8 @@ - #![allow(clippy::clippy::upper_case_acronyms)] + #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] +#![feature(const_evaluatable_checked)] diff --git a/14_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs index afa495060..f3ef29513 100644 --- a/14_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -108,7 +108,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs index afa495060..f3ef29513 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -108,7 +108,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs index afa495060..f3ef29513 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -108,7 +108,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index 19c864c4d..0dc87fa16 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -106,7 +106,7 @@ //! //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html -#![allow(clippy::clippy::upper_case_acronyms)] +#![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] From 80c88e4d6b4624c68f5da0cee5270d65cd3fd95d Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Thu, 29 Apr 2021 23:35:23 +0200 Subject: [PATCH 068/214] Bump toolchain version; Fix clippy warnings --- 06_uart_chainloader/demo_payload_rpi3.img | Bin 7024 -> 6920 bytes 06_uart_chainloader/demo_payload_rpi4.img | Bin 6872 -> 6760 bytes .../README.md | 2 +- .../aarch64/memory/mmu/translation_table.rs | 2 +- .../aarch64/memory/mmu/translation_table.rs | 2 +- .../aarch64/memory/mmu/translation_table.rs | 2 +- .../aarch64/memory/mmu/translation_table.rs | 2 +- 14_virtual_mem_part2_mmio_remap/README.md | 17 ++++++----------- .../aarch64/memory/mmu/translation_table.rs | 6 +++--- .../src/memory/mmu.rs | 2 +- .../src/memory/mmu/mapping_record.rs | 9 ++------- .../aarch64/memory/mmu/translation_table.rs | 6 +++--- .../src/memory/mmu.rs | 2 +- .../src/memory/mmu/mapping_record.rs | 9 ++------- .../aarch64/memory/mmu/translation_table.rs | 6 +++--- .../src/memory/mmu.rs | 2 +- .../src/memory/mmu/mapping_record.rs | 9 ++------- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8160 -> 8032 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6848 -> 6736 bytes rust-toolchain | 2 +- 20 files changed, 30 insertions(+), 50 deletions(-) diff --git a/06_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img index 9b1cf35bce83d123cb099774e0377f9ac1271a2b..a8d7f680ef624efc1753bb890807835d880967e8 100755 GIT binary patch delta 2758 zcmZ`*eN0=|6+ibqe>~eHVEk=9Y~&Cyz z-OTY^MwN(CXaZ$H;*`Guj0Yk37qy$4T|dbyUp{V7){YpJ)jpH5(rZ?h4+%78aC5{* zdku@yLq?cwo(B7T2;}CWXom&BDHtsy7NZyyh!}my0<#M!J4k#Z2aFKiParziH!JTr2a`4+OF`=y?nz6iwWX;e~v)n zCnm9UimSbEWRe$(L%3lvcq2o3F&f(h$*}9ZKWv02MH3`qMo13e|1~VeZUmR(9atDo zOP+UUo}fn4_RxOJ>sy%LKFqTR^WCegpX^pJfB6dEckP*WF?*6eXbMCVSec1+bL6Q< zx|C+VZbTrp?|aX}tk(>SfDsDSVu_eo>D?J1v1_7gZvR+SJ!>>=b4d5DcYbu@SeNoN z`l~g9d=9O00Et0|SjzEU#N=z)u=*2P#k%(@=QBqPv#-| zIsz&1{!D=Qi>poN{TW;f*eUGtlz<%yr@eBa=?_ZIOcX{Yl`E5n6)yGI*$*M}_Ggfo z{w#Hwtk4?aSm?-QEHuG_b4Vm!5kM?i1hKS)BwXf6CV(BLuL)jXD%;=~I z-n_{>>2={f;-u#-;RC~u&Uf??vg=js`vT^VNzZoI&zEX94-oE~p;; zq5nHlnon}Le-q&^phxDlkHF$MwOboP&Wx9l)*zN94e%zD?Zfrpn)Cwu&Gbj;cdcKS zPI1Br!(?-b9Y>A+YKd^ku<`WEeMBe@sA__Fh~e)U?5bpf#Yse0_TbFap+^XIDRYQ6 zj|^Ns)UEKF#L`M{7Z#*bxpm}^ll>65_NX4Uh*3vPAm8MznKO%h9nUw4)Mndm9OkSe zBlMfL8qS)d58LjOT>V{&d$*YNA*G@^r38&qF6^xPHl^6PywSBc$tWji^kMQc7{(0T zZx7-OVZiPL#X8AorE-p93TbtxZ=}`3chQRV;a^A}hMLveSrOt zruk$SuD_>I%Y)SN6mv+eJg)z}VT(FEO;+r5O`wNvMmj-6f)Ru)WR~friWB7U8#^}r zlSqKjJTmbh*(D>($8sS+mci7VjVQ%#kVn%|S!k%0=Ua`+0nB%y zo5;DJvN6cx803{=n+ZoDrz|uXl-gVBOKcPtw7`(bAxrZJAa+aj1hJVokquJ~6SegD z%6cIcl1U2WHF~LX|K4f0ynrBbW)oAIY7pz>90o6-)BP-HlTl$uC_G&&t2lG#LqwV9 zi98?TmBM;C=nUurVAf+-Nh&TpxK*=Z@*7q4&U8R7s0O7OaV%ZLco8TSsNL?hlAwGozg6?o zW^bswj(++#!yYI4VansLwBm)~%oH3@3ecoI=#E#(Yrh70oNLfLvdmbJi}t#mzh@w< z5cxjtu?hAC8l}I)y=OUAk+DG*{Ucsk)`2hC8v++uKmj6yINDjl;m)82{Gjf)t-A4% zpQ8>=^$3$QKMYuSI;dTWYyjo4ic36{=0gTWm5j=YWKvF}&4i&`5X;j&C03XgpO{^M zU?fJwtTo+JI*7-|nx?<2-s!^CQJf>AxFhX>r?h-ts0HOy6@hkR>|roRsbm zY9+sb9}gkCsm)MQB?A(e&2kuHcR~Tno_`1A%SdFgq~e`h26?-36HW5rV6!}jIuGc8 zm<~a%!j-Ug>27?zuLw;6G=^Sa&=~1nrqH(^uRd*x&NqEj~@t z9#6w@c5&h3*8T#V*4J1+?HQz5PfWV*Eo&L|-4DOAepg=`boo_%jaAd0L;kY;2Bb^6 zyz2kRFX(bSWbBE<8GQp!yY_t5;R#*7k;W7H+NA40rLQ;AIz+E}C0{{@1$t>d)z{5* zz!$KNy2_x>x#+NOpJhvBd3lO{?2C{AYWMfv-upI9`aO10H?Q0PNzeKp=ic&DYfZaF Ns41HZ^gvD1e*xMTcCr8f delta 2894 zcmZ`*eM}t36@RmPAG;T9K8}OCqi}Z+7Y+v#;=q|$3Jdnd4ylutl2A2Lfml_+wo0H; zv`OKq)IRw*rQWvSY`R@HImRKdoCvkTid zdkeK`Gt%j1-n@Cg-+S|$nH_2Rv}tSyxpAS{EqIn~aDo7w*Z&SQ6!g)g2=x7ny~4vp z(0%NvSl$|FfOtO)cSxXQ6JkIML?O=m@)n58Le4ISvA@~s*Fikolk4l~FS8l(;ie-s zRL9T>QP7MSQB&2RrelJ-0B|A*!9NQa|OU3|{T>TM;$xS#E@ zEyUXGFjF%Ph4WQFYx-+<$NlOlo8QXBjb)*+U zF+qNE^hF|P`q7XQy^+48(j3?XgOcZzU$Vo{lpQ()b{I_He-@K|6v^qLTT6(gwMnr= zJIe0Z>jHZ)uct7-DCW6aT^-n^V%~I0?4EnANr}JCqK-i95LRSr#rZ!7!xq9g{(puD z+YliumTC8}Z^M3{6X+?5>+E6dd!4YL19bLOwyC`sT`E+vKw9P$!`XD*FiH=j z?S6*OA}(Z|u?Sy~5ItH_EY_q=)}m!+35fX==mC99)PLiE;7I)L)v zLBn`_!T~d3B=p(;Ay7i2lxFtdGCI+}_O&L6zn+BV4={d8^l1U&%Pd!)@-rMuC5p%-pE|!?)qMr`m7rb))jSxI zlD&;OUlc(@O)T6IiN+Mk>mJ>#WHrG(I*;TJI^fbLqK6%neoZ`VyBu!R4$SZ9CZypU zrj*9|Z^9bJh%0lh9EwgCLFYTD^S5Q7zwrOCDv%al60h6cGl) z_G2e`9=(Rq`TZoj<$5w!V*jrMILMI=`o!HZ!QWq3fO}nvQvgjtypA}bH8Ol-SVvBt zhTzErF_95d75oI2Y?SBMX9NUDiKQrk(u|H0*cj|4%HD_fHbjopBY_ zHkLwhg~uPG4`DwK)tBCd*H>sKJw9Dd_hF8yYP*`+MX3G;zd3DMGM}hMOjOhL4)tb4 zP*V|`8d^0z;5R6s>UbSeCLm}!Dlmz_ZL3CUFxaN@OFpDj&@(X1w&(k$;1)`Tf!<@! z=kML!>ZNJqqB|YjZ!Dx&P#w9Xs%`3Ch&sPz3ULno|IsB_E zk^jiHM1W4Vc`0{xn+t-J2TXMV9t{|iLP7eSm+I&eg5hh!b{2%oRxkm*2pakPHG=QP zLbbVu&&LWXibElqGHhxM^0<9g#GE0UW)}*)uJI6^`(BM1XSN_z{#T6Cvl$wTF%EY= z(N>2~PO+A9LpH<;3xg$R3h3P*0zGl2(u~M6;~;wq%j>Uj5-y0mh&FyJ-or)x|3=$U z!K_FhN1qi{Jkus1D+~ox4k$sS508;qokYka-fqR6E!^^r579?)K!nfpLU3Z?-bx`oME-F!%L`NMQ{rg|Mp}pxcTGmK_!v^xBzvK# z-h-oqctspXORNzFpCP0Eyh(PlXxkwX7vj?}2E#RUzy3wqi!a}sJ=OTe6S{~XTsA!T zD87KQ8qL<)mG}Wq@%CCfzUFsvC(ssn58{EK_@^*lWYc(pUGQNCpL6}-K|6cY?Td{^ zv*C}T=A9zr5d@5lCk^qp(q6=93qC?+EGow}3&wrlf@OXOUBlQnEl$Q&GPAlJvNv!% zf(4WQadv3f371?LYaTbCoqFj*3oWJ;b^MSxhOyK2L}kBmA1lJb*7%S=`tIe$4%M># z#UZlD-YTBBf70BE)*9C0De1yHKZV6lUIlsu<*a0l$2s>0>ZgQr^_vP)?$>T{<$eTJ z3q4!Shn#=$@UJBEN;>Z;wgY}DZDrG*Zoed9vF~fB2eai#?1_@=PY3-EPA~9B2^&_> zY|){ifg$YOuC%+C!!WrMy5xyDpqSHqg^XI7ep7ApT&@Hb)E^DX;j*5Dry Tf@SQCze#@C0)3e!{MG*fK^mEM diff --git a/06_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img index 2357ef5e93fd8fe7d0716198726efb7c85810edc..2d181336afb87cade1e96e18297d2750aef5ff8e 100755 GIT binary patch delta 2839 zcmZ`)eM}t36`$F?J$BF7zO>rszmFk@0x>*Q9>rFh)+x0B3e=;+G*A* z)X?zN9|~S7O#0Xj- z7-o#>sXC%B#032xgp5^^%8$%8VS4qHsDE+Xq2E6u=__4Mefe;nzH~^|6{2SDh}5WG z-9JMI-J^P))lQ_x*-40gT-yyxj~u!WWn_+Yy% zzVzxR5?PGWz|A>IqCa!0nX^LazTwNFobJPd;_x^3CDm}tM$#YhT?~dKGN3w1v_&HQ zJ(%x{6~=CWONbt0PAq9iip9o{*{Zao<^U3X3JE`n#M@DT{rc*uHXQ}gCdIDVlTB*; z6g%jwh(%GV@zuQMpL{Q(*NbJtGA+I7zd)w_d1R3wLdvaXsB^G00ndcC%&KNy)89<9 z3jf7 zZQ?$P5%^0V>fOBN6|th>WD|*>NpYQw!30ZRSIu8Em!k>w8{c{qX`h4*1W#*BOef`MWC_6>@1@KJNW$;XqIoAYZ z34E)-c@Lo|IKst2%+CTR{~szO(GB@(=A7Ueo=1+)ImuhMMKAlK{0Hh~=ao>|xuf%Y zyC~gx1;kiDMZhj^jJncS%7|NK|5QqRf}2Ed1O5W+xsNVN7RQ+4s;coCerU@MHFMcP z-r{>tLhijEdxigV{bBaB>j~+sAP=8oFS#~6fqaJbFws>PoQA4f|rg?ucHzTa5?6qOi>(*Jh_v1}i1h43OyYI^4;XV>bQ!TBvHw=uj9v#` z6%~0Md3@+2?%0&rAA$TGi%caTQ%O1jG9@8@nzMzD$oldh^Nqtp8>gA9f>IQaWn+oG zmj66$ySjDb4N7%3zo{lhKpT$Ke1}f=qOG_H8J!0MJLVqOK=08+4FSE##M}m|r`w1& z(i7I?-vqSzZIXTvSueCvE%9^S9!0cAQBUu3qD2b&LY+e|T{F+~7MZfb6C(k7CIO>m z%`B`XzmAXKp{1rhh02N%JirvL#^COTS!%q%E%PgG3?wFKg)SF=8E zupNb4V-xb!X3Yn^-z8yp*n&EW_JV(^djS7LB&6LbAsM7FDz}zuyxnvEW=ztT5#s>l zv@1?lF2#DzCTn;!`WaV*>gvpYA?Oyv(d_rPSV>!;Vqb#!F95Cp2=53S(BD~W zk=Iup!h+WjDH%(g5%hbe`PscjcC=`-aJrnGDJsu@@aj?R8hf+ILtkX?6>TrX^4(j= z4j*@BFzWUTQ)O(kyFA8CE@js*04Ul&dSI?<?S_CF zUm%suEmV!WcD83aaLrshr3_y`YSuM%pqdg`zm1YO`@OqXfzAkY@^`X-yB~{nZ_xyl zCNf#AiI{(&)_`JETR|jFDXadq?HRn5@pE|f4*I;i@I=HFVstA1B5_V>OQ#Gvun$$W zTb)M1{UMM^cU{&$sn>d7zvGLn-<)o9%P+_ZZ%rSdVXWFgbN6Iz7s-5tHCa5}DZhJt zpb~AA8*ZEOuo=$_)pV=IuSTj)1kb?b8e&&a%JGby@V*?YLf^O5EU0(~EOk)x0efJY zd!Q%4E#t>x7R;|-03ePiP zFZ$})gij5QC8G9X!w#K7+?-#S` zl6|pef7TplZ;i%BCnEu+BJc*BO1L+P{k3;)kPwfDW*{cn*^`OAxe_F#=i z9e!-@=koZIEjw-fSzFHK|3-i{`d=tL5X`#mw&Q)nmd`?Fzx9PU?&6LZp^Jt*hv=IG5M_G8@JaB>~+Kzg2$+z^#_}kUOQqhdox(~KcU%u Aj{pDw delta 2879 zcmZ`)eM}t36@RmPd$)VR=HpoI0Gm5Z0Uy|Luzf5>-hZt35@H54%;BMzKm<5m)LT$3>M6E+oZZ;Ov_1 zo4w`Y(q^R9zIpTJz2AH9H#4bS*LROJlD}w8ACmD40;whP-yLw80K8opFVPa8f|le# zd!4p$RpgNiPjSVBi+JgCeD!V-l#IU(V(i}wkq)UkD@z>8L0jI4^7gD8>_VAN@Y}e` z8u|qWC2O*v^f1Y((G91P0QMNXOO68tr%FM46vd3@-5}Uo{1oKStaiP7?_C zIhD*$Ice|kcZrnlMMaADg?f@olYavYi1OJoQGmg?0AarX1Jn4=@P6a#2rj{UjA`Cm zm*h)~SLmFu-Twe4dOs%IhKcXjSNr$rSO_i7cTPRipv0b~-*o!bBUqyN9oPRvh}aR{ z!2dEvxECX2_;TYm$_|u!T%bK?wp=T{({k-N%03s&Cji1-TUzyQ^ez@E86eG5is@>& zYMS2TsJk2C0gMY7ml|Rlrg_;&^2LU@!QQmu3LQJps^7$5&AgljCxm;7luQC`rg>l8 zMJH&?dB8MxjX7a5h=7*=4>&J^dm6z_Ill^=>S{QT=opBn5#f3uu!uk&Yt>f}h{^fV z3ie|x2c)D9v(5n#T`=h<-t?gxW*F`3o@s#Cvq@+=gZ@)|so^K3>6O~EWfbR(Xh^8H z#e0zn!3nRHs(oKixQL$MLHmNsvX(VUc`x#DSMr@33Hg&7qn8%wdDqbcoqxc-5+pFE zvy&l-jKrL=$b*5Ln7@gHj$$X8QI6pMNh}VNqXiHi-l${@&NDoN`5bn_yK}ru4~c&! zGOd?_n~lDi#!f=+8^)N(Usb{DUG0&msO<+|PV;5}3b8e=Sv3 z$%YTDhmgEu4tSTXwc)(ymU@cG%zm6sOM8SdP8>F9rMvtgX4_1}^ku}TVuOY-YSvaZ zHsRh#ju;+e@!?jTM{+D3Y|&RcuZ(qpf2z;+okrhp(Vx5b2nOdK9;Y9<%ekK=>8yLN zS|WVed*&{>j+od6E6gu*t@q?O7zbzw;HAStuZ3kDp&eOXnw*>tAmky74(#ZatmoyE zlN<&}@m}n`H=V%VuXpwt@??&88)}pvcrk5;FJZMYaQHW}(s)&dB6WfvVD)(vYySC~ zE$XT$$=e~vamRkfCQ(7+5QHEaH)&hmaY8S=pLdxMgGM)2sR>sMZ#o!BA-96jo!UJp z=OOuK_p#e+k9MNFTzd#JP1Oo|Y9G-OFM>ABJB(yL(M^@tXLmUD&qAD@3OV%3Rr3tH zQ2|NEs*$q7f{{Q1CNX^Ls_6{`T6ODsPn2tKLWI`mS8$iN&~N9r77fVAbcfE6xDi1+ z*+kFgZz>w8jBL^p2`I1j3Fy1N09%T~I71>rhc_I0>9iyu#_rrunbs7pS|ioNEox+QN#w z+?z?XIQ>qcha90#6>cj)_03N)JN%flx9A&%J}&B~?-f?4Y|FW<{xt-uI3bh+N&*(p zT-ELZxCZPb?iqDtDw)yWwoIL%#D?+B z?30VXQ77O-k2(pA4(Xo|V+^!<^(fQ9Nbrd^>q?RETmsG^5Y+Gpj1}dQ@T6ui$AyT94i`cV#izxhHC}xp&oCHZ1p}cJA0gLv*l1 zEgBI#QGnO{LFK~N)DU0N(}!`cB*MOMl{#5&wH5P>xuB7If9+CEhJ-^)+17YGLy z#~N*egZ+aj?X&gBTWhShYxL6Jmh4r}`?A}y!`66n-TW8!+`@+zzq038Ol$bdvisLi zzHF}-e8u{Jy1C>f%pEF@jG7q8QnvfBRuczlS< diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index e33329fd8..75ff9c05c 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -602,7 +602,7 @@ diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + + STAGE1_PAGE_DESCRIPTOR::AF::True + + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + + STAGE1_PAGE_DESCRIPTOR::VALID::True -+ + attribute_fields.clone().into(), ++ + (*attribute_fields).into(), + ); + + Self { value: val.get() } diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs index f38d0895c..eba8e12b5 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -235,7 +235,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs index f38d0895c..eba8e12b5 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -235,7 +235,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } diff --git a/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs index 73a93ff73..665631f49 100644 --- a/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -235,7 +235,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs index 73a93ff73..665631f49 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -235,7 +235,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 6d35086fb..85aa5138a 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -659,7 +659,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + phys_pages: &PageSliceDescriptor, + attr: &AttributeFields, + ) -> Result<(), &'static str> { -+ assert_eq!(self.initialized, true, "Translation tables not initialized"); ++ assert!(self.initialized, "Translation tables not initialized"); + + let p = phys_pages.as_slice(); + let v = virt_pages.as_slice(); @@ -700,7 +700,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + &mut self, + num_pages: usize, + ) -> Result, &'static str> { -+ assert_eq!(self.initialized, true, "Translation tables not initialized"); ++ assert!(self.initialized, "Translation tables not initialized"); + + if num_pages == 0 { + return Err("num_pages == 0"); @@ -2213,7 +2213,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/main.rs 14_virtual_mem_part2_m diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs --- 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs +++ 14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs -@@ -0,0 +1,221 @@ +@@ -0,0 +1,216 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -2334,12 +2334,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + ); + info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); + -+ for i in self -+ .inner -+ .iter() -+ .filter(|x| x.is_some()) -+ .map(|x| x.unwrap()) -+ { ++ for i in self.inner.iter().flatten() { + let virt_start = i.virt_start_addr; + let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; + let phys_start = i.phys_pages.start_addr(); @@ -2418,7 +2413,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu/mapping_record.rs 1 + mmio_descriptor: &MMIODescriptor, + new_user: &'static str, +) -> Option> { -+ let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); ++ let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); + + KERNEL_MAPPING_RECORD.write(|mr| { + let dup = mr.find_duplicate(&phys_pages)?; @@ -3068,7 +3063,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 14_virtual_mem_p + name: &'static str, + mmio_descriptor: &MMIODescriptor, +) -> Result, &'static str> { -+ let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); ++ let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); + let offset_into_start_page = + mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; + diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs index 15784a1b6..df7413614 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -237,7 +237,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } @@ -364,7 +364,7 @@ impl memory::mmu::translation_table::interface::Transla phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); let p = phys_pages.as_slice(); let v = virt_pages.as_slice(); @@ -399,7 +399,7 @@ impl memory::mmu::translation_table::interface::Transla &mut self, num_pages: usize, ) -> Result, &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); if num_pages == 0 { return Err("num_pages == 0"); diff --git a/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs index 360e149ae..9f10ad72b 100644 --- a/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs +++ b/14_virtual_mem_part2_mmio_remap/src/memory/mmu.rs @@ -182,7 +182,7 @@ pub unsafe fn kernel_map_mmio( name: &'static str, mmio_descriptor: &MMIODescriptor, ) -> Result, &'static str> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); let offset_into_start_page = mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; diff --git a/14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs b/14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs index eab62fb39..16c27e38c 100644 --- a/14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs +++ b/14_virtual_mem_part2_mmio_remap/src/memory/mmu/mapping_record.rs @@ -118,12 +118,7 @@ impl MappingRecord { ); info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); - for i in self - .inner - .iter() - .filter(|x| x.is_some()) - .map(|x| x.unwrap()) - { + for i in self.inner.iter().flatten() { let virt_start = i.virt_start_addr; let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; let phys_start = i.phys_pages.start_addr(); @@ -202,7 +197,7 @@ pub fn kernel_find_and_insert_mmio_duplicate( mmio_descriptor: &MMIODescriptor, new_user: &'static str, ) -> Option> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); KERNEL_MAPPING_RECORD.write(|mr| { let dup = mr.find_duplicate(&phys_pages)?; diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs index 5f781bdd5..6b7227b3b 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -236,7 +236,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } @@ -374,7 +374,7 @@ impl memory::mmu::translation_table::interface::Transla phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); let p = phys_pages.as_slice(); let v = virt_pages.as_slice(); @@ -409,7 +409,7 @@ impl memory::mmu::translation_table::interface::Transla &mut self, num_pages: usize, ) -> Result, &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); if num_pages == 0 { return Err("num_pages == 0"); diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs index 969ae6dd0..106d3a902 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs @@ -208,7 +208,7 @@ pub unsafe fn kernel_map_mmio( name: &'static str, mmio_descriptor: &MMIODescriptor, ) -> Result, &'static str> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); let offset_into_start_page = mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs index eab62fb39..16c27e38c 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/memory/mmu/mapping_record.rs @@ -118,12 +118,7 @@ impl MappingRecord { ); info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); - for i in self - .inner - .iter() - .filter(|x| x.is_some()) - .map(|x| x.unwrap()) - { + for i in self.inner.iter().flatten() { let virt_start = i.virt_start_addr; let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; let phys_start = i.phys_pages.start_addr(); @@ -202,7 +197,7 @@ pub fn kernel_find_and_insert_mmio_duplicate( mmio_descriptor: &MMIODescriptor, new_user: &'static str, ) -> Option> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); KERNEL_MAPPING_RECORD.write(|mr| { let dup = mr.find_duplicate(&phys_pages)?; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs index a58a3bb9c..a2758b157 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -236,7 +236,7 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::AF::True + STAGE1_PAGE_DESCRIPTOR::TYPE::Page + STAGE1_PAGE_DESCRIPTOR::VALID::True - + attribute_fields.clone().into(), + + (*attribute_fields).into(), ); Self { value: val.get() } @@ -401,7 +401,7 @@ impl phys_pages: &PageSliceDescriptor, attr: &AttributeFields, ) -> Result<(), &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); let p = phys_pages.as_slice(); let v = virt_pages.as_slice(); @@ -436,7 +436,7 @@ impl &mut self, num_pages: usize, ) -> Result, &'static str> { - assert_eq!(self.initialized, true, "Translation tables not initialized"); + assert!(self.initialized, "Translation tables not initialized"); if num_pages == 0 { return Err("num_pages == 0"); diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs index d1a5a3e65..f9c40d35f 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs @@ -213,7 +213,7 @@ pub unsafe fn kernel_map_mmio( name: &'static str, mmio_descriptor: &MMIODescriptor, ) -> Result, &'static str> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); let offset_into_start_page = mmio_descriptor.start_addr().into_usize() & bsp::memory::mmu::KernelGranule::MASK; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs index eab62fb39..16c27e38c 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory/mmu/mapping_record.rs @@ -118,12 +118,7 @@ impl MappingRecord { ); info!(" -------------------------------------------------------------------------------------------------------------------------------------------"); - for i in self - .inner - .iter() - .filter(|x| x.is_some()) - .map(|x| x.unwrap()) - { + for i in self.inner.iter().flatten() { let virt_start = i.virt_start_addr; let virt_end_inclusive = virt_start + i.phys_pages.size() - 1; let phys_start = i.phys_pages.start_addr(); @@ -202,7 +197,7 @@ pub fn kernel_find_and_insert_mmio_duplicate( mmio_descriptor: &MMIODescriptor, new_user: &'static str, ) -> Option> { - let phys_pages: PageSliceDescriptor = mmio_descriptor.clone().into(); + let phys_pages: PageSliceDescriptor = (*mmio_descriptor).into(); KERNEL_MAPPING_RECORD.write(|mr| { let dup = mr.find_duplicate(&phys_pages)?; diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 30ce719cf6080dcd104be8b5465aa8aa91e66e95..96d8bfb1817aef0dfb5458871a1ea93582acea84 100755 GIT binary patch delta 2776 zcmb7GZA@F&89wJ8e_R^~AN~UKVH*;_q*)5#GpKMeG!VLq#zR|GrNb=hLPEO?`eUoM z)XpYV6llfip$#FaO4@bJHOW#5rqxq5NSW3w?XqQAvgOCr3*9suhjgJN!xXrCuYDZQ7&YA&pyyD#C%2`w(dmS?qS7$fwy^f96?fdGIJJn{ zFDwS$y}PJ;4DV%{W>A$#-@S&*bqGg`z}ppRo|RY<48e0yyZ1JuSLB+9cp)IQO_~> zU4d#U5Hd@|43*hR=7=@;Erf;zj4G?4dWmF~EVVrdFq%aFaz`WJ@`$TP8pL(|p5=HA z`5tkWSb}rJ6jTgJiV#zbzijJ6j_W5Xh0rK~;dKFTBKT&~zDg7tnL{;=kleSbuX^R1 z9&tpEXr>r;TmTfB}C%6h#XrPKda5yD5T(z$?Tpjt2 zM^qA?QocbD2>}}>AqEverCB8#G2Vw5^^HwEW6N^;y)3u@2wK)xb)|0j97$9ecl?*W z>QhMCK}=B}_r<_Z>kw6JoU&Zrr9)Se0a$p862uCLH)A;O=&P2AqvQlsJ_?op6o7<@ zJ?Oyp=yK6(_Sabh|75|zcY#U_{BzI-MC6eWGw?zzp+63Sj z)vZI53&hMm0kb5Vwnf!bfx3NdXCkn$r>Q;*UG zd|XlGJds`z3%*x6q2HI<8iDIg7)pyPLd%>f8UUg1jSWIS_64OCR&Ro$3&a>y82mUY zw@(4fFih<~vk29ffnEOv(&K^sqj9xuptdb4f!cGhzqR2DHiW&LywfrU5gOry=LNVu zV8jU@Tw*tpzmFRyyeaQt+`%lV1#B?25a+Xc&C9GM?I-@uHTn2@wh)sErV4&}24CsY z2E6mK{LctnL9NG1=}mZ-3J~-Y8k$YQVzd!1K)jpsr?=6_cCEM4q9Cg&`$utC;MvI@pK0+V!ue=?kcB84xH(|IfKQ^Yu~>5Bp~?R(Ug-) zAotPpC*hiABx45_S!qV;@uAgDdmcaG0)UF*GiZnl^zK<5vlP(J`zd`mAX zfSXOZh5Og#FgUE-6OLXP5WC4sfz@>p6NnrT21h~ z-ZmB2=y^@RYdYN=>(U9KVvBhfTr45~D=5??%g1>%;_Vtmp$2$YzWGO4o!M4;qB-V% zwK+b7257jA^hl_soe<3<)zUkkC9)_5x?g3_neW3p*@W5C1jWxor$~BTLdZZFm7qhy zpnQwc8AVz#?t{yy?B~Qt0Tv_BZIJ>Zenmw-$!aWLEmR3EHe4KyJAm+t01MlZc=(sk zds~xV@4XC!k6Rpg7yH0cZz-p(z-=TO#ae&9C0l7%g-3@q%Rde+2a{CM;Uw2&n2F7T zHfj5zTF}PXe{c;T2SE>|brl5VqT`0EHrWdz@J<=odq$<%qZoHed*>drt>?vD) z%H#KVJf?W~6CTBihhFt?Ra2Ny;8z3uDmOnG;^9AluwN_A%^*%xS%J)?9EIYo6Z#Q2*X)6e2Qgd{XQcz z4|3k~+{j@oS(V(yELLW(EbGcnoC3x1=Y!nD9>yU4eA)ZWY&L4oYxsZNy45skYSEzh1KI0b)UvMa~wvqHF3Jt%KCHm7|y08w%=tpa*Fb2auPeM8u5m@huMrj z@8q!j+`5{JYCZpJeIKg!WejJZXDxY^$)3iCoz0KzZA0in9{W+=*UL6=ysZ+Ou4Ytz j1z2P1_f+-!`8;NH)Uih##W;;UqOL-|w8?x$miss(-kKGn+}>89X^|#wFPPni;)>5qeAgJ0eBML!vw) zZ_*~tgLjTU#pPh^p&>rM$z6dW&!BL-fTY9}?^fIoqX-*|51@$1>Fc5w#%acQC5m`E z^|1=}BK?qG&NbE1D||g(KEA@#jvr~NBVN3m#7cfAs}?M)5V_8TUMfU|f1cnwE=cc& zKkkekn^o<9nLTjg`YcQc%!s zM!iLFPXOvlgtAmVQ<>tk%c8tP*@XZeM!p)zgvvY1ZnUid7|n9%c1HkkHOceOo*HF1 zJBtJI6mM6ee3rtD_@z>U87{W#MD7baeNto>fDlr`?Yh%@!Mc4YvLlLWzYakUd~W$S zuwkfEx zoHH&#ZATY?ILd(a0uaH1CfztY?wQqsU|+TqQoIXN{2~O1@i~ec=frQ9>~{^&Ny~RU z9lwR*V$c{TC#5b(iR&P)8z3$aNm5v}Q*dSurb!8f%N(2r1lv;3+n0Dd9TYymcDhOQ zu2y=cYdSEl7z120Ac>`rh!D2M$MTWwfDHv-G)uKV77;mXN4BA-T$@6DeD62uF)Ddsk6m%Xj($+@s0qEzVm!=h`&noY8_&QsMn!AAd6W|d720CW-NnlfGnIupBIGNCwqk&8ZPvlCPNMC) z$TZF#ht>=BCrit^#BsmIzBb80fEb?*bdP{;e7eF8PV#9I}X;}_yu@C4wWfGqt2 zczp!<5533tvaEd-^lf6G4#Ru!2*C7nGayyQw=c9sft_`xtMf za+)WoS|YND&~OaSGjBx|Spwr!IALd+5!rUEo(vr4#*o71uj3j;D7?88NcrSzTj)p$3(l>TL3Upi?KKOC{bgVrKHxuR)TuRPXBu8wo$ z+Ez?1ms{i+3rdvDwVZ2x{_H-$Jv?;KOE+b#z&;wvaKr0hf5u8YM*omeQB_7*MVCzc ze$V>v{d2YhFNupH3NpTDfZw)*Wh6d^ ze?H%KLe2yq1Ez3$cXVw+G)s{ddsdZbMiDl5zlV+a{2Iwqz2Lj!2)UnxoCi38@v~(E$bm%LH${Q zHIT*`ke_Nv(KQfWMO2Vqti&X*gG9}M^pbn%@-HQ&}4x zG)h$}o*qeV0L@&6l3iMXg#K%F0O(4iIvUSS&R3ztjW&ziKpPxR zE~A)kcC1{w2Mh#AcmFFW@)JrN8>>8|+l3;G{dBc1F7H%5us0fA zB2UtcZ2!`ZMV_8yo+v<(*kqD8+g`E*x~k6Y)Wzs&Jf#&Y$i&<@Gpdczu8EBU*B_XBy< zF~YOR}E3m&h$8^6>5 xnPEg?cL^@1vA)b$FLBd71&uhI%(pq=PzW^A0@cRG& diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 5e1e1bf7d6716daa5b88a735299e89c268faf47d..76531e6ff99de9580fcf1d1b05c8eed4404543ef 100755 GIT binary patch delta 2232 zcmZuyYitx%6h3$EK4!NqrMo-b2anxtt!s-273f2voo*?H#t5#ff>C$D2c@WFYYfI8 z&Z5DTwgSVgQV0nJqjm;w^y9m+a7n5IKzNpX5l=>#8w-9A&Bd1KpsB-xUlQOZ| ztniD)R0)W=4s$ZHfE$R|h`6U1K{pXd^kST?q}%5w^E)q5PMx5jUZ%bK8E+|;x-DX^ znmBu&BDU-%0?~Gpr@m`MZ1k5vf)@_@cr$dyEfDpaA+ZhHDe7030l3oCr4Cbf%?K@5 zHvsgB1nL@)gEaNjOssB2U@H({3jz%*(_5OA8SBc)d{Px*Wta-;2z9DxCn>Z<%>AEn z9Q>#E)rqP72!^*LBL(L-f_BwGs(%C;TQGQh+FJkKGhwCQ3bMxv&yh(KV{THjuJZBHS~{JtyJ`b!uzMxEI>caz_T(D5r4kaeSg zrY&u^z$vVoD>F-XQjURdI0^h~t)Ha+ntho0e$4zs1d^dX)lYoc$VJ22Jp#zP1v_?Niboy|3H%uvhKNr7n)|(J>VXV#8*$&mj>m zQElF_KSucMdQFS!9-6R!0!t%VV0;kK6+F1qdFT-Wy0~11xLOQtg^>^x;KuI;ztp-YSCj@7d(E^Pm#5supZKJnh4`~ZC)R4)Q3BpMtzLn`24lK zXhWG?yhs>84=uRD^CI3r1R?Y41pA}-HL~RB!zCvO>0@Q3m8`3@3a_rCpOrQeY5i4g z@5~%Ar%@v7=(UsNR6AZgW14nh6fd#1M6G-l`{_zR{lV_xBBEqlK<*uHl=)$gJi5TF ztU@NnT8Nx}3*=!+RZXHwc8NvFgg9j^WKtS2hJF!d@{mAH>0>I?rmPqmOFg*;k#s{diN(}4vRVN;mBOJ%TXcaUjTwH0Fk%PXnT?L zWw8MLLqqxitzoE2=CBQmcgjJUj66PF$RWQrP+_bEWr{Y>4h1o9BEO&&Y;>M@Z)?tG z7E|4(WMq}#T4FQfI|6FN_79|Vp4a7#%FqantA*D5S+HTH^8rkuLV$sgydE9Y8^N#J zZih+z?EHY|XtNPiFWk%$$c-}DDG(?7$oU4>d QI$ynQjS+ec+vf}Y2LqztT&P+_L24r7Dor)9#_g(o2Iy1J1|f~FrkPdS zXoZ>`uh!Q75Pq02qir?l4`P)?ZCVriptksX*Ar7Ccv~a>D6FPtqI^jWfIr0aRqXfzAoohHZ=E8Qmg%mB|k(el` zMqTV%kAi+{k#1W#u)T(z3HB`A_SS+;&yuIOx*0T1p5b;l=&j6y^c&Mzscwes#SS@E zd`^j2uo8h{+X*`0M($t3q^3_i6a1w%dU7ZtTpeob9v))8vBopXy?3xOTEi)K))8f7 zl}X{B7P~S4PJ=nzh^`wuAtuIGiTcfD*H7MHl!16D!L9-6F9lmz~>}&v=Ao=Qj zuuWjEw;=g_ZON6qhfA((wMa;ITF_twp#Vcy2Zi};F@bPZpQu^NFKe1(EBGE?Z50#_ zk6J1oUk#t3&q3gS?|Gkc8;VsCA!FM&lTfy`n^cnj(+ zz>av~kQRUmCEI}F?LhH9C=ergY5~rRj}{+s?WI3hHWt>Lh2~;V9;dKPVCBWcFeqyT zlm#LgyiKr5U*(;)mP|3Ba<-Q10AWK4IyFE9x|{zF3v`v$yIgI!zOV-4xyJxlKWHKY zh>&n<{MZvHr!5Bs_L;@XPpnA3ETEj$w_GnsJ4n;(;2!`?{g9Dmt-)yUH0`l^3I){( z^Y?(3J51;l`&Z3ceNWoYY!-ivno}3O+rjZI3{#pQF5F;tG85`V^O~@IgJzIun_}$ zlAUII#y(kcFZXg#AVwU}RYyDmU7eil1}I>Fq^gYZU2!eY4P6NXA724yKSDR89@514 zSp%K8IUWnXLRyqQM+n1=W$=gRML5tfK;YHev_5?+-qjgRKacT2dNi{z5V3SAR)j(^ zIP<)oGC2)wc4qZ2Me>IBMRIGqOI`^pjTM`f*itMXD0nVvj~pY&Rdz4cuqMu$MVI5MzG8%iY2iiH3CwN0;bqQ&EaB<|dM?{p zv{;UpVI}PY^^1XFBoCBv%7%w;UxD?6TH8xxkap7S+0%6iCOGwF$Mh%LT*`!87?I7S&{0Otbd^{#tIAMmop~{c~@2^wQmIh zB@11gse`EX%5{*$V}O{5{RoY+y>bcmO}$ZAyMTMg%1~kq7R8gDe*R*2vm5q4n^TGf zxgR)6x>^y1Wl04dW#rig>WHXK=K`E zZQDI<{y*4o7wK%Y6(wUB8@m7m(*>}s@+KxnUcEKO7#0&7lV4z3ES512`sts!8*v9+ zEjaLTS}**x!kGs!9(%kFeGF|$KGn_5jm==$48Ps*PmbdqbfC+I;h3Us>jS!;T>J{+Y@&SKm{KX Date: Thu, 29 Apr 2021 23:41:05 +0200 Subject: [PATCH 069/214] Add chainloader test to devtool --- utils/devtool.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/utils/devtool.rb b/utils/devtool.rb index 494de032b..7639a1b41 100755 --- a/utils/devtool.rb +++ b/utils/devtool.rb @@ -188,6 +188,7 @@ def ready_for_publish clean make_xtra + test_xtra test_unit test_integration clean From f6c48b54744e8f8b690c5ed45df6e6636e848e63 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 30 Apr 2021 00:03:55 +0200 Subject: [PATCH 070/214] Remove -i; Add chainloader test to CI --- .github/workflows/test_xtra.yml | 47 +++++++++++++++++++++++++++++++++ 06_uart_chainloader/Makefile | 2 +- 06_uart_chainloader/README.md | 2 +- 07_timestamps/README.md | 2 +- 4 files changed, 50 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/test_xtra.yml diff --git a/.github/workflows/test_xtra.yml b/.github/workflows/test_xtra.yml new file mode 100644 index 000000000..bfa7b0290 --- /dev/null +++ b/.github/workflows/test_xtra.yml @@ -0,0 +1,47 @@ +name: Xtra-Tests + +on: + push: + branches: + - master + paths-ignore: + - "utils/**" + - "doc/**" + - "docker/**" + pull_request: + branches: + - master + paths-ignore: + - "utils/**" + - "doc/**" + - "docker/**" + schedule: + - cron: "0 5 * * *" + +jobs: + build: + name: Run xtra tests + runs-on: ubuntu-20.04 + + steps: + - uses: actions/checkout@v2 + - name: Set up Ruby 2.x + uses: actions/setup-ruby@v1 + with: + ruby-version: "2.7" + - name: Set up Rust nightly + run: | + rm rust-toolchain + rustup self update + rustup toolchain install nightly --component llvm-tools-preview + rustup default nightly + rustup target add aarch64-unknown-none-softfloat + cargo install cargo-binutils + - name: Set up Ruby + run: | + gem install bundler + bundle config set without 'uart' + bundle install --retry 3 + - name: Make all + run: | + bundle exec ruby utils/devtool.rb test_xtra diff --git a/06_uart_chainloader/Makefile b/06_uart_chainloader/Makefile index 6805ed1cf..a8d38ffca 100644 --- a/06_uart_chainloader/Makefile +++ b/06_uart_chainloader/Makefile @@ -70,7 +70,7 @@ DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index ba6fc0c13..44fc10282 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -157,7 +157,7 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -+DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) ++DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux diff --git a/07_timestamps/README.md b/07_timestamps/README.md index aa6d237c8..88b829086 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -82,7 +82,7 @@ diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) --DOCKER_TEST = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) +-DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) # Dockerize commands that require USB device passthrough only on Linux From aa0e194c75c902765e68f0887143412260734766 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 30 Apr 2021 09:07:23 +0200 Subject: [PATCH 071/214] CI: Use rust-toolchain Latest nightly broke a dependency. Need to wait for it to be fixed. Pin the toolchain for now. --- .github/workflows/build_rpi3.yml | 5 ----- .github/workflows/build_rpi4.yml | 5 ----- .github/workflows/sanity.yml | 7 ------- .github/workflows/test_integration.yml | 5 ----- .github/workflows/test_unit.yml | 5 ----- .github/workflows/test_xtra.yml | 5 ----- 6 files changed, 32 deletions(-) diff --git a/.github/workflows/build_rpi3.yml b/.github/workflows/build_rpi3.yml index c3bc3992e..a9e4db3c2 100644 --- a/.github/workflows/build_rpi3.yml +++ b/.github/workflows/build_rpi3.yml @@ -31,11 +31,6 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component llvm-tools-preview - rustup default nightly - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils - name: Set up Ruby run: | diff --git a/.github/workflows/build_rpi4.yml b/.github/workflows/build_rpi4.yml index 8d479e8d9..9f9730dbb 100644 --- a/.github/workflows/build_rpi4.yml +++ b/.github/workflows/build_rpi4.yml @@ -31,11 +31,6 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component llvm-tools-preview - rustup default nightly - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils - name: Set up Ruby run: | diff --git a/.github/workflows/sanity.yml b/.github/workflows/sanity.yml index 2c4551f2d..c8d8999b6 100644 --- a/.github/workflows/sanity.yml +++ b/.github/workflows/sanity.yml @@ -31,13 +31,6 @@ jobs: - name: Set up Prettier run: | npm install prettier - - name: Set up Rust nightly - run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component rustfmt clippy - rustup default nightly - rustup target add aarch64-unknown-none-softfloat - name: Setup misspell run: | curl -L -o ./install-misspell.sh https://git.io/misspell diff --git a/.github/workflows/test_integration.yml b/.github/workflows/test_integration.yml index fa2fda987..4105894b9 100644 --- a/.github/workflows/test_integration.yml +++ b/.github/workflows/test_integration.yml @@ -31,11 +31,6 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component llvm-tools-preview - rustup default nightly - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils - name: Set up Ruby run: | diff --git a/.github/workflows/test_unit.yml b/.github/workflows/test_unit.yml index 741ff352b..48c1a758f 100644 --- a/.github/workflows/test_unit.yml +++ b/.github/workflows/test_unit.yml @@ -31,11 +31,6 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component llvm-tools-preview - rustup default nightly - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils - name: Set up Ruby run: | diff --git a/.github/workflows/test_xtra.yml b/.github/workflows/test_xtra.yml index bfa7b0290..8a7287a81 100644 --- a/.github/workflows/test_xtra.yml +++ b/.github/workflows/test_xtra.yml @@ -31,11 +31,6 @@ jobs: ruby-version: "2.7" - name: Set up Rust nightly run: | - rm rust-toolchain - rustup self update - rustup toolchain install nightly --component llvm-tools-preview - rustup default nightly - rustup target add aarch64-unknown-none-softfloat cargo install cargo-binutils - name: Set up Ruby run: | From 668e63fea55bf90c091de7f4059e3cae0f452daf Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 30 Apr 2021 09:14:33 +0200 Subject: [PATCH 072/214] CI: Explicitly add clippy --- .github/workflows/sanity.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/sanity.yml b/.github/workflows/sanity.yml index c8d8999b6..3bb343092 100644 --- a/.github/workflows/sanity.yml +++ b/.github/workflows/sanity.yml @@ -23,6 +23,9 @@ jobs: uses: actions/setup-ruby@v1 with: ruby-version: "2.7" + - name: Set up Rust nightly + run: | + rustup component add clippy - name: Set up Bundler run: | gem install bundler From 39a066c246d729f6d3be0060f705c33200af0b1a Mon Sep 17 00:00:00 2001 From: Alex Date: Sat, 15 May 2021 23:14:32 +0800 Subject: [PATCH 073/214] Update Dockerfile Use https when git clone qemu to avoid connect timeout errors. --- docker/rustembedded-osdev-utils/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/rustembedded-osdev-utils/Dockerfile b/docker/rustembedded-osdev-utils/Dockerfile index 47f69e438..1366abf2d 100644 --- a/docker/rustembedded-osdev-utils/Dockerfile +++ b/docker/rustembedded-osdev-utils/Dockerfile @@ -44,7 +44,7 @@ RUN set -ex; \ gem install bundler; \ bundle install --retry 3 --without development; \ # QEMU - git clone git://git.qemu.org/qemu.git; \ + git clone https://git.qemu.org/git/qemu.git; \ cd qemu; \ git checkout tags/v5.2.0; \ ./configure --target-list=aarch64-softmmu --enable-modules \ From 7f666000ceb65cce8134cd0f0e0b4cf066523daf Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Fri, 2 Jul 2021 23:04:18 +0200 Subject: [PATCH 074/214] Init DRAM in assembly instead of Rust See https://github.com/rust-embedded/cortex-m-rt/issues/300 --- 02_runtime_init/README.md | 200 ++++-------------- 02_runtime_init/src/_arch/aarch64/cpu/boot.rs | 8 +- 02_runtime_init/src/_arch/aarch64/cpu/boot.s | 21 +- 02_runtime_init/src/bsp/raspberrypi.rs | 1 - 02_runtime_init/src/bsp/raspberrypi/link.ld | 10 +- 02_runtime_init/src/bsp/raspberrypi/memory.rs | 37 ---- 02_runtime_init/src/main.rs | 6 +- 02_runtime_init/src/memory.rs | 30 --- 02_runtime_init/src/runtime_init.rs | 37 ---- 03_hacky_hello_world/README.md | 14 +- .../src/_arch/aarch64/cpu/boot.rs | 8 +- .../src/_arch/aarch64/cpu/boot.s | 21 +- 03_hacky_hello_world/src/bsp/raspberrypi.rs | 1 - .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 37 ---- 03_hacky_hello_world/src/main.rs | 6 +- 03_hacky_hello_world/src/memory.rs | 30 --- 03_hacky_hello_world/src/runtime_init.rs | 37 ---- 04_safe_globals/README.md | 8 +- 04_safe_globals/src/_arch/aarch64/cpu/boot.rs | 8 +- 04_safe_globals/src/_arch/aarch64/cpu/boot.s | 21 +- 04_safe_globals/src/bsp/raspberrypi.rs | 1 - 04_safe_globals/src/bsp/raspberrypi/link.ld | 10 +- 04_safe_globals/src/bsp/raspberrypi/memory.rs | 37 ---- 04_safe_globals/src/main.rs | 6 +- 04_safe_globals/src/memory.rs | 30 --- 04_safe_globals/src/runtime_init.rs | 37 ---- 05_drivers_gpio_uart/README.md | 33 ++- .../src/_arch/aarch64/cpu/boot.rs | 8 +- .../src/_arch/aarch64/cpu/boot.s | 21 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 32 --- 05_drivers_gpio_uart/src/main.rs | 6 +- 05_drivers_gpio_uart/src/memory.rs | 30 --- 05_drivers_gpio_uart/src/runtime_init.rs | 37 ---- 06_uart_chainloader/README.md | 78 +++---- 06_uart_chainloader/demo_payload_rpi3.img | Bin 6920 -> 6696 bytes 06_uart_chainloader/demo_payload_rpi4.img | Bin 6760 -> 6552 bytes .../src/_arch/aarch64/cpu/boot.rs | 8 +- .../src/_arch/aarch64/cpu/boot.s | 24 ++- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 28 --- 06_uart_chainloader/src/main.rs | 6 +- 06_uart_chainloader/src/memory.rs | 30 --- 06_uart_chainloader/src/runtime_init.rs | 37 ---- 07_timestamps/README.md | 82 +++---- 07_timestamps/src/_arch/aarch64/cpu/boot.rs | 8 +- 07_timestamps/src/_arch/aarch64/cpu/boot.s | 21 +- 07_timestamps/src/bsp/raspberrypi/link.ld | 10 +- 07_timestamps/src/bsp/raspberrypi/memory.rs | 32 --- 07_timestamps/src/main.rs | 6 +- 07_timestamps/src/memory.rs | 30 --- 07_timestamps/src/runtime_init.rs | 37 ---- .../src/_arch/aarch64/cpu/boot.rs | 8 +- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s | 21 +- 08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 32 --- 08_hw_debug_JTAG/src/main.rs | 6 +- 08_hw_debug_JTAG/src/memory.rs | 30 --- 08_hw_debug_JTAG/src/runtime_init.rs | 37 ---- 09_privilege_level/README.md | 57 +++-- .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 32 --- 09_privilege_level/src/main.rs | 6 +- 09_privilege_level/src/memory.rs | 30 --- 09_privilege_level/src/runtime_init.rs | 37 ---- .../README.md | 72 ++++--- .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 25 +-- .../src/main.rs | 5 +- .../src/memory.rs | 25 --- .../src/runtime_init.rs | 37 ---- 11_exceptions_part1_groundwork/README.md | 4 +- .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 25 +-- 11_exceptions_part1_groundwork/src/main.rs | 5 +- 11_exceptions_part1_groundwork/src/memory.rs | 25 --- .../src/runtime_init.rs | 37 ---- 12_integrated_testing/README.md | 125 ++++------- .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/_arch/aarch64/memory/mmu.rs | 13 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 25 +-- 12_integrated_testing/src/lib.rs | 12 +- 12_integrated_testing/src/memory.rs | 62 ------ 12_integrated_testing/src/runtime_init.rs | 40 ---- 13_exceptions_part2_peripheral_IRQs/README.md | 8 +- .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/_arch/aarch64/memory/mmu.rs | 13 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 25 +-- .../src/lib.rs | 12 +- .../src/memory.rs | 62 ------ .../src/runtime_init.rs | 40 ---- 14_virtual_mem_part2_mmio_remap/README.md | 148 +++++++------ .../src/_arch/aarch64/cpu/boot.rs | 10 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 24 +-- .../src/bsp/raspberrypi/memory/mmu.rs | 13 +- 14_virtual_mem_part2_mmio_remap/src/lib.rs | 12 +- 14_virtual_mem_part2_mmio_remap/src/memory.rs | 58 +---- .../src/runtime_init.rs | 40 ---- .../README.md | 38 ++-- .../src/_arch/aarch64/cpu/boot.rs | 11 +- .../src/_arch/aarch64/cpu/boot.s | 23 +- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 24 +-- .../src/lib.rs | 12 +- .../src/memory.rs | 58 +---- .../src/runtime_init.rs | 40 ---- .../README.md | 65 +++--- .../src/_arch/aarch64/cpu/boot.rs | 15 +- .../src/_arch/aarch64/cpu/boot.s | 25 ++- .../src/bsp/raspberrypi/link.ld | 10 +- .../src/bsp/raspberrypi/memory.rs | 24 +-- .../src/lib.rs | 7 +- .../src/memory.rs | 58 +---- .../src/runtime_init.rs | 41 ---- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 8032 -> 7856 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6736 -> 6584 bytes X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs | 8 +- X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s | 21 +- X1_JTAG_boot/src/bsp/raspberrypi/link.ld | 10 +- X1_JTAG_boot/src/bsp/raspberrypi/memory.rs | 32 --- X1_JTAG_boot/src/main.rs | 6 +- X1_JTAG_boot/src/memory.rs | 30 --- X1_JTAG_boot/src/runtime_init.rs | 37 ---- 136 files changed, 872 insertions(+), 2585 deletions(-) delete mode 100644 02_runtime_init/src/bsp/raspberrypi/memory.rs delete mode 100644 02_runtime_init/src/memory.rs delete mode 100644 02_runtime_init/src/runtime_init.rs delete mode 100644 03_hacky_hello_world/src/bsp/raspberrypi/memory.rs delete mode 100644 03_hacky_hello_world/src/memory.rs delete mode 100644 03_hacky_hello_world/src/runtime_init.rs delete mode 100644 04_safe_globals/src/bsp/raspberrypi/memory.rs delete mode 100644 04_safe_globals/src/memory.rs delete mode 100644 04_safe_globals/src/runtime_init.rs delete mode 100644 05_drivers_gpio_uart/src/memory.rs delete mode 100644 05_drivers_gpio_uart/src/runtime_init.rs delete mode 100644 06_uart_chainloader/src/memory.rs delete mode 100644 06_uart_chainloader/src/runtime_init.rs delete mode 100644 07_timestamps/src/memory.rs delete mode 100644 07_timestamps/src/runtime_init.rs delete mode 100644 08_hw_debug_JTAG/src/memory.rs delete mode 100644 08_hw_debug_JTAG/src/runtime_init.rs delete mode 100644 09_privilege_level/src/memory.rs delete mode 100644 09_privilege_level/src/runtime_init.rs delete mode 100644 10_virtual_mem_part1_identity_mapping/src/runtime_init.rs delete mode 100644 11_exceptions_part1_groundwork/src/runtime_init.rs delete mode 100644 12_integrated_testing/src/runtime_init.rs delete mode 100644 13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs delete mode 100644 14_virtual_mem_part2_mmio_remap/src/runtime_init.rs delete mode 100644 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs delete mode 100644 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs delete mode 100644 X1_JTAG_boot/src/memory.rs delete mode 100644 X1_JTAG_boot/src/runtime_init.rs diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 5cba10fb3..3dc0b3d36 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -2,8 +2,9 @@ ## tl;dr -- We extend `boot.s` to call into Rust code for the first time. There, we zero the [bss] section - before execution is halted with a call to `panic()`. +- We extend `boot.s` to call into Rust code for the first time. Before the jump + to Rust happens, a bit of runtime init work is done. +- The Rust code being called just halts execution with a call to `panic!()`. - Check out `make qemu` again to see the additional code run. ## Notable additions @@ -12,11 +13,11 @@ - New sections: `.rodata`, `.got`, `.data`, `.bss`. - A dedicated place for linking boot-time arguments that need to be read by `_start()`. - `_start()` in `_arch/__arch_name__/cpu/boot.s`: - 1. Halt core if core != core0. - 1. Set up the `stack pointer`. - 1. Jump to the `_start_rust()` function, defined in `arch/__arch_name__/cpu/boot.rs`. -- `runtime_init()` in `runtime_init.rs`: - - Zeros the `.bss` section. + 1. Halts core if core != core0. + 1. Initializes the `DRAM` by zeroing the [bss] section. + 1. Sets up the `stack pointer`. + 1. Jumps to the `_start_rust()` function, defined in `arch/__arch_name__/cpu/boot.rs`. +- `_start_rust()`: - Calls `kernel_init()`, which calls `panic!()`, which eventually halts core0 as well. - The library now uses the [cortex-a] crate, which provides zero-overhead abstractions and wraps `unsafe` parts when dealing with the CPU's resources. @@ -64,12 +65,8 @@ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.rs 02_runtime_init/src/_arch/aarch64/cpu/boot.rs --- 01_wait_forever/src/_arch/aarch64/cpu/boot.rs +++ 02_runtime_init/src/_arch/aarch64/cpu/boot.rs -@@ -11,5 +11,23 @@ - //! - //! crate::cpu::boot::arch_boot +@@ -13,3 +13,15 @@ -+use crate::runtime_init; -+ // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); + @@ -80,13 +77,9 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.rs 02_runtime_init/src/_arc +/// The Rust entry of the `kernel` binary. +/// +/// The function is called from the assembly `_start` function. -+/// -+/// # Safety -+/// -+/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. +#[no_mangle] +pub unsafe fn _start_rust() -> ! { -+ runtime_init::runtime_init() ++ crate::kernel_init() +} diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch/aarch64/cpu/boot.s @@ -117,7 +110,7 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch // Public Code //-------------------------------------------------------------------------------------------------- .section .text._start -@@ -11,6 +29,22 @@ +@@ -11,9 +29,38 @@ // fn _start() //------------------------------------------------------------------------------ _start: @@ -126,10 +119,22 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 -+ b.ne 1f ++ b.ne parking_loop ++ ++ // If execution reaches here, it is the boot core. ++ ++ // Initialize DRAM. ++ ADR_REL x0, __bss_start ++ ADR_REL x1, __bss_end_exclusive + -+ // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. ++bss_init_loop: ++ cmp x0, x1 ++ b.eq prepare_rust ++ stp xzr, xzr, [x0], #16 ++ b bss_init_loop + ++ // Prepare the jump to Rust code. ++prepare_rust: + // Set the stack pointer. + ADR_REL x0, __boot_core_stack_end_exclusive + mov sp, x0 @@ -138,8 +143,14 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + b _start_rust + // Infinitely wait for events (aka "park the core"). - 1: wfe - b 1b +-1: wfe +- b 1b ++parking_loop: ++ wfe ++ b parking_loop + + .size _start, . - _start + .type _start, function diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs --- 01_wait_forever/src/_arch/aarch64/cpu.rs @@ -194,7 +205,7 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/cpu.rs 02_runtime_init/src/bsp/ras diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/raspberrypi/link.ld --- 01_wait_forever/src/bsp/raspberrypi/link.ld +++ 02_runtime_init/src/bsp/raspberrypi/link.ld -@@ -11,17 +11,45 @@ +@@ -11,17 +11,43 @@ PHDRS { segment_rx PT_LOAD FLAGS(5); /* 5 == RX */ @@ -230,70 +241,25 @@ diff -uNr 01_wait_forever/src/bsp/raspberrypi/link.ld 02_runtime_init/src/bsp/ra + ***********************************************************************************************/ + .data : { *(.data*) } :segment_rw + -+ /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -+ .bss : ALIGN(8) ++ /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ ++ .bss : ALIGN(16) + { + __bss_start = .; + *(.bss*); -+ . = ALIGN(8); -+ -+ . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ -+ __bss_end_inclusive = . - 8; ++ . = ALIGN(16); ++ __bss_end_exclusive = .; + } :NONE } -diff -uNr 01_wait_forever/src/bsp/raspberrypi/memory.rs 02_runtime_init/src/bsp/raspberrypi/memory.rs ---- 01_wait_forever/src/bsp/raspberrypi/memory.rs -+++ 02_runtime_init/src/bsp/raspberrypi/memory.rs -@@ -0,0 +1,37 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! BSP Memory Management. -+ -+use core::{cell::UnsafeCell, ops::RangeInclusive}; -+ -+//-------------------------------------------------------------------------------------------------- -+// Private Definitions -+//-------------------------------------------------------------------------------------------------- -+ -+// Symbols from the linker script. -+extern "Rust" { -+ static __bss_start: UnsafeCell; -+ static __bss_end_inclusive: UnsafeCell; -+} -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Return the inclusive range spanning the .bss section. -+/// -+/// # Safety -+/// -+/// - Values are provided by the linker script and must be trusted as-is. -+/// - The linker-provided addresses must be u64 aligned. -+pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { -+ let range; -+ unsafe { -+ range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); -+ } -+ assert!(!range.is_empty()); -+ -+ range -+} - diff -uNr 01_wait_forever/src/bsp/raspberrypi.rs 02_runtime_init/src/bsp/raspberrypi.rs --- 01_wait_forever/src/bsp/raspberrypi.rs +++ 02_runtime_init/src/bsp/raspberrypi.rs -@@ -4,4 +4,5 @@ +@@ -4,4 +4,4 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -// Coming soon. +pub mod cpu; -+pub mod memory; diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs --- 01_wait_forever/src/cpu.rs @@ -316,24 +282,19 @@ diff -uNr 01_wait_forever/src/cpu.rs 02_runtime_init/src/cpu.rs diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs --- 01_wait_forever/src/main.rs +++ 02_runtime_init/src/main.rs -@@ -102,14 +102,25 @@ +@@ -102,8 +102,8 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -+//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -+//! -+//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html ++//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. -#![feature(asm)] #![feature(global_asm)] #![no_main] #![no_std] - - mod bsp; +@@ -112,4 +112,11 @@ mod cpu; -+mod memory; mod panic_wait; -+mod runtime_init; -// Kernel code coming next tutorial. +/// Early init code. @@ -345,41 +306,6 @@ diff -uNr 01_wait_forever/src/main.rs 02_runtime_init/src/main.rs + panic!() +} -diff -uNr 01_wait_forever/src/memory.rs 02_runtime_init/src/memory.rs ---- 01_wait_forever/src/memory.rs -+++ 02_runtime_init/src/memory.rs -@@ -0,0 +1,30 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Memory Management. -+ -+use core::ops::RangeInclusive; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Zero out an inclusive memory range. -+/// -+/// # Safety -+/// -+/// - `range.start` and `range.end` must be valid. -+/// - `range.start` and `range.end` must be `T` aligned. -+pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -+where -+ T: From, -+{ -+ let mut ptr = *range.start(); -+ let end_inclusive = *range.end(); -+ -+ while ptr <= end_inclusive { -+ core::ptr::write_volatile(ptr, T::from(0)); -+ ptr = ptr.offset(1); -+ } -+} - diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs --- 01_wait_forever/src/panic_wait.rs +++ 02_runtime_init/src/panic_wait.rs @@ -396,46 +322,4 @@ diff -uNr 01_wait_forever/src/panic_wait.rs 02_runtime_init/src/panic_wait.rs + cpu::wait_forever() } -diff -uNr 01_wait_forever/src/runtime_init.rs 02_runtime_init/src/runtime_init.rs ---- 01_wait_forever/src/runtime_init.rs -+++ 02_runtime_init/src/runtime_init.rs -@@ -0,0 +1,37 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2021 Andre Richter -+ -+//! Rust runtime initialization code. -+ -+use crate::{bsp, memory}; -+ -+//-------------------------------------------------------------------------------------------------- -+// Private Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Zero out the .bss section. -+/// -+/// # Safety -+/// -+/// - Must only be called pre `kernel_init()`. -+#[inline(always)] -+unsafe fn zero_bss() { -+ memory::zero_volatile(bsp::memory::bss_range_inclusive()); -+} -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -+/// init code. -+/// -+/// # Safety -+/// -+/// - Only a single core must be active and running this function. -+pub unsafe fn runtime_init() -> ! { -+ zero_bss(); -+ -+ crate::kernel_init() -+} - ``` diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.rs b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.rs +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.s b/02_runtime_init/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.s +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/02_runtime_init/src/bsp/raspberrypi.rs b/02_runtime_init/src/bsp/raspberrypi.rs index 10535d7b8..8fed4b83c 100644 --- a/02_runtime_init/src/bsp/raspberrypi.rs +++ b/02_runtime_init/src/bsp/raspberrypi.rs @@ -5,4 +5,3 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. pub mod cpu; -pub mod memory; diff --git a/02_runtime_init/src/bsp/raspberrypi/link.ld b/02_runtime_init/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/02_runtime_init/src/bsp/raspberrypi/link.ld +++ b/02_runtime_init/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/02_runtime_init/src/bsp/raspberrypi/memory.rs b/02_runtime_init/src/bsp/raspberrypi/memory.rs deleted file mode 100644 index 5c6b1ea5b..000000000 --- a/02_runtime_init/src/bsp/raspberrypi/memory.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! BSP Memory Management. - -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/02_runtime_init/src/main.rs b/02_runtime_init/src/main.rs index faad6b091..cf15402fd 100644 --- a/02_runtime_init/src/main.rs +++ b/02_runtime_init/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![feature(global_asm)] #![no_main] @@ -112,9 +110,7 @@ mod bsp; mod cpu; -mod memory; mod panic_wait; -mod runtime_init; /// Early init code. /// diff --git a/02_runtime_init/src/memory.rs b/02_runtime_init/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/02_runtime_init/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/02_runtime_init/src/runtime_init.rs b/02_runtime_init/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/02_runtime_init/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 6958b8bc5..01bbf54cb 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -117,13 +117,12 @@ diff -uNr 02_runtime_init/src/bsp/raspberrypi/console.rs 03_hacky_hello_world/sr diff -uNr 02_runtime_init/src/bsp/raspberrypi.rs 03_hacky_hello_world/src/bsp/raspberrypi.rs --- 02_runtime_init/src/bsp/raspberrypi.rs +++ 03_hacky_hello_world/src/bsp/raspberrypi.rs -@@ -4,5 +4,6 @@ +@@ -4,4 +4,5 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. +pub mod console; pub mod cpu; - pub mod memory; diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs --- 02_runtime_init/src/console.rs @@ -152,9 +151,9 @@ diff -uNr 02_runtime_init/src/console.rs 03_hacky_hello_world/src/console.rs diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs --- 02_runtime_init/src/main.rs +++ 03_hacky_hello_world/src/main.rs -@@ -106,14 +106,18 @@ - //! - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -104,13 +104,17 @@ + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. + //! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. +#![feature(format_args_nl)] #![feature(global_asm)] @@ -165,13 +164,12 @@ diff -uNr 02_runtime_init/src/main.rs 03_hacky_hello_world/src/main.rs mod bsp; +mod console; mod cpu; - mod memory; mod panic_wait; +mod print; - mod runtime_init; /// Early init code. -@@ -122,5 +126,7 @@ + /// +@@ -118,5 +122,7 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/03_hacky_hello_world/src/bsp/raspberrypi.rs b/03_hacky_hello_world/src/bsp/raspberrypi.rs index fdb7551a1..73863b174 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi.rs +++ b/03_hacky_hello_world/src/bsp/raspberrypi.rs @@ -6,4 +6,3 @@ pub mod console; pub mod cpu; -pub mod memory; diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/03_hacky_hello_world/src/bsp/raspberrypi/link.ld +++ b/03_hacky_hello_world/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs b/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs deleted file mode 100644 index 5c6b1ea5b..000000000 --- a/03_hacky_hello_world/src/bsp/raspberrypi/memory.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! BSP Memory Management. - -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/03_hacky_hello_world/src/main.rs b/03_hacky_hello_world/src/main.rs index bc290ddde..de9a05768 100644 --- a/03_hacky_hello_world/src/main.rs +++ b/03_hacky_hello_world/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![feature(format_args_nl)] #![feature(global_asm)] @@ -115,10 +113,8 @@ mod bsp; mod console; mod cpu; -mod memory; mod panic_wait; mod print; -mod runtime_init; /// Early init code. /// diff --git a/03_hacky_hello_world/src/memory.rs b/03_hacky_hello_world/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/03_hacky_hello_world/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/03_hacky_hello_world/src/runtime_init.rs b/03_hacky_hello_world/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/03_hacky_hello_world/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/04_safe_globals/README.md b/04_safe_globals/README.md index 272edfce6..6b7f52c4e 100644 --- a/04_safe_globals/README.md +++ b/04_safe_globals/README.md @@ -224,7 +224,7 @@ diff -uNr 03_hacky_hello_world/src/console.rs 04_safe_globals/src/console.rs diff -uNr 03_hacky_hello_world/src/main.rs 04_safe_globals/src/main.rs --- 03_hacky_hello_world/src/main.rs +++ 04_safe_globals/src/main.rs -@@ -109,6 +109,7 @@ +@@ -107,6 +107,7 @@ #![feature(format_args_nl)] #![feature(global_asm)] #![feature(panic_info_message)] @@ -232,15 +232,15 @@ diff -uNr 03_hacky_hello_world/src/main.rs 04_safe_globals/src/main.rs #![no_main] #![no_std] -@@ -119,6 +120,7 @@ +@@ -115,6 +116,7 @@ + mod cpu; mod panic_wait; mod print; - mod runtime_init; +mod synchronization; /// Early init code. /// -@@ -126,7 +128,15 @@ +@@ -122,7 +124,15 @@ /// /// - Only a single core must be active and running this function. unsafe fn kernel_init() -> ! { diff --git a/04_safe_globals/src/_arch/aarch64/cpu/boot.rs b/04_safe_globals/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/04_safe_globals/src/_arch/aarch64/cpu/boot.rs +++ b/04_safe_globals/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/04_safe_globals/src/_arch/aarch64/cpu/boot.s b/04_safe_globals/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/04_safe_globals/src/_arch/aarch64/cpu/boot.s +++ b/04_safe_globals/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/04_safe_globals/src/bsp/raspberrypi.rs b/04_safe_globals/src/bsp/raspberrypi.rs index fdb7551a1..73863b174 100644 --- a/04_safe_globals/src/bsp/raspberrypi.rs +++ b/04_safe_globals/src/bsp/raspberrypi.rs @@ -6,4 +6,3 @@ pub mod console; pub mod cpu; -pub mod memory; diff --git a/04_safe_globals/src/bsp/raspberrypi/link.ld b/04_safe_globals/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/04_safe_globals/src/bsp/raspberrypi/link.ld +++ b/04_safe_globals/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/04_safe_globals/src/bsp/raspberrypi/memory.rs b/04_safe_globals/src/bsp/raspberrypi/memory.rs deleted file mode 100644 index 5c6b1ea5b..000000000 --- a/04_safe_globals/src/bsp/raspberrypi/memory.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! BSP Memory Management. - -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/04_safe_globals/src/main.rs b/04_safe_globals/src/main.rs index 48687ace2..82262ea17 100644 --- a/04_safe_globals/src/main.rs +++ b/04_safe_globals/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![feature(format_args_nl)] #![feature(global_asm)] @@ -116,10 +114,8 @@ mod bsp; mod console; mod cpu; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; /// Early init code. diff --git a/04_safe_globals/src/memory.rs b/04_safe_globals/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/04_safe_globals/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/04_safe_globals/src/runtime_init.rs b/04_safe_globals/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/04_safe_globals/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index b5895cd77..4125583e3 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -1120,10 +1120,14 @@ diff -uNr 04_safe_globals/src/bsp/raspberrypi/driver.rs 05_drivers_gpio_uart/src diff -uNr 04_safe_globals/src/bsp/raspberrypi/memory.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs --- 04_safe_globals/src/bsp/raspberrypi/memory.rs +++ 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs -@@ -17,6 +17,38 @@ - } - - //-------------------------------------------------------------------------------------------------- +@@ -0,0 +1,37 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter ++ ++//! BSP Memory Management. ++ ++//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + @@ -1154,21 +1158,16 @@ diff -uNr 04_safe_globals/src/bsp/raspberrypi/memory.rs 05_drivers_gpio_uart/src + pub const PL011_UART_START: usize = START + UART_OFFSET; + } +} -+ -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- - diff -uNr 04_safe_globals/src/bsp/raspberrypi.rs 05_drivers_gpio_uart/src/bsp/raspberrypi.rs --- 04_safe_globals/src/bsp/raspberrypi.rs +++ 05_drivers_gpio_uart/src/bsp/raspberrypi.rs -@@ -6,4 +6,33 @@ +@@ -6,3 +6,33 @@ pub mod console; pub mod cpu; +pub mod driver; - pub mod memory; ++pub mod memory; + +//-------------------------------------------------------------------------------------------------- +// Global instances @@ -1321,24 +1320,24 @@ diff -uNr 04_safe_globals/src/driver.rs 05_drivers_gpio_uart/src/driver.rs diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs --- 04_safe_globals/src/main.rs +++ 05_drivers_gpio_uart/src/main.rs -@@ -106,6 +106,8 @@ - //! - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -104,6 +104,8 @@ + //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. + //! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. +#![allow(clippy::upper_case_acronyms)] +#![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] #![feature(panic_info_message)] -@@ -116,6 +118,7 @@ +@@ -114,6 +116,7 @@ mod bsp; mod console; mod cpu; +mod driver; - mod memory; mod panic_wait; mod print; -@@ -127,16 +130,54 @@ + mod synchronization; +@@ -123,16 +126,54 @@ /// # Safety /// /// - Only a single core must be active and running this function. diff --git a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs +++ b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s +++ b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld b/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld +++ b/05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs index c6d65e36b..ef397a6f9 100644 --- a/05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ b/05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -47,23 +35,3 @@ pub(super) mod map { pub const PL011_UART_START: usize = START + UART_OFFSET; } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/05_drivers_gpio_uart/src/main.rs b/05_drivers_gpio_uart/src/main.rs index d72919118..0261a136e 100644 --- a/05_drivers_gpio_uart/src/main.rs +++ b/05_drivers_gpio_uart/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] @@ -119,10 +117,8 @@ mod bsp; mod console; mod cpu; mod driver; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; /// Early init code. diff --git a/05_drivers_gpio_uart/src/memory.rs b/05_drivers_gpio_uart/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/05_drivers_gpio_uart/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/05_drivers_gpio_uart/src/runtime_init.rs b/05_drivers_gpio_uart/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/05_drivers_gpio_uart/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 44fc10282..18741f043 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -232,26 +232,36 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- -@@ -34,20 +45,31 @@ - and x1, x1, _core_id_mask - ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs - cmp x1, x2 -- b.ne 1f -+ b.ne 2f -+ -+ // If execution reaches here, it is the boot core. +@@ -39,23 +50,35 @@ + // If execution reaches here, it is the boot core. + + // Initialize DRAM. +- ADR_REL x0, __bss_start +- ADR_REL x1, __bss_end_exclusive ++ ADR_ABS x0, __bss_start ++ ADR_ABS x1, __bss_end_exclusive + + bss_init_loop: + cmp x0, x1 +- b.eq prepare_rust ++ b.eq relocate_binary + stp xzr, xzr, [x0], #16 + b bss_init_loop -- // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // Next, relocate the binary. ++relocate_binary: + ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. + ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. + ADR_ABS x2, __binary_nonzero_end_exclusive + -+1: ldr x3, [x0], #8 ++copy_loop: ++ ldr x3, [x0], #8 + str x3, [x1], #8 + cmp x1, x2 -+ b.lo 1b - ++ b.lo copy_loop ++ + // Prepare the jump to Rust code. +-prepare_rust: // Set the stack pointer. - ADR_REL x0, __boot_core_stack_end_exclusive + ADR_ABS x0, __boot_core_stack_end_exclusive @@ -264,13 +274,7 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ + br x1 // Infinitely wait for events (aka "park the core"). --1: wfe -- b 1b -+2: wfe -+ b 2b - - .size _start, . - _start - .type _start, function + parking_loop: diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -350,7 +354,7 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 06_uart_chainloader/s .text : { KEEP(*(.text._start)) -@@ -42,8 +44,12 @@ +@@ -42,6 +44,10 @@ ***********************************************************************************************/ .data : { *(.data*) } :segment_rw @@ -358,17 +362,14 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/link.ld 06_uart_chainloader/s + . = ALIGN(8); + __binary_nonzero_end_exclusive = .; + - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -- .bss : ALIGN(8) -+ .bss : + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { - __bss_start = .; - *(.bss*); diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 06_uart_chainloader/src/bsp/raspberrypi/memory.rs --- 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs +++ 06_uart_chainloader/src/bsp/raspberrypi/memory.rs -@@ -23,9 +23,10 @@ +@@ -11,9 +11,10 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { @@ -381,34 +382,33 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 06_uart_chainloader /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -52,7 +53,13 @@ - // Public Code - //-------------------------------------------------------------------------------------------------- - --/// Return the inclusive range spanning the .bss section. +@@ -35,3 +36,13 @@ + pub const PL011_UART_START: usize = START + UART_OFFSET; + } + } ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ +/// The address on which the Raspberry firmware loads every binary by default. +#[inline(always)] +pub fn board_default_load_addr() -> *const u64 { + map::BOARD_DEFAULT_LOAD_ADDRESS as _ +} -+ -+/// Return the inclusive range spanning the relocated .bss section. - /// - /// # Safety - /// diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs --- 05_drivers_gpio_uart/src/main.rs +++ 06_uart_chainloader/src/main.rs -@@ -107,6 +107,7 @@ - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -105,6 +105,7 @@ + //! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] -@@ -146,38 +147,56 @@ +@@ -142,38 +143,56 @@ kernel_main() } diff --git a/06_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img index a8d7f680ef624efc1753bb890807835d880967e8..4e640c636357f92152d3c180e30879ff3abcb130 100755 GIT binary patch delta 1586 zcmZ8hZ%kWN6hHU97TOL+|Cg3QTMF136aEM*+l)NA;1<-VjOkG0S}<{mA5iwi55^Xk z2opZ^dReB%m=Qm~YfKy#lVPHPEiwAZEGU`ZUcnCq5+kC+s_;Db^^zEF@_OI-{m%K_ zbIv`l6UXN|I%c5jr?5$RKOw<|ZjiQS2n+zgO<0Ojf_oVEAdsyjz(DY=vAI26_fdDF zv>?x)bV{DDOfb#ev~{52B@bPbfsW);QX`QvUN$bjz&xf_smGT;W9lPPDw1dBUx_;v zp|NEzEk5{-s#OwI!=RP?waQjAS*rf(6+l*tfNH@r3P*^#bsFeY z+X-q(1=#x0K1mC(mTLb+H_|(DFgHEaOw;ecHke>wLUK()lcGi)N(o5!>g5E`kghLq zT&qOMrnDM+Kq+amB^N^8iW%P(A%%enB0kviVUZ1H)cimYY%7`>c`())M> zc8r~}$7=WX_3N4wI7km=ujE|LubVvKKC=~B#%NTwj;}(ryHVFlij(Z3rc?iOkYB5SJ!{j#wGP?15obAb1Pi| zx+~e~-=6}VP6Um^&Y~|r|BR!qDigy!b(dOta^_hX%9PBw~iwCXO>cBaLNV*f= zc!Ep@svPViXZ&O`LK912I;=F%0-m#ry8JHnJLK;EBT(8#djj*Z{4zV5se}a^p0j~D z^-&kgyFPK^ut)u{$WuRSq}{}Z-C-%X&)#*<{+~693P)JC$6JMiU&K1r?f@;al*c{D zlL$4^;Tw&#fZ&@V3H$=H1P1|>X1XL_T-9*43;6JN0HyTOzV5R$Dk?T8$LF^oTXPwX zVcdJBOuyC4es`XF*xXwRUW3Kphp@;aZv!k8Wyr{=J+g1;B&FK07=aQ`8@x`(d zD&Tqr%KHUjCr&>VdgjU2wzkmuo__p)m|$VAk4&)>-X8|<)s%gWK>YFZEO1H2FF^0; z=*TF(%L4Zb{IK6&-g4X#!H=_)uOYNNBYrki^EOGcXZ=2#y}F#;Tn!ub zAD71bEa!i$%(WkIuZs#CLXhtYK1qRB3mk>p1CNI-fwJT+LED9`3EQ~n_V3(fuX|!o WiVLq1Vr3TKW4ZkHnatJ#&Hn)+exp4A delta 1879 zcmZuxZEO=|9RELe*Y!4#wcWb5gRNKh)`NsG+1Nzmj^T`%7;w(07{xKCEIJr1s7naf z#+ZOErVlqICPX*=Ft4A)L1fVwR)S`JA@~9xOiVld&~5Pr7Z(p?=l{7rGiLEgo<8?~ z|NYS!yeMHhubr5)>9fXB(fF9ahD`cdzdwqicM+rs=z{lC2fJPDlIx7M_ zHklM^iJ%3I332aMkz6fd3U1NNUjnK^qPr_sPqV0)6NR*`6Eoic=DaSLKLCs>pnP{q zY7$6ja;wxugk1fk?)sIKqluaf(tN`gX4XwiW^PX=-CJCwZ}OA18A7fYN7vn3O`7Oh zfK;{yOc|r*)(~@HC(x1P-BgZjFs`iY62>+d>q|pApcoQ0lq7)O#K4OpN$UbEfCN2q zt!9>M2-d4vSy4_ar2Sf6gmhLcPv@8Ib9pzIi92;uzC*JBWyR<#{~Dp{bOoJ9WLbBS z$*L03s%RF&=NlyRK}>wU!DTXCb$Y{kYGcN8HN>0~37xCK>fE+Zv+Ofqd|6u8obqEH z@{VeHK2t%_C{>16ovx zN9r9L)*jZTS&X-IU2v+-Kr!F;w=PpnhH2iG%qa2}ClF67SeWeTga=xfP6b{To5@w8 z&}*~xPRm&u@3NLU7&yjA|sSOz7~`V{iAVp;kYS|yFAy+HvLUhzi46U@B~tp|!Ui$PO&C{Bu9 z+{vfjx@X=&9jypjnfzba)5`xrHf`NyF6l+)NwgW03Z*L^?zC75H-@ySc~Dy0i8fV| z?6kUEKueOwc3-0(sTZrjH-a3!1r9$%-mF}kG>-dPBiVY|q1Dm8i*-2Ug#L+sS!C-vPUMAR0zEjbDFH8JRoaUl zIuaXc)!zlLP6e3m>+l!TpRw(pUwB0O61lp(6|BohpTO$KKTpIGZa9Z4nR3rz=eou# zjNg3k`EWR*A(&#mSS!?I)Of!vA}roEGG!C`f22_+9mfT(0Xpvn%hn|vW09>}O2#eC zV=ZSd0sS1uS+nfs7389W9_upJK>L&^&7saNj*{sR996ieQdWhm7*6N#vTp#*YT2@E z2iQ{LoZ%A6C@fC*ATI-OjxDBIuC;YYaHH4{ZR40eGt`X3{(w3pATT`lL*Z@XbjQwz zl|8$*G&L!Y?dZnxo-xk)17ysY@n3oJLa3nY7~;>#K!L^vIHxlfI~2pOIOpSLIA^uq z0WM#|4{DjN0c!Gbpz=YF$_zRYQf%V#)^t#Quo%7Q%(wR6sMxrfuM<>OqAyU5O? z@D^v_ABuBU8@$TpD`^dJx`^w)%;}YK?5;Ha2po^(IWFMU`<>HKPCNbZj`$1c&+u0> za-o3tm{{PB@E3K1fs)$N0{2A7I2GI`^i~>Q2k#?}?5-8r9v;QZ!Le6km-9VIy#5{dO5e4&5yKzv{S;LwrbsOlJ8;O14_yaQj>uz0Mm vaesWMG1eO&+TS0G4G#@84)zZoiVbg$#ZfM=FF;;-0hD&fg~xe~K&AH|E4Bxf diff --git a/06_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img index 2d181336afb87cade1e96e18297d2750aef5ff8e..7e017a42beb30832cd2c7beb0d8e180310445b27 100755 GIT binary patch delta 1403 zcmZ8gU1%It6h8OPZZb_{CbK)UGrQ(zW)pXtDw@`mv_;JBCTfipO01{|bwP`Y50(_A zqI7qq5JW9|dn?rkwIMz@c~aLt6ft&FeTWZMt+o$&=wyWwT@XtoH9|94rNZ|C_Fg=v_4JXAOL&@#S|kbZIr!0>Un^Jy>FC0z2$iZ z)B5Qb65IPddA<@RX`+7^g42O=l)1kFt4Y9K;aS>8l*(bM5t`BX>(U{9TWF9Hy)qIS zC)BnpZ-@URnr*XzlE#+PbB^0e92ZS5SKe(M43mqMx%PQN-s7c?`v~K69Y%|~hsn0d z7dyu3S(B%gBu)17N0eNos-lOGvy9el81;D7}oELRikMJP@*oO^|^vx*n&L%iD8;$X&Q zyBKAE&j54rt4V~duX|_Q2<-B0nZ*`J!xo&?tF&=XSuWkqo?e-eoQKiL?;bk69Co%f zE*OTjro*6wgw@emu@iQ!NFo^0#RgC}hK{ZzZ9ii_sKQ1t{8DEsR=b0_k~VuC;IunM zotvKb*Ug&3x2g$ByLnpep~WbFNbSa=9#-#a0IsHJwu^riP>X6JQ4Oxz$oqd+D3MrC zb<8i%_IT#}A7BlSkj(kEg|W5qFxGz|#78508}J>3?FgqLcbk^YD8`*D_A&|OvtY3$ zEZ9Fh8;u$_FjHB2b3*?1n@Ot$zTAQucTTJPrRZ^5P4arQryUFJ&Jb3+2y8powG)LU z%P+=qS_&$QgX9%(1h7jRh3vzwpusx!|Lnis5Ds=3nt`}>5J*1BmQd#;hR~Q1fXv!! z>z;w#oRvcv97uWgBgDQf^fQ7Ty&yevg`0YNtKZVE$M|BToA>KoVN`-V&+5j&5sDnH z=OB=~1 zQE`rQ;IGYkF9?3C;HP*sc2D7|5qR^1MCRvB_&b3^!fHWa?1jIsiRiGn{yHc4PJuV` z(}LFm{xLWy4)BfnYqP;I!8g-7B5;e)e^uaSIj)&}-Z)P3{BzUTA}c|5CjZGCrZ-I9 z7Qa8Bb~Y(jB%lg^4Vl63M{%x*bM$*E{-Av<9(dd)WKqZiLi&B)S=0ipvXFVvZ&Nr6 PSbT|6u=_;fxkTzek9%Fk delta 1731 zcmZ8hUrbw782`?_Ewl{Rmj17_w*2erl=9D1h>=pnY(x{R82?5nvpS3nY)K3+DKA78=kyIa0gD^~Vz9`v56JJ^ulRA>QWXVR+asBSSr#^6!+wXjTe&6?< z-*-+gw--BX%kbo{0bTTy7=c&1LD*de7^EEy!itW*_7kDGMiM0UH61JxfRzB)Y5peA z9TDg(O9=I(Gj~p~5n+;7s*x?hUF_oskjL=1k>^UL<7xZctI4ha)wC7AV{|| zg7gQ#q96P>*q+!b%s6t#b^Sz0G~_iq#aXm{TVe zF`v?Be~*Vu>8xK=E27R@IpqJGO3kFh)dm>q}YC0C8 zcb!ne$_s{`T4pl)1hBKP+2h!%r`O1ITsi8!-vp&MM0(kf7LhS~>SnwuIS%w}k2I)F z$r-(LFH<2Q&kM*i`^MaqsS6=@wya2CF@Ii=?#uF>`)y{r{~^%*A&^R+11(`Y7wUl8 z5Vkn$qs8l@bQ*at$~sB@TQ8kZ_7N!Yw0wMot{upi@>LLd9sU1Up_CV`*|~qwhIQ23 zn3oM(yvms}mL_4t%f7Jq-c>FDR^1T^{S-4qD9p!uxTZ@{=TWUjHC@x38E&8gB4g`yR780y zT95aN&^^qm3{EAwv9Dy*M^``6O~+5#ZS6)qtQsL3H~tB$>*CP18so(n#D@ZXrEX%_ zH4+#E)L}TN<%eT5f%wOf4<&CoHqw0#_7uQ1Ssr}Aiz-RwO|o^+4y7AtGl;Ws!}s5% z{1xz}0zAH9WjAZiK1Q18No5v65!t~zSg*a0Vo+ZmIWkM=&L+@@*vHnnOaLDN2dGWs zqfmi(PyQb!CQ$$dytzyJa))*ZPLjV-BWbCT7BJ?0-Y-D3^u~eg#BpQ|x+p#_Lv{vr z6&npwf;U$sDla~gE#Sw#wfRYu6>QGXfPkIgdImJ<;(Dz2dH=w3Ev>EomwMtzwEz67%dN$#K>Yd3q3JkCb@ioE<0;%-=XQJ(>N*_vBljQS{);T(2xlTLEt{TO z+YafCT-R}bi0e2RbzR}zA*1TDbHADEhy8^6@wqAMMY!$M8o<82}?G96dkY{YpsaKfXd9p?20xPF1_>O3q4yB6i*GRj_XJ1vu1->zRyY|J-kTz$_$_g}iQBQqtFC7=iF7)dnj9TJ<9|Cb`d;#_ zE`K1EIP*@jc{Dwg9GjS&Iu=p^F5bJB!(%9wd8JdQ!XxR4a4M0W7)hp5Qxl`%vE ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s index 10aebb343..c4ae4219e 100644 --- a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s +++ b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -45,20 +45,33 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 2f + b.ne parking_loop // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_ABS x0, __bss_start + ADR_ABS x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq relocate_binary + stp xzr, xzr, [x0], #16 + b bss_init_loop + // Next, relocate the binary. +relocate_binary: ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. ADR_ABS x2, __binary_nonzero_end_exclusive -1: ldr x3, [x0], #8 +copy_loop: + ldr x3, [x0], #8 str x3, [x1], #8 cmp x1, x2 - b.lo 1b + b.lo copy_loop + // Prepare the jump to Rust code. // Set the stack pointer. ADR_ABS x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -68,8 +81,9 @@ _start: br x1 // Infinitely wait for events (aka "park the core"). -2: wfe - b 2b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/06_uart_chainloader/src/bsp/raspberrypi/link.ld b/06_uart_chainloader/src/bsp/raspberrypi/link.ld index 324172b19..40added4a 100644 --- a/06_uart_chainloader/src/bsp/raspberrypi/link.ld +++ b/06_uart_chainloader/src/bsp/raspberrypi/link.ld @@ -48,14 +48,12 @@ SECTIONS . = ALIGN(8); __binary_nonzero_end_exclusive = .; - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/06_uart_chainloader/src/bsp/raspberrypi/memory.rs b/06_uart_chainloader/src/bsp/raspberrypi/memory.rs index 9a66cc8ac..852b89050 100644 --- a/06_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ b/06_uart_chainloader/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -58,19 +46,3 @@ pub(super) mod map { pub fn board_default_load_addr() -> *const u64 { map::BOARD_DEFAULT_LOAD_ADDRESS as _ } - -/// Return the inclusive range spanning the relocated .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/06_uart_chainloader/src/main.rs b/06_uart_chainloader/src/main.rs index aed10f7a8..5c45cb5f0 100644 --- a/06_uart_chainloader/src/main.rs +++ b/06_uart_chainloader/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(asm)] @@ -120,10 +118,8 @@ mod bsp; mod console; mod cpu; mod driver; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; /// Early init code. diff --git a/06_uart_chainloader/src/memory.rs b/06_uart_chainloader/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/06_uart_chainloader/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/06_uart_chainloader/src/runtime_init.rs b/06_uart_chainloader/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/06_uart_chainloader/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/07_timestamps/README.md b/07_timestamps/README.md index 88b829086..25d08f847 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -154,26 +154,36 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_ar .equ _core_id_mask, 0b11 //-------------------------------------------------------------------------------------------------- -@@ -45,31 +34,20 @@ - and x1, x1, _core_id_mask - ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs - cmp x1, x2 -- b.ne 2f -- -- // If execution reaches here, it is the boot core. -+ b.ne 1f +@@ -50,35 +39,23 @@ + // If execution reaches here, it is the boot core. + + // Initialize DRAM. +- ADR_ABS x0, __bss_start +- ADR_ABS x1, __bss_end_exclusive ++ ADR_REL x0, __bss_start ++ ADR_REL x1, __bss_end_exclusive + + bss_init_loop: + cmp x0, x1 +- b.eq relocate_binary ++ b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop - // Next, relocate the binary. +-relocate_binary: - ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. - ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. - ADR_ABS x2, __binary_nonzero_end_exclusive - --1: ldr x3, [x0], #8 +-copy_loop: +- ldr x3, [x0], #8 - str x3, [x1], #8 - cmp x1, x2 -- b.lo 1b -+ // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. - +- b.lo copy_loop +- + // Prepare the jump to Rust code. ++prepare_rust: // Set the stack pointer. - ADR_ABS x0, __boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive @@ -186,13 +196,7 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_ar + b _start_rust // Infinitely wait for events (aka "park the core"). --2: wfe -- b 2b -+1: wfe -+ b 1b - - .size _start, . - _start - .type _start, function + parking_loop: diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu.rs 07_timestamps/src/_arch/aarch64/cpu.rs --- 06_uart_chainloader/src/_arch/aarch64/cpu.rs @@ -438,7 +442,7 @@ diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/link.ld 07_timestamps/src/bsp/ .text : { KEEP(*(.text._start)) -@@ -44,12 +42,8 @@ +@@ -44,10 +42,6 @@ ***********************************************************************************************/ .data : { *(.data*) } :segment_rw @@ -446,17 +450,14 @@ diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/link.ld 07_timestamps/src/bsp/ - . = ALIGN(8); - __binary_nonzero_end_exclusive = .; - - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -- .bss : -+ .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { - __bss_start = .; - *(.bss*); diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/memory.rs 07_timestamps/src/bsp/raspberrypi/memory.rs --- 06_uart_chainloader/src/bsp/raspberrypi/memory.rs +++ 07_timestamps/src/bsp/raspberrypi/memory.rs -@@ -23,10 +23,9 @@ +@@ -11,10 +11,9 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { @@ -469,21 +470,20 @@ diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/memory.rs 07_timestamps/src/bs /// Physical devices. #[cfg(feature = "bsp_rpi3")] -@@ -53,13 +52,7 @@ - // Public Code - //-------------------------------------------------------------------------------------------------- - +@@ -36,13 +35,3 @@ + pub const PL011_UART_START: usize = START + UART_OFFSET; + } + } +- +-//-------------------------------------------------------------------------------------------------- +-// Public Code +-//-------------------------------------------------------------------------------------------------- +- -/// The address on which the Raspberry firmware loads every binary by default. -#[inline(always)] -pub fn board_default_load_addr() -> *const u64 { - map::BOARD_DEFAULT_LOAD_ADDRESS as _ -} -- --/// Return the inclusive range spanning the relocated .bss section. -+/// Return the inclusive range spanning the .bss section. - /// - /// # Safety - /// diff -uNr 06_uart_chainloader/src/cpu.rs 07_timestamps/src/cpu.rs --- 06_uart_chainloader/src/cpu.rs @@ -499,23 +499,23 @@ diff -uNr 06_uart_chainloader/src/cpu.rs 07_timestamps/src/cpu.rs diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs --- 06_uart_chainloader/src/main.rs +++ 07_timestamps/src/main.rs -@@ -107,7 +107,6 @@ - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -105,7 +105,6 @@ + //! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] -#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] #![feature(format_args_nl)] #![feature(global_asm)] -@@ -125,6 +124,7 @@ +@@ -121,6 +120,7 @@ + mod panic_wait; mod print; - mod runtime_init; mod synchronization; +mod time; /// Early init code. /// -@@ -147,56 +147,38 @@ +@@ -143,56 +143,38 @@ kernel_main() } diff --git a/07_timestamps/src/_arch/aarch64/cpu/boot.rs b/07_timestamps/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/07_timestamps/src/_arch/aarch64/cpu/boot.rs +++ b/07_timestamps/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/07_timestamps/src/_arch/aarch64/cpu/boot.s b/07_timestamps/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/07_timestamps/src/_arch/aarch64/cpu/boot.s +++ b/07_timestamps/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/07_timestamps/src/bsp/raspberrypi/link.ld b/07_timestamps/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/07_timestamps/src/bsp/raspberrypi/link.ld +++ b/07_timestamps/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/07_timestamps/src/bsp/raspberrypi/memory.rs b/07_timestamps/src/bsp/raspberrypi/memory.rs index c6d65e36b..ef397a6f9 100644 --- a/07_timestamps/src/bsp/raspberrypi/memory.rs +++ b/07_timestamps/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -47,23 +35,3 @@ pub(super) mod map { pub const PL011_UART_START: usize = START + UART_OFFSET; } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/07_timestamps/src/main.rs b/07_timestamps/src/main.rs index 54b69275c..1e6bf16a4 100644 --- a/07_timestamps/src/main.rs +++ b/07_timestamps/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] @@ -119,10 +117,8 @@ mod bsp; mod console; mod cpu; mod driver; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/07_timestamps/src/memory.rs b/07_timestamps/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/07_timestamps/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/07_timestamps/src/runtime_init.rs b/07_timestamps/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/07_timestamps/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s +++ b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld b/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld +++ b/08_hw_debug_JTAG/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs index c6d65e36b..ef397a6f9 100644 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs +++ b/08_hw_debug_JTAG/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -47,23 +35,3 @@ pub(super) mod map { pub const PL011_UART_START: usize = START + UART_OFFSET; } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/08_hw_debug_JTAG/src/main.rs b/08_hw_debug_JTAG/src/main.rs index 54b69275c..1e6bf16a4 100644 --- a/08_hw_debug_JTAG/src/main.rs +++ b/08_hw_debug_JTAG/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] @@ -119,10 +117,8 @@ mod bsp; mod console; mod cpu; mod driver; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/08_hw_debug_JTAG/src/memory.rs b/08_hw_debug_JTAG/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/08_hw_debug_JTAG/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/08_hw_debug_JTAG/src/runtime_init.rs b/08_hw_debug_JTAG/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/08_hw_debug_JTAG/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index 869c39600..7b4c0d3c0 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -62,7 +62,7 @@ Afterwards, we continue with preparing the `EL2` -> `EL1` transition by calling pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } ``` @@ -125,19 +125,16 @@ SPSR_EL2.write( + SPSR_EL2::M::EL1h, ); -// Second, let the link register point to runtime_init(). -ELR_EL2.set(runtime_init::runtime_init as *const () as u64); +// Second, let the link register point to kernel_init(). +ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. SP_EL1.set(phys_boot_core_stack_end_exclusive_addr); ``` -As you can see, we are populating `ELR_EL2` with the address of the [runtime_init()] function that -we earlier used to call directly from the entrypoint. Finally, we set the stack pointer for -`SP_EL1`. - -[runtime_init()]: src/runtime_init.rs +As you can see, we are populating `ELR_EL2` with the address of the `kernel_init()` function that we +earlier used to call directly from the entrypoint. Finally, we set the stack pointer for `SP_EL1`. You might have noticed that the stack's address was supplied as a function argument. As you might remember, in `_start()` in `boot.s`, we are already setting up the stack for `EL2`. Since there @@ -151,7 +148,7 @@ Lastly, back in `_start_rust()` a call to `ERET` is made: pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } ``` @@ -214,12 +211,12 @@ diff -uNr 08_hw_debug_JTAG/Cargo.toml 09_privilege_level/Cargo.toml diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/_arch/aarch64/cpu/boot.rs --- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ 09_privilege_level/src/_arch/aarch64/cpu/boot.rs -@@ -12,11 +12,53 @@ +@@ -11,17 +11,67 @@ + //! //! crate::cpu::boot::arch_boot - use crate::runtime_init; +use cortex_a::{asm, regs::*}; - ++ // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -256,8 +253,8 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/ + + SPSR_EL2::M::EL1h, + ); + -+ // Second, let the link register point to runtime_init(). -+ ELR_EL2.set(runtime_init::runtime_init as *const () as u64); ++ // Second, let the link register point to kernel_init(). ++ ELR_EL2.set(crate::kernel_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there + // are no plans to ever return to EL2, just re-use the same stack. @@ -268,18 +265,20 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/ // Public Code //-------------------------------------------------------------------------------------------------- -@@ -27,7 +69,11 @@ - /// # Safety + /// The Rust entry of the `kernel` binary. /// - /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -+/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. + /// The function is called from the assembly `_start` function. ++/// ++/// # Safety ++/// ++/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] -pub unsafe fn _start_rust() -> ! { -- runtime_init::runtime_init() +- crate::kernel_init() +pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { + prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + -+ // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. ++ // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. + asm::eret() } @@ -301,15 +300,15 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_ + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 -+ b.ne 1f ++ b.ne parking_loop + // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask -@@ -38,11 +44,11 @@ - - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. +@@ -50,11 +56,11 @@ + // Prepare the jump to Rust code. + prepare_rust: - // Set the stack pointer. + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive @@ -499,15 +498,15 @@ diff -uNr 08_hw_debug_JTAG/src/exception.rs 09_privilege_level/src/exception.rs diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs --- 08_hw_debug_JTAG/src/main.rs +++ 09_privilege_level/src/main.rs -@@ -119,6 +119,7 @@ +@@ -117,6 +117,7 @@ mod console; mod cpu; mod driver; +mod exception; - mod memory; mod panic_wait; mod print; -@@ -149,6 +150,8 @@ + mod synchronization; +@@ -145,6 +146,8 @@ /// The main function running after the early init. fn kernel_main() -> ! { @@ -516,7 +515,7 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs use core::time::Duration; use driver::interface::DriverManager; use time::interface::TimeManager; -@@ -160,6 +163,12 @@ +@@ -156,6 +159,12 @@ ); info!("Booting on: {}", bsp::board_name()); @@ -529,7 +528,7 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -174,11 +183,15 @@ +@@ -170,11 +179,15 @@ info!(" {}. {}", i + 1, driver.compatible()); } diff --git a/09_privilege_level/src/_arch/aarch64/cpu/boot.rs b/09_privilege_level/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/09_privilege_level/src/_arch/aarch64/cpu/boot.rs +++ b/09_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/09_privilege_level/src/_arch/aarch64/cpu/boot.s b/09_privilege_level/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/09_privilege_level/src/_arch/aarch64/cpu/boot.s +++ b/09_privilege_level/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/09_privilege_level/src/bsp/raspberrypi/link.ld b/09_privilege_level/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/09_privilege_level/src/bsp/raspberrypi/link.ld +++ b/09_privilege_level/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/09_privilege_level/src/bsp/raspberrypi/memory.rs b/09_privilege_level/src/bsp/raspberrypi/memory.rs index c6d65e36b..ef397a6f9 100644 --- a/09_privilege_level/src/bsp/raspberrypi/memory.rs +++ b/09_privilege_level/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -47,23 +35,3 @@ pub(super) mod map { pub const PL011_UART_START: usize = START + UART_OFFSET; } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/09_privilege_level/src/main.rs b/09_privilege_level/src/main.rs index 5cede7f98..74902def1 100644 --- a/09_privilege_level/src/main.rs +++ b/09_privilege_level/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] @@ -120,10 +118,8 @@ mod console; mod cpu; mod driver; mod exception; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/09_privilege_level/src/memory.rs b/09_privilege_level/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/09_privilege_level/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/09_privilege_level/src/runtime_init.rs b/09_privilege_level/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/09_privilege_level/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index 75ff9c05c..f573de03a 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -939,26 +939,28 @@ diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory/mmu.rs 10_virtual_mem_pa diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory.rs 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs --- 09_privilege_level/src/bsp/raspberrypi/memory.rs +++ 10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs -@@ -4,6 +4,8 @@ +@@ -4,6 +4,20 @@ //! BSP Memory Management. +pub mod mmu; + - use core::{cell::UnsafeCell, ops::RangeInclusive}; - - //-------------------------------------------------------------------------------------------------- -@@ -12,6 +14,9 @@ - - // Symbols from the linker script. - extern "Rust" { ++use core::cell::UnsafeCell; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++// Symbols from the linker script. ++extern "Rust" { + static __rx_start: UnsafeCell<()>; + static __rx_end_exclusive: UnsafeCell<()>; ++} + - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; - } -@@ -23,6 +28,20 @@ + //-------------------------------------------------------------------------------------------------- + // Public Definitions + //-------------------------------------------------------------------------------------------------- +@@ -11,6 +25,20 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { @@ -979,7 +981,7 @@ diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory.rs 10_virtual_mem_part1_ pub const GPIO_OFFSET: usize = 0x0020_0000; pub const UART_OFFSET: usize = 0x0020_1000; -@@ -35,6 +54,7 @@ +@@ -23,6 +51,7 @@ pub const START: usize = 0x3F00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -987,15 +989,15 @@ diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory.rs 10_virtual_mem_part1_ } /// Physical devices. -@@ -45,10 +65,35 @@ +@@ -33,5 +62,30 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; + pub const END_INCLUSIVE: usize = 0xFF84_FFFF; } } - - //-------------------------------------------------------------------------------------------------- ++ ++//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + @@ -1018,11 +1020,6 @@ diff -uNr 09_privilege_level/src/bsp/raspberrypi/memory.rs 10_virtual_mem_part1_ +fn rx_end_exclusive() -> usize { + unsafe { __rx_end_exclusive.get() as usize } +} -+ -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- - diff -uNr 09_privilege_level/src/bsp.rs 10_virtual_mem_part1_identity_mapping/src/bsp.rs --- 09_privilege_level/src/bsp.rs @@ -1040,8 +1037,8 @@ diff -uNr 09_privilege_level/src/bsp.rs 10_virtual_mem_part1_identity_mapping/sr diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/src/main.rs --- 09_privilege_level/src/main.rs +++ 10_virtual_mem_part1_identity_mapping/src/main.rs -@@ -107,7 +107,11 @@ - //! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +@@ -105,7 +105,11 @@ + //! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] +#![allow(incomplete_features)] @@ -1052,7 +1049,15 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s #![feature(format_args_nl)] #![feature(global_asm)] #![feature(panic_info_message)] -@@ -132,9 +136,17 @@ +@@ -118,6 +122,7 @@ + mod cpu; + mod driver; + mod exception; ++mod memory; + mod panic_wait; + mod print; + mod synchronization; +@@ -128,9 +133,17 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1071,7 +1076,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s for i in bsp::driver::driver_manager().all_device_drivers().iter() { if let Err(x) = i.init() { -@@ -163,6 +175,9 @@ +@@ -159,6 +172,9 @@ ); info!("Booting on: {}", bsp::board_name()); @@ -1081,7 +1086,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -186,6 +201,13 @@ +@@ -182,6 +198,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); @@ -1387,14 +1392,13 @@ diff -uNr 09_privilege_level/src/memory/mmu.rs 10_virtual_mem_part1_identity_map diff -uNr 09_privilege_level/src/memory.rs 10_virtual_mem_part1_identity_mapping/src/memory.rs --- 09_privilege_level/src/memory.rs +++ 10_virtual_mem_part1_identity_mapping/src/memory.rs -@@ -4,6 +4,8 @@ - - //! Memory Management. - -+pub mod mmu; +@@ -0,0 +1,7 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2018-2021 Andre Richter + - use core::ops::RangeInclusive; - - //-------------------------------------------------------------------------------------------------- ++//! Memory Management. ++ ++pub mod mmu; ``` diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld index bda0da5e9..485ba49b0 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/link.ld @@ -46,14 +46,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs index 81233775d..48b1d603a 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/memory.rs @@ -6,7 +6,7 @@ pub mod mmu; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -16,9 +16,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; extern "Rust" { static __rx_start: UnsafeCell<()>; static __rx_end_exclusive: UnsafeCell<()>; - - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- @@ -92,23 +89,3 @@ fn rx_start() -> usize { fn rx_end_exclusive() -> usize { unsafe { __rx_end_exclusive.get() as usize } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/10_virtual_mem_part1_identity_mapping/src/main.rs b/10_virtual_mem_part1_identity_mapping/src/main.rs index 35f41c0e7..3a1958d6a 100644 --- a/10_virtual_mem_part1_identity_mapping/src/main.rs +++ b/10_virtual_mem_part1_identity_mapping/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -127,7 +125,6 @@ mod exception; mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/10_virtual_mem_part1_identity_mapping/src/memory.rs b/10_virtual_mem_part1_identity_mapping/src/memory.rs index e8cc752f5..7959c555e 100644 --- a/10_virtual_mem_part1_identity_mapping/src/memory.rs +++ b/10_virtual_mem_part1_identity_mapping/src/memory.rs @@ -5,28 +5,3 @@ //! Memory Management. pub mod mmu; - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/10_virtual_mem_part1_identity_mapping/src/runtime_init.rs b/10_virtual_mem_part1_identity_mapping/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/10_virtual_mem_part1_identity_mapping/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/11_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md index f9c70824b..77a45ec6d 100644 --- a/11_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -967,7 +967,7 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/exception.rs 11_exceptions_p diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_groundwork/src/main.rs --- 10_virtual_mem_part1_identity_mapping/src/main.rs +++ 11_exceptions_part1_groundwork/src/main.rs -@@ -144,6 +144,8 @@ +@@ -141,6 +141,8 @@ use driver::interface::DriverManager; use memory::mmu::interface::MMU; @@ -976,7 +976,7 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_ if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } -@@ -201,13 +203,28 @@ +@@ -198,13 +200,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld index bda0da5e9..485ba49b0 100644 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld +++ b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/link.ld @@ -46,14 +46,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs index 81233775d..48b1d603a 100644 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs +++ b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/memory.rs @@ -6,7 +6,7 @@ pub mod mmu; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -16,9 +16,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; extern "Rust" { static __rx_start: UnsafeCell<()>; static __rx_end_exclusive: UnsafeCell<()>; - - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- @@ -92,23 +89,3 @@ fn rx_start() -> usize { fn rx_end_exclusive() -> usize { unsafe { __rx_end_exclusive.get() as usize } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/11_exceptions_part1_groundwork/src/main.rs b/11_exceptions_part1_groundwork/src/main.rs index 5d00393c4..cd9d0786e 100644 --- a/11_exceptions_part1_groundwork/src/main.rs +++ b/11_exceptions_part1_groundwork/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -127,7 +125,6 @@ mod exception; mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/11_exceptions_part1_groundwork/src/memory.rs b/11_exceptions_part1_groundwork/src/memory.rs index e8cc752f5..7959c555e 100644 --- a/11_exceptions_part1_groundwork/src/memory.rs +++ b/11_exceptions_part1_groundwork/src/memory.rs @@ -5,28 +5,3 @@ //! Memory Management. pub mod mmu; - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/11_exceptions_part1_groundwork/src/runtime_init.rs b/11_exceptions_part1_groundwork/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/11_exceptions_part1_groundwork/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index cff8587b9..d252d0c23 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -215,25 +215,24 @@ kernel boot: | | Function | File | | - | - | - | -| 1. | `_start()` | `lib.rs` | -| 2. | (some more aarch64 code) | `lib.rs` | -| 3. | `runtime_init()` | `lib.rs` | -| 4. | `kernel_init()` | `main.rs` | -| 5. | `kernel_main()` | `main.rs` | +| 1. | `_start()` | The library's `boot.s` | +| 2. | (some more aarch64 code) | The library's `boot.rs` | +| 3. | `kernel_init()` | `main.rs` | +| 4. | `kernel_main()` | `main.rs` | A function named `main` is never called. Hence, the `main()` function generated by `cargo test` -would be silently dropped, and therefore the tests would never be executed. As you can see, -`runtime_init()` is the last function residing in our carved-out `lib.rs`, and it calls into -`kernel_init()`. So in order to get the tests to execute, we add a test-environment version of -`kernel_init()` to `lib.rs` as well (conditional compilation ensures it is only present when the -test flag is set), and call the `cargo test` generated `main()` function from there. +would be silently dropped, and therefore the tests would never be executed. As you can see, the +first function getting called in our carved-out `main.rs` is `kernel_init()`. So in order to get the +tests to execute, we add a test-environment version of `kernel_init()` to `lib.rs` as well +(conditional compilation ensures it is only present when the test flag is set), and call the `cargo +test` generated `main()` function from there. This is where `#![reexport_test_harness_main = "test_main"]` finally comes into picture. It declares the name of the generated main function so that we can manually call it. Here is the final implementation in `lib.rs`: ```rust -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { @@ -1073,7 +1072,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translatio diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs --- 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs -@@ -162,3 +162,22 @@ +@@ -162,3 +162,33 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } @@ -1085,12 +1084,23 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 12_inte +#[cfg(test)] +mod tests { + use super::*; ++ use core::{cell::UnsafeCell, ops::Range}; + use test_macros::kernel_test; + + /// Check if KERNEL_TABLES is in .bss. + #[kernel_test] + fn kernel_tables_in_bss() { -+ let bss_range = bsp::memory::bss_range_inclusive(); ++ extern "Rust" { ++ static __bss_start: UnsafeCell; ++ static __bss_end_exclusive: UnsafeCell; ++ } ++ ++ let bss_range = unsafe { ++ Range { ++ start: __bss_start.get(), ++ end: __bss_end_exclusive.get(), ++ } ++ }; + let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; + + assert!(bss_range.contains(&kernel_tables_addr)); @@ -1207,7 +1217,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/exception.rs 12_integrated_testing/ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/lib.rs --- 11_exceptions_part1_groundwork/src/lib.rs +++ 12_integrated_testing/src/lib.rs -@@ -0,0 +1,184 @@ +@@ -0,0 +1,186 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1314,9 +1324,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li +//! +//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. +//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -+//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -+//! -+//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html ++//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. + +#![allow(clippy::upper_case_acronyms)] +#![allow(incomplete_features)] @@ -1337,7 +1345,6 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li +#![test_runner(crate::test_runner)] + +mod panic_wait; -+mod runtime_init; +mod synchronization; + +pub mod bsp; @@ -1362,6 +1369,11 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li + ) +} + ++#[cfg(not(test))] ++extern "Rust" { ++ fn kernel_init() -> !; ++} ++ +//-------------------------------------------------------------------------------------------------- +// Testing +//-------------------------------------------------------------------------------------------------- @@ -1381,7 +1393,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li + } +} + -+/// The `kernel_init()` for unit tests. Called from `runtime_init()`. ++/// The `kernel_init()` for unit tests. +#[cfg(test)] +#[no_mangle] +unsafe fn kernel_init() -> ! { @@ -1396,7 +1408,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/main.rs --- 11_exceptions_part1_groundwork/src/main.rs +++ 12_integrated_testing/src/main.rs -@@ -6,130 +6,12 @@ +@@ -6,127 +6,12 @@ #![doc(html_logo_url = "https://git.io/JeGIp")] //! The `kernel` binary. @@ -1496,9 +1508,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m -//! -//! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. -//! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. --//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. --//! --//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +-//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. - -#![allow(clippy::upper_case_acronyms)] -#![allow(incomplete_features)] @@ -1522,14 +1532,13 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m -mod memory; -mod panic_wait; -mod print; --mod runtime_init; -mod synchronization; -mod time; +use libkernel::{bsp, console, driver, exception, info, memory, time}; /// Early init code. /// -@@ -140,6 +22,7 @@ +@@ -137,6 +22,7 @@ /// - MMU + Data caching must be activated at the earliest. Without it, any atomic operations, /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. @@ -1537,7 +1546,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m unsafe fn kernel_init() -> ! { use driver::interface::DriverManager; use memory::mmu::interface::MMU; -@@ -166,15 +49,9 @@ +@@ -163,15 +49,9 @@ fn kernel_main() -> ! { use bsp::console::console; use console::interface::All; @@ -1554,7 +1563,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/main.rs 12_integrated_testing/src/m info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); -@@ -200,31 +77,6 @@ +@@ -197,31 +77,6 @@ info!(" {}. {}", i + 1, driver.compatible()); } @@ -1609,51 +1618,6 @@ diff -uNr 11_exceptions_part1_groundwork/src/memory/mmu.rs 12_integrated_testing + } } -diff -uNr 11_exceptions_part1_groundwork/src/memory.rs 12_integrated_testing/src/memory.rs ---- 11_exceptions_part1_groundwork/src/memory.rs -+++ 12_integrated_testing/src/memory.rs -@@ -30,3 +30,40 @@ - ptr = ptr.offset(1); - } - } -+ -+//-------------------------------------------------------------------------------------------------- -+// Testing -+//-------------------------------------------------------------------------------------------------- -+ -+#[cfg(test)] -+mod tests { -+ use super::*; -+ use test_macros::kernel_test; -+ -+ /// Check `zero_volatile()`. -+ #[kernel_test] -+ fn zero_volatile_works() { -+ let mut x: [usize; 3] = [10, 11, 12]; -+ let x_range = x.as_mut_ptr_range(); -+ let x_range_inclusive = -+ RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); -+ -+ unsafe { zero_volatile(x_range_inclusive) }; -+ -+ assert_eq!(x, [0, 0, 0]); -+ } -+ -+ /// Check `bss` section layout. -+ #[kernel_test] -+ fn bss_section_is_sane() { -+ use crate::bsp::memory::bss_range_inclusive; -+ use core::mem; -+ -+ let start = *bss_range_inclusive().start() as usize; -+ let end = *bss_range_inclusive().end() as usize; -+ -+ assert_eq!(start modulo mem::size_of::(), 0); -+ assert_eq!(end modulo mem::size_of::(), 0); -+ assert!(end >= start); -+ } -+} - diff -uNr 11_exceptions_part1_groundwork/src/panic_wait.rs 12_integrated_testing/src/panic_wait.rs --- 11_exceptions_part1_groundwork/src/panic_wait.rs +++ 12_integrated_testing/src/panic_wait.rs @@ -1689,23 +1653,6 @@ diff -uNr 11_exceptions_part1_groundwork/src/panic_wait.rs 12_integrated_testing + _panic_exit() } -diff -uNr 11_exceptions_part1_groundwork/src/runtime_init.rs 12_integrated_testing/src/runtime_init.rs ---- 11_exceptions_part1_groundwork/src/runtime_init.rs -+++ 12_integrated_testing/src/runtime_init.rs -@@ -31,7 +31,10 @@ - /// - /// - Only a single core must be active and running this function. - pub unsafe fn runtime_init() -> ! { -- zero_bss(); -+ extern "Rust" { -+ fn kernel_init() -> !; -+ } - -- crate::kernel_init() -+ zero_bss(); -+ kernel_init() - } - diff -uNr 11_exceptions_part1_groundwork/test-macros/Cargo.toml 12_integrated_testing/test-macros/Cargo.toml --- 11_exceptions_part1_groundwork/test-macros/Cargo.toml +++ 12_integrated_testing/test-macros/Cargo.toml diff --git a/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs b/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs +++ b/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s +++ b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs index 29e8125d7..a706290bd 100644 --- a/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -170,12 +170,23 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { #[cfg(test)] mod tests { use super::*; + use core::{cell::UnsafeCell, ops::Range}; use test_macros::kernel_test; /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { - let bss_range = bsp::memory::bss_range_inclusive(); + extern "Rust" { + static __bss_start: UnsafeCell; + static __bss_end_exclusive: UnsafeCell; + } + + let bss_range = unsafe { + Range { + start: __bss_start.get(), + end: __bss_end_exclusive.get(), + } + }; let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; assert!(bss_range.contains(&kernel_tables_addr)); diff --git a/12_integrated_testing/src/bsp/raspberrypi/link.ld b/12_integrated_testing/src/bsp/raspberrypi/link.ld index bda0da5e9..485ba49b0 100644 --- a/12_integrated_testing/src/bsp/raspberrypi/link.ld +++ b/12_integrated_testing/src/bsp/raspberrypi/link.ld @@ -46,14 +46,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/12_integrated_testing/src/bsp/raspberrypi/memory.rs b/12_integrated_testing/src/bsp/raspberrypi/memory.rs index 81233775d..48b1d603a 100644 --- a/12_integrated_testing/src/bsp/raspberrypi/memory.rs +++ b/12_integrated_testing/src/bsp/raspberrypi/memory.rs @@ -6,7 +6,7 @@ pub mod mmu; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -16,9 +16,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; extern "Rust" { static __rx_start: UnsafeCell<()>; static __rx_end_exclusive: UnsafeCell<()>; - - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- @@ -92,23 +89,3 @@ fn rx_start() -> usize { fn rx_end_exclusive() -> usize { unsafe { __rx_end_exclusive.get() as usize } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/12_integrated_testing/src/lib.rs b/12_integrated_testing/src/lib.rs index 1e315b4da..9890351f9 100644 --- a/12_integrated_testing/src/lib.rs +++ b/12_integrated_testing/src/lib.rs @@ -104,9 +104,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -127,7 +125,6 @@ #![test_runner(crate::test_runner)] mod panic_wait; -mod runtime_init; mod synchronization; pub mod bsp; @@ -152,6 +149,11 @@ pub fn version() -> &'static str { ) } +#[cfg(not(test))] +extern "Rust" { + fn kernel_init() -> !; +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- @@ -171,7 +173,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { } } -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { diff --git a/12_integrated_testing/src/memory.rs b/12_integrated_testing/src/memory.rs index 1ef0285a6..7959c555e 100644 --- a/12_integrated_testing/src/memory.rs +++ b/12_integrated_testing/src/memory.rs @@ -5,65 +5,3 @@ //! Memory Management. pub mod mmu; - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check `zero_volatile()`. - #[kernel_test] - fn zero_volatile_works() { - let mut x: [usize; 3] = [10, 11, 12]; - let x_range = x.as_mut_ptr_range(); - let x_range_inclusive = - RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); - - unsafe { zero_volatile(x_range_inclusive) }; - - assert_eq!(x, [0, 0, 0]); - } - - /// Check `bss` section layout. - #[kernel_test] - fn bss_section_is_sane() { - use crate::bsp::memory::bss_range_inclusive; - use core::mem; - - let start = *bss_range_inclusive().start() as usize; - let end = *bss_range_inclusive().end() as usize; - - assert_eq!(start % mem::size_of::(), 0); - assert_eq!(end % mem::size_of::(), 0); - assert!(end >= start); - } -} diff --git a/12_integrated_testing/src/runtime_init.rs b/12_integrated_testing/src/runtime_init.rs deleted file mode 100644 index 0a1c685cd..000000000 --- a/12_integrated_testing/src/runtime_init.rs +++ /dev/null @@ -1,40 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - } - - zero_bss(); - kernel_init() -} diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 2a44ecdac..1174ca9e4 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -2104,7 +2104,7 @@ diff -uNr 12_integrated_testing/src/bsp/raspberrypi/exception.rs 13_exceptions_p diff -uNr 12_integrated_testing/src/bsp/raspberrypi/memory.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs --- 12_integrated_testing/src/bsp/raspberrypi/memory.rs +++ 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs -@@ -51,10 +51,12 @@ +@@ -48,10 +48,12 @@ pub mod mmio { use super::*; @@ -2121,7 +2121,7 @@ diff -uNr 12_integrated_testing/src/bsp/raspberrypi/memory.rs 13_exceptions_part } /// Physical devices. -@@ -65,6 +67,8 @@ +@@ -62,6 +64,8 @@ pub const START: usize = 0xFE00_0000; pub const GPIO_START: usize = START + GPIO_OFFSET; pub const PL011_UART_START: usize = START + UART_OFFSET; @@ -2376,7 +2376,7 @@ diff -uNr 12_integrated_testing/src/exception/asynchronous.rs 13_exceptions_part diff -uNr 12_integrated_testing/src/lib.rs 13_exceptions_part2_peripheral_IRQs/src/lib.rs --- 12_integrated_testing/src/lib.rs +++ 13_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -110,6 +110,7 @@ +@@ -108,6 +108,7 @@ #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -2384,7 +2384,7 @@ diff -uNr 12_integrated_testing/src/lib.rs 13_exceptions_part2_peripheral_IRQs/s #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -137,6 +138,7 @@ +@@ -134,6 +135,7 @@ pub mod exception; pub mod memory; pub mod print; diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index 29e8125d7..a706290bd 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -170,12 +170,23 @@ impl memory::mmu::interface::MMU for MemoryManagementUnit { #[cfg(test)] mod tests { use super::*; + use core::{cell::UnsafeCell, ops::Range}; use test_macros::kernel_test; /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { - let bss_range = bsp::memory::bss_range_inclusive(); + extern "Rust" { + static __bss_start: UnsafeCell; + static __bss_end_exclusive: UnsafeCell; + } + + let bss_range = unsafe { + Range { + start: __bss_start.get(), + end: __bss_end_exclusive.get(), + } + }; let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; assert!(bss_range.contains(&kernel_tables_addr)); diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld index bda0da5e9..485ba49b0 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld @@ -46,14 +46,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs index 8a58a17eb..ede648f67 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs @@ -6,7 +6,7 @@ pub mod mmu; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -16,9 +16,6 @@ use core::{cell::UnsafeCell, ops::RangeInclusive}; extern "Rust" { static __rx_start: UnsafeCell<()>; static __rx_end_exclusive: UnsafeCell<()>; - - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; } //-------------------------------------------------------------------------------------------------- @@ -96,23 +93,3 @@ fn rx_start() -> usize { fn rx_end_exclusive() -> usize { unsafe { __rx_end_exclusive.get() as usize } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/13_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs index 9a787e60e..ca761361d 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -104,9 +104,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -128,7 +126,6 @@ #![test_runner(crate::test_runner)] mod panic_wait; -mod runtime_init; mod synchronization; pub mod bsp; @@ -154,6 +151,11 @@ pub fn version() -> &'static str { ) } +#[cfg(not(test))] +extern "Rust" { + fn kernel_init() -> !; +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- @@ -173,7 +175,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { } } -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { diff --git a/13_exceptions_part2_peripheral_IRQs/src/memory.rs b/13_exceptions_part2_peripheral_IRQs/src/memory.rs index 1ef0285a6..7959c555e 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/memory.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/memory.rs @@ -5,65 +5,3 @@ //! Memory Management. pub mod mmu; - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check `zero_volatile()`. - #[kernel_test] - fn zero_volatile_works() { - let mut x: [usize; 3] = [10, 11, 12]; - let x_range = x.as_mut_ptr_range(); - let x_range_inclusive = - RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); - - unsafe { zero_volatile(x_range_inclusive) }; - - assert_eq!(x, [0, 0, 0]); - } - - /// Check `bss` section layout. - #[kernel_test] - fn bss_section_is_sane() { - use crate::bsp::memory::bss_range_inclusive; - use core::mem; - - let start = *bss_range_inclusive().start() as usize; - let end = *bss_range_inclusive().end() as usize; - - assert_eq!(start % mem::size_of::(), 0); - assert_eq!(end % mem::size_of::(), 0); - assert!(end >= start); - } -} diff --git a/13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs b/13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs deleted file mode 100644 index 0a1c685cd..000000000 --- a/13_exceptions_part2_peripheral_IRQs/src/runtime_init.rs +++ /dev/null @@ -1,40 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - } - - zero_bss(); - kernel_init() -} diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 85aa5138a..73ef83a3a 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -806,7 +806,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 self.configure_translation_control(); -@@ -162,22 +153,3 @@ +@@ -162,33 +153,3 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } @@ -818,12 +818,23 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 -#[cfg(test)] -mod tests { - use super::*; +- use core::{cell::UnsafeCell, ops::Range}; - use test_macros::kernel_test; - - /// Check if KERNEL_TABLES is in .bss. - #[kernel_test] - fn kernel_tables_in_bss() { -- let bss_range = bsp::memory::bss_range_inclusive(); +- extern "Rust" { +- static __bss_start: UnsafeCell; +- static __bss_end_exclusive: UnsafeCell; +- } +- +- let bss_range = unsafe { +- Range { +- start: __bss_start.get(), +- end: __bss_end_exclusive.get(), +- } +- }; - let kernel_tables_addr = unsafe { &KERNEL_TABLES as *const _ as usize as *mut u64 }; - - assert!(bss_range.contains(&kernel_tables_addr)); @@ -1489,10 +1500,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/link.ld 14_vir + __rw_start = .; .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ -@@ -56,4 +52,23 @@ - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ +@@ -54,4 +50,23 @@ + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE + + . = ALIGN(64K); /* Align to page boundary */ @@ -1554,16 +1565,16 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// The translation granule chosen by this BSP. This will be used everywhere else in the kernel to +/// derive respective data structures and their sizes. For example, the `crate::memory::mmu::Page`. +pub type KernelGranule = TranslationGranule<{ 64 * 1024 }>; -+ + +-const NUM_MEM_RANGES: usize = 2; +/// The kernel's virtual address space defined by this BSP. +pub type KernelVirtAddrSpace = AddressSpace<{ 8 * 1024 * 1024 * 1024 }>; --const NUM_MEM_RANGES: usize = 2; +-/// The virtual memory layout. +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- - --/// The virtual memory layout. ++ +/// The kernel translation tables. /// -/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. @@ -1642,15 +1653,15 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs +/// The Read+Execute (RX) pages of the kernel binary. +fn phys_rx_page_desc() -> PageSliceDescriptor { + virt_rx_page_desc().into() ++} ++ ++/// The Read+Write (RW) pages of the kernel binary. ++fn phys_rw_page_desc() -> PageSliceDescriptor { ++ virt_rw_page_desc().into() } -fn mmio_range_inclusive() -> RangeInclusive { - RangeInclusive::new(memory_map::mmio::START, memory_map::mmio::END_INCLUSIVE) -+/// The Read+Write (RW) pages of the kernel binary. -+fn phys_rw_page_desc() -> PageSliceDescriptor { -+ virt_rw_page_desc().into() -+} -+ +/// The boot core's stack. +fn phys_boot_core_stack_page_desc() -> PageSliceDescriptor { + virt_boot_core_stack_page_desc().into() @@ -1726,15 +1737,17 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs } //-------------------------------------------------------------------------------------------------- -@@ -82,14 +176,18 @@ +@@ -77,19 +171,24 @@ + #[cfg(test)] + mod tests { + use super::*; ++ use core::{cell::UnsafeCell, ops::Range}; + use test_macros::kernel_test; + /// Check alignment of the kernel's virtual memory layout sections. #[kernel_test] fn virt_mem_layout_sections_are_64KiB_aligned() { - const SIXTYFOUR_KIB: usize = 65536; -- -- for i in LAYOUT.inner().iter() { -- let start: usize = *(i.virtual_range)().start(); -- let end: usize = *(i.virtual_range)().end() + 1; + for i in [ + virt_rx_page_desc, + virt_rw_page_desc, @@ -1745,6 +1758,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + let start: usize = i().start_addr().into_usize(); + let end: usize = i().end_addr().into_usize(); +- for i in LAYOUT.inner().iter() { +- let start: usize = *(i.virtual_range)().start(); +- let end: usize = *(i.virtual_range)().end() + 1; +- - assert_eq!(start modulo SIXTYFOUR_KIB, 0); - assert_eq!(end modulo SIXTYFOUR_KIB, 0); + assert_eq!(start modulo KernelGranule::SIZE, 0); @@ -1752,7 +1769,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs assert!(end >= start); } } -@@ -97,18 +195,28 @@ +@@ -97,18 +196,38 @@ /// Ensure the kernel's virtual memory layout is free of overlaps. #[kernel_test] fn virt_mem_layout_has_no_overlaps() { @@ -1786,7 +1803,17 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory/mmu.rs + /// Check if KERNEL_TABLES is in .bss. + #[kernel_test] + fn kernel_tables_in_bss() { -+ let bss_range = super::super::bss_range_inclusive(); ++ extern "Rust" { ++ static __bss_start: UnsafeCell; ++ static __bss_end_exclusive: UnsafeCell; ++ } ++ ++ let bss_range = unsafe { ++ Range { ++ start: __bss_start.get(), ++ end: __bss_end_exclusive.get(), ++ } ++ }; + let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; + + assert!(bss_range.contains(&kernel_tables_addr)); @@ -1834,16 +1861,15 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v pub mod mmu; +use crate::memory::{Address, Physical, Virtual}; - use core::{cell::UnsafeCell, ops::RangeInclusive}; + use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- -@@ -17,8 +48,16 @@ +@@ -16,6 +47,15 @@ + extern "Rust" { static __rx_start: UnsafeCell<()>; static __rx_end_exclusive: UnsafeCell<()>; - ++ + static __rw_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; + static __rw_end_exclusive: UnsafeCell<()>; + + static __boot_core_stack_start: UnsafeCell<()>; @@ -1854,7 +1880,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v } //-------------------------------------------------------------------------------------------------- -@@ -28,35 +67,26 @@ +@@ -25,35 +65,26 @@ /// The board's physical memory map. #[rustfmt::skip] pub(super) mod map { @@ -1904,7 +1930,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v } /// Physical devices. -@@ -64,13 +94,22 @@ +@@ -61,13 +92,22 @@ pub mod mmio { use super::*; @@ -1933,7 +1959,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v } //-------------------------------------------------------------------------------------------------- -@@ -83,18 +122,69 @@ +@@ -80,16 +120,67 @@ /// /// - Value is provided by the linker script and must be trusted as-is. #[inline(always)] @@ -1941,21 +1967,19 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v - unsafe { __rx_start.get() as usize } +fn virt_rx_start() -> Address { + Address::new(unsafe { __rx_start.get() as usize }) - } - --/// Exclusive end address of the Read+Execute (RX) range. ++} ++ +/// Size of the Read+Execute (RX) range. - /// - /// # Safety - /// - /// - Value is provided by the linker script and must be trusted as-is. - #[inline(always)] --fn rx_end_exclusive() -> usize { -- unsafe { __rx_end_exclusive.get() as usize } ++/// ++/// # Safety ++/// ++/// - Value is provided by the linker script and must be trusted as-is. ++#[inline(always)] +fn rx_size() -> usize { + unsafe { (__rx_end_exclusive.get() as usize) - (__rx_start.get() as usize) } -+} -+ + } + +-/// Exclusive end address of the Read+Execute (RX) range. +/// Start address of the Read+Write (RW) range. +#[inline(always)] +fn virt_rw_start() -> Address { @@ -1963,11 +1987,13 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v +} + +/// Size of the Read+Write (RW) range. -+/// -+/// # Safety -+/// -+/// - Value is provided by the linker script and must be trusted as-is. -+#[inline(always)] + /// + /// # Safety + /// + /// - Value is provided by the linker script and must be trusted as-is. + #[inline(always)] +-fn rx_end_exclusive() -> usize { +- unsafe { __rx_end_exclusive.get() as usize } +fn rw_size() -> usize { + unsafe { (__rw_end_exclusive.get() as usize) - (__rw_start.get() as usize) } +} @@ -2007,8 +2033,6 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi/memory.rs 14_v + map::END } - //-------------------------------------------------------------------------------------------------- - diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs --- 13_exceptions_part2_peripheral_IRQs/src/bsp/raspberrypi.rs +++ 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi.rs @@ -2131,7 +2155,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/driver.rs 14_virtual_mem_part2 diff -uNr 13_exceptions_part2_peripheral_IRQs/src/lib.rs 14_virtual_mem_part2_mmio_remap/src/lib.rs --- 13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 14_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -111,6 +111,8 @@ +@@ -109,6 +109,8 @@ #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] @@ -2140,7 +2164,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/lib.rs 14_virtual_mem_part2_mm #![feature(const_fn_fn_ptr_basics)] #![feature(const_generics)] #![feature(const_panic)] -@@ -132,6 +134,7 @@ +@@ -129,6 +131,7 @@ mod synchronization; pub mod bsp; @@ -3131,16 +3155,16 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory/mmu.rs 14_virtual_mem_p diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory.rs 14_virtual_mem_part2_mmio_remap/src/memory.rs --- 13_exceptions_part2_peripheral_IRQs/src/memory.rs +++ 14_virtual_mem_part2_mmio_remap/src/memory.rs -@@ -6,12 +6,136 @@ +@@ -5,3 +5,133 @@ + //! Memory Management. pub mod mmu; - --use core::ops::RangeInclusive; ++ +use crate::common; +use core::{ + fmt, + marker::PhantomData, -+ ops::{AddAssign, RangeInclusive, SubAssign}, ++ ops::{AddAssign, SubAssign}, +}; + +//-------------------------------------------------------------------------------------------------- @@ -3164,11 +3188,11 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory.rs 14_virtual_mem_part2 + value: usize, + _address_type: PhantomData ATYPE>, +} - - //-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- - ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ +impl AddressType for Physical {} +impl AddressType for Virtual {} + @@ -3265,10 +3289,6 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/memory.rs 14_virtual_mem_part2 + write!(f, "{:04x}", q1) + } +} -+ - /// Zero out an inclusive memory range. - /// - /// # Safety diff -uNr 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs --- 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs index 02798fdc0..b743418e6 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. @@ -50,8 +49,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -68,12 +67,11 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s index 5696220d0..d1666919a 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -52,8 +64,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld index 64545edf2..698ad85da 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/link.ld @@ -42,15 +42,13 @@ SECTIONS __rw_start = .; .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE . = ALIGN(64K); /* Align to page boundary */ diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs index 7f571e080..e51b00fa1 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory.rs @@ -37,7 +37,7 @@ pub mod mmu; use crate::memory::{Address, Physical, Virtual}; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -49,8 +49,6 @@ extern "Rust" { static __rx_end_exclusive: UnsafeCell<()>; static __rw_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; static __rw_end_exclusive: UnsafeCell<()>; static __boot_core_stack_start: UnsafeCell<()>; @@ -186,23 +184,3 @@ fn boot_core_stack_guard_page_size() -> usize { fn phys_addr_space_end() -> Address { map::END } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs index 88fc80bd5..813fa0b4d 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs @@ -171,6 +171,7 @@ pub unsafe fn kernel_map_binary() -> Result<(), &'static str> { #[cfg(test)] mod tests { use super::*; + use core::{cell::UnsafeCell, ops::Range}; use test_macros::kernel_test; /// Check alignment of the kernel's virtual memory layout sections. @@ -214,7 +215,17 @@ mod tests { /// Check if KERNEL_TABLES is in .bss. #[kernel_test] fn kernel_tables_in_bss() { - let bss_range = super::super::bss_range_inclusive(); + extern "Rust" { + static __bss_start: UnsafeCell; + static __bss_end_exclusive: UnsafeCell; + } + + let bss_range = unsafe { + Range { + start: __bss_start.get(), + end: __bss_end_exclusive.get(), + } + }; let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; assert!(bss_range.contains(&kernel_tables_addr)); diff --git a/14_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs index f3ef29513..1b2cb367a 100644 --- a/14_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -104,9 +104,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -130,7 +128,6 @@ #![test_runner(crate::test_runner)] mod panic_wait; -mod runtime_init; mod synchronization; pub mod bsp; @@ -157,6 +154,11 @@ pub fn version() -> &'static str { ) } +#[cfg(not(test))] +extern "Rust" { + fn kernel_init() -> !; +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- @@ -176,7 +178,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { } } -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { diff --git a/14_virtual_mem_part2_mmio_remap/src/memory.rs b/14_virtual_mem_part2_mmio_remap/src/memory.rs index 515731eb7..3de1d6077 100644 --- a/14_virtual_mem_part2_mmio_remap/src/memory.rs +++ b/14_virtual_mem_part2_mmio_remap/src/memory.rs @@ -10,7 +10,7 @@ use crate::common; use core::{ fmt, marker::PhantomData, - ops::{AddAssign, RangeInclusive, SubAssign}, + ops::{AddAssign, SubAssign}, }; //-------------------------------------------------------------------------------------------------- @@ -135,59 +135,3 @@ impl fmt::Display for Address { write!(f, "{:04x}", q1) } } - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check `zero_volatile()`. - #[kernel_test] - fn zero_volatile_works() { - let mut x: [usize; 3] = [10, 11, 12]; - let x_range = x.as_mut_ptr_range(); - let x_range_inclusive = - RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); - - unsafe { zero_volatile(x_range_inclusive) }; - - assert_eq!(x, [0, 0, 0]); - } - - /// Check `bss` section layout. - #[kernel_test] - fn bss_section_is_sane() { - use crate::bsp::memory::bss_range_inclusive; - use core::mem; - - let start = *bss_range_inclusive().start() as usize; - let end = *bss_range_inclusive().end() as usize; - - assert_eq!(start % mem::size_of::(), 0); - assert_eq!(end % mem::size_of::(), 0); - assert!(end >= start); - } -} diff --git a/14_virtual_mem_part2_mmio_remap/src/runtime_init.rs b/14_virtual_mem_part2_mmio_remap/src/runtime_init.rs deleted file mode 100644 index 0a1c685cd..000000000 --- a/14_virtual_mem_part2_mmio_remap/src/runtime_init.rs +++ /dev/null @@ -1,40 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - } - - zero_bss(); - kernel_init() -} diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index bb9a50bb6..b06b222df 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -530,7 +530,7 @@ pub unsafe extern "C" fn _start_rust( cpu::wait_forever(); } - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } ``` @@ -796,19 +796,18 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/Makefile 15_virtual_mem_part3_precompu diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs --- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs -@@ -11,7 +11,8 @@ +@@ -11,6 +11,8 @@ //! //! crate::cpu::boot::arch_boot --use crate::runtime_init; -+use crate::{cpu, memory, memory::Address, runtime_init}; ++use crate::{cpu, memory, memory::Address}; +use core::intrinsics::unlikely; use cortex_a::{asm, regs::*}; // Assembly counterpart to this file. -@@ -71,9 +72,18 @@ - /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. - /// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +@@ -69,9 +71,18 @@ + /// + /// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] -pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! { +pub unsafe extern "C" fn _start_rust( @@ -823,17 +822,17 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs 15_virtu + cpu::wait_forever(); + } + - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s --- 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s +++ 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s -@@ -44,11 +44,14 @@ - - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. +@@ -56,11 +56,14 @@ + // Prepare the jump to Rust code. + prepare_rust: + // Load the base address of the kernel's translation tables. + ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + @@ -1204,7 +1203,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs 15_v "Kernel boot-core stack", &virt_boot_core_stack_page_desc(), &phys_boot_core_stack_page_desc(), -@@ -159,64 +183,5 @@ +@@ -159,75 +183,5 @@ acc_perms: AccessPermissions::ReadWrite, execute_never: true, }, @@ -1220,6 +1219,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs 15_v -#[cfg(test)] -mod tests { - use super::*; +- use core::{cell::UnsafeCell, ops::Range}; - use test_macros::kernel_test; - - /// Check alignment of the kernel's virtual memory layout sections. @@ -1263,7 +1263,17 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/bsp/raspberrypi/memory/mmu.rs 15_v - /// Check if KERNEL_TABLES is in .bss. - #[kernel_test] - fn kernel_tables_in_bss() { -- let bss_range = super::super::bss_range_inclusive(); +- extern "Rust" { +- static __bss_start: UnsafeCell; +- static __bss_end_exclusive: UnsafeCell; +- } +- +- let bss_range = unsafe { +- Range { +- start: __bss_start.get(), +- end: __bss_end_exclusive.get(), +- } +- }; - let kernel_tables_addr = &KERNEL_TABLES as *const _ as usize as *mut u64; - - assert!(bss_range.contains(&kernel_tables_addr)); @@ -1511,7 +1521,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/memory.rs 15_virtual_mem_part3_pre + convert::TryFrom, fmt, marker::PhantomData, - ops::{AddAssign, RangeInclusive, SubAssign}, + ops::{AddAssign, SubAssign}, @@ -67,6 +68,14 @@ } } diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs index 5e0ecfc6c..f275c792d 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,7 @@ //! //! crate::cpu::boot::arch_boot -use crate::{cpu, memory, memory::Address, runtime_init}; +use crate::{cpu, memory, memory::Address}; use core::intrinsics::unlikely; use cortex_a::{asm, regs::*}; @@ -51,8 +51,8 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(crate::kernel_init as *const () as u64); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -69,8 +69,7 @@ unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust( phys_kernel_tables_base_addr: u64, @@ -84,6 +83,6 @@ pub unsafe extern "C" fn _start_rust( cpu::wait_forever(); } - // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. asm::eret() } diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s index 286462616..9e7fc619a 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s @@ -33,17 +33,29 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Load the base address of the kernel's translation tables. ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs @@ -55,8 +67,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld index 7ad75a40c..aa127cc01 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/link.ld @@ -45,15 +45,13 @@ SECTIONS __rw_start = .; .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE . = ALIGN(64K); /* Align to page boundary */ diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs index 7f571e080..e51b00fa1 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory.rs @@ -37,7 +37,7 @@ pub mod mmu; use crate::memory::{Address, Physical, Virtual}; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -49,8 +49,6 @@ extern "Rust" { static __rx_end_exclusive: UnsafeCell<()>; static __rw_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; static __rw_end_exclusive: UnsafeCell<()>; static __boot_core_stack_start: UnsafeCell<()>; @@ -186,23 +184,3 @@ fn boot_core_stack_guard_page_size() -> usize { fn phys_addr_space_end() -> Address { map::END } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs index f3ef29513..1b2cb367a 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -104,9 +104,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -130,7 +128,6 @@ #![test_runner(crate::test_runner)] mod panic_wait; -mod runtime_init; mod synchronization; pub mod bsp; @@ -157,6 +154,11 @@ pub fn version() -> &'static str { ) } +#[cfg(not(test))] +extern "Rust" { + fn kernel_init() -> !; +} + //-------------------------------------------------------------------------------------------------- // Testing //-------------------------------------------------------------------------------------------------- @@ -176,7 +178,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { } } -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { diff --git a/15_virtual_mem_part3_precomputed_tables/src/memory.rs b/15_virtual_mem_part3_precomputed_tables/src/memory.rs index 87ffa81f5..4ac969040 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/memory.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/memory.rs @@ -11,7 +11,7 @@ use core::{ convert::TryFrom, fmt, marker::PhantomData, - ops::{AddAssign, RangeInclusive, SubAssign}, + ops::{AddAssign, SubAssign}, }; //-------------------------------------------------------------------------------------------------- @@ -144,59 +144,3 @@ impl fmt::Display for Address { write!(f, "{:04x}", q1) } } - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check `zero_volatile()`. - #[kernel_test] - fn zero_volatile_works() { - let mut x: [usize; 3] = [10, 11, 12]; - let x_range = x.as_mut_ptr_range(); - let x_range_inclusive = - RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); - - unsafe { zero_volatile(x_range_inclusive) }; - - assert_eq!(x, [0, 0, 0]); - } - - /// Check `bss` section layout. - #[kernel_test] - fn bss_section_is_sane() { - use crate::bsp::memory::bss_range_inclusive; - use core::mem; - - let start = *bss_range_inclusive().start() as usize; - let end = *bss_range_inclusive().end() as usize; - - assert_eq!(start % mem::size_of::(), 0); - assert_eq!(end % mem::size_of::(), 0); - assert!(end >= start); - } -} diff --git a/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs b/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs deleted file mode 100644 index 0a1c685cd..000000000 --- a/15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs +++ /dev/null @@ -1,40 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - } - - zero_bss(); - kernel_init() -} diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index 17d71e2b9..0a4381841 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -99,7 +99,7 @@ by the time the CPU enters `EL1`, virtual memory will be active, and the CPU mus new higher-half `virtual addresses` for everything it does. Specifically, this means the address from which the CPU should execute upon entering `EL1` (function -`runtime_init()`) must be a valid _virtual address_, same as the stack pointer's address. Both of +`kernel_init()`) must be a valid _virtual address_, same as the stack pointer's address. Both of them are programmed in function `fn prepare_el2_to_el1_transition(...)`, so we must ensure now that _link-time_ addresses are used here. For this reason, retrieval of these addresses happens in `assembly` in `boot.s`, where we can explicitly enforce generation of **absolute** addresses: @@ -108,7 +108,7 @@ _link-time_ addresses are used here. For this reason, retrieval of these address // Load the _absolute_ addresses of the following symbols. Since the kernel is linked at // the top of the 64 bit address space, these are effectively virtual addresses. ADR_ABS x1, __boot_core_stack_end_exclusive -ADR_ABS x2, runtime_init +ADR_ABS x2, kernel_init ``` Both values are forwarded to the Rust entry point function `_start_rust()`, which in turn forwards @@ -246,15 +246,6 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/Cargo.toml 16_virtual_mem_part diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs --- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs -@@ -11,7 +11,7 @@ - //! - //! crate::cpu::boot::arch_boot - --use crate::{cpu, memory, memory::Address, runtime_init}; -+use crate::{cpu, memory, memory::Address}; - use core::intrinsics::unlikely; - use cortex_a::{asm, regs::*}; - @@ -29,7 +29,10 @@ /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. @@ -262,7 +253,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs -unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) { +unsafe fn prepare_el2_to_el1_transition( + virt_boot_core_stack_end_exclusive_addr: u64, -+ virt_runtime_init_addr: u64, ++ virt_kernel_init_addr: u64, +) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -270,9 +261,9 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs @@ -52,11 +55,11 @@ ); - // Second, let the link register point to runtime_init(). -- ELR_EL2.set(runtime_init::runtime_init as *const () as u64); -+ ELR_EL2.set(virt_runtime_init_addr); + // Second, let the link register point to kernel_init(). +- ELR_EL2.set(crate::kernel_init as *const () as u64); ++ ELR_EL2.set(virt_kernel_init_addr); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -281,29 +272,29 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs } //-------------------------------------------------------------------------------------------------- -@@ -74,9 +77,13 @@ +@@ -73,9 +76,13 @@ #[no_mangle] pub unsafe extern "C" fn _start_rust( phys_kernel_tables_base_addr: u64, - phys_boot_core_stack_end_exclusive_addr: u64, + virt_boot_core_stack_end_exclusive_addr: u64, -+ virt_runtime_init_addr: u64, ++ virt_kernel_init_addr: u64, ) -> ! { - prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr); + prepare_el2_to_el1_transition( + virt_boot_core_stack_end_exclusive_addr, -+ virt_runtime_init_addr, ++ virt_kernel_init_addr, + ); // Turn on the MMU for EL1. let addr = Address::new(phys_kernel_tables_base_addr as usize); -@@ -84,6 +91,7 @@ +@@ -83,6 +90,7 @@ cpu::wait_forever(); } -- // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. +- // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1. + // Use `eret` to "return" to EL1. Since virtual memory will already be enabled, this results in -+ // execution of runtime_init() in EL1 from its _virtual address_. ++ // execution of kernel_init() in EL1 from its _virtual address_. asm::eret() } @@ -329,7 +320,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s 1 .equ _EL2, 0x8 .equ _core_id_mask, 0b11 -@@ -47,11 +59,23 @@ +@@ -59,11 +71,23 @@ // Load the base address of the kernel's translation tables. ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs @@ -339,7 +330,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s 1 + // Load the _absolute_ addresses of the following symbols. Since the kernel is linked at + // the top of the 64 bit address space, these are effectively virtual addresses. + ADR_ABS x1, __boot_core_stack_end_exclusive -+ ADR_ABS x2, runtime_init ++ ADR_ABS x2, kernel_init + + // Load the PC-relative address of the stack and set the stack pointer. + // @@ -590,6 +581,22 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/bsp/raspberrypi/memory/mmu //-------------------------------------------------------------------------------------------------- // Public Definitions +diff -uNr 15_virtual_mem_part3_precomputed_tables/src/lib.rs 16_virtual_mem_part4_higher_half_kernel/src/lib.rs +--- 15_virtual_mem_part3_precomputed_tables/src/lib.rs ++++ 16_virtual_mem_part4_higher_half_kernel/src/lib.rs +@@ -154,11 +154,6 @@ + ) + } + +-#[cfg(not(test))] +-extern "Rust" { +- fn kernel_init() -> !; +-} +- + //-------------------------------------------------------------------------------------------------- + // Testing + //-------------------------------------------------------------------------------------------------- + diff -uNr 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs --- 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/memory/mmu.rs @@ -606,18 +613,6 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/memory/mmu.rs 16_virtual_m type TableStartFromBottom; } -diff -uNr 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs ---- 15_virtual_mem_part3_precomputed_tables/src/runtime_init.rs -+++ 16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs -@@ -30,6 +30,7 @@ - /// # Safety - /// - /// - Only a single core must be active and running this function. -+#[no_mangle] - pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - diff -uNr 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs --- 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs +++ 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs index 4f0ef0b4d..fe8e90600 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs @@ -31,7 +31,7 @@ global_asm!(include_str!("boot.s")); #[inline(always)] unsafe fn prepare_el2_to_el1_transition( virt_boot_core_stack_end_exclusive_addr: u64, - virt_runtime_init_addr: u64, + virt_kernel_init_addr: u64, ) { // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); @@ -54,8 +54,8 @@ unsafe fn prepare_el2_to_el1_transition( + SPSR_EL2::M::EL1h, ); - // Second, let the link register point to runtime_init(). - ELR_EL2.set(virt_runtime_init_addr); + // Second, let the link register point to kernel_init(). + ELR_EL2.set(virt_kernel_init_addr); // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there // are no plans to ever return to EL2, just re-use the same stack. @@ -72,17 +72,16 @@ unsafe fn prepare_el2_to_el1_transition( /// /// # Safety /// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. -/// - Exception return from EL2 must must continue execution in EL1 with `runtime_init()`. +/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] pub unsafe extern "C" fn _start_rust( phys_kernel_tables_base_addr: u64, virt_boot_core_stack_end_exclusive_addr: u64, - virt_runtime_init_addr: u64, + virt_kernel_init_addr: u64, ) -> ! { prepare_el2_to_el1_transition( virt_boot_core_stack_end_exclusive_addr, - virt_runtime_init_addr, + virt_kernel_init_addr, ); // Turn on the MMU for EL1. @@ -92,6 +91,6 @@ pub unsafe extern "C" fn _start_rust( } // Use `eret` to "return" to EL1. Since virtual memory will already be enabled, this results in - // execution of runtime_init() in EL1 from its _virtual address_. + // execution of kernel_init() in EL1 from its _virtual address_. asm::eret() } diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s index 638259d47..32a5c3ff6 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s @@ -45,24 +45,36 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne 1f + b.ne parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Load the base address of the kernel's translation tables. ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs // Load the _absolute_ addresses of the following symbols. Since the kernel is linked at // the top of the 64 bit address space, these are effectively virtual addresses. ADR_ABS x1, __boot_core_stack_end_exclusive - ADR_ABS x2, runtime_init + ADR_ABS x2, kernel_init // Load the PC-relative address of the stack and set the stack pointer. // @@ -79,8 +91,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld index 962638d00..a3179f1a3 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/link.ld @@ -55,15 +55,13 @@ SECTIONS __rw_start = .; .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE . = ALIGN(64K); /* Align to page boundary */ diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs index 7f571e080..e51b00fa1 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/raspberrypi/memory.rs @@ -37,7 +37,7 @@ pub mod mmu; use crate::memory::{Address, Physical, Virtual}; -use core::{cell::UnsafeCell, ops::RangeInclusive}; +use core::cell::UnsafeCell; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -49,8 +49,6 @@ extern "Rust" { static __rx_end_exclusive: UnsafeCell<()>; static __rw_start: UnsafeCell<()>; - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; static __rw_end_exclusive: UnsafeCell<()>; static __boot_core_stack_start: UnsafeCell<()>; @@ -186,23 +184,3 @@ fn boot_core_stack_guard_page_size() -> usize { fn phys_addr_space_end() -> Address { map::END } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs index f3ef29513..3e3e08d8f 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -104,9 +104,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] @@ -130,7 +128,6 @@ #![test_runner(crate::test_runner)] mod panic_wait; -mod runtime_init; mod synchronization; pub mod bsp; @@ -176,7 +173,7 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { } } -/// The `kernel_init()` for unit tests. Called from `runtime_init()`. +/// The `kernel_init()` for unit tests. #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { diff --git a/16_virtual_mem_part4_higher_half_kernel/src/memory.rs b/16_virtual_mem_part4_higher_half_kernel/src/memory.rs index 87ffa81f5..4ac969040 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/memory.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/memory.rs @@ -11,7 +11,7 @@ use core::{ convert::TryFrom, fmt, marker::PhantomData, - ops::{AddAssign, RangeInclusive, SubAssign}, + ops::{AddAssign, SubAssign}, }; //-------------------------------------------------------------------------------------------------- @@ -144,59 +144,3 @@ impl fmt::Display for Address { write!(f, "{:04x}", q1) } } - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} - -//-------------------------------------------------------------------------------------------------- -// Testing -//-------------------------------------------------------------------------------------------------- - -#[cfg(test)] -mod tests { - use super::*; - use test_macros::kernel_test; - - /// Check `zero_volatile()`. - #[kernel_test] - fn zero_volatile_works() { - let mut x: [usize; 3] = [10, 11, 12]; - let x_range = x.as_mut_ptr_range(); - let x_range_inclusive = - RangeInclusive::new(x_range.start, unsafe { x_range.end.offset(-1) }); - - unsafe { zero_volatile(x_range_inclusive) }; - - assert_eq!(x, [0, 0, 0]); - } - - /// Check `bss` section layout. - #[kernel_test] - fn bss_section_is_sane() { - use crate::bsp::memory::bss_range_inclusive; - use core::mem; - - let start = *bss_range_inclusive().start() as usize; - let end = *bss_range_inclusive().end() as usize; - - assert_eq!(start % mem::size_of::(), 0); - assert_eq!(end % mem::size_of::(), 0); - assert!(end >= start); - } -} diff --git a/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs b/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs deleted file mode 100644 index 2c0a62e67..000000000 --- a/16_virtual_mem_part4_higher_half_kernel/src/runtime_init.rs +++ /dev/null @@ -1,41 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -#[no_mangle] -pub unsafe fn runtime_init() -> ! { - extern "Rust" { - fn kernel_init() -> !; - } - - zero_bss(); - kernel_init() -} diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 96d8bfb1817aef0dfb5458871a1ea93582acea84..9122433173b872ffac01c7ef68ea4d3f32a5243e 100755 GIT binary patch delta 1571 zcmb7ET})eL7=FKV3bY7Y3;h8K{gIzii0itoP}zVME|9W`DH!~T0cD%bc4N3(;!@^? zVai26K6I0rj5?z`IO0q-+L_EUVuHHN7&g7}qOe5Uq72l;RBey%={Y*17wVUMeZQaQ zectzZzxOoRb-8n26uh_WD&I^G2aj|Ew-F^U3;>rQ(Lo7XJ6bD{jWK{>>$9gW=0wp% z^$p}JdjDtSLac~5h~aP%%u3kgZJ>6r(RChZg!#F8G7vk$RTE-kVg3n^8X;vkC8*lp zNS*v-FtZG_6()nzQ4h5z8fg+;#V`?44qnK(K=_Ukm6OR{(8tWGDrlsmdwj;B+LYI7hrvAVeCkRBkCc^cv`V1<5ovqZq+ zvfCAVO6NVa$F_@(&+lT^%17( z*7x*b{0!DCSyC)W%=UzfCVK(vsS+N*2pLC_Am!!syYLM-Aze>|G*4){o)<=x5UT*Q zY(?MSq{wX6tjraBuzZ^+_xn~5j$Jt&43xb#aRX(y8!X(?ZZ=}*GA_Ak05zt~kPW=I z!<@!UBt35#s{3x{#53~%o>3crOT-3y<5yV_He!Us$>W5(qUjoDbtQinL&#(hydGvX z){!9VG*)rM!bXhum0v}W8qNl%UWWLGj2aK^wb4)w#3i}ucKI2c{&tkB5|S5Wsi}IP zPP|WHIo6kfKF@Tf?(!G^!Zal@g=MB@i3xi|E`o<9(OY7ZiiQmzQDf7l!ZOnk%1mU% zegBX()<(yZZFCKzb^a;Km%Wh_P3i`pu+Ec(B(Gjr<24Hh^P(v~_>rK-4>EFEH5shz z@5_dYeeq_4es8E4;$f8fpptB84&np2md?Vs&JQkKl$zukz`H3P7MyyYw?_rr8UuKv zqBtzwVRD5r#{5+#=QA9Nz;PI4>s9tWy@zZM9o+49+aC4@@egB#ovJdC>+F@P&vl_X z8C!6~A16Ce-A<<1JJs#(X0v>UFIpI81&jSy7=QOY!m4Ym{{QRMszlK$tovgpY6i)@ zV~Pmn6#fYMwN_3lVP=ojHn}Tn<*iY{Ih8$jVqvJ2{XNPamDRf3Rf>}Ts;rjj>V76V zrm=Jf{w*=E$QkLBh?4c;+)`rhRL;dZHe>0sM-;eEv1b(r9U&PO~Yapn7TvV9pY_h}>NF|MbpN2+iNJuiKe&PIJE z>9ta?N-M*KL@}05M^x1~gMO=Oe%z`eRm!`=5XgYTSWI>}Zg=NQ5!v-g$PbN}Y+K~) zrW}Q`_NrDK-SCyfnOS`%C6MF5k)M79iKWO|(T+ZiPG zH*Uy)3}5Zne99`o)+g<8#*S5lT4c64lSX}_MTX|QZ=t!995_Q zxXW?15ab{q6jv_!a=!?LeyUNhYQQ~U^y7mm7P=H`j{%W3p|F0rjOEie_T*5#TC2x7 zg!4++U@B&V3?{);hRqgJoPn@ut-9(7QF@bdaX9Q9O1cdt`2HATFMr(UbR2%q{RNaa zSPJ?>NzL6$BRd<2>^?x`5kA_z>Xo+M6LdpeQrnwL98i#80tz`kI+75Ci1<$SZ(!(V zOcYR{?<^5);F#PTw8+>G_qZV&Mvv7SnEM>Z>HeuAFH)CAo$_btpwBVs-xt*42^jY{ z(Zf7Cq)AFhgL@=LGlNihJ{R>jx7+~;!fZ6qWzD0J17PRNb{)#~I#p^&Yli7<4K(u7LDbtKa9gbeF~ z=u9i(R>d>?f#ESLo}urA&iBkGc|n!FOm~QmNzL0-F}=6)k$?R~yVx z@jgntpGv0dNhMyF$J6BQY`uEL(Clh`uIbrAgVF7}a;Yi1)l=Wp-qG1*GKUE&QQahw zZih7jrpMiAx#;PzxLux(i%o8KS4XR*y{Y}GyDQu60egeUJ6I7il!l*Bg;#nrIr?`P Cl<*G# diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 76531e6ff99de9580fcf1d1b05c8eed4404543ef..25e8aedfdf849d61eb75103719199b31f4b9c2d0 100755 GIT binary patch delta 1053 zcmZ8fT}TvB6h3!m)k)2Eb#+HecgB@f6Y?MWqt>-R${-h|qz5+=DTIpcp+L(u3@Qxv zb|a-8T8p3&qGWprD$_LRB`V75aaRcz|0?RmN~g0kw}-rN=ezfu@BG~F%-fpB)ipz~ zc}lf-u6hMHxD$lAAp*?+@DQMw5=0fz31rR>(CqB)y|=9E4r(tV?vX7iLiV5qt{JRg z7LCpTofd&!je zSkz|AL7QjGVhJH^G2=RWYI$ZyT4}GH#*0U_XbRC1bT7M-)GVkdB>143n1i0R@x~FIlUtoy6Fv&EY27>0C9@(#{UwT=)TW zp<2*7mpk6$YeW6LZn_rneqA2>ZQCT&t!HJKg&l@U1XUvR!iqi+2B>XQ#3>y$J@CgY zfZB&ZQK*3E9V?(B;t`LN#xSN+H#zzQdACvxOKEtQ#PdFB@_9f>hDU3VTQ+24+p^9H z<0^ZSRgj*jcn2#^0NusDWcBsLoU|0BPe`4RMM7=$9(;CBl7=G+sfhp=4iW?8M2o-l zn*UmgDrQT5mBx@G@Rk5POKe4V;M0aBiw9>U4QJxBj973{LV~_AvF#&^+tX8w!moF+ z2buZor+q^blEBYw*^bga0e!gI!Nf*>_tx)I_BU2kRw^xZKKwr?SyQ%yykf_)-?U1y z$u$D;b1BEXqk_iT9SEEV;OP{%=WzRK-cjyPS$IpEb#j)X!EkHtt{n*)5F7?1sF<^i}=yg?aZjo-ctO+#zmX$L+UdR+xL7NQ~xI qlyKW%u9?4qW0>Pfj)taHLA%KcarAR-WuC?KVnxwBg+;b8&;1wdoEY2y delta 1280 zcmZ8fO>7%g5T3VdC+mh1+p)88q3N#E#IaKHgM(;6A{&=Tw18ABgQ#*y-Nr#to5Wd{ zLk`F$qKKj(03lGjj)Dm^q3r3vYU6Td+m*Ho3|R%0+Lv(x@ArYQu?Uc)y*tU*=ML5i!Lu%yBLLs2T7a5Yfc`B4 z{dDO=K_LPJ*}Rx)-GW}TRy4|MK%|YF16Hz5mC|WsUlp|nVXD_(g$tr8EKgDji zE(Q=1?GMs&)}MWYtS&Xz50dSZJ@kfU z-7#ChkN{>&b;Q`%3v}l09x9dCDes%Yp%A;`4GYsdSgkMaR~xW^%P3=V>~VJg$00W8 zYxcr6YGRo5$eLF``6J0r`i6tok^7}IwJyU9@ zTTz}R&#?}Fr$C-(gZ^;NW`!Utgr2Ba^BRGdh7@t#vMd=84J)lwKpEHCs4C_Y@BQNx zwS5a=?o+9Vm?JAHaz^Ngg=nl4kfXx_c7o@1aO5to z$M?Ri9C)j{r$>3WKY^^yF*PWY(`--h=YxeN2Wto7XSvbQap3kD(Dex&|IToGBey?| zJI(!d+&{xoja@wj*~x~hVV`ffj+ENx>FVFI)yCUu_cq=W?`=2N+qrJP*7sMyL6y1G z*knldWkkoC=Tc+mWO+-$>sUW)V!z34!aE`MP<{u yr5kTnKhkwLo9WV1+05azu1{q~yT;OE6Z%xAo<;uw9`{%RC>1+}*O9Cc^8O2m9AoDI diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs index c85bb94b8..7513df074 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.rs @@ -11,8 +11,6 @@ //! //! crate::cpu::boot::arch_boot -use crate::runtime_init; - // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -23,11 +21,7 @@ global_asm!(include_str!("boot.s")); /// The Rust entry of the `kernel` binary. /// /// The function is called from the assembly `_start` function. -/// -/// # Safety -/// -/// - The `bss` section is not initialized yet. The code must not use or reference it in any way. #[no_mangle] pub unsafe fn _start_rust() -> ! { - runtime_init::runtime_init() + crate::kernel_init() } diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s index bfa94abf3..f4162c877 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s @@ -34,10 +34,22 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne 1f + b.ne parking_loop - // If execution reaches here, it is the boot core. Now, prepare the jump to Rust code. + // If execution reaches here, it is the boot core. + // Initialize DRAM. + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + +bss_init_loop: + cmp x0, x1 + b.eq prepare_rust + stp xzr, xzr, [x0], #16 + b bss_init_loop + + // Prepare the jump to Rust code. +prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -46,8 +58,9 @@ _start: b _start_rust // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +parking_loop: + wfe + b parking_loop .size _start, . - _start .type _start, function diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld index 97ea6d69c..cf63a4a64 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/link.ld +++ b/X1_JTAG_boot/src/bsp/raspberrypi/link.ld @@ -42,14 +42,12 @@ SECTIONS ***********************************************************************************************/ .data : { *(.data*) } :segment_rw - /* Section is zeroed in u64 chunks, align start and end to 8 bytes */ - .bss : ALIGN(8) + /* Section is zeroed in pairs of u64. Align start and end to 16 bytes */ + .bss : ALIGN(16) { __bss_start = .; *(.bss*); - . = ALIGN(8); - - . += 8; /* Fill for the bss == 0 case, so that __bss_start <= __bss_end_inclusive holds */ - __bss_end_inclusive = . - 8; + . = ALIGN(16); + __bss_end_exclusive = .; } :NONE } diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs index c6d65e36b..ef397a6f9 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/memory.rs @@ -4,18 +4,6 @@ //! BSP Memory Management. -use core::{cell::UnsafeCell, ops::RangeInclusive}; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -// Symbols from the linker script. -extern "Rust" { - static __bss_start: UnsafeCell; - static __bss_end_inclusive: UnsafeCell; -} - //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -47,23 +35,3 @@ pub(super) mod map { pub const PL011_UART_START: usize = START + UART_OFFSET; } } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return the inclusive range spanning the .bss section. -/// -/// # Safety -/// -/// - Values are provided by the linker script and must be trusted as-is. -/// - The linker-provided addresses must be u64 aligned. -pub fn bss_range_inclusive() -> RangeInclusive<*mut u64> { - let range; - unsafe { - range = RangeInclusive::new(__bss_start.get(), __bss_end_inclusive.get()); - } - assert!(!range.is_empty()); - - range -} diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index 0dc87fa16..23221621b 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -102,9 +102,7 @@ //! //! 1. The kernel's entry point is the function `cpu::boot::arch_boot::_start()`. //! - It is implemented in `src/_arch/__arch_name__/cpu/boot.s`. -//! 2. Once finished with architectural setup, the arch code calls [`runtime_init::runtime_init()`]. -//! -//! [`runtime_init::runtime_init()`]: runtime_init/fn.runtime_init.html +//! 2. Once finished with architectural setup, the arch code calls `kernel_init()`. #![allow(clippy::upper_case_acronyms)] #![feature(const_fn_fn_ptr_basics)] @@ -119,10 +117,8 @@ mod bsp; mod console; mod cpu; mod driver; -mod memory; mod panic_wait; mod print; -mod runtime_init; mod synchronization; mod time; diff --git a/X1_JTAG_boot/src/memory.rs b/X1_JTAG_boot/src/memory.rs deleted file mode 100644 index 1f79e0c95..000000000 --- a/X1_JTAG_boot/src/memory.rs +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Memory Management. - -use core::ops::RangeInclusive; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out an inclusive memory range. -/// -/// # Safety -/// -/// - `range.start` and `range.end` must be valid. -/// - `range.start` and `range.end` must be `T` aligned. -pub unsafe fn zero_volatile(range: RangeInclusive<*mut T>) -where - T: From, -{ - let mut ptr = *range.start(); - let end_inclusive = *range.end(); - - while ptr <= end_inclusive { - core::ptr::write_volatile(ptr, T::from(0)); - ptr = ptr.offset(1); - } -} diff --git a/X1_JTAG_boot/src/runtime_init.rs b/X1_JTAG_boot/src/runtime_init.rs deleted file mode 100644 index ee0946863..000000000 --- a/X1_JTAG_boot/src/runtime_init.rs +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2021 Andre Richter - -//! Rust runtime initialization code. - -use crate::{bsp, memory}; - -//-------------------------------------------------------------------------------------------------- -// Private Code -//-------------------------------------------------------------------------------------------------- - -/// Zero out the .bss section. -/// -/// # Safety -/// -/// - Must only be called pre `kernel_init()`. -#[inline(always)] -unsafe fn zero_bss() { - memory::zero_volatile(bsp::memory::bss_range_inclusive()); -} - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel -/// init code. -/// -/// # Safety -/// -/// - Only a single core must be active and running this function. -pub unsafe fn runtime_init() -> ! { - zero_bss(); - - crate::kernel_init() -} From 5d6b68d7107ed921f046fdf45ace252f63dbcc6a Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Jul 2021 15:26:46 +0200 Subject: [PATCH 075/214] rust-analyzer: Fix error with recent version --- 01_wait_forever/.vscode/settings.json | 6 ++++-- 01_wait_forever/build.rs | 2 +- 02_runtime_init/.vscode/settings.json | 6 ++++-- 02_runtime_init/build.rs | 2 +- 03_hacky_hello_world/.vscode/settings.json | 6 ++++-- 03_hacky_hello_world/build.rs | 2 +- 04_safe_globals/.vscode/settings.json | 6 ++++-- 04_safe_globals/build.rs | 2 +- 05_drivers_gpio_uart/.vscode/settings.json | 6 ++++-- 05_drivers_gpio_uart/build.rs | 2 +- 06_uart_chainloader/.vscode/settings.json | 6 ++++-- 06_uart_chainloader/build.rs | 2 +- 07_timestamps/.vscode/settings.json | 6 ++++-- 07_timestamps/build.rs | 2 +- 08_hw_debug_JTAG/.vscode/settings.json | 6 ++++-- 08_hw_debug_JTAG/build.rs | 2 +- 09_privilege_level/.vscode/settings.json | 6 ++++-- 09_privilege_level/build.rs | 2 +- 10_virtual_mem_part1_identity_mapping/.vscode/settings.json | 6 ++++-- 10_virtual_mem_part1_identity_mapping/build.rs | 2 +- 11_exceptions_part1_groundwork/.vscode/settings.json | 6 ++++-- 11_exceptions_part1_groundwork/build.rs | 2 +- 12_integrated_testing/.vscode/settings.json | 6 ++++-- 12_integrated_testing/build.rs | 2 +- 13_exceptions_part2_peripheral_IRQs/.vscode/settings.json | 6 ++++-- 13_exceptions_part2_peripheral_IRQs/build.rs | 2 +- 14_virtual_mem_part2_mmio_remap/.vscode/settings.json | 6 ++++-- 14_virtual_mem_part2_mmio_remap/build.rs | 2 +- .../.vscode/settings.json | 6 ++++-- 15_virtual_mem_part3_precomputed_tables/build.rs | 2 +- .../.vscode/settings.json | 6 ++++-- 16_virtual_mem_part4_higher_half_kernel/build.rs | 2 +- X1_JTAG_boot/.vscode/settings.json | 6 ++++-- X1_JTAG_boot/build.rs | 2 +- 34 files changed, 85 insertions(+), 51 deletions(-) diff --git a/01_wait_forever/.vscode/settings.json b/01_wait_forever/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/01_wait_forever/.vscode/settings.json +++ b/01_wait_forever/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/01_wait_forever/build.rs b/01_wait_forever/build.rs index 3f0a29110..883c59556 100644 --- a/01_wait_forever/build.rs +++ b/01_wait_forever/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/02_runtime_init/.vscode/settings.json b/02_runtime_init/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/02_runtime_init/.vscode/settings.json +++ b/02_runtime_init/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/02_runtime_init/build.rs b/02_runtime_init/build.rs index 3f0a29110..883c59556 100644 --- a/02_runtime_init/build.rs +++ b/02_runtime_init/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/03_hacky_hello_world/.vscode/settings.json b/03_hacky_hello_world/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/03_hacky_hello_world/.vscode/settings.json +++ b/03_hacky_hello_world/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/03_hacky_hello_world/build.rs b/03_hacky_hello_world/build.rs index 3f0a29110..883c59556 100644 --- a/03_hacky_hello_world/build.rs +++ b/03_hacky_hello_world/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/04_safe_globals/.vscode/settings.json b/04_safe_globals/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/04_safe_globals/.vscode/settings.json +++ b/04_safe_globals/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/04_safe_globals/build.rs b/04_safe_globals/build.rs index 3f0a29110..883c59556 100644 --- a/04_safe_globals/build.rs +++ b/04_safe_globals/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/05_drivers_gpio_uart/.vscode/settings.json b/05_drivers_gpio_uart/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/05_drivers_gpio_uart/.vscode/settings.json +++ b/05_drivers_gpio_uart/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/05_drivers_gpio_uart/build.rs b/05_drivers_gpio_uart/build.rs index 3f0a29110..883c59556 100644 --- a/05_drivers_gpio_uart/build.rs +++ b/05_drivers_gpio_uart/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/06_uart_chainloader/.vscode/settings.json b/06_uart_chainloader/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/06_uart_chainloader/.vscode/settings.json +++ b/06_uart_chainloader/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/06_uart_chainloader/build.rs b/06_uart_chainloader/build.rs index 3f0a29110..883c59556 100644 --- a/06_uart_chainloader/build.rs +++ b/06_uart_chainloader/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/07_timestamps/.vscode/settings.json b/07_timestamps/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/07_timestamps/.vscode/settings.json +++ b/07_timestamps/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/07_timestamps/build.rs b/07_timestamps/build.rs index 3f0a29110..883c59556 100644 --- a/07_timestamps/build.rs +++ b/07_timestamps/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/08_hw_debug_JTAG/.vscode/settings.json b/08_hw_debug_JTAG/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/08_hw_debug_JTAG/.vscode/settings.json +++ b/08_hw_debug_JTAG/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/08_hw_debug_JTAG/build.rs b/08_hw_debug_JTAG/build.rs index 3f0a29110..883c59556 100644 --- a/08_hw_debug_JTAG/build.rs +++ b/08_hw_debug_JTAG/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/09_privilege_level/.vscode/settings.json b/09_privilege_level/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/09_privilege_level/.vscode/settings.json +++ b/09_privilege_level/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/09_privilege_level/build.rs b/09_privilege_level/build.rs index 3f0a29110..883c59556 100644 --- a/09_privilege_level/build.rs +++ b/09_privilege_level/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/10_virtual_mem_part1_identity_mapping/.vscode/settings.json b/10_virtual_mem_part1_identity_mapping/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/10_virtual_mem_part1_identity_mapping/.vscode/settings.json +++ b/10_virtual_mem_part1_identity_mapping/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/10_virtual_mem_part1_identity_mapping/build.rs b/10_virtual_mem_part1_identity_mapping/build.rs index 3f0a29110..883c59556 100644 --- a/10_virtual_mem_part1_identity_mapping/build.rs +++ b/10_virtual_mem_part1_identity_mapping/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/11_exceptions_part1_groundwork/.vscode/settings.json b/11_exceptions_part1_groundwork/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/11_exceptions_part1_groundwork/.vscode/settings.json +++ b/11_exceptions_part1_groundwork/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/11_exceptions_part1_groundwork/build.rs b/11_exceptions_part1_groundwork/build.rs index 3f0a29110..883c59556 100644 --- a/11_exceptions_part1_groundwork/build.rs +++ b/11_exceptions_part1_groundwork/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/12_integrated_testing/.vscode/settings.json b/12_integrated_testing/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/12_integrated_testing/.vscode/settings.json +++ b/12_integrated_testing/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/12_integrated_testing/build.rs b/12_integrated_testing/build.rs index 3f0a29110..883c59556 100644 --- a/12_integrated_testing/build.rs +++ b/12_integrated_testing/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/13_exceptions_part2_peripheral_IRQs/.vscode/settings.json b/13_exceptions_part2_peripheral_IRQs/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/13_exceptions_part2_peripheral_IRQs/.vscode/settings.json +++ b/13_exceptions_part2_peripheral_IRQs/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/13_exceptions_part2_peripheral_IRQs/build.rs b/13_exceptions_part2_peripheral_IRQs/build.rs index 3f0a29110..883c59556 100644 --- a/13_exceptions_part2_peripheral_IRQs/build.rs +++ b/13_exceptions_part2_peripheral_IRQs/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/14_virtual_mem_part2_mmio_remap/.vscode/settings.json b/14_virtual_mem_part2_mmio_remap/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/14_virtual_mem_part2_mmio_remap/.vscode/settings.json +++ b/14_virtual_mem_part2_mmio_remap/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/14_virtual_mem_part2_mmio_remap/build.rs b/14_virtual_mem_part2_mmio_remap/build.rs index 3f0a29110..883c59556 100644 --- a/14_virtual_mem_part2_mmio_remap/build.rs +++ b/14_virtual_mem_part2_mmio_remap/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json b/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json +++ b/15_virtual_mem_part3_precomputed_tables/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/15_virtual_mem_part3_precomputed_tables/build.rs b/15_virtual_mem_part3_precomputed_tables/build.rs index 3f0a29110..883c59556 100644 --- a/15_virtual_mem_part3_precomputed_tables/build.rs +++ b/15_virtual_mem_part3_precomputed_tables/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json b/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json +++ b/16_virtual_mem_part4_higher_half_kernel/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/16_virtual_mem_part4_higher_half_kernel/build.rs b/16_virtual_mem_part4_higher_half_kernel/build.rs index 3f0a29110..883c59556 100644 --- a/16_virtual_mem_part4_higher_half_kernel/build.rs +++ b/16_virtual_mem_part4_higher_half_kernel/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); diff --git a/X1_JTAG_boot/.vscode/settings.json b/X1_JTAG_boot/.vscode/settings.json index a0d6a9202..0a8d7c090 100644 --- a/X1_JTAG_boot/.vscode/settings.json +++ b/X1_JTAG_boot/.vscode/settings.json @@ -1,7 +1,9 @@ { "editor.formatOnSave": true, "editor.rulers": [100], - "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat", - "rust-analyzer.cargo.features": ["bsp_rpi3"] + "rust-analyzer.cargo.features": ["bsp_rpi3"], + "rust-analyzer.checkOnSave.overrideCommand": ["make", "check"], + "rust-analyzer.lens.debug": false, + "rust-analyzer.lens.run": false } diff --git a/X1_JTAG_boot/build.rs b/X1_JTAG_boot/build.rs index 3f0a29110..883c59556 100644 --- a/X1_JTAG_boot/build.rs +++ b/X1_JTAG_boot/build.rs @@ -1,7 +1,7 @@ use std::env; fn main() { - let linker_file = env::var("LINKER_FILE").unwrap(); + let linker_file = env::var("LINKER_FILE").unwrap_or_default(); println!("cargo:rerun-if-changed={}", linker_file); println!("cargo:rerun-if-changed=build.rs"); From 8c2cec00be2e158148cecb1ba4fe5330ffbbf93b Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Sun, 4 Jul 2021 15:50:46 +0200 Subject: [PATCH 076/214] Assembly: Use local label syntax for named labels This prevents that they show up in the symbol table. --- 01_wait_forever/src/_arch/aarch64/cpu/boot.s | 5 +++-- 02_runtime_init/README.md | 22 +++++++------------ 02_runtime_init/src/_arch/aarch64/cpu/boot.s | 14 ++++++------ .../src/_arch/aarch64/cpu/boot.s | 14 ++++++------ 04_safe_globals/src/_arch/aarch64/cpu/boot.s | 14 ++++++------ .../src/_arch/aarch64/cpu/boot.s | 14 ++++++------ 06_uart_chainloader/README.md | 18 +++++++-------- .../src/_arch/aarch64/cpu/boot.s | 18 +++++++-------- 07_timestamps/README.md | 18 +++++++-------- 07_timestamps/src/_arch/aarch64/cpu/boot.s | 14 ++++++------ 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s | 14 ++++++------ 09_privilege_level/README.md | 4 ++-- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../README.md | 2 +- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- .../src/_arch/aarch64/cpu/boot.s | 16 +++++++------- X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s | 14 ++++++------ 22 files changed, 154 insertions(+), 159 deletions(-) diff --git a/01_wait_forever/src/_arch/aarch64/cpu/boot.s b/01_wait_forever/src/_arch/aarch64/cpu/boot.s index fb05382cd..a6d8ca565 100644 --- a/01_wait_forever/src/_arch/aarch64/cpu/boot.s +++ b/01_wait_forever/src/_arch/aarch64/cpu/boot.s @@ -12,8 +12,9 @@ //------------------------------------------------------------------------------ _start: // Infinitely wait for events (aka "park the core"). -1: wfe - b 1b +.L_parking_loop: + wfe + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 3dc0b3d36..044a2d090 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -110,7 +110,7 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch // Public Code //-------------------------------------------------------------------------------------------------- .section .text._start -@@ -11,9 +29,38 @@ +@@ -11,6 +29,34 @@ // fn _start() //------------------------------------------------------------------------------ _start: @@ -119,7 +119,7 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + and x1, x1, _core_id_mask + ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs + cmp x1, x2 -+ b.ne parking_loop ++ b.ne .L_parking_loop + + // If execution reaches here, it is the boot core. + @@ -127,14 +127,14 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive + -+bss_init_loop: ++.L_bss_init_loop: + cmp x0, x1 -+ b.eq prepare_rust ++ b.eq .L_prepare_rust + stp xzr, xzr, [x0], #16 -+ b bss_init_loop ++ b .L_bss_init_loop + + // Prepare the jump to Rust code. -+prepare_rust: ++.L_prepare_rust: + // Set the stack pointer. + ADR_REL x0, __boot_core_stack_end_exclusive + mov sp, x0 @@ -143,14 +143,8 @@ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.s 02_runtime_init/src/_arch + b _start_rust + // Infinitely wait for events (aka "park the core"). --1: wfe -- b 1b -+parking_loop: -+ wfe -+ b parking_loop - - .size _start, . - _start - .type _start, function + .L_parking_loop: + wfe diff -uNr 01_wait_forever/src/_arch/aarch64/cpu.rs 02_runtime_init/src/_arch/aarch64/cpu.rs --- 01_wait_forever/src/_arch/aarch64/cpu.rs diff --git a/02_runtime_init/src/_arch/aarch64/cpu/boot.s b/02_runtime_init/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/02_runtime_init/src/_arch/aarch64/cpu/boot.s +++ b/02_runtime_init/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s +++ b/03_hacky_hello_world/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/04_safe_globals/src/_arch/aarch64/cpu/boot.s b/04_safe_globals/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/04_safe_globals/src/_arch/aarch64/cpu/boot.s +++ b/04_safe_globals/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s +++ b/05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 18741f043..a7239e8b3 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -241,27 +241,27 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ + ADR_ABS x0, __bss_start + ADR_ABS x1, __bss_end_exclusive - bss_init_loop: + .L_bss_init_loop: cmp x0, x1 -- b.eq prepare_rust -+ b.eq relocate_binary +- b.eq .L_prepare_rust ++ b.eq .L_relocate_binary stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop + // Next, relocate the binary. -+relocate_binary: ++.L_relocate_binary: + ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. + ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. + ADR_ABS x2, __binary_nonzero_end_exclusive + -+copy_loop: ++.L_copy_loop: + ldr x3, [x0], #8 + str x3, [x1], #8 + cmp x1, x2 -+ b.lo copy_loop ++ b.lo .L_copy_loop + // Prepare the jump to Rust code. --prepare_rust: +-.L_prepare_rust: // Set the stack pointer. - ADR_REL x0, __boot_core_stack_end_exclusive + ADR_ABS x0, __boot_core_stack_end_exclusive @@ -274,7 +274,7 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ + br x1 // Infinitely wait for events (aka "park the core"). - parking_loop: + .L_parking_loop: diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs diff --git a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s index c4ae4219e..e88b18867 100644 --- a/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s +++ b/06_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -45,7 +45,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -53,23 +53,23 @@ _start: ADR_ABS x0, __bss_start ADR_ABS x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq relocate_binary + b.eq .L_relocate_binary stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Next, relocate the binary. -relocate_binary: +.L_relocate_binary: ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. ADR_ABS x2, __binary_nonzero_end_exclusive -copy_loop: +.L_copy_loop: ldr x3, [x0], #8 str x3, [x1], #8 cmp x1, x2 - b.lo copy_loop + b.lo .L_copy_loop // Prepare the jump to Rust code. // Set the stack pointer. @@ -81,9 +81,9 @@ copy_loop: br x1 // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/07_timestamps/README.md b/07_timestamps/README.md index 25d08f847..cfaa65d54 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -163,27 +163,27 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_ar + ADR_REL x0, __bss_start + ADR_REL x1, __bss_end_exclusive - bss_init_loop: + .L_bss_init_loop: cmp x0, x1 -- b.eq relocate_binary -+ b.eq prepare_rust +- b.eq .L_relocate_binary ++ b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop - // Next, relocate the binary. --relocate_binary: +-.L_relocate_binary: - ADR_REL x0, __binary_nonzero_start // The address the binary got loaded to. - ADR_ABS x1, __binary_nonzero_start // The address the binary was linked to. - ADR_ABS x2, __binary_nonzero_end_exclusive - --copy_loop: +-.L_copy_loop: - ldr x3, [x0], #8 - str x3, [x1], #8 - cmp x1, x2 -- b.lo copy_loop +- b.lo .L_copy_loop - // Prepare the jump to Rust code. -+prepare_rust: ++.L_prepare_rust: // Set the stack pointer. - ADR_ABS x0, __boot_core_stack_end_exclusive + ADR_REL x0, __boot_core_stack_end_exclusive @@ -196,7 +196,7 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_ar + b _start_rust // Infinitely wait for events (aka "park the core"). - parking_loop: + .L_parking_loop: diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu.rs 07_timestamps/src/_arch/aarch64/cpu.rs --- 06_uart_chainloader/src/_arch/aarch64/cpu.rs diff --git a/07_timestamps/src/_arch/aarch64/cpu/boot.s b/07_timestamps/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/07_timestamps/src/_arch/aarch64/cpu/boot.s +++ b/07_timestamps/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s +++ b/08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index 7b4c0d3c0..42ad905e8 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -300,7 +300,7 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_ + // Only proceed if the core executes in EL2. Park it otherwise. + mrs x0, CurrentEL + cmp x0, _EL2 -+ b.ne parking_loop ++ b.ne .L_parking_loop + // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 @@ -308,7 +308,7 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_ @@ -50,11 +56,11 @@ // Prepare the jump to Rust code. - prepare_rust: + .L_prepare_rust: - // Set the stack pointer. + // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive diff --git a/09_privilege_level/src/_arch/aarch64/cpu/boot.s b/09_privilege_level/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/09_privilege_level/src/_arch/aarch64/cpu/boot.s +++ b/09_privilege_level/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/12_integrated_testing/src/_arch/aarch64/cpu/boot.s +++ b/12_integrated_testing/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s index d1666919a..fa1984d66 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -64,9 +64,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index b06b222df..c04cb3d2d 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -832,7 +832,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.s 15_virtua @@ -56,11 +56,14 @@ // Prepare the jump to Rust code. - prepare_rust: + .L_prepare_rust: + // Load the base address of the kernel's translation tables. + ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs + diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s index 9e7fc619a..6b167d172 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s @@ -33,14 +33,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -48,14 +48,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Load the base address of the kernel's translation tables. ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs @@ -67,9 +67,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s index 32a5c3ff6..d37b4623a 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.s @@ -45,14 +45,14 @@ _start: // Only proceed if the core executes in EL2. Park it otherwise. mrs x0, CurrentEL cmp x0, _EL2 - b.ne parking_loop + b.ne .L_parking_loop // Only proceed on the boot core. Park it otherwise. mrs x1, MPIDR_EL1 and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -60,14 +60,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Load the base address of the kernel's translation tables. ldr x0, PHYS_KERNEL_TABLES_BASE_ADDR // provided by bsp/__board_name__/memory/mmu.rs @@ -91,9 +91,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function diff --git a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s index f4162c877..28bc1a70a 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s +++ b/X1_JTAG_boot/src/_arch/aarch64/cpu/boot.s @@ -34,7 +34,7 @@ _start: and x1, x1, _core_id_mask ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs cmp x1, x2 - b.ne parking_loop + b.ne .L_parking_loop // If execution reaches here, it is the boot core. @@ -42,14 +42,14 @@ _start: ADR_REL x0, __bss_start ADR_REL x1, __bss_end_exclusive -bss_init_loop: +.L_bss_init_loop: cmp x0, x1 - b.eq prepare_rust + b.eq .L_prepare_rust stp xzr, xzr, [x0], #16 - b bss_init_loop + b .L_bss_init_loop // Prepare the jump to Rust code. -prepare_rust: +.L_prepare_rust: // Set the stack pointer. ADR_REL x0, __boot_core_stack_end_exclusive mov sp, x0 @@ -58,9 +58,9 @@ prepare_rust: b _start_rust // Infinitely wait for events (aka "park the core"). -parking_loop: +.L_parking_loop: wfe - b parking_loop + b .L_parking_loop .size _start, . - _start .type _start, function From 04e0f5bae08214c948c3595d84b592260f13349c Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 5 Jul 2021 23:59:18 +0200 Subject: [PATCH 077/214] Switch to tock-registers: Sources for tutorial 16 --- .../Cargo.lock | 25 ++++++------------- .../Cargo.toml | 10 ++++---- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/cpu/smp.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 10 +++++--- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 6 ++++- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 6 ++++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../peripheral_ic.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- .../src/lib.rs | 2 +- 15 files changed, 67 insertions(+), 43 deletions(-) diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock index 5703573b5..b9a33f4ec 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Cargo.lock +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -17,9 +17,9 @@ version = "0.16.0" dependencies = [ "cortex-a", "qemu-exit", - "register", "test-macros", "test-types", + "tock-registers", ] [[package]] @@ -33,9 +33,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.2" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" +checksum = "e179da212ad772061d4120ad2a86a019f929362a477582d8c8014dbf7509cd83" [[package]] name = "quote" @@ -46,15 +46,6 @@ dependencies = [ "proc-macro2", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "syn" version = "1.0.68" @@ -82,9 +73,9 @@ version = "0.1.0" [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" [[package]] name = "unicode-xid" diff --git a/16_virtual_mem_part4_higher_half_kernel/Cargo.toml b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml index f7147ecd3..4fd433bf7 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Cargo.toml +++ b/16_virtual_mem_part4_higher_half_kernel/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -21,12 +21,12 @@ test_build = ["qemu-exit"] test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -qemu-exit = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } +qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } ##-------------------------------------------------------------------------------------------------- ## Testing diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs index fe8e90600..176c628a3 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs @@ -13,7 +13,8 @@ use crate::{cpu, memory, memory::Address}; use core::intrinsics::unlikely; -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs index b9fdd0f73..0ea9b8764 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/smp.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::smp::arch_smp -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs index 4a2c8de96..1aac65454 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception.rs @@ -17,8 +17,11 @@ use crate::{ memory::Address, }; use core::{cell::UnsafeCell, fmt}; -use cortex_a::{barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs index a4b1a548e..4e79cc8a1 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/exception/asynchronous.rs @@ -11,7 +11,8 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::{Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -22,7 +23,7 @@ mod daif_bits { } trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -35,25 +36,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs index f9e3d59bd..830fb9daf 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::{mmu::TranslationGranule, Address, Physical, Virtual}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs index a2758b157..662ee3ee5 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -24,7 +24,11 @@ use crate::{ }, }; use core::convert::{self, TryInto}; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -180,7 +184,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. @@ -426,7 +430,7 @@ impl return Err("Virtual page is already mapped"); } - *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), attr); } Ok(()) diff --git a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs index a1279f956..abfadb0c0 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -7,7 +7,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index 5f56e54a1..ae7861c98 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -12,7 +12,11 @@ use crate::{ state, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index e444812ba..9bdc73f8a 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -9,7 +9,11 @@ use crate::{ synchronization::IRQSafeNullLock, }; use core::sync::atomic::{AtomicUsize, Ordering}; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index abc3af711..5934e6dd1 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -10,7 +10,11 @@ use crate::{ driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_structs, + registers::{ReadOnly, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 3adb9137d..13664712c 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -17,7 +17,11 @@ use core::{ fmt, sync::atomic::{AtomicUsize, Ordering}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -84,6 +88,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs index 3e3e08d8f..7b5cdb09b 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -110,8 +110,8 @@ #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] -#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_fn_trait_bound)] #![feature(const_generics)] #![feature(const_panic)] #![feature(core_intrinsics)] From 69eac31e05a0989133d6c9f80d4cdfb8a2870ace Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 5 Jul 2021 23:59:42 +0200 Subject: [PATCH 078/214] Switch to tock-registers: Sources for tutorial 15 --- .../Cargo.lock | 25 ++++++------------- .../Cargo.toml | 10 ++++---- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/cpu/smp.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 10 +++++--- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 6 ++++- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 6 ++++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../peripheral_ic.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- .../src/lib.rs | 2 +- 15 files changed, 67 insertions(+), 43 deletions(-) diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.lock b/15_virtual_mem_part3_precomputed_tables/Cargo.lock index cc4041447..d91bc03d3 100644 --- a/15_virtual_mem_part3_precomputed_tables/Cargo.lock +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -17,9 +17,9 @@ version = "0.15.0" dependencies = [ "cortex-a", "qemu-exit", - "register", "test-macros", "test-types", + "tock-registers", ] [[package]] @@ -33,9 +33,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.2" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" +checksum = "e179da212ad772061d4120ad2a86a019f929362a477582d8c8014dbf7509cd83" [[package]] name = "quote" @@ -46,15 +46,6 @@ dependencies = [ "proc-macro2", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "syn" version = "1.0.68" @@ -82,9 +73,9 @@ version = "0.1.0" [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" [[package]] name = "unicode-xid" diff --git a/15_virtual_mem_part3_precomputed_tables/Cargo.toml b/15_virtual_mem_part3_precomputed_tables/Cargo.toml index 4449da657..8ce927a2f 100644 --- a/15_virtual_mem_part3_precomputed_tables/Cargo.toml +++ b/15_virtual_mem_part3_precomputed_tables/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -21,12 +21,12 @@ test_build = ["qemu-exit"] test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -qemu-exit = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } +qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } ##-------------------------------------------------------------------------------------------------- ## Testing diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs index f275c792d..d22b61877 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs @@ -13,7 +13,8 @@ use crate::{cpu, memory, memory::Address}; use core::intrinsics::unlikely; -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs index b9fdd0f73..0ea9b8764 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/smp.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::smp::arch_smp -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs index 4a2c8de96..1aac65454 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception.rs @@ -17,8 +17,11 @@ use crate::{ memory::Address, }; use core::{cell::UnsafeCell, fmt}; -use cortex_a::{barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs index a4b1a548e..4e79cc8a1 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/exception/asynchronous.rs @@ -11,7 +11,8 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::{Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -22,7 +23,7 @@ mod daif_bits { } trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -35,25 +36,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs index 49e6630ed..86b5337c4 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::{mmu::TranslationGranule, Address, Physical, Virtual}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs index 6b7227b3b..d4fa1f39b 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -24,7 +24,11 @@ use crate::{ }, }; use core::convert::{self, TryInto}; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -180,7 +184,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. @@ -399,7 +403,7 @@ impl memory::mmu::translation_table::interface::Transla return Err("Virtual page is already mapped"); } - *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), attr); } Ok(()) diff --git a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs index a1279f956..abfadb0c0 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -7,7 +7,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs index 5f56e54a1..ae7861c98 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -12,7 +12,11 @@ use crate::{ state, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index e444812ba..9bdc73f8a 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -9,7 +9,11 @@ use crate::{ synchronization::IRQSafeNullLock, }; use core::sync::atomic::{AtomicUsize, Ordering}; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index abc3af711..5934e6dd1 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -10,7 +10,11 @@ use crate::{ driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_structs, + registers::{ReadOnly, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 3adb9137d..13664712c 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -17,7 +17,11 @@ use core::{ fmt, sync::atomic::{AtomicUsize, Ordering}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -84,6 +88,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs index 1b2cb367a..8a0297385 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -110,8 +110,8 @@ #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] -#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_fn_trait_bound)] #![feature(const_generics)] #![feature(const_panic)] #![feature(core_intrinsics)] From 7c0272c365d30132d52dd333c3b86c5321e1ab11 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 6 Jul 2021 00:07:55 +0200 Subject: [PATCH 079/214] Switch to tock-registers: Sources for tutorial 14 --- 14_virtual_mem_part2_mmio_remap/Cargo.lock | 25 ++++++------------- 14_virtual_mem_part2_mmio_remap/Cargo.toml | 10 ++++---- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/cpu/smp.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 10 +++++--- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 6 ++++- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 6 ++++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../peripheral_ic.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 14_virtual_mem_part2_mmio_remap/src/lib.rs | 2 +- 15 files changed, 67 insertions(+), 43 deletions(-) diff --git a/14_virtual_mem_part2_mmio_remap/Cargo.lock b/14_virtual_mem_part2_mmio_remap/Cargo.lock index 24a35e1d8..6257551ab 100644 --- a/14_virtual_mem_part2_mmio_remap/Cargo.lock +++ b/14_virtual_mem_part2_mmio_remap/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -17,9 +17,9 @@ version = "0.14.0" dependencies = [ "cortex-a", "qemu-exit", - "register", "test-macros", "test-types", + "tock-registers", ] [[package]] @@ -33,9 +33,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.2" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" +checksum = "e179da212ad772061d4120ad2a86a019f929362a477582d8c8014dbf7509cd83" [[package]] name = "quote" @@ -46,15 +46,6 @@ dependencies = [ "proc-macro2", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "syn" version = "1.0.68" @@ -82,9 +73,9 @@ version = "0.1.0" [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" [[package]] name = "unicode-xid" diff --git a/14_virtual_mem_part2_mmio_remap/Cargo.toml b/14_virtual_mem_part2_mmio_remap/Cargo.toml index ad9a78151..002b7bbd8 100644 --- a/14_virtual_mem_part2_mmio_remap/Cargo.toml +++ b/14_virtual_mem_part2_mmio_remap/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -21,12 +21,12 @@ test_build = ["qemu-exit"] test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -qemu-exit = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } +qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } ##-------------------------------------------------------------------------------------------------- ## Testing diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs index b9fdd0f73..0ea9b8764 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/smp.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::smp::arch_smp -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs index 4a2c8de96..1aac65454 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception.rs @@ -17,8 +17,11 @@ use crate::{ memory::Address, }; use core::{cell::UnsafeCell, fmt}; -use cortex_a::{barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs index a4b1a548e..4e79cc8a1 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/exception/asynchronous.rs @@ -11,7 +11,8 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::{Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -22,7 +23,7 @@ mod daif_bits { } trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -35,25 +36,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs index 223115168..2cad10b76 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::{mmu::TranslationGranule, Address, Physical}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs index df7413614..0a9f6d52c 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -24,7 +24,11 @@ use crate::{ }, }; use core::convert; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -181,7 +185,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. @@ -389,7 +393,7 @@ impl memory::mmu::translation_table::interface::Transla return Err("Virtual page is already mapped"); } - *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); + *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), attr); } Ok(()) diff --git a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs +++ b/14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs index a1279f956..abfadb0c0 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -7,7 +7,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs index 5f56e54a1..ae7861c98 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -12,7 +12,11 @@ use crate::{ state, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index e444812ba..9bdc73f8a 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -9,7 +9,11 @@ use crate::{ synchronization::IRQSafeNullLock, }; use core::sync::atomic::{AtomicUsize, Ordering}; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index abc3af711..5934e6dd1 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -10,7 +10,11 @@ use crate::{ driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_structs, + registers::{ReadOnly, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 3adb9137d..13664712c 100644 --- a/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -17,7 +17,11 @@ use core::{ fmt, sync::atomic::{AtomicUsize, Ordering}, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -84,6 +88,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/14_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs index 1b2cb367a..8a0297385 100644 --- a/14_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -110,8 +110,8 @@ #![allow(incomplete_features)] #![feature(asm)] #![feature(const_evaluatable_checked)] -#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_fn_trait_bound)] #![feature(const_generics)] #![feature(const_panic)] #![feature(core_intrinsics)] From e1728636c11e1edb7a430fabda4514919ef68663 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 6 Jul 2021 22:57:44 +0200 Subject: [PATCH 080/214] Switch to tock-registers: Sources for tutorials 13..02 --- 02_runtime_init/Cargo.lock | 19 ++++---------- 02_runtime_init/Cargo.toml | 3 ++- 03_hacky_hello_world/Cargo.lock | 19 ++++---------- 03_hacky_hello_world/Cargo.toml | 3 ++- 04_safe_globals/Cargo.lock | 19 ++++---------- 04_safe_globals/Cargo.toml | 3 ++- 05_drivers_gpio_uart/Cargo.lock | 19 ++++---------- 05_drivers_gpio_uart/Cargo.toml | 9 ++++--- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 06_uart_chainloader/Cargo.lock | 19 ++++---------- 06_uart_chainloader/Cargo.toml | 9 ++++--- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 07_timestamps/Cargo.lock | 19 ++++---------- 07_timestamps/Cargo.toml | 9 ++++--- 07_timestamps/src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 08_hw_debug_JTAG/Cargo.lock | 19 ++++---------- 08_hw_debug_JTAG/Cargo.toml | 9 ++++--- 08_hw_debug_JTAG/src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 09_privilege_level/Cargo.lock | 19 ++++---------- 09_privilege_level/Cargo.toml | 9 ++++--- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 3 ++- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- 09_privilege_level/src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- .../Cargo.lock | 19 ++++---------- .../Cargo.toml | 9 ++++--- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 3 ++- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 8 ++++-- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 11_exceptions_part1_groundwork/Cargo.lock | 19 ++++---------- 11_exceptions_part1_groundwork/Cargo.toml | 9 ++++--- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 8 ++++-- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 12_integrated_testing/Cargo.lock | 25 ++++++------------- 12_integrated_testing/Cargo.toml | 10 ++++---- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 8 ++++-- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- .../Cargo.lock | 25 ++++++------------- .../Cargo.toml | 10 ++++---- .../src/_arch/aarch64/cpu/boot.rs | 3 ++- .../src/_arch/aarch64/cpu/smp.rs | 3 ++- .../src/_arch/aarch64/exception.rs | 7 ++++-- .../_arch/aarch64/exception/asynchronous.rs | 13 +++++----- .../src/_arch/aarch64/memory/mmu.rs | 3 ++- .../aarch64/memory/mmu/translation_table.rs | 8 ++++-- .../src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/arm/gicv2/gicc.rs | 6 ++++- .../src/bsp/device_driver/arm/gicv2/gicd.rs | 6 ++++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../peripheral_ic.rs | 8 ++++-- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- .../src/lib.rs | 1 + X1_JTAG_boot/Cargo.lock | 19 ++++---------- X1_JTAG_boot/Cargo.toml | 9 ++++--- X1_JTAG_boot/src/_arch/aarch64/time.rs | 3 ++- .../src/bsp/device_driver/bcm/bcm2xxx_gpio.rs | 6 ++++- .../device_driver/bcm/bcm2xxx_pl011_uart.rs | 7 +++++- 82 files changed, 368 insertions(+), 321 deletions(-) diff --git a/02_runtime_init/Cargo.lock b/02_runtime_init/Cargo.lock index 090fdaee4..c2773baab 100644 --- a/02_runtime_init/Cargo.lock +++ b/02_runtime_init/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -18,17 +18,8 @@ dependencies = [ "cortex-a", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/02_runtime_init/Cargo.toml b/02_runtime_init/Cargo.toml index 69f9f06d6..ebf536a79 100644 --- a/02_runtime_init/Cargo.toml +++ b/02_runtime_init/Cargo.toml @@ -24,4 +24,5 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/03_hacky_hello_world/Cargo.lock b/03_hacky_hello_world/Cargo.lock index 09dfcc720..6779dc472 100644 --- a/03_hacky_hello_world/Cargo.lock +++ b/03_hacky_hello_world/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -18,17 +18,8 @@ dependencies = [ "cortex-a", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/03_hacky_hello_world/Cargo.toml b/03_hacky_hello_world/Cargo.toml index ffe9206f2..3d3e28ab4 100644 --- a/03_hacky_hello_world/Cargo.toml +++ b/03_hacky_hello_world/Cargo.toml @@ -24,4 +24,5 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/04_safe_globals/Cargo.lock b/04_safe_globals/Cargo.lock index ccfbefbf3..4b5f5c722 100644 --- a/04_safe_globals/Cargo.lock +++ b/04_safe_globals/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -18,17 +18,8 @@ dependencies = [ "cortex-a", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/04_safe_globals/Cargo.toml b/04_safe_globals/Cargo.toml index fc077af93..09fbaa913 100644 --- a/04_safe_globals/Cargo.toml +++ b/04_safe_globals/Cargo.toml @@ -24,4 +24,5 @@ path = "src/main.rs" # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/05_drivers_gpio_uart/Cargo.lock b/05_drivers_gpio_uart/Cargo.lock index 87044397a..b39e584f2 100644 --- a/05_drivers_gpio_uart/Cargo.lock +++ b/05_drivers_gpio_uart/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.5.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/05_drivers_gpio_uart/Cargo.toml b/05_drivers_gpio_uart/Cargo.toml index 05abef4d4..92c98e0c5 100644 --- a/05_drivers_gpio_uart/Cargo.toml +++ b/05_drivers_gpio_uart/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 771151e76..dd207221a 100644 --- a/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/06_uart_chainloader/Cargo.lock b/06_uart_chainloader/Cargo.lock index 5d2602a8e..e390c9597 100644 --- a/06_uart_chainloader/Cargo.lock +++ b/06_uart_chainloader/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.6.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/06_uart_chainloader/Cargo.toml b/06_uart_chainloader/Cargo.toml index f3b9ba2f9..d21dd6a8f 100644 --- a/06_uart_chainloader/Cargo.toml +++ b/06_uart_chainloader/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 9d41900d7..cd2cf2af9 100644 --- a/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 17ca3543b..2befbeebe 100644 --- a/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/07_timestamps/Cargo.lock b/07_timestamps/Cargo.lock index 311b6c211..098fe83c6 100644 --- a/07_timestamps/Cargo.lock +++ b/07_timestamps/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.7.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/07_timestamps/Cargo.toml b/07_timestamps/Cargo.toml index 8d97be1f1..881f1d1af 100644 --- a/07_timestamps/Cargo.toml +++ b/07_timestamps/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/07_timestamps/src/_arch/aarch64/time.rs b/07_timestamps/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/07_timestamps/src/_arch/aarch64/time.rs +++ b/07_timestamps/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/08_hw_debug_JTAG/Cargo.lock b/08_hw_debug_JTAG/Cargo.lock index 011adc0f3..c93636780 100644 --- a/08_hw_debug_JTAG/Cargo.lock +++ b/08_hw_debug_JTAG/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.8.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/08_hw_debug_JTAG/Cargo.toml b/08_hw_debug_JTAG/Cargo.toml index 7b1adb5b3..07dc9b5b6 100644 --- a/08_hw_debug_JTAG/Cargo.toml +++ b/08_hw_debug_JTAG/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/08_hw_debug_JTAG/src/_arch/aarch64/time.rs b/08_hw_debug_JTAG/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/08_hw_debug_JTAG/src/_arch/aarch64/time.rs +++ b/08_hw_debug_JTAG/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/08_hw_debug_JTAG/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/09_privilege_level/Cargo.lock b/09_privilege_level/Cargo.lock index 753961f7f..606b7819a 100644 --- a/09_privilege_level/Cargo.lock +++ b/09_privilege_level/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.9.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/09_privilege_level/Cargo.toml b/09_privilege_level/Cargo.toml index 7af3572f6..e9965b2f6 100644 --- a/09_privilege_level/Cargo.toml +++ b/09_privilege_level/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/09_privilege_level/src/_arch/aarch64/cpu/boot.rs b/09_privilege_level/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/09_privilege_level/src/_arch/aarch64/cpu/boot.rs +++ b/09_privilege_level/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/09_privilege_level/src/_arch/aarch64/exception.rs b/09_privilege_level/src/_arch/aarch64/exception.rs index d8c617b2a..10fa86a93 100644 --- a/09_privilege_level/src/_arch/aarch64/exception.rs +++ b/09_privilege_level/src/_arch/aarch64/exception.rs @@ -11,7 +11,8 @@ //! //! crate::exception::arch_exception -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs b/09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs index b63b00fe7..c447d4557 100644 --- a/09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs +++ b/09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs @@ -11,14 +11,15 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -31,25 +32,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/09_privilege_level/src/_arch/aarch64/time.rs b/09_privilege_level/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/09_privilege_level/src/_arch/aarch64/time.rs +++ b/09_privilege_level/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/09_privilege_level/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/10_virtual_mem_part1_identity_mapping/Cargo.lock b/10_virtual_mem_part1_identity_mapping/Cargo.lock index 5022ffadb..a3e7eb266 100644 --- a/10_virtual_mem_part1_identity_mapping/Cargo.lock +++ b/10_virtual_mem_part1_identity_mapping/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.10.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/10_virtual_mem_part1_identity_mapping/Cargo.toml b/10_virtual_mem_part1_identity_mapping/Cargo.toml index 9d47f28da..fac86dd5a 100644 --- a/10_virtual_mem_part1_identity_mapping/Cargo.toml +++ b/10_virtual_mem_part1_identity_mapping/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs index d8c617b2a..10fa86a93 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs @@ -11,7 +11,8 @@ //! //! crate::exception::arch_exception -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs index b63b00fe7..c447d4557 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception/asynchronous.rs @@ -11,14 +11,15 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -31,25 +32,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs index 2f7bf615a..eaeff3219 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs index eba8e12b5..dac6e32f8 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -21,7 +21,11 @@ use crate::{ }, }; use core::convert; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -182,7 +186,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. diff --git a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs +++ b/10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/11_exceptions_part1_groundwork/Cargo.lock b/11_exceptions_part1_groundwork/Cargo.lock index c956c58cd..6fca12525 100644 --- a/11_exceptions_part1_groundwork/Cargo.lock +++ b/11_exceptions_part1_groundwork/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.11.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/11_exceptions_part1_groundwork/Cargo.toml b/11_exceptions_part1_groundwork/Cargo.toml index 2f6683e5a..adafedfac 100644 --- a/11_exceptions_part1_groundwork/Cargo.toml +++ b/11_exceptions_part1_groundwork/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs index 21f0c68cf..d51d7f014 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs @@ -12,8 +12,11 @@ //! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; -use cortex_a::{asm, barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm, asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs index b63b00fe7..c447d4557 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/exception/asynchronous.rs @@ -11,14 +11,15 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -31,25 +32,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs index 2f7bf615a..eaeff3219 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs index eba8e12b5..dac6e32f8 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -21,7 +21,11 @@ use crate::{ }, }; use core::convert; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -182,7 +186,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. diff --git a/11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs b/11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs +++ b/11_exceptions_part1_groundwork/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_exceptions_part1_groundwork/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/12_integrated_testing/Cargo.lock b/12_integrated_testing/Cargo.lock index 009654700..18175739d 100644 --- a/12_integrated_testing/Cargo.lock +++ b/12_integrated_testing/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -17,9 +17,9 @@ version = "0.12.0" dependencies = [ "cortex-a", "qemu-exit", - "register", "test-macros", "test-types", + "tock-registers", ] [[package]] @@ -33,9 +33,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.2" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" +checksum = "e179da212ad772061d4120ad2a86a019f929362a477582d8c8014dbf7509cd83" [[package]] name = "quote" @@ -46,15 +46,6 @@ dependencies = [ "proc-macro2", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "syn" version = "1.0.68" @@ -82,9 +73,9 @@ version = "0.1.0" [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" [[package]] name = "unicode-xid" diff --git a/12_integrated_testing/Cargo.toml b/12_integrated_testing/Cargo.toml index 686df7640..42013b264 100644 --- a/12_integrated_testing/Cargo.toml +++ b/12_integrated_testing/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -21,12 +21,12 @@ test_build = ["qemu-exit"] test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -qemu-exit = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } +qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } ##-------------------------------------------------------------------------------------------------- ## Testing diff --git a/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs b/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs +++ b/12_integrated_testing/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/12_integrated_testing/src/_arch/aarch64/exception.rs b/12_integrated_testing/src/_arch/aarch64/exception.rs index 1ee0a1982..6ba47300f 100644 --- a/12_integrated_testing/src/_arch/aarch64/exception.rs +++ b/12_integrated_testing/src/_arch/aarch64/exception.rs @@ -12,8 +12,11 @@ //! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; -use cortex_a::{barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs b/12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs index b63b00fe7..c447d4557 100644 --- a/12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ b/12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs @@ -11,14 +11,15 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -31,25 +32,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs index a706290bd..99991b6fe 100644 --- a/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs +++ b/12_integrated_testing/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs b/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs index 665631f49..1e1173f2d 100644 --- a/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -21,7 +21,11 @@ use crate::{ }, }; use core::convert; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -182,7 +186,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. diff --git a/12_integrated_testing/src/_arch/aarch64/time.rs b/12_integrated_testing/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/12_integrated_testing/src/_arch/aarch64/time.rs +++ b/12_integrated_testing/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/13_exceptions_part2_peripheral_IRQs/Cargo.lock b/13_exceptions_part2_peripheral_IRQs/Cargo.lock index ed9501003..7fd778588 100644 --- a/13_exceptions_part2_peripheral_IRQs/Cargo.lock +++ b/13_exceptions_part2_peripheral_IRQs/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -17,9 +17,9 @@ version = "0.13.0" dependencies = [ "cortex-a", "qemu-exit", - "register", "test-macros", "test-types", + "tock-registers", ] [[package]] @@ -33,9 +33,9 @@ dependencies = [ [[package]] name = "qemu-exit" -version = "1.0.2" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "702abe3c3255be20a4d67bda326c4117081a49a57afaf752de4845091bc6b476" +checksum = "e179da212ad772061d4120ad2a86a019f929362a477582d8c8014dbf7509cd83" [[package]] name = "quote" @@ -46,15 +46,6 @@ dependencies = [ "proc-macro2", ] -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ - "tock-registers", -] - [[package]] name = "syn" version = "1.0.68" @@ -82,9 +73,9 @@ version = "0.1.0" [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" [[package]] name = "unicode-xid" diff --git a/13_exceptions_part2_peripheral_IRQs/Cargo.toml b/13_exceptions_part2_peripheral_IRQs/Cargo.toml index 343bdfda3..4c6de8378 100644 --- a/13_exceptions_part2_peripheral_IRQs/Cargo.toml +++ b/13_exceptions_part2_peripheral_IRQs/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] test_build = ["qemu-exit"] ##-------------------------------------------------------------------------------------------------- @@ -21,12 +21,12 @@ test_build = ["qemu-exit"] test-types = { path = "test-types" } # Optional dependencies -register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -qemu-exit = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } +qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } ##-------------------------------------------------------------------------------------------------- ## Testing diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs index b743418e6..4f0068622 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/boot.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::boot::arch_boot -use cortex_a::{asm, regs::*}; +use cortex_a::{asm, registers::*}; +use tock_registers::interfaces::Writeable; // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs index b9fdd0f73..0ea9b8764 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs @@ -11,7 +11,8 @@ //! //! crate::cpu::smp::arch_smp -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::Readable; //-------------------------------------------------------------------------------------------------- // Public Code diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs index 5c44abb70..d755c51db 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs @@ -13,8 +13,11 @@ use crate::{bsp, exception}; use core::{cell::UnsafeCell, fmt}; -use cortex_a::{barrier, regs::*}; -use register::InMemoryRegister; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +}; // Assembly counterpart to this file. global_asm!(include_str!("exception.s")); diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs index a4b1a548e..4e79cc8a1 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs @@ -11,7 +11,8 @@ //! //! crate::exception::asynchronous::arch_asynchronous -use cortex_a::regs::*; +use cortex_a::registers::*; +use tock_registers::interfaces::{Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -22,7 +23,7 @@ mod daif_bits { } trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } struct Debug; @@ -35,25 +36,25 @@ struct FIQ; //-------------------------------------------------------------------------------------------------- impl DaifField for Debug { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::D } } impl DaifField for SError { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::A } } impl DaifField for IRQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::I } } impl DaifField for FIQ { - fn daif_field() -> register::Field { + fn daif_field() -> tock_registers::fields::Field { DAIF::F } } diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs index a706290bd..99991b6fe 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs @@ -18,7 +18,8 @@ use crate::{ memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, }; use core::intrinsics::unlikely; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs index 665631f49..1e1173f2d 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/translation_table.rs @@ -21,7 +21,11 @@ use crate::{ }, }; use core::convert; -use register::{register_bitfields, InMemoryRegister}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, + registers::InMemoryRegister, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -182,7 +186,7 @@ impl TableDescriptor { /// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. impl convert::From - for register::FieldValue + for tock_registers::fields::FieldValue { fn from(attribute_fields: AttributeFields) -> Self { // Memory attributes. diff --git a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs index 424bafba3..5f1d6d53b 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs @@ -5,7 +5,11 @@ //! GICC Driver - GIC CPU interface. use crate::{bsp::device_driver::common::MMIODerefWrapper, exception}; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs index 32c428089..38fa0ed6c 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -11,7 +11,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, state, synchronization, synchronization::IRQSafeNullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index aaf9e1caf..a2f7eb93d 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::IRQSafeNullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 79814d8c3..4ca5988c4 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2020-2021 Andre Richter -//! Peripheral Interrupt regsler Driver. +//! Peripheral Interrupt Controller Driver. use super::{InterruptController, PendingIRQs, PeripheralIRQ}; use crate::{ @@ -10,7 +10,11 @@ use crate::{ exception, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; -use register::{mmio::*, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_structs, + registers::{ReadOnly, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index c8026dcf5..11d9745e5 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization, synchronization::IRQSafeNullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, diff --git a/13_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs index ca761361d..4ca9f03ee 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -110,6 +110,7 @@ #![allow(incomplete_features)] #![feature(asm)] #![feature(const_fn_fn_ptr_basics)] +#![feature(const_fn_trait_bound)] #![feature(const_generics)] #![feature(const_panic)] #![feature(core_intrinsics)] diff --git a/X1_JTAG_boot/Cargo.lock b/X1_JTAG_boot/Cargo.lock index 011adc0f3..c93636780 100644 --- a/X1_JTAG_boot/Cargo.lock +++ b/X1_JTAG_boot/Cargo.lock @@ -4,11 +4,11 @@ version = 3 [[package]] name = "cortex-a" -version = "5.1.6" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ecefc30975eb87afc5a810d4b2305c0ec29e607ea97e51b2ecd80766e9268d28" +checksum = "d25a057b35117d030453cd322f7adbdd331e886bc5bde46cdae656952c4feba6" dependencies = [ - "register", + "tock-registers", ] [[package]] @@ -16,20 +16,11 @@ name = "mingo" version = "0.8.0" dependencies = [ "cortex-a", - "register", -] - -[[package]] -name = "register" -version = "1.0.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4a247de29ab7cc8f5006cfe775c4a81c704f9914c5e2a79696862e643135433" -dependencies = [ "tock-registers", ] [[package]] name = "tock-registers" -version = "0.6.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f521a79accce68c417c9c77ce22108056b626126da1932f7e2e9b5bbffee0cea" +checksum = "4ee8fba06c1f4d0b396ef61a54530bb6b28f0dc61c38bc8bc5a5a48161e6282e" diff --git a/X1_JTAG_boot/Cargo.toml b/X1_JTAG_boot/Cargo.toml index 7b1adb5b3..07dc9b5b6 100644 --- a/X1_JTAG_boot/Cargo.toml +++ b/X1_JTAG_boot/Cargo.toml @@ -9,8 +9,8 @@ lto = true [features] default = [] -bsp_rpi3 = ["register"] -bsp_rpi4 = ["register"] +bsp_rpi3 = ["tock-registers"] +bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -23,8 +23,9 @@ path = "src/main.rs" [dependencies] # Optional dependencies -register = { version = "1.x.x", optional = true } +tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] -cortex-a = { version = "5.x.x" } +cortex-a = { version = "6.x.x" } + diff --git a/X1_JTAG_boot/src/_arch/aarch64/time.rs b/X1_JTAG_boot/src/_arch/aarch64/time.rs index 63017305b..7a7647ef7 100644 --- a/X1_JTAG_boot/src/_arch/aarch64/time.rs +++ b/X1_JTAG_boot/src/_arch/aarch64/time.rs @@ -13,7 +13,8 @@ use crate::{time, warn}; use core::time::Duration; -use cortex_a::{barrier, regs::*}; +use cortex_a::{asm::barrier, registers::*}; +use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 41d8b861c..8033381e9 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -8,7 +8,11 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, synchronization::NullLock, }; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, + registers::ReadWrite, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions diff --git a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index d2aae4d24..70b3839e8 100644 --- a/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/X1_JTAG_boot/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -14,7 +14,11 @@ use crate::{ synchronization::NullLock, }; use core::fmt; -use register::{mmio::*, register_bitfields, register_structs}; +use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, + registers::{ReadOnly, ReadWrite, WriteOnly}, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -81,6 +85,7 @@ register_bitfields! { LCR_H [ /// Word length. These bits indicate the number of data bits transmitted or received in a /// frame. + #[allow(clippy::enum_variant_names)] WLEN OFFSET(5) NUMBITS(2) [ FiveBit = 0b00, SixBit = 0b01, From b23c4318f2edf83cc76ba5487afe7ffbfa2eb649 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 6 Jul 2021 23:02:20 +0200 Subject: [PATCH 081/214] Switch to tock-registers: Documentation and misc updates --- 02_runtime_init/README.md | 5 +- 05_drivers_gpio_uart/README.md | 25 ++-- 06_uart_chainloader/README.md | 8 +- 06_uart_chainloader/demo_payload_rpi3.img | Bin 6696 -> 6696 bytes 06_uart_chainloader/demo_payload_rpi4.img | Bin 6552 -> 6528 bytes 07_timestamps/README.md | 13 +- 09_privilege_level/README.md | 25 ++-- .../README.md | 15 ++- 11_exceptions_part1_groundwork/README.md | 14 +- 12_integrated_testing/README.md | 31 +++-- 13_exceptions_part2_peripheral_IRQs/README.md | 80 +++++++----- 14_virtual_mem_part2_mmio_remap/README.md | 123 +++++++++--------- .../README.md | 28 ++-- .../README.md | 28 ++-- X1_JTAG_boot/jtag_boot_rpi3.img | Bin 7856 -> 7840 bytes X1_JTAG_boot/jtag_boot_rpi4.img | Bin 6584 -> 6568 bytes 16 files changed, 216 insertions(+), 179 deletions(-) diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 044a2d090..11ad00cc9 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -40,14 +40,15 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml authors = ["Andre Richter "] edition = "2018" -@@ -21,3 +21,7 @@ +@@ -21,3 +21,8 @@ ##-------------------------------------------------------------------------------------------------- [dependencies] + +# Platform specific dependencies +[target.'cfg(target_arch = "aarch64")'.dependencies] -+cortex-a = { version = "5.x.x" } ++cortex-a = { version = "6.x.x" } ++ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile --- 01_wait_forever/Makefile diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index 4125583e3..7f0bbda5b 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -126,8 +126,8 @@ diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml default = [] -bsp_rpi3 = [] -bsp_rpi4 = [] -+bsp_rpi3 = ["register"] -+bsp_rpi4 = ["register"] ++bsp_rpi3 = ["tock-registers"] ++bsp_rpi4 = ["tock-registers"] [[bin]] name = "kernel" @@ -136,11 +136,11 @@ diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml [dependencies] +# Optional dependencies -+register = { version = "1.x.x", optional = true } ++tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } + # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] - cortex-a = { version = "5.x.x" } + cortex-a = { version = "6.x.x" } diff -uNr 04_safe_globals/Makefile 05_drivers_gpio_uart/Makefile --- 04_safe_globals/Makefile @@ -220,7 +220,7 @@ diff -uNr 04_safe_globals/src/_arch/aarch64/cpu.rs 05_drivers_gpio_uart/src/_arc diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -@@ -0,0 +1,221 @@ +@@ -0,0 +1,225 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -231,7 +231,11 @@ diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 05_drivers_g + bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, + synchronization::NullLock, +}; -+use register::{mmio::*, register_bitfields, register_structs}; ++use tock_registers::{ ++ interfaces::{ReadWriteable, Writeable}, ++ register_bitfields, register_structs, ++ registers::ReadWrite, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -446,7 +450,7 @@ diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 05_drivers_g diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -0,0 +1,403 @@ +@@ -0,0 +1,408 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -463,7 +467,11 @@ diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 05_dri + synchronization::NullLock, +}; +use core::fmt; -+use register::{mmio::*, register_bitfields, register_structs}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ register_bitfields, register_structs, ++ registers::{ReadOnly, ReadWrite, WriteOnly}, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -530,6 +538,7 @@ diff -uNr 04_safe_globals/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 05_dri + LCR_H [ + /// Word length. These bits indicate the number of data bits transmitted or received in a + /// frame. ++ #[allow(clippy::enum_variant_names)] + WLEN OFFSET(5) NUMBITS(2) [ + FiveBit = 0b00, + SixBit = 0b01, diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index a7239e8b3..0f9e4aa61 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -279,7 +279,7 @@ diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/ diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -@@ -144,7 +144,7 @@ +@@ -148,7 +148,7 @@ // Make an educated guess for a good delay value (Sequence described in the BCM2837 // peripherals PDF). // @@ -292,7 +292,7 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 06_uart diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -279,7 +279,7 @@ +@@ -284,7 +284,7 @@ } /// Retrieve a character. @@ -301,7 +301,7 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 // If RX FIFO is empty, if self.registers.FR.matches_all(FR::RXFE::SET) { // immediately return in non-blocking mode. -@@ -294,12 +294,7 @@ +@@ -299,12 +299,7 @@ } // Read one character. @@ -315,7 +315,7 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 // Update statistics. self.chars_read += 1; -@@ -379,14 +374,14 @@ +@@ -384,14 +379,14 @@ impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { self.inner diff --git a/06_uart_chainloader/demo_payload_rpi3.img b/06_uart_chainloader/demo_payload_rpi3.img index 4e640c636357f92152d3c180e30879ff3abcb130..5e3bf103b1fecffa26150f846a9608f5b22fe04a 100755 GIT binary patch delta 1694 zcmah}du&r>6hGg+Exos6?fP1KncD7UZDZnNbEWdf?m803%pyV|@=8Yt2^$DjBtEen z#3&Bgt{)K@CSo*!dsX7HX;@5TbJ0vRn1PTOHQaVFxod%BEJ*9ZuIKLIA4cO%zMJoH z&hPxr@0{;DU4Od1_bJlfzrjaL``P3nx{tINlQB9%&XFYTVpbc=RrE3=8WV0nU`>oE z1XJf}rFk>cRZCOmHqQ*}SLy-WD(g}*tT&ux%atU}nD>x%^vNX^BtV~DQqeZUa*BqQ zCbZ*zW;HP0WaYf|c zzL>5#b5q^=%S~vyMo(BWB_UG!{Z2DXg~(F+Iwx;xBZY{}z+fYB<*Sgzo1K%(6NEfu z#d!|dP2#Yj7Aa!Qe|$GM(@uyVF^kc+c#wYZfb(#hdu^(VO~!ElhLJnsG|lSh>mKOl zlVl?%qgW82@GrtHM`1miV{&=DI3%F3(FI{Ory-XIg)RSESSw;&jjc=;*mwI3@iiM= zQ}VFYkLsO7u9}k`DOr{>s9r<)uCrFfpf#G;+SChXFm*tc&m-fqS$7QY4+*SdGGL-!(mLx;lul!% za+*NIhsXwMC3fdk#*?q2yltoBO9Dr;Xf^*TxRt&1HQW8d_=D0?lwGX`*|=<(5+diD zK{`z6d0SWr%<9LCJEqOm7!(yNo5fl;(^7kI6`%vO-z$+cNWTLdokpt^19>n(oU0LK zw+K?mVKg|W7~C}0KyWMCtUjn&Ku%YsF+B*0NUrbtI53 zgGcZY=q<-VaNLJg2{qj#T{%!o2g+*6uk`1#3i1cNRkqeMjP@$F#U`{d5}mfXa)dgZAJv?!mSV$|(l)V5%CJPbwUDdvp6H`we6|jXH|S;OSWRWM zq<%=GTm)M!qdU4Re^K zSk&=(hF-j0y(|ten6%eO+teB)y@!#cr0RR2?IQ7se?=fPchkT7fT0k23+2{TMJqv3>$9+%m nCj0#99vTvYHtXZtpWVHESKyhgyF0d3(mjHoc{D(KgyVk!V%A<* delta 1744 zcmZ`)ZERCz6h800?RvYfZvF0nw4UhL^l@0)Vnztz#-qn`noys1XG43W&`g+9z0(4i!g2W?UP*k)u zprzRek}w^Yh`?L zN{>E{v=m_2H)RUMEb~4NCCl3t=GRj)1s)(~H*c#b^l%uM)|=Wli4RSIDc;-mUMGyd zpM~Ig46X7FikCDLrdm!lD7Y2?zA)n@^!laNlv7e$F3D*Vk<&a#H#vj^-_MhL_wD{& zCvJc;aUY_o`~6@4Nl%$FRYyq84?B%el!=)R3i761WGY}LFcu=V!X(mT{~TuX;f~z$ zZbFuwBM{AF$~7oWf>ajHEdr}(h3H2{F?hrb(lrNIhqg7WO|Tl^`+JyQ9%@R_n^kXIgN#~#tzij`H#j(*#2u|Wt`POu+WWF&nI-w zR_MoEA2;7y0TPqfr|TWD3b@Yl{tflK$5$#W$GZhRI)OYykpPo(&G1lQ}! zTDwqdS0RmBXL0@eyf51D`B~rZNp)L4KxgzbA_j*r05=D+(_FiD^)n6aJJ6szyfQw5 zU|p+f5kYWf9&7#bhq>T_@JuqT`Xxbl< z!-EOAlm%&H7tUmjsFmkNGj%p&(3tbT$~gIdm0@bkt0VHj21+TRA~DWg#>y;5Uxrnjtyw0ipT{FGqj9es%`TuoY9Z5xm3$-07@E*pTz=(D6zDJ?dZS`gW)YtNr{6{ z1QcL^vz3!wQa?+4Zv=DJ^)dE7Z9Kj-q~rT%TzNtw?WKuQrip zDIdUwK0_ouVUTlvURM1eu^v)ejB@r0NM4$~{KWm@rYD`>;@70tFub%pR=bR8pg^sqZ*KH$ML@MobJ_ZoAZWp=fP o20bm_-E(`SYynWYdfVH5+uri+>FwLTb%l@idAy{N9`&5~8{RT*cK`qY diff --git a/06_uart_chainloader/demo_payload_rpi4.img b/06_uart_chainloader/demo_payload_rpi4.img index 7e017a42beb30832cd2c7beb0d8e180310445b27..a8c1362c19ab56f4e4d1857fe201b383f5618cbf 100755 GIT binary patch delta 3309 zcmai0e{2)y8Gg@aJ9kcSV#i7Bgw}Qfqc&lM8semA&BaiSgrb&^vT1Zpq82F$q0BI~ z`vJ3oN&^zQ+%6gw-PR=hfzB(8s>wtqF#?&?cI%LUjjh@r*g;c+4Q+w7*dTIyzVn$? zk*0mp<^0|I`+1-5{qklg9vbq9^fj3yI^`$1wjp+sr(qxY{yN2XUeVUjMS-N8R!Wo2 zWQvXvb)-Vn6rgx}z(Nx{h+NI&i$*x=2<*0QvX#Z#gDwjBw^l#F^wME!YK88HJWagJ zbR{V|23-+)On2$-l1;&SYbXGl?%o<^6Vr{ZucoG67>XvE*bKwKZl~_N7)B+`aBWGb z9)^0b?F;o_83ellyL2-(9b~p5ji*B-w{qeDdp~0#d6XK%OBa$e&6gxfI4Ku`@3%QI zGh(8S^%kkAO^lJet%98OW#n82#j|_(=s^fO4W$hgQsW88w^vk$n4{P^K6=cMCz%t- znSBKuXLxTg&5Mbz`ugC3)-m9y4zb`a=-f1P9{OVr$DnlB;Rfp9Cn zHo=! z6nvcrvi(37{nvFEvT5jS1F{1^wvT(-7VHGHaj=v*DL~r~Xg>n9eSp>mXxjjz4bZZz zyR%vcgOb5WI7gH$SKi{^FpavDrRKCbK)3#+cq}iNQnku!mJJp6d2eDR;wJg>M2fEp z&hV9qsPeI82lu?vYOQ(1XQhr=rX7 zY>*}wB9UJf(XAxHCe>z zgt-7>6b6SR{`^vW(aGD%?tk8NNkm5jFiq8-D) z9_-@`)*@JhwMmfB5)L&9i95=QlA72$LCW3XO$pIFb*T@-F?HOermsK`X7F}3_lj#Ha6dw zQ$_x}*%aoPfZ()PQhmsL%1q(!fERAey1NmQQ#eMb-qll?N5!k7a^UDMK-*m?hM|)E zKWO+&6z*9BxL6PtkOjhc3A+)91@VKkS4|a}T7|Q3b70;MvoV{W4z9#5Wskl7(b45{ z3A&K1Bb6P&R57P!){s1Su~8PzRVnY=7jr)4roGC(W=^Z}`68-5wI#(_T4y?6r-Vx* zj*SkSRaW%r*@4kmhV(ij%6ZrEA}$)3fvF>K8B?6%-~+&HSI{!NvohSB&_hsA9C^bSv2&^ zznfV8LxNvMF%-(kuX0_U$nPM$`7_a_~Y4~xQ@m#@`skn%8WBszt<E! zsv{#~%fr4Tk8$$MG|9`A!{x8XMsZUZ26|(0bLfU>=0iN;gc+DwY4&# zF1O=#6Fy+QO!@aSiddnwo_6PVx(I)wmH9n@)7R5GckbAUUt5gtps2p)>piDztZa^* zWp^eP=8(RAKW~4+*qaUcyT%@n>+2IE8(^icUm5Z;V?SSh(2#@6Y_-#?#s{2Sea$!M zGUSCazHRKShW?AjzEJ)rN_W)}F>5zKQaE?m^A97*()lrz7WcDH6?ES>v>b*OdrNo< zfETQ7>$mQFWoy6hd)=?}zT{V~y1k2@^w}b7Hu%2zTnb{ zf{w`sCXKH~dAg|>)jra8f9$za#WvU!X)6hAlW41Iz)Gb_6HbsOJ09CoD9KBO-+uSm zrdw21nxuBn=%QN>0Tn5ur@~3L<|ug@)Q0B_oII+Z9{u+B|qlL#99Cr&vBti7k*@ zD&$mk=m_Kl&xt2`sBZf<=hC%ZtL(8E=oBCeopdb!C?!%*s)thKkaxQfcHvQ$wB0uz z-CP%o2t_Cf`7SssbW`FO^a?T3!b~}t!~5A98VwRF<{XqP$drvXPHfn+jJzjkKAw}E z*7^tT-r9(pi&M6Qq!~vXQ(_4|#)!<)KYr2qPK1U4C`LHt=>gw#I^!^3c%= zZvt-xJOw;I3>M{JG$*rc)rM3ulPf)~vAkRx6;6GSnW@NBZC$))CZlPOA3`om$OWN1 zgu0yd+qbT6jYXIe4fbpv$^b@TL;^n;lHZHAsN-*$55*&l##46HP;8{7>|}_bpPDz5 zIoK%m_2?+4W|dWDR}_@A)FJCRnKa3s>Hi^+5>1c}XM9@jCTY`Pd`qIy(o)o3W|m^9 z?}DC3R-zxmB%RF3O4j{fL;R@-r~m&De=5RT{}+fSe=6W5z(}f}pO%Lu*2p_+W@=W@ z>^r>C_GKv?=KF1ru?0S9^Ra#WJ=@opC5cAI?m6Pmf2CKC$jeSDtnKZ<8M+d1&@Az% z>`PZK0A}<-wg&ZfqPEJto@*QN9C)r*Ke%2UZ^PaXAf3a}OmW@5v}MAox-UB=H%F)C zrYw4VG&Jvsr}&iJ=NX)H#Bamifb6rnKE-X0*iQ9BM|>Wg^1^x=|MoN`-Z0JPyxf`% zvex2!=Z{;npcSO%#G{16TGwH}mhXI1&pDXYMc2?Q)xnY3d*&)-FML4B$qzEWxy!j@ zs`f>;-IE`sD%Anh!009TabglJn9@`PtJjGKzZ&H;EMyk z#=%~_=@E6j1@I**@F_mk$jOZWt|>~a`?_j6)+1s6jo!^e0$#<Ebt6Q`gr4!iKN2gBfV7zVq+AHXmeE1K!#CQ%zjMI9JISF(3XX0y3T zx>D*OW8qnwK0jdB@9uEucls24hoTy{WhtxFOkCZCc{%l(q!(^d!j#D{Ystv@$e2^; zT&i9*zWJo82U%nBS4WP$e7%;=9doIASDk7IA0faKs-yURZ(211L?bUhOyTR?a>RO=uVm)Lw38o%E>O9YIL)J)dVPBw$MR(;Z-k5 z3C3zDnd|7)drqeHxoOhAvMPP zv(NFp^{bX^4{LQOKn}o&6FODP^diih@+34R9N@pMU%}e=U+SCdyU`b+P_J%X_hVdg zcS!9I@J@HHXVQbiT!ZOO8m^tQ+a~!7?qGx3qUB(yc+Wg7j&_3U?#Sw%R;>{t=PyEK zN7*?W z@2^rwn?fA(uvwk#<|THe6eD96dy0IdU-{jIZ-nf04nuNel5Vy4k@sM~$$I*>UON@%#T1@jStRjNjn3zwrBY}ckB zGhB&^@G)`_eGSAg)5Hu^vOfZC>CyulLwAX5*^AF0oS9%&5LOfYmJjo3qQv>YibVfZz}Ji_pEiiLIXK#rFf#LJN>@6E3q(xtaWt} z{rd`yJKb8BD>QJl){Tm9AY-lb6@Ej7Ud(@`!bg*e)lb8f4ID{pEmk;C;aAhxSE20{ z`R6NiHLV+bYx8NR;x4BZ;DzS6bG8zsn @@ -237,7 +237,8 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/time.rs 07_timestamps/src/_arch/ + +use crate::{time, warn}; +use core::time::Duration; -+use cortex_a::{barrier, regs::*}; ++use cortex_a::{asm::barrier, registers::*}; ++use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -344,7 +345,7 @@ diff -uNr 06_uart_chainloader/src/_arch/aarch64/time.rs 07_timestamps/src/_arch/ diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -@@ -139,25 +139,19 @@ +@@ -143,25 +143,19 @@ /// Disable pull-up/down on pins 14 and 15. #[cfg(feature = "bsp_rpi3")] fn disable_pud_14_15_bcm2837(&mut self) { @@ -380,7 +381,7 @@ diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 07_times diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 07_timestamps/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -279,7 +279,7 @@ +@@ -284,7 +284,7 @@ } /// Retrieve a character. @@ -389,7 +390,7 @@ diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07 // If RX FIFO is empty, if self.registers.FR.matches_all(FR::RXFE::SET) { // immediately return in non-blocking mode. -@@ -294,7 +294,12 @@ +@@ -299,7 +299,12 @@ } // Read one character. @@ -403,7 +404,7 @@ diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07 // Update statistics. self.chars_read += 1; -@@ -374,14 +379,14 @@ +@@ -379,14 +384,14 @@ impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { self.inner diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index 42ad905e8..4b4c538d8 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -211,11 +211,12 @@ diff -uNr 08_hw_debug_JTAG/Cargo.toml 09_privilege_level/Cargo.toml diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/_arch/aarch64/cpu/boot.rs --- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs +++ 09_privilege_level/src/_arch/aarch64/cpu/boot.rs -@@ -11,17 +11,67 @@ +@@ -11,17 +11,68 @@ //! //! crate::cpu::boot::arch_boot -+use cortex_a::{asm, regs::*}; ++use cortex_a::{asm, registers::*}; ++use tock_registers::interfaces::Writeable; + // Assembly counterpart to this file. global_asm!(include_str!("boot.s")); @@ -323,7 +324,7 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs --- 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs +++ 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs -@@ -0,0 +1,81 @@ +@@ -0,0 +1,82 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -337,14 +338,15 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privil +//! +//! crate::exception::asynchronous::arch_asynchronous + -+use cortex_a::regs::*; ++use cortex_a::registers::*; ++use tock_registers::interfaces::Readable; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +trait DaifField { -+ fn daif_field() -> register::Field; ++ fn daif_field() -> tock_registers::fields::Field; +} + +struct Debug; @@ -357,25 +359,25 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privil +//-------------------------------------------------------------------------------------------------- + +impl DaifField for Debug { -+ fn daif_field() -> register::Field { ++ fn daif_field() -> tock_registers::fields::Field { + DAIF::D + } +} + +impl DaifField for SError { -+ fn daif_field() -> register::Field { ++ fn daif_field() -> tock_registers::fields::Field { + DAIF::A + } +} + +impl DaifField for IRQ { -+ fn daif_field() -> register::Field { ++ fn daif_field() -> tock_registers::fields::Field { + DAIF::I + } +} + +impl DaifField for FIQ { -+ fn daif_field() -> register::Field { ++ fn daif_field() -> tock_registers::fields::Field { + DAIF::F + } +} @@ -409,7 +411,7 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privil diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs 09_privilege_level/src/_arch/aarch64/exception.rs --- 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs +++ 09_privilege_level/src/_arch/aarch64/exception.rs -@@ -0,0 +1,30 @@ +@@ -0,0 +1,31 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -423,7 +425,8 @@ diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs 09_privilege_level/src +//! +//! crate::exception::arch_exception + -+use cortex_a::regs::*; ++use cortex_a::registers::*; ++use tock_registers::interfaces::Readable; + +//-------------------------------------------------------------------------------------------------- +// Public Code diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index f573de03a..9b16747d1 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -364,7 +364,7 @@ diff -uNr 09_privilege_level/Cargo.toml 10_virtual_mem_part1_identity_mapping/Ca diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs --- 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu/translation_table.rs -@@ -0,0 +1,288 @@ +@@ -0,0 +1,292 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2021 Andre Richter @@ -388,7 +388,11 @@ diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + }, +}; +use core::convert; -+use register::{register_bitfields, InMemoryRegister}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ register_bitfields, ++ registers::InMemoryRegister, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -549,7 +553,7 @@ diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 + +/// Convert the kernel's generic memory attributes to HW-specific attributes of the MMU. +impl convert::From -+ for register::FieldValue ++ for tock_registers::fields::FieldValue +{ + fn from(attribute_fields: AttributeFields) -> Self { + // Memory attributes. @@ -657,7 +661,7 @@ diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu/translation_table.rs 1 diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu.rs 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs --- 09_privilege_level/src/_arch/aarch64/memory/mmu.rs +++ 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/memory/mmu.rs -@@ -0,0 +1,164 @@ +@@ -0,0 +1,165 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -678,7 +682,8 @@ diff -uNr 09_privilege_level/src/_arch/aarch64/memory/mmu.rs 10_virtual_mem_part + memory::mmu::{translation_table::KernelTranslationTable, TranslationGranule}, +}; +use core::intrinsics::unlikely; -+use cortex_a::{barrier, regs::*}; ++use cortex_a::{asm::barrier, registers::*}; ++use tock_registers::interfaces::{ReadWriteable, Readable, Writeable}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions diff --git a/11_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md index 77a45ec6d..df8ecd6f9 100644 --- a/11_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -492,14 +492,18 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/Cargo.toml 11_exceptions_part1_g diff -uNr 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs --- 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs +++ 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs -@@ -11,7 +11,224 @@ +@@ -11,8 +11,227 @@ //! //! crate::exception::arch_exception --use cortex_a::regs::*; +-use cortex_a::registers::*; +-use tock_registers::interfaces::Readable; +use core::{cell::UnsafeCell, fmt}; -+use cortex_a::{asm, barrier, regs::*}; -+use register::InMemoryRegister; ++use cortex_a::{asm, asm::barrier, registers::*}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ registers::InMemoryRegister, ++}; + +// Assembly counterpart to this file. +global_asm!(include_str!("exception.s")); @@ -718,7 +722,7 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/_arch/aarch64/exception.rs 1 //-------------------------------------------------------------------------------------------------- // Public Code -@@ -28,3 +245,23 @@ +@@ -29,3 +248,23 @@ _ => (PrivilegeLevel::Unknown, "Unknown"), } } diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index d252d0c23..bb1677e98 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -825,10 +825,10 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. authors = ["Andre Richter "] edition = "2018" -@@ -11,20 +11,46 @@ +@@ -11,21 +11,46 @@ default = [] - bsp_rpi3 = ["register"] - bsp_rpi4 = ["register"] + bsp_rpi3 = ["tock-registers"] + bsp_rpi4 = ["tock-registers"] - -[[bin]] -name = "kernel" @@ -843,14 +843,13 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. +test-types = { path = "test-types" } # Optional dependencies --register = { version = "1.x.x", optional = true } -+register = { version = "1.x.x", features = ["no_std_unit_tests"], optional = true } -+qemu-exit = { version = "1.x.x", optional = true } + tock-registers = { version = "0.7.x", default-features = false, features = ["register_types"], optional = true } ++qemu-exit = { version = "2.x.x", optional = true } # Platform specific dependencies [target.'cfg(target_arch = "aarch64")'.dependencies] - cortex-a = { version = "5.x.x" } -+ + cortex-a = { version = "6.x.x" } + +##-------------------------------------------------------------------------------------------------- +## Testing +##-------------------------------------------------------------------------------------------------- @@ -1010,12 +1009,12 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 12_integ //! crate::exception::arch_exception use core::{cell::UnsafeCell, fmt}; --use cortex_a::{asm, barrier, regs::*}; -+use cortex_a::{barrier, regs::*}; - use register::InMemoryRegister; - - // Assembly counterpart to this file. -@@ -87,16 +87,6 @@ +-use cortex_a::{asm, asm::barrier, registers::*}; ++use cortex_a::{asm::barrier, registers::*}; + use tock_registers::{ + interfaces::{Readable, Writeable}, + registers::InMemoryRegister, +@@ -90,16 +90,6 @@ #[no_mangle] unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) { @@ -1036,7 +1035,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/exception.rs 12_integ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs 12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs --- 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 12_integrated_testing/src/_arch/aarch64/memory/mmu/translation_table.rs -@@ -286,3 +286,31 @@ +@@ -290,3 +290,31 @@ self.lvl2.phys_start_addr_u64() } } @@ -1072,7 +1071,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu/translatio diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs --- 11_exceptions_part1_groundwork/src/_arch/aarch64/memory/mmu.rs +++ 12_integrated_testing/src/_arch/aarch64/memory/mmu.rs -@@ -162,3 +162,33 @@ +@@ -163,3 +163,33 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 1174ca9e4..02341a12a 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -761,7 +761,7 @@ diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/C diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs --- 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs -@@ -0,0 +1,29 @@ +@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -775,7 +775,8 @@ diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2 +//! +//! crate::cpu::smp::arch_smp + -+use cortex_a::regs::*; ++use cortex_a::registers::*; ++use tock_registers::interfaces::Readable; + +//-------------------------------------------------------------------------------------------------- +// Public Code @@ -795,7 +796,14 @@ diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2 diff -uNr 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs --- 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs +++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception/asynchronous.rs -@@ -17,6 +17,10 @@ +@@ -12,12 +12,16 @@ + //! crate::exception::asynchronous::arch_asynchronous + + use cortex_a::registers::*; +-use tock_registers::interfaces::Readable; ++use tock_registers::interfaces::{Readable, Writeable}; + + //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- @@ -804,9 +812,9 @@ diff -uNr 12_integrated_testing/src/_arch/aarch64/exception/asynchronous.rs 13_e +} + trait DaifField { - fn daif_field() -> register::Field; + fn daif_field() -> tock_registers::fields::Field; } -@@ -65,6 +69,71 @@ +@@ -66,6 +70,71 @@ // Public Code //-------------------------------------------------------------------------------------------------- @@ -888,9 +896,9 @@ diff -uNr 12_integrated_testing/src/_arch/aarch64/exception.rs 13_exceptions_par +use crate::{bsp, exception}; use core::{cell::UnsafeCell, fmt}; - use cortex_a::{barrier, regs::*}; - use register::InMemoryRegister; -@@ -91,8 +92,11 @@ + use cortex_a::{asm::barrier, registers::*}; + use tock_registers::{ +@@ -94,8 +95,11 @@ } #[no_mangle] @@ -908,7 +916,7 @@ diff -uNr 12_integrated_testing/src/_arch/aarch64/exception.rs 13_exceptions_par diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs --- 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs +++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicc.rs -@@ -0,0 +1,137 @@ +@@ -0,0 +1,141 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -916,7 +924,11 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 13_excep +//! GICC Driver - GIC CPU interface. + +use crate::{bsp::device_driver::common::MMIODerefWrapper, exception}; -+use register::{mmio::*, register_bitfields, register_structs}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ register_bitfields, register_structs, ++ registers::ReadWrite, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -1050,7 +1062,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicc.rs 13_excep diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs --- 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs +++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gicd.rs -@@ -0,0 +1,195 @@ +@@ -0,0 +1,199 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter @@ -1064,7 +1076,11 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/arm/gicv2/gicd.rs 13_excep + bsp::device_driver::common::MMIODerefWrapper, state, synchronization, + synchronization::IRQSafeNullLock, +}; -+use register::{mmio::*, register_bitfields, register_structs}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ register_bitfields, register_structs, ++ registers::{ReadOnly, ReadWrite}, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -1495,9 +1511,9 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 13_exc - synchronization::NullLock, + synchronization::IRQSafeNullLock, }; - use register::{mmio::*, register_bitfields, register_structs}; - -@@ -117,7 +117,7 @@ + use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, +@@ -121,7 +121,7 @@ /// Representation of the GPIO HW. pub struct GPIO { @@ -1506,7 +1522,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 13_exc } //-------------------------------------------------------------------------------------------------- -@@ -193,7 +193,7 @@ +@@ -197,7 +197,7 @@ /// - The user must ensure to provide a correct MMIO start address. pub const unsafe fn new(mmio_start_addr: usize) -> Self { Self { @@ -1519,12 +1535,12 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 13_exc diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs --- 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs -@@ -0,0 +1,163 @@ +@@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2021 Andre Richter + -+//! Peripheral Interrupt regsler Driver. ++//! Peripheral Interrupt Controller Driver. + +use super::{InterruptController, PendingIRQs, PeripheralIRQ}; +use crate::{ @@ -1532,7 +1548,11 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_interrupt_cont + exception, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, +}; -+use register::{mmio::*, register_structs}; ++use tock_registers::{ ++ interfaces::{Readable, Writeable}, ++ register_structs, ++ registers::{ReadOnly, WriteOnly}, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -1833,8 +1853,8 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs + synchronization, synchronization::IRQSafeNullLock, }; use core::fmt; - use register::{mmio::*, register_bitfields, register_structs}; -@@ -135,6 +135,52 @@ + use tock_registers::{ +@@ -140,6 +140,52 @@ ] ], @@ -1887,7 +1907,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs /// Interrupt Clear Register. ICR [ /// Meta field for all pending interrupts. -@@ -153,7 +199,10 @@ +@@ -158,7 +204,10 @@ (0x28 => FBRD: WriteOnly), (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), @@ -1899,7 +1919,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs (0x44 => ICR: WriteOnly), (0x48 => @END), } -@@ -183,7 +232,8 @@ +@@ -188,7 +237,8 @@ /// Representation of the UART. pub struct PL011Uart { @@ -1909,7 +1929,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } //-------------------------------------------------------------------------------------------------- -@@ -251,6 +301,14 @@ +@@ -256,6 +306,14 @@ .LCR_H .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); @@ -1924,7 +1944,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs // Turn the UART on. self.registers .CR -@@ -333,9 +391,13 @@ +@@ -338,9 +396,13 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -1940,7 +1960,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } } } -@@ -355,6 +417,21 @@ +@@ -360,6 +422,21 @@ Ok(()) } @@ -1962,7 +1982,7 @@ diff -uNr 12_integrated_testing/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs } impl console::interface::Write for PL011Uart { -@@ -401,3 +478,24 @@ +@@ -406,3 +483,24 @@ self.inner.lock(|inner| inner.chars_read) } } @@ -2376,15 +2396,17 @@ diff -uNr 12_integrated_testing/src/exception/asynchronous.rs 13_exceptions_part diff -uNr 12_integrated_testing/src/lib.rs 13_exceptions_part2_peripheral_IRQs/src/lib.rs --- 12_integrated_testing/src/lib.rs +++ 13_exceptions_part2_peripheral_IRQs/src/lib.rs -@@ -108,6 +108,7 @@ +@@ -108,7 +108,9 @@ #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] +#![feature(asm)] #![feature(const_fn_fn_ptr_basics)] ++#![feature(const_fn_trait_bound)] #![feature(const_generics)] #![feature(const_panic)] -@@ -134,6 +135,7 @@ + #![feature(core_intrinsics)] +@@ -134,6 +136,7 @@ pub mod exception; pub mod memory; pub mod print; diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 73ef83a3a..6cb7e16f1 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -397,9 +397,9 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 14_ + memory::Address, +}; use core::{cell::UnsafeCell, fmt}; - use cortex_a::{barrier, regs::*}; - use register::InMemoryRegister; -@@ -50,6 +54,20 @@ + use cortex_a::{asm::barrier, registers::*}; + use tock_registers::{ +@@ -53,6 +57,20 @@ // Private Code //-------------------------------------------------------------------------------------------------- @@ -420,7 +420,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/exception.rs 14_ /// Prints verbose information about the exception and then panics. fn default_exception_handler(e: &ExceptionContext) { panic!( -@@ -166,7 +184,9 @@ +@@ -169,7 +187,9 @@ writeln!(f, " - {}", ec_translation)?; // Raw print of instruction specific syndrome. @@ -451,7 +451,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans }, }; use core::convert; -@@ -117,12 +120,9 @@ +@@ -121,12 +124,9 @@ } trait StartAddr { @@ -465,7 +465,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- -@@ -137,10 +137,13 @@ +@@ -141,10 +141,13 @@ /// Table descriptors, covering 512 MiB windows. lvl2: [TableDescriptor; NUM_TABLES], @@ -482,7 +482,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans //-------------------------------------------------------------------------------------------------- // Private Code -@@ -148,12 +151,8 @@ +@@ -152,12 +155,8 @@ // The binary is still identity mapped, so we don't need to convert here. impl StartAddr for [T; N] { @@ -497,7 +497,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans } } -@@ -166,10 +165,10 @@ +@@ -170,10 +169,10 @@ } /// Create an instance pointing to the supplied address. @@ -510,7 +510,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans val.write( STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64) + STAGE1_TABLE_DESCRIPTOR::TYPE::Table -@@ -226,7 +225,10 @@ +@@ -230,7 +229,10 @@ } /// Create an instance. @@ -522,7 +522,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans let val = InMemoryRegister::::new(0); let shifted = phys_output_addr as u64 >> Granule64KiB::SHIFT; -@@ -240,50 +242,193 @@ +@@ -244,50 +246,193 @@ Self { value: val.get() } } @@ -687,7 +687,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans + return Err("Virtual page is already mapped"); } + -+ *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), &attr); ++ *page_descriptor = PageDescriptor::from_output_addr(phys_page.as_ptr(), attr); } Ok(()) @@ -733,7 +733,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu/trans } } -@@ -292,6 +437,9 @@ +@@ -296,6 +441,9 @@ //-------------------------------------------------------------------------------------------------- #[cfg(test)] @@ -755,8 +755,8 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 + memory::{mmu::TranslationGranule, Address, Physical}, }; use core::intrinsics::unlikely; - use cortex_a::{barrier, regs::*}; -@@ -45,13 +45,6 @@ + use cortex_a::{asm::barrier, registers::*}; +@@ -46,13 +46,6 @@ // Global instances //-------------------------------------------------------------------------------------------------- @@ -770,7 +770,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 static MMU: MemoryManagementUnit = MemoryManagementUnit; //-------------------------------------------------------------------------------------------------- -@@ -86,7 +79,7 @@ +@@ -87,7 +80,7 @@ /// Configure various settings of stage 1 of the EL1 translation regime. fn configure_translation_control(&self) { @@ -779,7 +779,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 TCR_EL1.write( TCR_EL1::TBI0::Used -@@ -118,7 +111,10 @@ +@@ -119,7 +112,10 @@ use memory::mmu::MMUEnableError; impl memory::mmu::interface::MMU for MemoryManagementUnit { @@ -791,7 +791,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 if unlikely(self.is_enabled()) { return Err(MMUEnableError::AlreadyEnabled); } -@@ -133,13 +129,8 @@ +@@ -134,13 +130,8 @@ // Prepare the memory attribute indirection register. self.set_up_mair(); @@ -806,7 +806,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/memory/mmu.rs 14 self.configure_translation_control(); -@@ -162,33 +153,3 @@ +@@ -163,33 +154,3 @@ SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } } @@ -852,10 +852,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi +use crate::{ + bsp::device_driver::common::MMIODerefWrapper, exception, synchronization::InitStateLock, +}; - use register::{mmio::*, register_bitfields, register_structs}; - - //-------------------------------------------------------------------------------------------------- -@@ -56,12 +58,13 @@ + use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, +@@ -60,12 +62,13 @@ /// Representation of the GIC CPU interface. pub struct GICC { @@ -870,7 +870,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi impl GICC { /// Create an instance. -@@ -71,10 +74,15 @@ +@@ -75,10 +78,15 @@ /// - The user must ensure to provide a correct MMIO start address. pub const unsafe fn new(mmio_start_addr: usize) -> Self { Self { @@ -887,7 +887,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi /// Accept interrupts of any priority. /// /// Quoting the GICv2 Architecture Specification: -@@ -87,7 +95,9 @@ +@@ -91,7 +99,9 @@ /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead /// of `&mut self`. pub fn priority_accept_all(&self) { @@ -898,7 +898,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } /// Enable the interface - start accepting IRQs. -@@ -97,7 +107,9 @@ +@@ -101,7 +111,9 @@ /// - GICC MMIO registers are banked per CPU core. It is therefore safe to have `&self` instead /// of `&mut self`. pub fn enable(&self) { @@ -909,7 +909,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } /// Extract the number of the highest-priority pending IRQ. -@@ -113,7 +125,8 @@ +@@ -117,7 +129,8 @@ &self, _ic: &exception::asynchronous::IRQContext<'irq_context>, ) -> usize { @@ -919,7 +919,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } /// Complete handling of the currently active IRQ. -@@ -132,6 +145,8 @@ +@@ -136,6 +149,8 @@ irq_number: u32, _ic: &exception::asynchronous::IRQContext<'irq_context>, ) { @@ -943,9 +943,9 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi + state, synchronization, + synchronization::{IRQSafeNullLock, InitStateLock}, }; - use register::{mmio::*, register_bitfields, register_structs}; - -@@ -79,7 +80,7 @@ + use tock_registers::{ + interfaces::{Readable, Writeable}, +@@ -83,7 +84,7 @@ shared_registers: IRQSafeNullLock, /// Access to banked registers is unguarded. @@ -954,7 +954,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } //-------------------------------------------------------------------------------------------------- -@@ -116,6 +117,7 @@ +@@ -120,6 +121,7 @@ //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -962,7 +962,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi use synchronization::interface::Mutex; impl GICD { -@@ -127,10 +129,17 @@ +@@ -131,10 +133,17 @@ pub const unsafe fn new(mmio_start_addr: usize) -> Self { Self { shared_registers: IRQSafeNullLock::new(SharedRegisters::new(mmio_start_addr)), @@ -981,7 +981,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi /// Use a banked ITARGETSR to retrieve the executing core's GIC target mask. /// /// Quoting the GICv2 Architecture Specification: -@@ -138,7 +147,8 @@ +@@ -142,7 +151,8 @@ /// "GICD_ITARGETSR0 to GICD_ITARGETSR7 are read-only, and each field returns a value that /// corresponds only to the processor reading the register." fn local_gic_target_mask(&self) -> u32 { @@ -991,7 +991,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/arm/gicv2/gi } /// Route all SPIs to the boot core and enable the distributor. -@@ -177,10 +187,10 @@ +@@ -181,10 +191,10 @@ // Check if we are handling a private or shared IRQ. match irq_num { // Private. @@ -1096,10 +1096,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ synchronization::IRQSafeNullLock, }; +use core::sync::atomic::{AtomicUsize, Ordering}; - use register::{mmio::*, register_bitfields, register_structs}; - - //-------------------------------------------------------------------------------------------------- -@@ -117,6 +118,8 @@ + use tock_registers::{ + interfaces::{ReadWriteable, Writeable}, + register_bitfields, register_structs, +@@ -121,6 +122,8 @@ /// Representation of the GPIO HW. pub struct GPIO { @@ -1108,7 +1108,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ inner: IRQSafeNullLock, } -@@ -136,6 +139,19 @@ +@@ -140,6 +143,19 @@ } } @@ -1128,7 +1128,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Disable pull-up/down on pins 14 and 15. #[cfg(feature = "bsp_rpi3")] fn disable_pud_14_15_bcm2837(&mut self) { -@@ -190,10 +206,12 @@ +@@ -194,10 +210,12 @@ /// /// # Safety /// @@ -1144,7 +1144,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } } -@@ -212,4 +230,26 @@ +@@ -216,4 +234,26 @@ fn compatible(&self) -> &'static str { "BCM GPIO" } @@ -1175,13 +1175,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs --- 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ 14_virtual_mem_part2_mmio_remap/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs -@@ -2,12 +2,12 @@ - // - // Copyright (c) 2020-2021 Andre Richter - --//! Peripheral Interrupt regsler Driver. -+//! Peripheral Interrupt Controller Driver. - +@@ -7,7 +7,7 @@ use super::{InterruptController, PendingIRQs, PeripheralIRQ}; use crate::{ bsp::device_driver::common::MMIODerefWrapper, @@ -1189,8 +1183,8 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + driver, exception, memory, synchronization, synchronization::{IRQSafeNullLock, InitStateLock}, }; - use register::{mmio::*, register_structs}; -@@ -51,11 +51,13 @@ + use tock_registers::{ +@@ -55,11 +55,13 @@ /// Representation of the peripheral interrupt controller. pub struct PeripheralIC { @@ -1205,7 +1199,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, -@@ -70,21 +72,26 @@ +@@ -74,21 +76,26 @@ /// /// # Safety /// @@ -1239,7 +1233,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } } -@@ -93,6 +100,24 @@ +@@ -97,6 +104,24 @@ //------------------------------------------------------------------------------ use synchronization::interface::{Mutex, ReadWriteEx}; @@ -1322,10 +1316,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ + fmt, + sync::atomic::{AtomicUsize, Ordering}, +}; - use register::{mmio::*, register_bitfields, register_structs}; - - //-------------------------------------------------------------------------------------------------- -@@ -232,6 +235,8 @@ + use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, register_structs, +@@ -237,6 +240,8 @@ /// Representation of the UART. pub struct PL011Uart { @@ -1334,7 +1328,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ inner: IRQSafeNullLock, irq_number: bsp::device_driver::IRQNumber, } -@@ -271,7 +276,15 @@ +@@ -276,7 +281,15 @@ /// genrated baud rate of `48_000_000 / (16 * 3.25) = 923_077`. /// /// Error = `((923_077 - 921_600) / 921_600) * 100 = 0.16modulo`. @@ -1351,7 +1345,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ // Execution can arrive here while there are still characters queued in the TX FIFO and // actively being sent out by the UART hardware. If the UART is turned off in this case, // those queued characters would be lost. -@@ -313,6 +326,8 @@ +@@ -318,6 +331,8 @@ self.registers .CR .write(CR::UARTEN::Enabled + CR::TXE::Enabled + CR::RXE::Enabled); @@ -1360,7 +1354,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ } /// Send a character. -@@ -390,13 +405,18 @@ +@@ -395,13 +410,18 @@ /// /// # Safety /// @@ -1382,7 +1376,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ irq_number, } } -@@ -413,7 +433,13 @@ +@@ -418,7 +438,13 @@ } unsafe fn init(&self) -> Result<(), &'static str> { @@ -1397,7 +1391,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/bsp/device_driver/bcm/bcm2xxx_ Ok(()) } -@@ -432,6 +458,16 @@ +@@ -437,6 +463,16 @@ Ok(()) } @@ -2155,16 +2149,15 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/src/driver.rs 14_virtual_mem_part2 diff -uNr 13_exceptions_part2_peripheral_IRQs/src/lib.rs 14_virtual_mem_part2_mmio_remap/src/lib.rs --- 13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ 14_virtual_mem_part2_mmio_remap/src/lib.rs -@@ -109,6 +109,8 @@ +@@ -109,6 +109,7 @@ #![allow(clippy::upper_case_acronyms)] #![allow(incomplete_features)] #![feature(asm)] +#![feature(const_evaluatable_checked)] -+#![feature(const_fn)] #![feature(const_fn_fn_ptr_basics)] + #![feature(const_fn_trait_bound)] #![feature(const_generics)] - #![feature(const_panic)] -@@ -129,6 +131,7 @@ +@@ -130,6 +131,7 @@ mod synchronization; pub mod bsp; diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index c04cb3d2d..581dcefca 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -802,10 +802,10 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs 15_virtu +use crate::{cpu, memory, memory::Address}; +use core::intrinsics::unlikely; - use cortex_a::{asm, regs::*}; + use cortex_a::{asm, registers::*}; + use tock_registers::interfaces::Writeable; - // Assembly counterpart to this file. -@@ -69,9 +71,18 @@ +@@ -70,9 +72,18 @@ /// /// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`. #[no_mangle] @@ -857,10 +857,10 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translati }; -use core::convert; +use core::convert::{self, TryInto}; - use register::{register_bitfields, InMemoryRegister}; - - //-------------------------------------------------------------------------------------------------- -@@ -120,7 +120,7 @@ + use tock_registers::{ + interfaces::{Readable, Writeable}, + register_bitfields, +@@ -124,7 +124,7 @@ } trait StartAddr { @@ -869,7 +869,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translati } //-------------------------------------------------------------------------------------------------- -@@ -149,9 +149,8 @@ +@@ -153,9 +153,8 @@ // Private Code //-------------------------------------------------------------------------------------------------- @@ -880,7 +880,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translati Address::new(self as *const _ as usize) } } -@@ -269,7 +268,7 @@ +@@ -273,7 +272,7 @@ /// Create an instance. #[allow(clippy::assertions_on_constants)] @@ -889,7 +889,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translati assert!(bsp::memory::mmu::KernelGranule::SIZE == Granule64KiB::SIZE); // Can't have a zero-sized address space. -@@ -278,11 +277,20 @@ +@@ -282,11 +281,20 @@ Self { lvl3: [[PageDescriptor::new_zeroed(); 8192]; NUM_TABLES], lvl2: [TableDescriptor::new_zeroed(); NUM_TABLES], @@ -912,7 +912,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu/translati /// The start address of the table's MMIO range. #[inline(always)] fn mmio_start_addr(&self) -> Address { -@@ -338,24 +346,26 @@ +@@ -342,24 +350,26 @@ impl memory::mmu::translation_table::interface::TranslationTable for FixedSizeTranslationTable { @@ -958,8 +958,8 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs 15_vir + memory::{mmu::TranslationGranule, Address, Physical, Virtual}, }; use core::intrinsics::unlikely; - use cortex_a::{barrier, regs::*}; -@@ -108,7 +108,7 @@ + use cortex_a::{asm::barrier, registers::*}; +@@ -109,7 +109,7 @@ //------------------------------------------------------------------------------ // OS Interface Code //------------------------------------------------------------------------------ @@ -968,7 +968,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/memory/mmu.rs 15_vir impl memory::mmu::interface::MMU for MemoryManagementUnit { unsafe fn enable_mmu_and_caching( -@@ -152,4 +152,31 @@ +@@ -153,4 +153,31 @@ fn is_enabled(&self) -> bool { SCTLR_EL1.matches_all(SCTLR_EL1::M::Enable) } diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index 0a4381841..fa594703a 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -246,7 +246,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/Cargo.toml 16_virtual_mem_part diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs --- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/cpu/boot.rs -@@ -29,7 +29,10 @@ +@@ -30,7 +30,10 @@ /// - The `bss` section is not initialized yet. The code must not use or reference it in any way. /// - The HW state of EL1 must be prepared in a sound way. #[inline(always)] @@ -258,7 +258,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs // Enable timer counter registers for EL1. CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); -@@ -52,11 +55,11 @@ +@@ -53,11 +56,11 @@ ); // Second, let the link register point to kernel_init(). @@ -272,7 +272,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs } //-------------------------------------------------------------------------------------------------- -@@ -73,9 +76,13 @@ +@@ -74,9 +77,13 @@ #[no_mangle] pub unsafe extern "C" fn _start_rust( phys_kernel_tables_base_addr: u64, @@ -288,7 +288,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs // Turn on the MMU for EL1. let addr = Address::new(phys_kernel_tables_base_addr as usize); -@@ -83,6 +90,7 @@ +@@ -84,6 +91,7 @@ cpu::wait_forever(); } @@ -352,7 +352,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.s 1 diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs --- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/translation_table.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu/translation_table.rs -@@ -131,7 +131,7 @@ +@@ -135,7 +135,7 @@ /// aligned, so the lvl3 is put first. #[repr(C)] #[repr(align(65536))] @@ -361,7 +361,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t /// Page descriptors, covering 64 KiB windows per entry. lvl3: [[PageDescriptor; 8192]; NUM_TABLES], -@@ -258,14 +258,23 @@ +@@ -262,14 +262,23 @@ where [u8; Self::SIZE >> Granule512MiB::SHIFT]: Sized, { @@ -387,7 +387,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t /// Create an instance. #[allow(clippy::assertions_on_constants)] const fn _new(for_precompute: bool) -> Self { -@@ -294,20 +303,32 @@ +@@ -298,20 +307,32 @@ /// The start address of the table's MMIO range. #[inline(always)] fn mmio_start_addr(&self) -> Address { @@ -424,7 +424,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t } /// Helper to calculate the lvl2 and lvl3 indices from an address. -@@ -316,7 +337,12 @@ +@@ -320,7 +341,12 @@ &self, addr: *const Page, ) -> Result<(usize, usize), &'static str> { @@ -438,7 +438,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t let lvl2_index = addr >> Granule512MiB::SHIFT; let lvl3_index = (addr & Granule512MiB::MASK) >> Granule64KiB::SHIFT; -@@ -343,8 +369,9 @@ +@@ -347,8 +373,9 @@ // OS Interface Code //------------------------------------------------------------------------------ @@ -450,7 +450,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t { fn init(&mut self) -> Result<(), &'static str> { if self.initialized { -@@ -419,12 +446,16 @@ +@@ -423,12 +450,16 @@ return Err("Not enough MMIO space left"); } @@ -468,7 +468,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t Ok(PageSliceDescriptor::from_addr(addr, num_pages)) } -@@ -447,7 +478,7 @@ +@@ -451,7 +482,7 @@ //-------------------------------------------------------------------------------------------------- #[cfg(test)] @@ -481,7 +481,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu/t diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs --- 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.rs +++ 16_virtual_mem_part4_higher_half_kernel/src/_arch/aarch64/memory/mmu.rs -@@ -65,6 +65,7 @@ +@@ -66,6 +66,7 @@ impl MemoryManagementUnit { /// Setup function for the MAIR_EL1 register. @@ -489,7 +489,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.r fn set_up_mair(&self) { // Define the memory types being mapped. MAIR_EL1.write( -@@ -78,20 +79,21 @@ +@@ -79,20 +80,21 @@ } /// Configure various settings of stage 1 of the EL1 translation regime. @@ -521,7 +521,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/memory/mmu.r ); } } -@@ -130,7 +132,7 @@ +@@ -131,7 +133,7 @@ self.set_up_mair(); // Set the "Translation Table Base Register". diff --git a/X1_JTAG_boot/jtag_boot_rpi3.img b/X1_JTAG_boot/jtag_boot_rpi3.img index 9122433173b872ffac01c7ef68ea4d3f32a5243e..5e9581f57bcc227a275f045d352db907902e89c0 100755 GIT binary patch delta 2517 zcmb7Fe{2)?6@Oo!9iN?Fc5KIXOkz8M1v@G!17Z?Za&egmNm;ZIsuk8bZkmxFv|5r$ zt3*>f-9HJWQ?B0#XxIuAP0@J^Adz->n?|zyu`N_=BU`1de{fDF)sBHc(1tw-w|DlX zLe-{iC!OxT@B7~8z0c>~_ub{^&OJ9~*M;kN0&v+K=F4ly0{_x=a+Yo)oO+Py+ef>z zFD#uRXD#g-7?)keCE(pNPJWRA{#_bWRgk^^CL*61BVqbJ(`zA{cn=e9U7iEJ3c{_U zLq6Uere*3~q=6n%C)FOfKTnN@W;(91Z^2E^tc#CV{z?$FL=Z8@KNuKyf$L8*OyCnv z_wLQ`#*3?B@Y7XjxJ^IR)TxGi)SBZVhiOAjrR6c>jO+j~i;OuEW31jqra00;2Xl^D z_5h5xKi2wAe#&Eh(aMFTu6&1NdK0F>Fxq9 z{oNNip`KWhAH6*?QIiYPc7U;msHSZeRhuj1|A`Gctg@a^X}I*S0j9+YzcAGEy%d|z zrDZ9?dhoH|4_uA=Aj&daZF~trwYs^$)h?7eu??d7$+v`M0@DHPjq@n4-l#j#gkS+; zPtM?4?uF$_5cHFSf|tODCZy}R^X7uCyF29}T-u8S*7uM92b^{VN0W&SeKD8_I_Bk6 z49_Zy$zc;GxUljWHTVu;U)Yv_f9MB3mXg|#y}7G6tBNzD%J9TVY~leVBde17FKW0{ z;Op9ma>+^LgmR)V0iS%D-ZH(y0;SB#oxQ4;_r75{%*BaSk^+?yS8tkY4GDSc^Eeuq;DIm$Pm3~bgE{Y z^!LVR$R)aFeAXSTG_=^dnBE83-s?V=}&yq4Kg z{_Z74NK`{whYrnyQ)dk2biT;h7~9PMbnV*<`Dh)UVjWFu{R>m)E9KEXwtNJl#}R6E zQY*-FJAMhG@ebhc{VYciFA#pY1=5Ri_`JXhuh6X~Tb}p~-_gE>k6lpld+9;b0b-+q z$wmBh-t@c?Ygm>g)#hXs!BK9unn^V;g;YYKf(QwQ@QDgWI867MU)_NM;dHrorwW9g zpO3UQ<$ch91I0?zBLx)5GkE>X$`Tgy0C<_RXRUdmcZe>U>$#z=d=pYiI3V5hVSyuo zN#*jzC4#>mf@7$E;dZq~h*xPniRv8TuWQn&a{xO6F6Pe?9@_{FT5M(OR50KdPoz`w ze*`cMA>md?ZKtO!r6mW@4qBKVM~!4v&3~Khk7F{Ak>0Ru$-jo%ESzo~J_*rZQeE*2 z+p&nQ5{NdUf5Mmj5+3&sUe;0ysZj$)h*w}d?#C-N!r~n8qjaRWK7aHnsFA0kHp@`h zGIDxPgD(RkB7_Wu-7&`EZ!sRqLcK%}S}VT3TOq@48>0uy4hmhhLbfKLN@4|Vb(PP(HE}#&Ke*_u@uuib=hgH-DiBm zp0(oX&0JrlL-xv$&5^Y-iZxe>#h%MtU&K17_*W?V1Fr>-9_{RZrTfT{&VIE0pLj*! zPyh`IVAYk{k|RX-2Sc{5%T#bwmNOf&t765;uB>tDXRefkt~z#aWtHOpS2B4txZ$B4 z%Cg{v3Rt1CCNx;uvU|Od6$RU`~lC5Es2FqIh1<6h@^8f$< delta 2580 zcmb7FZA@F|6+W-8?R(9)zpyc1b{QiVR9KpV18K@Nq+0{k5QH^dR}DzQFd>Ns*`KM4 zO#5Ra4Xu3pqBJB?3yIX~y2;up=_Y4WFpbnoO%!RN%YJlIjoCCC2NII5a8cpzT>H|p zs6VzVS=aA*-}5}@Jm;MEPV9VtXRyE&X%Gp(`<{sSNFDjB_~Yy3JS``J{v~Fxnf9mu z=;%fAmi>SM#?+(HIS@VI7Gn&ELE57$Bdy=!k^!T50JBq zu^N-zIs~S_W4C$S|7N`_OljqK9fVKEp#E)yjx(-MDRCuND&BIPqXzEt%vVYF^#KDc zNyJ3&a?(!@lNGOtz~x3_O-3+C^{qq9R}Ln&cM-DnT>_o+C`dLY7$6zRcZpTXRU`j7NFb$GtmvO1Be`|~i)y&5-6rD9WjInZ7x|-<?fVJI zsvJhf3q+_NMSq>fF<{Es)B31u+dOLVX;;M+BM2fH(G!Z{P$T0_w)K3S{_JXClRQY6c+sksQheNzEi`0I%8v6ME>3 zmNiMf69H*C2IBM0*f80GthRr*%+{47ra%9`mFdlAzLDmrov)jB^F0mXT|~ZXenPx= zNhdDtC*ooaE6uYYSFY`zX*qV|OcuJ1PSPQB6`7`QnoD)yrc>rei9{F7-}8hkL=4H6 z8Tsh-)}#Nr-$gQ$3B=Asr%$lR1C#Vg%MP9S3HqX?((Wk{7vE!~L=DKON3iwS#WQ*P z=oL%p(;i;Du=cYVoWS77MXs=AZD{7=Y=!z|2+sFG_!We@{|+mu*M9sx?8p5e-ur`5 zik>Ipd>hEIYxq9TNiR^3)tTk47WYq7i#QSD_vusCJ;X-Gt#0x%y<%-PV+`}E@~kl} zN`&Uv95!E#s046;%7~O;h?pp2q`kD>cAypoT10^ssz5q;<#?8UAQv;zy5 z$510xRog$_8HytrPv}*9b@o-<=HOJv$VmwQk;?XG9>*a3E(kxZ&Y(A43Et<${Pih4 z)!gG{=#TsH3WF582I8;jFY_C-Pi%r3*$lNsg~FCm>G_^o5gRoHr$VU>%i)(b9!Nv& zp-(x=+IN1Pj3D2{9_79xNT>?-Er$bP3O$22-+J3r%KNx}xWl^p2eRKNgtypj+x$id z2QZr-YT@`#L%8))*wIDI{$ixt^zf7~@R~;o($)fPXBQnTs0eH+Oj{YvTB!MA&!wJ+ zv3j-hW!nC}{=UB6p`(X;4jt+pLfij|XMrjWFi!*Aafx^J5!2uD(s4!TcwxmO)}r(( zUpvHWYy6I+9H0dZdjoQ;OoTcq0ZT|rh C|3-KK diff --git a/X1_JTAG_boot/jtag_boot_rpi4.img b/X1_JTAG_boot/jtag_boot_rpi4.img index 25e8aedfdf849d61eb75103719199b31f4b9c2d0..33045fe94e046489ca6227aa30056cd73e29f3fc 100755 GIT binary patch delta 1901 zcmah}eQZ-z6hF7GU0>U+Tff$J6WXrAY(Wi7)^&(`Es$&wg1{g$)D9?f-3s?6N;{I}TE>Z4dS`N}CQY~+;&3QpdzG>3RgTOC<$odXa!n-20NV!xo zV5W}zqjX%kQR#unA81~|YTBu?KY<#afSHe%ej|u#B8cdsih(XCIDhJALN__h`!^zw zp1&h*{p}6}#_2DrN(CvQrW_C1LYL)~TBaFCbP0f2WX#bRWAzR*h0zAOH>b_A0-&pY zTJ1abVo-QP4ZKYalTNBp7e7AD6i4qNPKmfiylImHLyyb^hE6mGg_Gr6Uz;pLZ4sch znyyn92ktQ+c1My@?So!iXkWam_EqO|DUq?EcT04bF-3t?-E^XS3O$4wQ=(oWTM^V{ zF8E9BOEM1c$#UrH9tQs>l4OlzEDlnfNS2-|IzwmFw{#wo_v3aIOnOKz{eu;@tRYFC zpn%>gVoH>ttwYVD6(K?vX>mV>K1OlsF_SRSCeF1|z6SPpd@Ntj@? zQ8}!^gj9POL$4=Ld)~rTT*HvBZ7bRk9>9=Mwkb`L4N0;kL?kJO=i9S*)EOMj;3McU zN;v;2#4FGTQsdpB(U%=2dT%q)dP9K~Es=y|tPP?8mgTV22!_u_r|8Mt;&3&~rKT7~ zRHN*_(Tt@VJF5`JM!+vIkdkze5>1f0!a{G8o=ff7#0gczlDPQx!R~S$^x6SB?;>%w z0VJp=pZ^^UWLX&#GdHKv;6F1r66d}?i0KOE|@T1&%~QU znx>|thj1w`CNO({=s)b#XYHuc*rgAHiJ)O#NXGCxvgjP47+IS2rbT@`C^N{?X{GAH$H6lN%!jdSVP5L&n zoBpn^$n=k~e|racnQks{&Rw^dAD5&*MK=g6K0-`Kgv4@eQtg#I8?(~m1@p+$^y>nv zeso5%I-NlQ+gYcpo(>nd$xAfHu-w2Ku=gm1IEI1s+qtTp))^cu*m)7{=xDoP|Geu< z`9v8Jbo-(oOiTekQKk^K&%82}i|QSL`FtL(pAqUcK1m3zGrIlhZb`6P68_MA>G)}Y zB_SvCvxIe5H!U5PTN@w-oK7AKwKEu*%QLt(0KI5jKw9XqF}(1MgRj9yjF&Qk6vpt} zz*XdZ z;+w^A%6pkc_B;BKxw1zzKSpXIHk3Rt_7-N&7Dg@vIAFlqzWZuAO6n>o z<2)$4>2~gE=F&hv<7D>1(GtFg$n+q;#UA!60Y1KR@&YLHA-;02Pf~ot)MI#rBX-(s zm^4(t)J00n%V@&rUWliGZkv*>{8UW|L`@-mgogtXNZ$_gEu-fu^>=qR(HcM6~>N}mztkUrWW zSOYL41fykgHpROzmOVMnTa!SRjLOqO@CNd=@YYN#$1tab8lUvL5S-z?RflU~=*T#P z-qAE?n)f8V#FLq+Jm%@AM7&b8kCc3|$p}*_G0}{uzOb3h1Wg3aEFsp+Fve#qhn%}N zPSmv!Qhka*(fDoNMafNxai@S##u9Q%=K>fZ3Un(h@@$Yq6P2`%ll_kKtfpUWxcM8%w$6e>23`(I|U=)+Iuv7eLTsyIMx)4N0t?L`7k53wvo3(d zk#*R3gJnt<+cI4x52b@Jls+$q(#V}&i2E8Cn&=P5)CqRyCYb6&IomAao`6x#E+MX5 z58&M;C;XsJ^kX0bXRTPCcx}h!=*H-bNaSn)6P`Wa^*=nV$jf+)|A~zeHO;XKK#5{- zsE}1wNy-R8)&qS8XiE7bYIsjfogN3}`3CHl5;@B*wlyo%Rv>BA@o!}u{6EStHD=Y3 zd0@XqavBwha;{vgu{sRQ0^;8joGySV32&bT+=UK4Q&@G~9q>-R`Oa~UOn|;r+C|#w zkEM0_>GC!`S@!JT6Nap%PnjiBP20`Rf`NKP^ttFBa|H>}0kgC8r`wuSk|HXOw}h{c zj+*_XpWZY#l#i6-JTj2ixdlpB-;2PIM7x}zP!3tikI@!x6Kcg?r7Lu|4EVnC7tc~=N2=n6G(Ldlt$WZ-TB~{S7Dc(uH;n_F(OV+EryxKjQJ`Uz>h;3 z0WMl?Tgri-cG*1SOM1{&kK+zt@N3%xBq1*5E zi_)-TN%d>aJQ~Z$Jze@9h0Q(pM%yts;1G!E2Xh{lrX$Wu(}12*86~b2D`wC7bZfaz z#a0>L#H_b&Kc??zLl!RE(AmD>6>00*_O|u)65a3e5+8lvRb4K+^HJS8vXM@?D&4)h Yb*aAg>FZWn?e?1Dwmc?5*SJ^w1#CDo=Kufz From 15a1e717fb21616f7115eb3416ad7ece406d08c1 Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Tue, 6 Jul 2021 23:12:41 +0200 Subject: [PATCH 082/214] Switch to tock-registers: Bump rust toolchain --- rust-toolchain | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-toolchain b/rust-toolchain index 268f9495a..70934f67f 100644 --- a/rust-toolchain +++ b/rust-toolchain @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2021-04-25" +channel = "nightly-2021-07-01" components = ["llvm-tools-preview"] targets = ["aarch64-unknown-none-softfloat"] From de3ba3e871d57ae6b6c0776292a22b6bb1d110ee Mon Sep 17 00:00:00 2001 From: Andre Richter Date: Mon, 12 Jul 2021 22:08:22 +0200 Subject: [PATCH 083/214] Refactor Tests and Makefile - Carve out common files for tests - Add boot tests starting tutorial 3 - Overhaul the Makefile for more structure --- .githooks/pre-commit | 3 +- 01_wait_forever/Makefile | 81 ++- 02_runtime_init/Makefile | 81 ++- 02_runtime_init/README.md | 4 +- 03_hacky_hello_world/Makefile | 109 +++- 03_hacky_hello_world/README.md | 70 ++- .../tests/boot_test_string.rb | 3 + 04_safe_globals/Makefile | 109 +++- 04_safe_globals/tests/boot_test_string.rb | 3 + 05_drivers_gpio_uart/Makefile | 124 +++- 05_drivers_gpio_uart/README.md | 66 ++- .../tests/boot_test_string.rb | 3 + 06_uart_chainloader/Makefile | 181 ++++-- 06_uart_chainloader/README.md | 235 +++++--- 06_uart_chainloader/tests/chainboot_test.rb | 80 +++ 06_uart_chainloader/tests/qemu_minipush.rb | 80 --- 07_timestamps/Makefile | 125 +++- 07_timestamps/README.md | 233 ++++---- 07_timestamps/tests/boot_test_string.rb | 3 + 08_hw_debug_JTAG/Makefile | 180 ++++-- 08_hw_debug_JTAG/README.md | 60 +- 08_hw_debug_JTAG/tests/boot_test_string.rb | 3 + 09_privilege_level/Makefile | 180 ++++-- 09_privilege_level/README.md | 9 + 09_privilege_level/tests/boot_test_string.rb | 3 + .../Makefile | 180 ++++-- .../tests/boot_test_string.rb | 3 + 11_exceptions_part1_groundwork/Makefile | 180 ++++-- 11_exceptions_part1_groundwork/README.md | 9 + .../tests/boot_test_string.rb | 3 + 12_integrated_testing/Makefile | 253 +++++--- 12_integrated_testing/README.md | 555 ++++++++---------- 12_integrated_testing/src/lib.rs | 3 +- 12_integrated_testing/src/panic_wait.rs | 4 +- .../tests/00_console_sanity.rb | 13 +- .../tests/00_console_sanity.rs | 9 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/boot_test_string.rb | 3 + 12_integrated_testing/tests/runner.rb | 143 ----- 13_exceptions_part2_peripheral_IRQs/Makefile | 253 +++++--- 13_exceptions_part2_peripheral_IRQs/README.md | 13 + .../src/lib.rs | 3 +- .../src/panic_wait.rs | 4 +- .../tests/00_console_sanity.rb | 13 +- .../tests/00_console_sanity.rs | 9 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/boot_test_string.rb | 3 + .../tests/runner.rb | 143 ----- 14_virtual_mem_part2_mmio_remap/Makefile | 253 +++++--- 14_virtual_mem_part2_mmio_remap/README.md | 2 +- 14_virtual_mem_part2_mmio_remap/src/lib.rs | 3 +- .../src/panic_wait.rs | 4 +- .../tests/00_console_sanity.rb | 13 +- .../tests/00_console_sanity.rs | 9 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/boot_test_string.rb | 3 + .../tests/runner.rb | 143 ----- .../Makefile | 258 +++++--- .../README.md | 24 +- .../src/lib.rs | 3 +- .../src/panic_wait.rs | 4 +- .../tests/00_console_sanity.rb | 13 +- .../tests/00_console_sanity.rs | 9 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/boot_test_string.rb | 3 + .../tests/runner.rb | 143 ----- .../Makefile | 258 +++++--- .../README.md | 2 +- .../src/lib.rs | 3 +- .../src/panic_wait.rs | 4 +- .../tests/00_console_sanity.rb | 13 +- .../tests/00_console_sanity.rs | 9 +- .../tests/02_exception_sync_page_fault.rs | 2 +- .../tests/boot_test_string.rb | 3 + .../tests/runner.rb | 143 ----- X1_JTAG_boot/Makefile | 125 +++- X1_JTAG_boot/tests/boot_test_string.rb | 3 + {utils => common}/color.mk.in | 0 {utils => common/serial}/minipush.rb | 33 +- .../serial}/minipush/progressbar_patch.rb | 0 {utils => common/serial}/miniterm.rb | 3 +- common/tests/boot_test.rb | 75 +++ common/tests/console_io_test.rb | 48 ++ common/tests/dispatch.rb | 35 ++ common/tests/exit_code_test.rb | 52 ++ common/tests/test.rb | 82 +++ devtool_completion.bash | 2 +- doc/12_demo.gif | Bin 0 -> 224703 bytes doc/13_demo.gif | Bin 441164 -> 0 bytes doc/demo_PS1.txt | 1 + utils/devtool.rb | 126 ++-- utils/devtool/copyright.rb | 2 + utils/update_copyright.rb | 2 + 93 files changed, 3558 insertions(+), 2190 deletions(-) create mode 100644 03_hacky_hello_world/tests/boot_test_string.rb create mode 100644 04_safe_globals/tests/boot_test_string.rb create mode 100644 05_drivers_gpio_uart/tests/boot_test_string.rb create mode 100644 06_uart_chainloader/tests/chainboot_test.rb delete mode 100644 06_uart_chainloader/tests/qemu_minipush.rb create mode 100644 07_timestamps/tests/boot_test_string.rb create mode 100644 08_hw_debug_JTAG/tests/boot_test_string.rb create mode 100644 09_privilege_level/tests/boot_test_string.rb create mode 100644 10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb create mode 100644 11_exceptions_part1_groundwork/tests/boot_test_string.rb create mode 100644 12_integrated_testing/tests/boot_test_string.rb delete mode 100755 12_integrated_testing/tests/runner.rb create mode 100644 13_exceptions_part2_peripheral_IRQs/tests/boot_test_string.rb delete mode 100755 13_exceptions_part2_peripheral_IRQs/tests/runner.rb create mode 100644 14_virtual_mem_part2_mmio_remap/tests/boot_test_string.rb delete mode 100755 14_virtual_mem_part2_mmio_remap/tests/runner.rb create mode 100644 15_virtual_mem_part3_precomputed_tables/tests/boot_test_string.rb delete mode 100755 15_virtual_mem_part3_precomputed_tables/tests/runner.rb create mode 100644 16_virtual_mem_part4_higher_half_kernel/tests/boot_test_string.rb delete mode 100755 16_virtual_mem_part4_higher_half_kernel/tests/runner.rb create mode 100644 X1_JTAG_boot/tests/boot_test_string.rb rename {utils => common}/color.mk.in (100%) rename {utils => common/serial}/minipush.rb (83%) rename {utils => common/serial}/minipush/progressbar_patch.rb (100%) rename {utils => common/serial}/miniterm.rb (99%) create mode 100644 common/tests/boot_test.rb create mode 100644 common/tests/console_io_test.rb create mode 100755 common/tests/dispatch.rb create mode 100644 common/tests/exit_code_test.rb create mode 100644 common/tests/test.rb create mode 100644 doc/12_demo.gif delete mode 100644 doc/13_demo.gif create mode 100644 doc/demo_PS1.txt diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 6636f0075..468906434 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -5,8 +5,6 @@ # # Copyright (c) 2018-2021 Andre Richter -require 'rubygems' -require 'bundler/setup' require_relative '../utils/devtool/copyright' def copyright_check(staged_files) @@ -14,6 +12,7 @@ def copyright_check(staged_files) staged_files = staged_files.select do |f| next if f.include?('build.rs') + next if f.include?('boot_test_string.rb') f.include?('Makefile') || f.include?('Dockerfile') || diff --git a/01_wait_forever/Makefile b/01_wait_forever/Makefile index 2e44e7ac5..16607fe62 100644 --- a/01_wait_forever/Makefile +++ b/01_wait_forever/Makefile @@ -2,12 +2,22 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 -# BSP-specific arguments + + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -32,11 +42,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -53,61 +70,99 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json diff --git a/02_runtime_init/Makefile b/02_runtime_init/Makefile index a20b283b4..ea979232c 100644 --- a/02_runtime_init/Makefile +++ b/02_runtime_init/Makefile @@ -2,12 +2,22 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 -# BSP-specific arguments + + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -32,11 +42,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -53,51 +70,84 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -106,10 +156,15 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json diff --git a/02_runtime_init/README.md b/02_runtime_init/README.md index 11ad00cc9..d37b3c9f9 100644 --- a/02_runtime_init/README.md +++ b/02_runtime_init/README.md @@ -53,7 +53,7 @@ diff -uNr 01_wait_forever/Cargo.toml 02_runtime_init/Cargo.toml diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile --- 01_wait_forever/Makefile +++ 02_runtime_init/Makefile -@@ -102,6 +102,8 @@ +@@ -152,6 +152,8 @@ $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ --section .text \ @@ -61,7 +61,7 @@ diff -uNr 01_wait_forever/Makefile 02_runtime_init/Makefile + --section .got \ $(KERNEL_ELF) | rustfilt - nm: $(KERNEL_ELF) + ##------------------------------------------------------------------------------ diff -uNr 01_wait_forever/src/_arch/aarch64/cpu/boot.rs 02_runtime_init/src/_arch/aarch64/cpu/boot.rs --- 01_wait_forever/src/_arch/aarch64/cpu/boot.rs diff --git a/03_hacky_hello_world/Makefile b/03_hacky_hello_world/Makefile index 2ed82a5f1..19e50b332 100644 --- a/03_hacky_hello_world/Makefile +++ b/03_hacky_hello_world/Makefile @@ -2,12 +2,22 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 -# BSP-specific arguments + + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -32,11 +42,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -53,51 +70,87 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -106,10 +159,40 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/03_hacky_hello_world/README.md b/03_hacky_hello_world/README.md index 01bbf54cb..ce262bc98 100644 --- a/03_hacky_hello_world/README.md +++ b/03_hacky_hello_world/README.md @@ -12,6 +12,10 @@ - `src/console.rs` introduces interface `Traits` for console commands. - `src/bsp/raspberrypi/console.rs` implements the interface for QEMU's emulated UART. - The panic handler makes use of the new `print!()` to display user error messages. +- There is a new Makefile target, `make test`, intended for automated testing. It boots the compiled + kernel in `QEMU`, and checks for an expected output string produced by the kernel. + - In this tutorial, it checks for the string `Stopping here`, which is emitted by the `panic!()` + at the end of `main.rs`. ## Test it @@ -43,7 +47,7 @@ diff -uNr 02_runtime_init/Cargo.toml 03_hacky_hello_world/Cargo.toml diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile --- 02_runtime_init/Makefile +++ 03_hacky_hello_world/Makefile -@@ -13,7 +13,7 @@ +@@ -23,7 +23,7 @@ KERNEL_BIN = kernel8.img QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = raspi3 @@ -52,7 +56,7 @@ diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf -@@ -24,7 +24,7 @@ +@@ -34,7 +34,7 @@ KERNEL_BIN = kernel8.img QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = @@ -61,6 +65,60 @@ diff -uNr 02_runtime_init/Makefile 03_hacky_hello_world/Makefile OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf +@@ -70,17 +70,20 @@ + --strip-all \ + -O binary + +-EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) ++EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) ++EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb + + ##------------------------------------------------------------------------------ + ## Dockerization + ##------------------------------------------------------------------------------ +-DOCKER_IMAGE = rustembedded/osdev-utils +-DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +-DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i ++DOCKER_IMAGE = rustembedded/osdev-utils ++DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial ++DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i ++DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common + + DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) + DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) ++DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + + + +@@ -168,3 +171,28 @@ + ##------------------------------------------------------------------------------ + check: + @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json ++ ++ ++ ++##-------------------------------------------------------------------------------------------------- ++## Testing targets ++##-------------------------------------------------------------------------------------------------- ++.PHONY: test test_boot ++ ++ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. ++ ++test_boot test : ++ $(call colorecho, "\n$(QEMU_MISSING_STRING)") ++ ++else # QEMU is supported. ++ ++##------------------------------------------------------------------------------ ++## Run boot test ++##------------------------------------------------------------------------------ ++test_boot: $(KERNEL_BIN) ++ $(call colorecho, "\nBoot test - $(BSP)") ++ @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) ++ ++test: test_boot ++ ++endif diff -uNr 02_runtime_init/src/bsp/raspberrypi/console.rs 03_hacky_hello_world/src/bsp/raspberrypi/console.rs --- 02_runtime_init/src/bsp/raspberrypi/console.rs @@ -245,4 +303,12 @@ diff -uNr 02_runtime_init/src/print.rs 03_hacky_hello_world/src/print.rs + }) +} +diff -uNr 02_runtime_init/tests/boot_test_string.rb 03_hacky_hello_world/tests/boot_test_string.rb +--- 02_runtime_init/tests/boot_test_string.rb ++++ 03_hacky_hello_world/tests/boot_test_string.rb +@@ -0,0 +1,3 @@ ++# frozen_string_literal: true ++ ++EXPECTED_PRINT = 'Stopping here' + ``` diff --git a/03_hacky_hello_world/tests/boot_test_string.rb b/03_hacky_hello_world/tests/boot_test_string.rb new file mode 100644 index 000000000..b0c595368 --- /dev/null +++ b/03_hacky_hello_world/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Stopping here' diff --git a/04_safe_globals/Makefile b/04_safe_globals/Makefile index 2ed82a5f1..19e50b332 100644 --- a/04_safe_globals/Makefile +++ b/04_safe_globals/Makefile @@ -2,12 +2,22 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 -# BSP-specific arguments + + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -32,11 +42,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -53,51 +70,87 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -106,10 +159,40 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/04_safe_globals/tests/boot_test_string.rb b/04_safe_globals/tests/boot_test_string.rb new file mode 100644 index 000000000..b0c595368 --- /dev/null +++ b/04_safe_globals/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Stopping here' diff --git a/05_drivers_gpio_uart/Makefile b/05_drivers_gpio_uart/Makefile index 1dd6cc2b9..c38e049eb 100644 --- a/05_drivers_gpio_uart/Makefile +++ b/05_drivers_gpio_uart/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -38,11 +45,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -59,64 +73,102 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINITERM = ruby ../common/serial/miniterm.rb -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DEV = --privileged -v /dev:/dev +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINITERM = ruby ../utils/miniterm.rb + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu miniterm clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) endif +##------------------------------------------------------------------------------ +## Connect to the target's serial +##------------------------------------------------------------------------------ miniterm: @$(DOCKER_MINITERM) $(EXEC_MINITERM) $(DEV_SERIAL) +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -125,10 +177,40 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index 7f0bbda5b..2ee89aa6d 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -145,55 +145,64 @@ diff -uNr 04_safe_globals/Cargo.toml 05_drivers_gpio_uart/Cargo.toml diff -uNr 04_safe_globals/Makefile 05_drivers_gpio_uart/Makefile --- 04_safe_globals/Makefile +++ 05_drivers_gpio_uart/Makefile -@@ -7,6 +7,12 @@ - # Default to the RPi3 +@@ -11,6 +11,9 @@ + # Default to the RPi3. BSP ?= rpi3 +# Default to a serial device name that is common in Linux. +DEV_SERIAL ?= /dev/ttyUSB0 + -+# Query the host system's kernel name -+UNAME_S = $(shell uname -s) -+ - # BSP-specific arguments - ifeq ($(BSP),rpi3) - TARGET = aarch64-unknown-none-softfloat -@@ -58,13 +64,23 @@ - DOCKER_IMAGE = rustembedded/osdev-utils - DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial - DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -+DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -+DOCKER_ARG_DEV = --privileged -v /dev:/dev + + + ##-------------------------------------------------------------------------------------------------- +@@ -72,6 +75,7 @@ + + EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) + EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb ++EXEC_MINITERM = ruby ../common/serial/miniterm.rb + + ##------------------------------------------------------------------------------ + ## Dockerization +@@ -80,17 +84,25 @@ + DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial + DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i + DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common ++DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) --EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -+# Dockerize commands that require USB device passthrough only on Linux -+ifeq ($(UNAME_S),Linux) ++# Dockerize commands, which require USB device passthrough, only on Linux. ++ifeq ($(shell uname -s),Linux) + DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) + -+ DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) ++ DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +endif + -+EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -+EXEC_MINITERM = ruby ../utils/miniterm.rb + + ##-------------------------------------------------------------------------------------------------- + ## Targets + ##-------------------------------------------------------------------------------------------------- -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu clippy clean readelf objdump nm check +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu miniterm clippy clean readelf objdump nm check all: $(KERNEL_BIN) -@@ -88,6 +104,9 @@ - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) +@@ -130,6 +142,12 @@ endif + ##------------------------------------------------------------------------------ ++## Connect to the target's serial ++##------------------------------------------------------------------------------ +miniterm: + @$(DOCKER_MINITERM) $(EXEC_MINITERM) $(DEV_SERIAL) + ++##------------------------------------------------------------------------------ + ## Run clippy + ##------------------------------------------------------------------------------ clippy: - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) - diff -uNr 04_safe_globals/src/_arch/aarch64/cpu.rs 05_drivers_gpio_uart/src/_arch/aarch64/cpu.rs --- 04_safe_globals/src/_arch/aarch64/cpu.rs @@ -1451,4 +1460,13 @@ diff -uNr 04_safe_globals/src/panic_wait.rs 05_drivers_gpio_uart/src/panic_wait. cpu::wait_forever() +diff -uNr 04_safe_globals/tests/boot_test_string.rb 05_drivers_gpio_uart/tests/boot_test_string.rb +--- 04_safe_globals/tests/boot_test_string.rb ++++ 05_drivers_gpio_uart/tests/boot_test_string.rb +@@ -1,3 +1,3 @@ + # frozen_string_literal: true + +-EXPECTED_PRINT = 'Stopping here' ++EXPECTED_PRINT = 'Echoing input now' + ``` diff --git a/05_drivers_gpio_uart/tests/boot_test_string.rb b/05_drivers_gpio_uart/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/05_drivers_gpio_uart/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/06_uart_chainloader/Makefile b/06_uart_chainloader/Makefile index a8d38ffca..3064e67a0 100644 --- a/06_uart_chainloader/Makefile +++ b/06_uart_chainloader/Makefile @@ -2,49 +2,63 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img - QEMU_BINARY = qemu-system-aarch64 - QEMU_MACHINE_TYPE = raspi3 - QEMU_RELEASE_ARGS = -serial stdio -display none - OBJDUMP_BINARY = aarch64-none-elf-objdump - NM_BINARY = aarch64-none-elf-nm - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = raspi3 + QEMU_RELEASE_ARGS = -serial stdio -display none + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img else ifeq ($(BSP),rpi4) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img - QEMU_BINARY = qemu-system-aarch64 - QEMU_MACHINE_TYPE = - QEMU_RELEASE_ARGS = -serial stdio -display none - OBJDUMP_BINARY = aarch64-none-elf-objdump - NM_BINARY = aarch64-none-elf-nm - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 + TARGET = aarch64-unknown-none-softfloat + KERNEL_BIN = kernel8.img + QEMU_BINARY = qemu-system-aarch64 + QEMU_MACHINE_TYPE = + QEMU_RELEASE_ARGS = -serial stdio -display none + OBJDUMP_BINARY = aarch64-none-elf-objdump + NM_BINARY = aarch64-none-elf-nm + READELF_BINARY = aarch64-none-elf-readelf + LINKER_FILE = src/bsp/raspberrypi/link.ld + RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -61,49 +75,69 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DEV = --privileged -v /dev:/dev +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_MINIPUSH = ruby tests/chainboot_test.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ - check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu qemuasm: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) @@ -112,26 +146,36 @@ qemuasm: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU with ASM output") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm -test: $(KERNEL_BIN) - $(call colorecho, "\nTesting chainloading - $(BSP)") - @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ - -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) - endif -chainboot: +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ +chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(CHAINBOOT_DEMO_PAYLOAD) +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -140,10 +184,41 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ + -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) + +test: test_boot + +endif diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 0f9e4aa61..d606f5369 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -137,57 +137,95 @@ Binary files 05_drivers_gpio_uart/demo_payload_rpi4.img and 06_uart_chainloader/ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile --- 05_drivers_gpio_uart/Makefile +++ 06_uart_chainloader/Makefile -@@ -25,6 +25,7 @@ - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 +@@ -22,27 +22,29 @@ + + # BSP-specific arguments. + ifeq ($(BSP),rpi3) +- TARGET = aarch64-unknown-none-softfloat +- KERNEL_BIN = kernel8.img +- QEMU_BINARY = qemu-system-aarch64 +- QEMU_MACHINE_TYPE = raspi3 +- QEMU_RELEASE_ARGS = -serial stdio -display none +- OBJDUMP_BINARY = aarch64-none-elf-objdump +- NM_BINARY = aarch64-none-elf-nm +- READELF_BINARY = aarch64-none-elf-readelf +- LINKER_FILE = src/bsp/raspberrypi/link.ld +- RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 ++ TARGET = aarch64-unknown-none-softfloat ++ KERNEL_BIN = kernel8.img ++ QEMU_BINARY = qemu-system-aarch64 ++ QEMU_MACHINE_TYPE = raspi3 ++ QEMU_RELEASE_ARGS = -serial stdio -display none ++ OBJDUMP_BINARY = aarch64-none-elf-objdump ++ NM_BINARY = aarch64-none-elf-nm ++ READELF_BINARY = aarch64-none-elf-readelf ++ LINKER_FILE = src/bsp/raspberrypi/link.ld ++ RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 + CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img else ifeq ($(BSP),rpi4) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img -@@ -36,6 +37,7 @@ - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 +- TARGET = aarch64-unknown-none-softfloat +- KERNEL_BIN = kernel8.img +- QEMU_BINARY = qemu-system-aarch64 +- QEMU_MACHINE_TYPE = +- QEMU_RELEASE_ARGS = -serial stdio -display none +- OBJDUMP_BINARY = aarch64-none-elf-objdump +- NM_BINARY = aarch64-none-elf-nm +- READELF_BINARY = aarch64-none-elf-readelf +- LINKER_FILE = src/bsp/raspberrypi/link.ld +- RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 ++ TARGET = aarch64-unknown-none-softfloat ++ KERNEL_BIN = kernel8.img ++ QEMU_BINARY = qemu-system-aarch64 ++ QEMU_MACHINE_TYPE = ++ QEMU_RELEASE_ARGS = -serial stdio -display none ++ OBJDUMP_BINARY = aarch64-none-elf-objdump ++ NM_BINARY = aarch64-none-elf-nm ++ READELF_BINARY = aarch64-none-elf-readelf ++ LINKER_FILE = src/bsp/raspberrypi/link.ld ++ RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 + CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img endif - # Export for build.rs -@@ -68,19 +70,22 @@ - DOCKER_ARG_DEV = --privileged -v /dev:/dev + QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +@@ -74,8 +76,8 @@ + -O binary - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -+DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +-EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +-EXEC_MINITERM = ruby ../common/serial/miniterm.rb ++EXEC_TEST_MINIPUSH = ruby tests/chainboot_test.rb ++EXEC_MINIPUSH = ruby ../common/serial/minipush.rb - # Dockerize commands that require USB device passthrough only on Linux - ifeq ($(UNAME_S),Linux) + ##------------------------------------------------------------------------------ + ## Dockerization +@@ -94,7 +96,7 @@ + ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) -- DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) -+ DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) +- DOCKER_MINITERM = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) ++ DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) endif --EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) --EXEC_MINITERM = ruby ../utils/miniterm.rb -+EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -+EXEC_MINIPUSH = ruby ../utils/minipush.rb -+EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb +@@ -102,7 +104,7 @@ + ##-------------------------------------------------------------------------------------------------- + ## Targets + ##-------------------------------------------------------------------------------------------------- -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu miniterm clippy clean readelf objdump nm check -+.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ -+ check ++.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) -@@ -96,16 +101,26 @@ - @$(DOC_CMD) --document-private-items --open +@@ -131,7 +133,7 @@ + ##------------------------------------------------------------------------------ + ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. - ifeq ($(QEMU_MACHINE_TYPE),) -qemu: -+qemu test: ++qemu qemuasm: $(call colorecho, "\n$(QEMU_MISSING_STRING)") - else + + else # QEMU is supported. +@@ -139,13 +141,18 @@ qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) @@ -195,21 +233,30 @@ diff -uNr 05_drivers_gpio_uart/Makefile 06_uart_chainloader/Makefile +qemuasm: $(KERNEL_BIN) + $(call colorecho, "\nLaunching QEMU with ASM output") + @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm -+ -+test: $(KERNEL_BIN) -+ $(call colorecho, "\nTesting chainloading - $(BSP)") -+ @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ -+ -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) + endif + ##------------------------------------------------------------------------------ +-## Connect to the target's serial ++## Push the kernel to the real HW target + ##------------------------------------------------------------------------------ -miniterm: - @$(DOCKER_MINITERM) $(EXEC_MINITERM) $(DEV_SERIAL) -+chainboot: ++chainboot: $(KERNEL_BIN) + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(CHAINBOOT_DEMO_PAYLOAD) - clippy: - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + ##------------------------------------------------------------------------------ + ## Run clippy +@@ -209,7 +216,8 @@ + ##------------------------------------------------------------------------------ + test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") +- @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) ++ @$(DOCKER_TEST) $(EXEC_TEST_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ ++ -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) + + test: test_boot + diff -uNr 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s --- 05_drivers_gpio_uart/src/_arch/aarch64/cpu/boot.s @@ -492,9 +539,17 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs + kernel() } -diff -uNr 05_drivers_gpio_uart/tests/qemu_minipush.rb 06_uart_chainloader/tests/qemu_minipush.rb ---- 05_drivers_gpio_uart/tests/qemu_minipush.rb -+++ 06_uart_chainloader/tests/qemu_minipush.rb +diff -uNr 05_drivers_gpio_uart/tests/boot_test_string.rb 06_uart_chainloader/tests/boot_test_string.rb +--- 05_drivers_gpio_uart/tests/boot_test_string.rb ++++ 06_uart_chainloader/tests/boot_test_string.rb +@@ -1,3 +0,0 @@ +-# frozen_string_literal: true +- +-EXPECTED_PRINT = 'Echoing input now' + +diff -uNr 05_drivers_gpio_uart/tests/chainboot_test.rb 06_uart_chainloader/tests/chainboot_test.rb +--- 05_drivers_gpio_uart/tests/chainboot_test.rb ++++ 06_uart_chainloader/tests/chainboot_test.rb @@ -0,0 +1,80 @@ +# frozen_string_literal: true + @@ -502,80 +557,80 @@ diff -uNr 05_drivers_gpio_uart/tests/qemu_minipush.rb 06_uart_chainloader/tests/ +# +# Copyright (c) 2020-2021 Andre Richter + -+require_relative '../../utils/minipush' -+require 'expect' -+require 'timeout' ++require_relative '../../common/serial/minipush' ++require_relative '../../common/tests/boot_test' ++require 'pty' + +# Match for the last print that 'demo_payload_rpiX.img' produces. +EXPECTED_PRINT = 'Echoing input now' + -+# The main class -+class QEMUMiniPush < MiniPush -+ TIMEOUT_SECS = 3 ++# Extend BootTest so that it listens on the output of a MiniPush instance, which is itself connected ++# to a QEMU instance instead of a real HW. ++class ChainbootTest < BootTest ++ MINIPUSH = '../common/serial/minipush.rb' ++ MINIPUSH_POWER_TARGET_REQUEST = 'Please power the target now' + -+ # override -+ def initialize(qemu_cmd, binary_image_path) -+ super(nil, binary_image_path) ++ def initialize(qemu_cmd, payload_path) ++ super(qemu_cmd, EXPECTED_PRINT) ++ ++ @test_name = 'Boot test using Minipush' + -+ @qemu_cmd = qemu_cmd ++ @payload_path = payload_path + end + + private + -+ def quit_qemu_graceful -+ Timeout.timeout(5) do -+ pid = @target_serial.pid -+ Process.kill('TERM', pid) -+ Process.wait(pid) -+ end -+ end -+ + # override -+ def open_serial -+ @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') ++ def post_process_and_add_output(output) ++ temp = output.join.split("\r\n") + -+ # Ensure all output is immediately flushed to the device. -+ @target_serial.sync = true ++ # Should a line have solo carriage returns, remove any overridden parts of the string. ++ temp.map! { |x| x.gsub(/.*\r/, '') } + -+ puts "[#{@name_short}] ✅ Serial connected" ++ @test_output += temp + end + -+ # override -+ def terminal -+ result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) -+ exit(1) if result.nil? -+ -+ puts result -+ -+ quit_qemu_graceful ++ def wait_for_minipush_power_request(mp_out) ++ output = [] ++ Timeout.timeout(MAX_WAIT_SECS) do ++ loop do ++ output << mp_out.gets ++ break if output.last.include?(MINIPUSH_POWER_TARGET_REQUEST) ++ end ++ end ++ rescue Timeout::Error ++ @test_error = 'Timed out waiting for power request' ++ rescue StandardError => e ++ @test_error = e.message ++ ensure ++ post_process_and_add_output(output) + end + + # override -+ def connetion_reset; end ++ def setup ++ pty_main, pty_secondary = PTY.open ++ mp_out, _mp_in = PTY.spawn("ruby #{MINIPUSH} #{pty_secondary.path} #{@payload_path}") + -+ # override -+ def handle_reconnect(error) -+ handle_unexpected(error) ++ # Wait until MiniPush asks for powering the target. ++ wait_for_minipush_power_request(mp_out) ++ ++ # Now is the time to start QEMU with the chainloader binary. QEMU's virtual tty is connected ++ # to the MiniPush instance spawned above, so that the two processes talk to each other. ++ Process.spawn(@qemu_cmd, in: pty_main, out: pty_main) ++ ++ # The remainder of the test is done by the parent class' run_concrete_test, which listens on ++ # @qemu_serial. Hence, point it to MiniPush's output. ++ @qemu_serial = mp_out + end +end + +##-------------------------------------------------------------------------------------------------- +## Execution starts here +##-------------------------------------------------------------------------------------------------- -+puts -+puts 'QEMUMiniPush 1.0'.cyan -+puts -+ -+# CTRL + C handler. Only here to suppress Ruby's default exception print. -+trap('INT') do -+ # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. -+ exit -+end -+ -+binary_image_path = ARGV.pop ++payload_path = ARGV.pop +qemu_cmd = ARGV.join(' ') + -+QEMUMiniPush.new(qemu_cmd, binary_image_path).run ++ChainbootTest.new(qemu_cmd, payload_path).run diff -uNr 05_drivers_gpio_uart/update.sh 06_uart_chainloader/update.sh --- 05_drivers_gpio_uart/update.sh diff --git a/06_uart_chainloader/tests/chainboot_test.rb b/06_uart_chainloader/tests/chainboot_test.rb new file mode 100644 index 000000000..3dee0f9c4 --- /dev/null +++ b/06_uart_chainloader/tests/chainboot_test.rb @@ -0,0 +1,80 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2020-2021 Andre Richter + +require_relative '../../common/serial/minipush' +require_relative '../../common/tests/boot_test' +require 'pty' + +# Match for the last print that 'demo_payload_rpiX.img' produces. +EXPECTED_PRINT = 'Echoing input now' + +# Extend BootTest so that it listens on the output of a MiniPush instance, which is itself connected +# to a QEMU instance instead of a real HW. +class ChainbootTest < BootTest + MINIPUSH = '../common/serial/minipush.rb' + MINIPUSH_POWER_TARGET_REQUEST = 'Please power the target now' + + def initialize(qemu_cmd, payload_path) + super(qemu_cmd, EXPECTED_PRINT) + + @test_name = 'Boot test using Minipush' + + @payload_path = payload_path + end + + private + + # override + def post_process_and_add_output(output) + temp = output.join.split("\r\n") + + # Should a line have solo carriage returns, remove any overridden parts of the string. + temp.map! { |x| x.gsub(/.*\r/, '') } + + @test_output += temp + end + + def wait_for_minipush_power_request(mp_out) + output = [] + Timeout.timeout(MAX_WAIT_SECS) do + loop do + output << mp_out.gets + break if output.last.include?(MINIPUSH_POWER_TARGET_REQUEST) + end + end + rescue Timeout::Error + @test_error = 'Timed out waiting for power request' + rescue StandardError => e + @test_error = e.message + ensure + post_process_and_add_output(output) + end + + # override + def setup + pty_main, pty_secondary = PTY.open + mp_out, _mp_in = PTY.spawn("ruby #{MINIPUSH} #{pty_secondary.path} #{@payload_path}") + + # Wait until MiniPush asks for powering the target. + wait_for_minipush_power_request(mp_out) + + # Now is the time to start QEMU with the chainloader binary. QEMU's virtual tty is connected + # to the MiniPush instance spawned above, so that the two processes talk to each other. + Process.spawn(@qemu_cmd, in: pty_main, out: pty_main) + + # The remainder of the test is done by the parent class' run_concrete_test, which listens on + # @qemu_serial. Hence, point it to MiniPush's output. + @qemu_serial = mp_out + end +end + +##-------------------------------------------------------------------------------------------------- +## Execution starts here +##-------------------------------------------------------------------------------------------------- +payload_path = ARGV.pop +qemu_cmd = ARGV.join(' ') + +ChainbootTest.new(qemu_cmd, payload_path).run diff --git a/06_uart_chainloader/tests/qemu_minipush.rb b/06_uart_chainloader/tests/qemu_minipush.rb deleted file mode 100644 index 73857cd6b..000000000 --- a/06_uart_chainloader/tests/qemu_minipush.rb +++ /dev/null @@ -1,80 +0,0 @@ -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2020-2021 Andre Richter - -require_relative '../../utils/minipush' -require 'expect' -require 'timeout' - -# Match for the last print that 'demo_payload_rpiX.img' produces. -EXPECTED_PRINT = 'Echoing input now' - -# The main class -class QEMUMiniPush < MiniPush - TIMEOUT_SECS = 3 - - # override - def initialize(qemu_cmd, binary_image_path) - super(nil, binary_image_path) - - @qemu_cmd = qemu_cmd - end - - private - - def quit_qemu_graceful - Timeout.timeout(5) do - pid = @target_serial.pid - Process.kill('TERM', pid) - Process.wait(pid) - end - end - - # override - def open_serial - @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') - - # Ensure all output is immediately flushed to the device. - @target_serial.sync = true - - puts "[#{@name_short}] ✅ Serial connected" - end - - # override - def terminal - result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) - exit(1) if result.nil? - - puts result - - quit_qemu_graceful - end - - # override - def connetion_reset; end - - # override - def handle_reconnect(error) - handle_unexpected(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Execution starts here -##-------------------------------------------------------------------------------------------------- -puts -puts 'QEMUMiniPush 1.0'.cyan -puts - -# CTRL + C handler. Only here to suppress Ruby's default exception print. -trap('INT') do - # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. - exit -end - -binary_image_path = ARGV.pop -qemu_cmd = ARGV.join(' ') - -QEMUMiniPush.new(qemu_cmd, binary_image_path).run diff --git a/07_timestamps/Makefile b/07_timestamps/Makefile index b5a56d075..8336ccb72 100644 --- a/07_timestamps/Makefile +++ b/07_timestamps/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -38,11 +45,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -59,64 +73,103 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DEV = --privileged -v /dev:/dev +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -125,10 +178,40 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/07_timestamps/README.md b/07_timestamps/README.md index 4e146bd0a..4fb36ddf2 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -62,76 +62,103 @@ Binary files 06_uart_chainloader/demo_payload_rpi4.img and 07_timestamps/demo_pa diff -uNr 06_uart_chainloader/Makefile 07_timestamps/Makefile --- 06_uart_chainloader/Makefile +++ 07_timestamps/Makefile -@@ -25,7 +25,6 @@ - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 +@@ -22,29 +22,27 @@ + + # BSP-specific arguments. + ifeq ($(BSP),rpi3) +- TARGET = aarch64-unknown-none-softfloat +- KERNEL_BIN = kernel8.img +- QEMU_BINARY = qemu-system-aarch64 +- QEMU_MACHINE_TYPE = raspi3 +- QEMU_RELEASE_ARGS = -serial stdio -display none +- OBJDUMP_BINARY = aarch64-none-elf-objdump +- NM_BINARY = aarch64-none-elf-nm +- READELF_BINARY = aarch64-none-elf-readelf +- LINKER_FILE = src/bsp/raspberrypi/link.ld +- RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi3.img ++ TARGET = aarch64-unknown-none-softfloat ++ KERNEL_BIN = kernel8.img ++ QEMU_BINARY = qemu-system-aarch64 ++ QEMU_MACHINE_TYPE = raspi3 ++ QEMU_RELEASE_ARGS = -serial stdio -display none ++ OBJDUMP_BINARY = aarch64-none-elf-objdump ++ NM_BINARY = aarch64-none-elf-nm ++ READELF_BINARY = aarch64-none-elf-readelf ++ LINKER_FILE = src/bsp/raspberrypi/link.ld ++ RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) - TARGET = aarch64-unknown-none-softfloat - KERNEL_BIN = kernel8.img -@@ -37,7 +36,6 @@ - READELF_BINARY = aarch64-none-elf-readelf - LINKER_FILE = src/bsp/raspberrypi/link.ld - RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 +- TARGET = aarch64-unknown-none-softfloat +- KERNEL_BIN = kernel8.img +- QEMU_BINARY = qemu-system-aarch64 +- QEMU_MACHINE_TYPE = +- QEMU_RELEASE_ARGS = -serial stdio -display none +- OBJDUMP_BINARY = aarch64-none-elf-objdump +- NM_BINARY = aarch64-none-elf-nm +- READELF_BINARY = aarch64-none-elf-readelf +- LINKER_FILE = src/bsp/raspberrypi/link.ld +- RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 - CHAINBOOT_DEMO_PAYLOAD = demo_payload_rpi4.img ++ TARGET = aarch64-unknown-none-softfloat ++ KERNEL_BIN = kernel8.img ++ QEMU_BINARY = qemu-system-aarch64 ++ QEMU_MACHINE_TYPE = ++ QEMU_RELEASE_ARGS = -serial stdio -display none ++ OBJDUMP_BINARY = aarch64-none-elf-objdump ++ NM_BINARY = aarch64-none-elf-nm ++ READELF_BINARY = aarch64-none-elf-readelf ++ LINKER_FILE = src/bsp/raspberrypi/link.ld ++ RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif - # Export for build.rs -@@ -70,7 +68,6 @@ - DOCKER_ARG_DEV = --privileged -v /dev:/dev + QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +@@ -76,7 +74,7 @@ + -O binary - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) --DOCKER_TEST = $(DOCKER_CMD) -t $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +-EXEC_TEST_MINIPUSH = ruby tests/chainboot_test.rb ++EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb + EXEC_MINIPUSH = ruby ../common/serial/minipush.rb - # Dockerize commands that require USB device passthrough only on Linux -@@ -80,12 +77,10 @@ - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - endif - --EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) --EXEC_MINIPUSH = ruby ../utils/minipush.rb --EXEC_QEMU_MINIPUSH = ruby tests/qemu_minipush.rb -+EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -+EXEC_MINIPUSH = ruby ../utils/minipush.rb - --.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu qemuasm chainboot clippy clean readelf objdump nm \ -- check -+.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check - - all: $(KERNEL_BIN) + ##------------------------------------------------------------------------------ +@@ -133,7 +131,7 @@ + ##------------------------------------------------------------------------------ + ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. -@@ -101,26 +96,16 @@ - @$(DOC_CMD) --document-private-items --open - - ifeq ($(QEMU_MACHINE_TYPE),) --qemu test: +-qemu qemuasm: +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") - else - qemu: $(KERNEL_BIN) + + else # QEMU is supported. +@@ -142,17 +140,13 @@ $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -- + -qemuasm: $(KERNEL_BIN) - $(call colorecho, "\nLaunching QEMU with ASM output") - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -d in_asm -- --test: $(KERNEL_BIN) -- $(call colorecho, "\nTesting chainloading - $(BSP)") -- @$(DOCKER_TEST) $(EXEC_QEMU_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ -- -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) - endif --chainboot: + ##------------------------------------------------------------------------------ + ## Push the kernel to the real HW target + ##------------------------------------------------------------------------------ + chainboot: $(KERNEL_BIN) - @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(CHAINBOOT_DEMO_PAYLOAD) -+chainboot: $(KERNEL_BIN) + @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) - clippy: - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) + ##------------------------------------------------------------------------------ + ## Run clippy +@@ -216,8 +210,7 @@ + ##------------------------------------------------------------------------------ + test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") +- @$(DOCKER_TEST) $(EXEC_TEST_MINIPUSH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) \ +- -kernel $(KERNEL_BIN) $(CHAINBOOT_DEMO_PAYLOAD) ++ @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + + test: test_boot + diff -uNr 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s 07_timestamps/src/_arch/aarch64/cpu/boot.s --- 06_uart_chainloader/src/_arch/aarch64/cpu/boot.s @@ -720,9 +747,17 @@ diff -uNr 06_uart_chainloader/src/time.rs 07_timestamps/src/time.rs + } +} -diff -uNr 06_uart_chainloader/tests/qemu_minipush.rb 07_timestamps/tests/qemu_minipush.rb ---- 06_uart_chainloader/tests/qemu_minipush.rb -+++ 07_timestamps/tests/qemu_minipush.rb +diff -uNr 06_uart_chainloader/tests/boot_test_string.rb 07_timestamps/tests/boot_test_string.rb +--- 06_uart_chainloader/tests/boot_test_string.rb ++++ 07_timestamps/tests/boot_test_string.rb +@@ -0,0 +1,3 @@ ++# frozen_string_literal: true ++ ++EXPECTED_PRINT = 'Spinning for 1 second' + +diff -uNr 06_uart_chainloader/tests/chainboot_test.rb 07_timestamps/tests/chainboot_test.rb +--- 06_uart_chainloader/tests/chainboot_test.rb ++++ 07_timestamps/tests/chainboot_test.rb @@ -1,80 +0,0 @@ -# frozen_string_literal: true - @@ -730,80 +765,80 @@ diff -uNr 06_uart_chainloader/tests/qemu_minipush.rb 07_timestamps/tests/qemu_mi -# -# Copyright (c) 2020-2021 Andre Richter - --require_relative '../../utils/minipush' --require 'expect' --require 'timeout' +-require_relative '../../common/serial/minipush' +-require_relative '../../common/tests/boot_test' +-require 'pty' - -# Match for the last print that 'demo_payload_rpiX.img' produces. -EXPECTED_PRINT = 'Echoing input now' - --# The main class --class QEMUMiniPush < MiniPush -- TIMEOUT_SECS = 3 +-# Extend BootTest so that it listens on the output of a MiniPush instance, which is itself connected +-# to a QEMU instance instead of a real HW. +-class ChainbootTest < BootTest +- MINIPUSH = '../common/serial/minipush.rb' +- MINIPUSH_POWER_TARGET_REQUEST = 'Please power the target now' - -- # override -- def initialize(qemu_cmd, binary_image_path) -- super(nil, binary_image_path) +- def initialize(qemu_cmd, payload_path) +- super(qemu_cmd, EXPECTED_PRINT) - -- @qemu_cmd = qemu_cmd +- @test_name = 'Boot test using Minipush' +- +- @payload_path = payload_path - end - - private - -- def quit_qemu_graceful -- Timeout.timeout(5) do -- pid = @target_serial.pid -- Process.kill('TERM', pid) -- Process.wait(pid) -- end -- end -- - # override -- def open_serial -- @target_serial = IO.popen(@qemu_cmd, 'r+', err: '/dev/null') +- def post_process_and_add_output(output) +- temp = output.join.split("\r\n") - -- # Ensure all output is immediately flushed to the device. -- @target_serial.sync = true +- # Should a line have solo carriage returns, remove any overridden parts of the string. +- temp.map! { |x| x.gsub(/.*\r/, '') } - -- puts "[#{@name_short}] ✅ Serial connected" +- @test_output += temp - end - -- # override -- def terminal -- result = @target_serial.expect(EXPECTED_PRINT, TIMEOUT_SECS) -- exit(1) if result.nil? -- -- puts result -- -- quit_qemu_graceful +- def wait_for_minipush_power_request(mp_out) +- output = [] +- Timeout.timeout(MAX_WAIT_SECS) do +- loop do +- output << mp_out.gets +- break if output.last.include?(MINIPUSH_POWER_TARGET_REQUEST) +- end +- end +- rescue Timeout::Error +- @test_error = 'Timed out waiting for power request' +- rescue StandardError => e +- @test_error = e.message +- ensure +- post_process_and_add_output(output) - end - - # override -- def connetion_reset; end +- def setup +- pty_main, pty_secondary = PTY.open +- mp_out, _mp_in = PTY.spawn("ruby #{MINIPUSH} #{pty_secondary.path} #{@payload_path}") - -- # override -- def handle_reconnect(error) -- handle_unexpected(error) +- # Wait until MiniPush asks for powering the target. +- wait_for_minipush_power_request(mp_out) +- +- # Now is the time to start QEMU with the chainloader binary. QEMU's virtual tty is connected +- # to the MiniPush instance spawned above, so that the two processes talk to each other. +- Process.spawn(@qemu_cmd, in: pty_main, out: pty_main) +- +- # The remainder of the test is done by the parent class' run_concrete_test, which listens on +- # @qemu_serial. Hence, point it to MiniPush's output. +- @qemu_serial = mp_out - end -end - -##-------------------------------------------------------------------------------------------------- -## Execution starts here -##-------------------------------------------------------------------------------------------------- --puts --puts 'QEMUMiniPush 1.0'.cyan --puts -- --# CTRL + C handler. Only here to suppress Ruby's default exception print. --trap('INT') do -- # The `ensure` block from `QEMUMiniPush::run` will run after exit, restoring console state. -- exit --end -- --binary_image_path = ARGV.pop +-payload_path = ARGV.pop -qemu_cmd = ARGV.join(' ') - --QEMUMiniPush.new(qemu_cmd, binary_image_path).run +-ChainbootTest.new(qemu_cmd, payload_path).run diff -uNr 06_uart_chainloader/update.sh 07_timestamps/update.sh --- 06_uart_chainloader/update.sh diff --git a/07_timestamps/tests/boot_test_string.rb b/07_timestamps/tests/boot_test_string.rb new file mode 100644 index 000000000..02c3c492a --- /dev/null +++ b/07_timestamps/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Spinning for 1 second' diff --git a/08_hw_debug_JTAG/Makefile b/08_hw_debug_JTAG/Makefile index e3dcaae86..0a2443acc 100644 --- a/08_hw_debug_JTAG/Makefile +++ b/08_hw_debug_JTAG/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -42,11 +49,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -63,85 +77,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ - clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -150,10 +189,69 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/08_hw_debug_JTAG/README.md b/08_hw_debug_JTAG/README.md index a30a296ed..74a942a80 100644 --- a/08_hw_debug_JTAG/README.md +++ b/08_hw_debug_JTAG/README.md @@ -320,7 +320,7 @@ diff -uNr 07_timestamps/Cargo.toml 08_hw_debug_JTAG/Cargo.toml diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile --- 07_timestamps/Makefile +++ 08_hw_debug_JTAG/Makefile -@@ -23,6 +23,8 @@ +@@ -30,6 +30,8 @@ OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf @@ -329,7 +329,7 @@ diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) -@@ -34,6 +36,8 @@ +@@ -41,6 +43,8 @@ OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf @@ -338,56 +338,66 @@ diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -@@ -65,9 +69,12 @@ - DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial - DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t - DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -+DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot - DOCKER_ARG_DEV = --privileged -v /dev:/dev -+DOCKER_ARG_NET = --network host +@@ -84,17 +88,24 @@ + DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial + DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i + DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common ++DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot + DOCKER_ARG_DEV = --privileged -v /dev:/dev ++DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -+DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) + DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) ++DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) - # Dockerize commands that require USB device passthrough only on Linux -@@ -75,12 +82,17 @@ + # Dockerize commands, which require USB device passthrough, only on Linux. + ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) -+ DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) ++ DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) +else + DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif - EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) - EXEC_MINIPUSH = ruby ../utils/minipush.rb --.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check -+.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ -+ clean readelf objdump nm check +@@ -193,6 +204,35 @@ - all: $(KERNEL_BIN) -@@ -107,6 +119,19 @@ - chainboot: $(KERNEL_BIN) - @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) ++##-------------------------------------------------------------------------------------------------- ++## Debugging targets ++##-------------------------------------------------------------------------------------------------- ++.PHONY: jtagboot openocd gdb gdb-opt0 ++ ++##------------------------------------------------------------------------------ ++## Push the JTAG boot image to the real HW target ++##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + ++##------------------------------------------------------------------------------ ++## Start OpenOCD session ++##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + ++##------------------------------------------------------------------------------ ++## Start GDB session ++##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + - clippy: - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) ++ ++ + ##-------------------------------------------------------------------------------------------------- + ## Testing targets + ##-------------------------------------------------------------------------------------------------- ``` diff --git a/08_hw_debug_JTAG/tests/boot_test_string.rb b/08_hw_debug_JTAG/tests/boot_test_string.rb new file mode 100644 index 000000000..02c3c492a --- /dev/null +++ b/08_hw_debug_JTAG/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Spinning for 1 second' diff --git a/09_privilege_level/Makefile b/09_privilege_level/Makefile index e3dcaae86..0a2443acc 100644 --- a/09_privilege_level/Makefile +++ b/09_privilege_level/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -42,11 +49,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -63,85 +77,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ - clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -150,10 +189,69 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index 4b4c538d8..645ec210b 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -552,4 +552,13 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs } } +diff -uNr 08_hw_debug_JTAG/tests/boot_test_string.rb 09_privilege_level/tests/boot_test_string.rb +--- 08_hw_debug_JTAG/tests/boot_test_string.rb ++++ 09_privilege_level/tests/boot_test_string.rb +@@ -1,3 +1,3 @@ + # frozen_string_literal: true + +-EXPECTED_PRINT = 'Spinning for 1 second' ++EXPECTED_PRINT = 'Echoing input now' + ``` diff --git a/09_privilege_level/tests/boot_test_string.rb b/09_privilege_level/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/09_privilege_level/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/10_virtual_mem_part1_identity_mapping/Makefile b/10_virtual_mem_part1_identity_mapping/Makefile index e3dcaae86..0a2443acc 100644 --- a/10_virtual_mem_part1_identity_mapping/Makefile +++ b/10_virtual_mem_part1_identity_mapping/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -42,11 +49,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -63,85 +77,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ - clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -150,10 +189,69 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb b/10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/11_exceptions_part1_groundwork/Makefile b/11_exceptions_part1_groundwork/Makefile index e3dcaae86..0a2443acc 100644 --- a/11_exceptions_part1_groundwork/Makefile +++ b/11_exceptions_part1_groundwork/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -42,11 +49,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -63,85 +77,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ - clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -150,10 +189,69 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/11_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md index df8ecd6f9..629f5f091 100644 --- a/11_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -1016,4 +1016,13 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_ // Discard any spurious received characters before going into echo mode. +diff -uNr 10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb 11_exceptions_part1_groundwork/tests/boot_test_string.rb +--- 10_virtual_mem_part1_identity_mapping/tests/boot_test_string.rb ++++ 11_exceptions_part1_groundwork/tests/boot_test_string.rb +@@ -1,3 +1,3 @@ + # frozen_string_literal: true + +-EXPECTED_PRINT = 'Echoing input now' ++EXPECTED_PRINT = 'lr : 0x' + ``` diff --git a/11_exceptions_part1_groundwork/tests/boot_test_string.rb b/11_exceptions_part1_groundwork/tests/boot_test_string.rb new file mode 100644 index 000000000..200cd9713 --- /dev/null +++ b/11_exceptions_part1_groundwork/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'lr : 0x' diff --git a/12_integrated_testing/Makefile b/12_integrated_testing/Makefile index 4c2fb069a..080049863 100644 --- a/12_integrated_testing/Makefile +++ b/12_integrated_testing/Makefile @@ -2,18 +2,32 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) +# Optional integration test name. +ifdef TEST + TEST_ARG = --test $(TEST) +else + TEST_ARG = --test '*' +endif + + -# BSP-specific arguments +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -44,20 +58,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -# Testing-specific arguments -ifdef TEST - ifeq ($(TEST),unit) - TEST_ARG = --lib - else - TEST_ARG = --test $(TEST) - endif -endif +KERNEL_ELF = target/$(TARGET)/release/kernel + -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -75,105 +87,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ - clippy clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -define KERNEL_TEST_RUNNER - #!/usr/bin/env bash - - TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') - TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') - - $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY -endef - -export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - $(call colorecho, "\nCompiling test(s) - $(BSP)") - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -182,10 +199,108 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot test_unit test_integration + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + +##------------------------------------------------------------------------------ +## Run integration test(s) +##------------------------------------------------------------------------------ +test_integration: + $(call colorecho, "\nCompiling integration test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +test: test_boot test_unit test_integration + +endif diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index bb1677e98..c25e30474 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -2,12 +2,14 @@ ## tl;dr -- We implement our own test framework using `Rust`'s [custom_test_frameworks] feature by enabling - `Unit Tests` and `Integration Tests` using `QEMU`. -- It is also possible to have test automation for the kernel's `console` (provided over `UART` in - our case): Sending strings/characters to the console and expecting specific answers in return. +- We implement our own integrated test framework using `Rust`'s [custom_test_frameworks] feature by + enabling `Unit Tests` and `Integration Tests` using `QEMU`. +- It is also possible to have test automation for I/O with the kernel's `console` (provided over + `UART` in our case). That is, sending strings/characters to the console and expecting specific + answers in return. +- The already existing basic `boot test` remains unchanged. - + ## Table of Contents @@ -40,13 +42,13 @@ functionality. For example: - Stalling execution during boot to test the kernel's timekeeping code by spinning for 1 second. - Willingly causing exceptions to see the exception handler running. -The feature set of the kernel is now rich enough so that it makes sense to introduce proper testing -modeled after Rust's [native testing framework]. This tutorial extends our kernel with three basic -testing facilities: +The feature set of the kernel is now rich enough so that it makes sense to introduce proper +integrated testing modeled after Rust's [native testing framework]. This tutorial extends our single +existing kernel test with three new testing facilities: - Classic `Unit Tests`. - [Integration Tests] (self-contained tests stored in the `$CRATE/tests/` directory). - - `Console Tests`. These are integration tests acting on external stimuli - aka `console` input. - Sending strings/characters to the console and expecting specific answers in return. + - `Console I/O Tests`. These are integration tests acting on external stimuli - aka `console` + input. Sending strings/characters to the console and expecting specific answers in return. [native testing framework]: https://doc.rust-lang.org/book/ch11-00-testing.html @@ -64,7 +66,7 @@ dependencies on the standard library, but comes at the cost of having a reduced of annotating functions with `#[test]`, the `#[test_case]` attribute must be used. Additionally, we need to write a `test_runner` function, which is supposed to execute all the functions annotated with `#[test_case]`. This is barely enough to get `Unit Tests` running, though. There will be some -more challenges that need solving for getting `Integration Tests` running as well. +more challenges that need be solved for getting `Integration Tests` running as well. Please note that for automation purposes, all testing will be done in `QEMU` and not on real hardware. @@ -82,15 +84,23 @@ additional insights. ## Implementation -We introduce a new `Makefile` target: +We introduce two new `Makefile` targets: ```console -$ make test +$ make test_unit +$ make test_integration ``` -In essence, `make test` will execute `cargo test` instead of `cargo rustc`. The details will be -explained in due course. The rest of the tutorial will explain as chronologically as possible what -happens when `make test` aka `cargo test` runs. +In essence, the `make test_*` targets will execute `cargo test` instead of `cargo rustc`. The +details will be explained in due course. The rest of the tutorial will explain as chronologically as +possible what happens when `make test_*` aka `cargo test` runs. + +Please note that the new targets are added to the existing `make test` target, so this is now your +one-stop target to execute all possible tests for the kernel: + +```Makefile +test: test_boot test_unit test_integration +``` ### Test Organization @@ -166,8 +176,9 @@ that we are supposed to provide. This is the one that will be called by the `car ```rust /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); @@ -255,7 +266,7 @@ opportunity to cut down on setup code. [tutorial 03]: ../03_hacky_hello_world As a matter of fact, for the `Raspberrys`, nothing needs to be done, so the function is empy. But -this might be different for other hardware emulated by QEMU, so it makes sense to introduce the +this might be different for other hardware emulated by `QEMU`, so it makes sense to introduce the function now to make it easier in case new `BSPs` are added to the kernel in the future. Next, the reexported `test_main()` is called, which will call our `test_runner()` which finally @@ -265,9 +276,10 @@ prints the unit test names and executes them. Let's recap where we are right now: -We've enabled `custom_test_frameworks` in `lib.rs` to a point where, when using `make test`, the -code gets compiled to a test kernel binary that eventually executes all the (yet-to-be-defined) -`UnitTest` instances by executing all the way from `_start()` to our `test_runner()` function. +We've enabled `custom_test_frameworks` in `lib.rs` to a point where, when using a `make test_unit` +target, the code gets compiled to a test kernel binary that eventually executes all the +(yet-to-be-defined) `UnitTest` instances by executing all the way from `_start()` to our +`test_runner()` function. Through mechanisms that are explained later, `cargo` will now instantiate a `QEMU` process that exectues this test kernel. The question now is: How is test success/failure communicated to `cargo`? @@ -339,30 +351,30 @@ concludes: #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } } ``` -In case none of the unit tests panicked, `lib.rs`'s `kernel_init()` calls `cpu::qemu_exit_success()` -to successfully conclude the unit test run. +In case _none_ of the unit tests panicked, `lib.rs`'s `kernel_init()` calls +`cpu::qemu_exit_success()` to successfully conclude the unit test run. ### Controlling Test Kernel Execution Now is a good time to catch up on how the test kernel binary is actually being executed. Normally, `cargo test` would try to execute the compiled binary as a normal child process. This would fail -horribly because we build a kernel, and not a userspace process. Also, chances are very high that -you sit in front of an `x86` machine, whereas the RPi kernel is `AArch64`. +horribly because we build a kernel, and not a userspace process. Also, chances are high that you sit +in front of an `x86` machine, whereas the RPi kernel is `AArch64`. Therefore, we need to install some hooks that make sure the test kernel gets executed inside `QEMU`, -quite like it is done for the existing `make qemu` target that is in place since tutorial 1. The +quite like it is done for the existing `make qemu` target that is in place since `tutorial 1`. The first step is to add a new file to the project, `.cargo/config.toml`: ```toml @@ -374,10 +386,13 @@ Instead of executing a compilation result directly, the `runner` flag will instr delegate the execution. Using the setting depicted above, `target/kernel_test_runner.sh` will be executed and given the full path to the compiled test kernel as the first command line argument. -The file `kernel_test_runner.sh` does not exist by default. We generate it on demand throguh the -`make test` target: +The file `kernel_test_runner.sh` does not exist by default. We generate it on demand when one of the +`make test_*` targets is called: ```Makefile +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER #!/usr/bin/env bash @@ -385,16 +400,26 @@ define KERNEL_TEST_RUNNER TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib ``` It first does the standard `objcopy` step to strip the `ELF` down to a raw binary. Just like in all @@ -403,44 +428,76 @@ provided to it by `cargo`, and finally compiles a `docker` command to execute th reference, here it is fully resolved for an `RPi3 BSP`: ```bash -docker run -i --rm -v /opt/rust-raspberrypi-OS-tutorials/12_integrated_testing:/work/tutorial -w /work/tutorial rustembedded/osdev-utils ruby tests/runner.rb qemu-system-aarch64 -M raspi3 -serial stdio -display none -semihosting -kernel $TEST_BINARY +docker run --rm -v /opt/rust-raspberrypi-OS-tutorials/12_integrated_testing:/work/tutorial -w /work/tutorial -v /opt/rust-raspberrypi-OS-tutorials/12_integrated_testing/../common:/work/common rustembedded/osdev-utils ruby ../common/tests/dispatch.rb qemu-system-aarch64 -M raspi3 -serial stdio -display none -semihosting -kernel $TEST_BINARY ``` -We're still not done with all the redirections. Spotted the `ruby tests/runner.rb` part that gets -excuted inside Docker? +This command is quite similar to the one used in the `make test_boot` target that we have since +`tutorial 3`. However, we never bothered explaining it, so lets take a closer look this time. One of +the key ingredients is that we execute this script: `ruby ../common/tests/dispatch.rb`. #### Wrapping QEMU Test Execution -`runner.rb` is a [Ruby] wrapper script around `QEMU` that, for unit tests, catches the case that a -test gets stuck, e.g. in an unintentional busy loop or a crash. If `runner.rb` does not observe any -output of the test kernel for `5 seconds`, it cancels the execution and reports a failure back to -`cargo`. If `QEMU` exited itself by means of `aarch64::exit_success() / aarch64::exit_failure()`, -the respective exit status code is passed through. The essential part happens here in `class -RawTest`: +`dispatch.rb` is a [Ruby] script which first determines what kind of test is due by inspecting the +`QEMU`-command that was given to it. In case of `unit tests`, we are only interested if they all +executed successfully, which can be checked by inspecting `QEMU`'s exit code. So the script takes +the provided qemu command it got from `ARGV`, and creates and runs an instance of `ExitCodeTest`: ```ruby -def exec - error = 'Timed out waiting for test' +require_relative 'boot_test' +require_relative 'console_io_test' +require_relative 'exit_code_test' + +qemu_cmd = ARGV.join(' ') +binary = ARGV.last +test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] + +case test_name +when 'kernel8.img' + load 'tests/boot_test_string.rb' # provides 'EXPECTED_PRINT' + BootTest.new(qemu_cmd, EXPECTED_PRINT).run # Doesn't return + +when 'libkernel' + ExitCodeTest.new(qemu_cmd, 'Kernel library unit tests').run # Doesn't return +``` + +The easy case is `QEMU` existing by itself by means of `aarch64::exit_success()` or +`aarch64::exit_failure()`. But the script can also catch the case of a test that gets stuck, e.g. in +an unintentional busy loop or a crash. If `ExitCodeTest` does not observe any output of the test +kernel for `MAX_WAIT_SECS`, it cancels the execution and marks the test as failed. Test success or +failure is finally reported back to `cargo`. + +Here is the essential part happening in `class ExitCodeTest` (If `QEMU` exits itself, an `EOFError` +is thrown): + +```ruby +def run_concrete_test io = IO.popen(@qemu_cmd) - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end + Timeout.timeout(MAX_WAIT_SECS) do + @test_output << io.read_nonblock(1024) while IO.select([io]) end +rescue EOFError + io.close + @test_error = $CHILD_STATUS.to_i.zero? ? false : 'QEMU exit status != 0' +rescue Timeout::Error + @test_error = 'Timed out waiting for test' +rescue StandardError => e + @test_error = e.message +ensure + post_process_output +end ``` +Please note that `dispatch.rb` and all its dependencies live in the shared folder +`../common/tests/`. + [Ruby]: https://www.ruby-lang.org/ ### Writing Unit Tests -Alright, that's a wrap for the whole chain from `make test` all the way to reporting the test exit -status back to `cargo test`. It is a lot to digest already, but we haven't even learned to write -`Unit Tests` yet. +Alright, that's a wrap for the whole chain from `make test_unit` all the way to reporting the test +exit status back to `cargo test`. It is a lot to digest already, but we haven't even learned to +write `Unit Tests` yet. In essence, it is almost like in `std` environments, with the difference that `#[test]` can't be used, because it is part of the standard library. The `no_std` replacement attribute provided by @@ -485,9 +542,9 @@ Since this is a bit boiler-platy with the const and name definition, let's write macro] named `#[kernel_test]` to simplify this. It should work this way: 1. Must be put before functions that take no arguments and return nothing. - 2. Automatically constructs a `const UnitTest` from attributed functions like shown above by: + 1. Automatically constructs a `const UnitTest` from attributed functions like shown above by: 1. Converting the function name to the `name` member of the `UnitTest` struct. - 2. Populating the `test_func` member with a closure that executes the body of the attributed + 1. Populating the `test_func` member with a closure that executes the body of the attributed function. For the sake of brevity, we're not going to discuss the macro implementation. [The source is in the @@ -609,12 +666,12 @@ function? This marks the function in `lib.rs` as a [weak symbol]. Let's look at #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } @@ -624,7 +681,7 @@ fn _panic_exit() -> ! { [weak symbol]: https://en.wikipedia.org/wiki/Weak_symbol This enables integration tests in `$CRATE/tests/` to override this function according to their -needs. This is useful because depending on the kind of test, a `panic!` could mean success or +needs. This is useful, because depending on the kind of test, a `panic!` could mean success or failure. For example, `tests/02_exception_sync_page_fault.rs` is intentionally causing a page fault, so the wanted outcome is a `panic!`. Here is the whole test (minus some inline comments): @@ -646,10 +703,10 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::mmu().init() { + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { println!("MMU: {}", string); cpu::qemu_exit_failure() } @@ -661,7 +718,6 @@ unsafe fn kernel_init() -> ! { // If execution reaches here, the memory access above did not cause a page fault exception. cpu::qemu_exit_failure() } - ``` The `_panic_exit()` version that makes `QEMU` return `0` (indicating test success) is pulled in by @@ -671,22 +727,21 @@ The `_panic_exit()` version that makes `QEMU` return `0` (indicating test succes As the kernel or OS grows, it will be more and more interesting to test user/kernel interaction through the serial console. That is, sending strings/characters to the console and expecting -specific answers in return. The `runner.rb` wrapper script provides infrastructure to do this with -little overhead. It basically works like this: +specific answers in return. The `dispatch.rb` wrapper script provides infrastructure to recognize +and dispatch console I/O tests with little overhead. It basically works like this: 1. For each integration test, check if a companion file to the `.rs` test file exists. - A companion file has the same name, but ends in `.rb`. - - The companion file contains one or more console subtests. - 2. If it exists, load the file to dynamically import the console subtests. - 3. Spawn `QEMU` and attach to the serial console. - 4. Run the console subtests. + - The companion file contains one or more console I/O subtests. + 1. If it exists, load the file to dynamically import the console subtests. + 1. Create a `ConsoleIOTest` instance and run it. + - This first spawns `QEMU` and attaches to `QEMU`'s serial console emulation. + - Then it runs all console subtests on it. Here is an excerpt from `00_console_sanity.rb` showing a subtest that does a handshake with the kernel over the console: ```ruby -TIMEOUT_SECS = 3 - # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -695,7 +750,7 @@ class TxRxHandshake def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end ``` @@ -727,20 +782,22 @@ unsafe fn kernel_init() -> ! { ## Test it -Believe it or not, that is all. There are three ways you can run tests: +Believe it or not, that is all. There are four ways you can run tests now: - 1. `make test` will run all tests back-to-back. - 2. `TEST=unit make test` will run `libkernel`'s unit tests. - 3. `TEST=TEST_NAME make test` will run a specficic integration test. - - For example, `TEST=01_timer_sanity make test` + 1. `make test` will run all tests back-to-back. That is, the ever existing `boot test` first, then + `unit tests`, then `integration tests`. + 1. `make test_unit` will run `libkernel`'s unit tests. + 1. `make test_integration` will run all integration tests back-to-back. + 1. `TEST=TEST_NAME make test_integration` will run a specficic integration test. + - For example, `TEST=01_timer_sanity make test_integration` ```console $ make test [...] - Running unittests (target/aarch64-unknown-none-softfloat/release/deps/libkernel-836110ac5dd535ba) + Running unittests (target/aarch64-unknown-none-softfloat/release/deps/libkernel-142a8d94bc9c615a) ------------------------------------------------------------------- - 🦀 Running 8 tests + 🦀 Running 6 tests ------------------------------------------------------------------- 1. virt_mem_layout_sections_are_64KiB_aligned................[ok] @@ -749,17 +806,18 @@ $ make test 4. kernel_tables_in_bss......................................[ok] 5. size_of_tabledescriptor_equals_64_bit.....................[ok] 6. size_of_pagedescriptor_equals_64_bit......................[ok] - 7. zero_volatile_works.......................................[ok] - 8. bss_section_is_sane.......................................[ok] ------------------------------------------------------------------- - ✅ Success: libkernel + ✅ Success: Kernel library unit tests ------------------------------------------------------------------- - Running tests/00_console_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/00_console_sanity-78c12c5472d40df7) + +Compiling integration test(s) - rpi3 + Finished release [optimized] target(s) in 0.00s + Running tests/00_console_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/00_console_sanity-c06130838f14dbff) ------------------------------------------------------------------- - 🦀 Running 3 console-based tests + 🦀 Running 3 console I/O tests ------------------------------------------------------------------- 1. Transmit and Receive handshake............................[ok] @@ -767,11 +825,11 @@ $ make test 3. Receive statistics........................................[ok] ------------------------------------------------------------------- - ✅ Success: 00_console_sanity + ✅ Success: 00_console_sanity.rs ------------------------------------------------------------------- - Running tests/01_timer_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/01_timer_sanity-4866734b14c83c9b) + Running tests/01_timer_sanity.rs (target/aarch64-unknown-none-softfloat/release/deps/01_timer_sanity-62a954d22239d1a3) ------------------------------------------------------------------- 🦀 Running 3 tests ------------------------------------------------------------------- @@ -781,11 +839,11 @@ $ make test 3. spin_accuracy_check_1_second..............................[ok] ------------------------------------------------------------------- - ✅ Success: 01_timer_sanity + ✅ Success: 01_timer_sanity.rs ------------------------------------------------------------------- - Running tests/02_exception_sync_page_fault.rs (target/aarch64-unknown-none-softfloat/release/deps/02_exception_sync_page_fault-f2d0885cada1105b) + Running tests/02_exception_sync_page_fault.rs (target/aarch64-unknown-none-softfloat/release/deps/02_exception_sync_page_fault-2d8ec603ef1c4d8e) ------------------------------------------------------------------- 🦀 Testing synchronous exception handling by causing a page fault ------------------------------------------------------------------- @@ -800,7 +858,7 @@ $ make test [...] ------------------------------------------------------------------- - ✅ Success: 02_exception_sync_page_fault + ✅ Success: 02_exception_sync_page_fault.rs ------------------------------------------------------------------- ``` @@ -880,7 +938,21 @@ diff -uNr 11_exceptions_part1_groundwork/Cargo.toml 12_integrated_testing/Cargo. diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile --- 11_exceptions_part1_groundwork/Makefile +++ 12_integrated_testing/Makefile -@@ -20,6 +20,7 @@ +@@ -14,6 +14,13 @@ + # Default to a serial device name that is common in Linux. + DEV_SERIAL ?= /dev/ttyUSB0 + ++# Optional integration test name. ++ifdef TEST ++ TEST_ARG = --test $(TEST) ++else ++ TEST_ARG = --test '*' ++endif ++ + + + ##-------------------------------------------------------------------------------------------------- +@@ -27,6 +34,7 @@ QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = raspi3 QEMU_RELEASE_ARGS = -serial stdio -display none @@ -888,7 +960,7 @@ diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf -@@ -33,6 +34,7 @@ +@@ -40,6 +48,7 @@ QEMU_BINARY = qemu-system-aarch64 QEMU_MACHINE_TYPE = QEMU_RELEASE_ARGS = -serial stdio -display none @@ -896,23 +968,7 @@ diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile OBJDUMP_BINARY = aarch64-none-elf-objdump NM_BINARY = aarch64-none-elf-nm READELF_BINARY = aarch64-none-elf-readelf -@@ -45,6 +47,15 @@ - # Export for build.rs - export LINKER_FILE - -+# Testing-specific arguments -+ifdef TEST -+ ifeq ($(TEST),unit) -+ TEST_ARG = --lib -+ else -+ TEST_ARG = --test $(TEST) -+ endif -+endif -+ - QEMU_MISSING_STRING = "This board is not yet supported for QEMU." - - RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) -@@ -59,6 +70,7 @@ +@@ -73,6 +82,7 @@ DOC_CMD = cargo doc $(COMPILER_ARGS) CLIPPY_CMD = cargo clippy $(COMPILER_ARGS) CHECK_CMD = cargo check $(COMPILER_ARGS) @@ -920,37 +976,28 @@ diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -@@ -75,6 +87,7 @@ - - DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) - DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -+DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) - DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) - - # Dockerize commands that require USB device passthrough only on Linux -@@ -91,8 +104,8 @@ - EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) - EXEC_MINIPUSH = ruby ../utils/minipush.rb +@@ -236,11 +246,11 @@ + ##-------------------------------------------------------------------------------------------------- + ## Testing targets + ##-------------------------------------------------------------------------------------------------- +-.PHONY: test test_boot ++.PHONY: test test_boot test_unit test_integration --.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot jtagboot openocd gdb gdb-opt0 clippy \ -- clean readelf objdump nm check -+.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ -+ clippy clean readelf objdump nm check + ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. - all: $(KERNEL_BIN) +-test_boot test : ++test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") -@@ -108,12 +121,31 @@ - @$(DOC_CMD) --document-private-items --open + else # QEMU is supported. +@@ -252,6 +262,45 @@ + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) - ifeq ($(QEMU_MACHINE_TYPE),) --qemu: -+qemu test: - $(call colorecho, "\n$(QEMU_MISSING_STRING)") - else - qemu: $(KERNEL_BIN) - $(call colorecho, "\nLaunching QEMU") - @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -+ +-test: test_boot ++##------------------------------------------------------------------------------ ++## Helpers for unit and integration test targets ++##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + @@ -958,20 +1005,38 @@ diff -uNr 11_exceptions_part1_groundwork/Makefile 12_integrated_testing/Makefile + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY -+ $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY ++ $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER -+test: FEATURES += --features test_build -+test: -+ $(call colorecho, "\nCompiling test(s) - $(BSP)") -+ @mkdir -p target -+ @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh -+ @chmod +x target/kernel_test_runner.sh ++ ++define test_prepare ++ @mkdir -p target ++ @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh ++ @chmod +x target/kernel_test_runner.sh ++endef ++ ++test_unit test_integration: FEATURES += --features test_build ++ ++##------------------------------------------------------------------------------ ++## Run unit test(s) ++##------------------------------------------------------------------------------ ++test_unit: ++ $(call colorecho, "\nCompiling unit test(s) - $(BSP)") ++ $(call test_prepare) ++ RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib ++ ++##------------------------------------------------------------------------------ ++## Run integration test(s) ++##------------------------------------------------------------------------------ ++test_integration: ++ $(call colorecho, "\nCompiling integration test(s) - $(BSP)") ++ $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) - endif ++ ++test: test_boot test_unit test_integration - chainboot: $(KERNEL_BIN) + endif diff -uNr 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs 12_integrated_testing/src/_arch/aarch64/cpu.rs --- 11_exceptions_part1_groundwork/src/_arch/aarch64/cpu.rs @@ -1216,7 +1281,7 @@ diff -uNr 11_exceptions_part1_groundwork/src/exception.rs 12_integrated_testing/ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/lib.rs --- 11_exceptions_part1_groundwork/src/lib.rs +++ 12_integrated_testing/src/lib.rs -@@ -0,0 +1,186 @@ +@@ -0,0 +1,187 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2021 Andre Richter @@ -1379,8 +1444,9 @@ diff -uNr 11_exceptions_part1_groundwork/src/lib.rs 12_integrated_testing/src/li + +/// The default runner for unit tests. +pub fn test_runner(tests: &[&test_types::UnitTest]) { ++ // This line will be printed as the test header. + println!("Running {} tests", tests.len()); -+ println!("-------------------------------------------------------------------\n"); ++ + for (i, test) in tests.iter().enumerate() { + print!("{:>3}. {:.<58}", i + 1, test.name); + @@ -1630,12 +1696,12 @@ diff -uNr 11_exceptions_part1_groundwork/src/panic_wait.rs 12_integrated_testing +#[linkage = "weak"] +#[no_mangle] +fn _panic_exit() -> ! { -+ #[cfg(not(test_build))] ++ #[cfg(not(feature = "test_build"))] + { + cpu::wait_forever() + } + -+ #[cfg(test_build)] ++ #[cfg(feature = "test_build")] + { + cpu::qemu_exit_failure() + } @@ -1708,7 +1774,7 @@ diff -uNr 11_exceptions_part1_groundwork/test-macros/src/lib.rs 12_integrated_te diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrated_testing/tests/00_console_sanity.rb --- 11_exceptions_part1_groundwork/tests/00_console_sanity.rb +++ 12_integrated_testing/tests/00_console_sanity.rb -@@ -0,0 +1,50 @@ +@@ -0,0 +1,57 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 @@ -1719,6 +1785,13 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate + +TIMEOUT_SECS = 3 + ++# Error class for when expect times out. ++class ExpectTimeoutError < StandardError ++ def initialize ++ super('Timeout while expecting string') ++ end ++end ++ +# Verify sending and receiving works as expected. +class TxRxHandshake + def name @@ -1727,7 +1800,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate + + def run(qemu_out, qemu_in) + qemu_in.write_nonblock('ABC') -+ raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? ++ raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + end +end + @@ -1738,7 +1811,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate + end + + def run(qemu_out, _qemu_in) -+ raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? ++ raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? + end +end + @@ -1749,7 +1822,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate + end + + def run(qemu_out, _qemu_in) -+ raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? ++ raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? + end +end + @@ -1763,7 +1836,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rb 12_integrate diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrated_testing/tests/00_console_sanity.rs --- 11_exceptions_part1_groundwork/tests/00_console_sanity.rs +++ 12_integrated_testing/tests/00_console_sanity.rs -@@ -0,0 +1,42 @@ +@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2019-2021 Andre Richter @@ -1797,14 +1870,7 @@ diff -uNr 11_exceptions_part1_groundwork/tests/00_console_sanity.rs 12_integrate + print!("{}", console().chars_read()); + + // The QEMU process running this test will be closed by the I/O test harness. -+ // cpu::wait_forever(); -+ -+ // For some reason, in this test, rustc or the linker produces an empty binary when -+ // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time -+ // being, the following lines are just a workaround to fix this compiler/linker weirdness. -+ use libkernel::time::interface::TimeManager; -+ libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); -+ cpu::qemu_exit_success() ++ cpu::wait_forever(); +} diff -uNr 11_exceptions_part1_groundwork/tests/01_timer_sanity.rs 12_integrated_testing/tests/01_timer_sanity.rs @@ -1893,8 +1959,8 @@ diff -uNr 11_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 + exception::handling_init(); + bsp::console::qemu_bring_up_console(); + ++ // This line will be printed as the test header. + println!("Testing synchronous exception handling by causing a page fault"); -+ println!("-------------------------------------------------------------------\n"); + + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { + println!("MMU: {}", string); @@ -1909,6 +1975,15 @@ diff -uNr 11_exceptions_part1_groundwork/tests/02_exception_sync_page_fault.rs 1 + cpu::qemu_exit_failure() +} +diff -uNr 11_exceptions_part1_groundwork/tests/boot_test_string.rb 12_integrated_testing/tests/boot_test_string.rb +--- 11_exceptions_part1_groundwork/tests/boot_test_string.rb ++++ 12_integrated_testing/tests/boot_test_string.rb +@@ -1,3 +1,3 @@ + # frozen_string_literal: true + +-EXPECTED_PRINT = 'lr : 0x' ++EXPECTED_PRINT = 'Echoing input now' + diff -uNr 11_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 12_integrated_testing/tests/panic_exit_success/mod.rs --- 11_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs +++ 12_integrated_testing/tests/panic_exit_success/mod.rs @@ -1923,154 +1998,6 @@ diff -uNr 11_exceptions_part1_groundwork/tests/panic_exit_success/mod.rs 12_inte + libkernel::cpu::qemu_exit_success() +} -diff -uNr 11_exceptions_part1_groundwork/tests/runner.rb 12_integrated_testing/tests/runner.rb ---- 11_exceptions_part1_groundwork/tests/runner.rb -+++ 12_integrated_testing/tests/runner.rb -@@ -0,0 +1,143 @@ -+#!/usr/bin/env ruby -+# frozen_string_literal: true -+ -+# SPDX-License-Identifier: MIT OR Apache-2.0 -+# -+# Copyright (c) 2019-2021 Andre Richter -+ -+require 'English' -+require 'pty' -+ -+# Test base class. -+class Test -+ INDENT = ' ' -+ -+ def print_border(status) -+ puts -+ puts "#{INDENT}-------------------------------------------------------------------" -+ puts status -+ puts "#{INDENT}-------------------------------------------------------------------\n\n\n" -+ end -+ -+ def print_error(error) -+ puts -+ print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") -+ end -+ -+ def print_success -+ print_border("#{INDENT}✅ Success: #{@test_name}") -+ end -+ -+ def print_output -+ puts "#{INDENT}-------------------------------------------------------------------" -+ print INDENT -+ print '🦀 ' -+ print @output.join.gsub("\n", "\n#{INDENT}") -+ end -+ -+ def finish(error) -+ print_output -+ -+ exit_code = if error -+ print_error(error) -+ false -+ else -+ print_success -+ true -+ end -+ -+ exit(exit_code) -+ end -+end -+ -+# Executes tests with console I/O. -+class ConsoleTest < Test -+ def initialize(binary, qemu_cmd, test_name, console_subtests) -+ super() -+ -+ @binary = binary -+ @qemu_cmd = qemu_cmd -+ @test_name = test_name -+ @console_subtests = console_subtests -+ @cur_subtest = 1 -+ @output = ["Running #{@console_subtests.length} console-based tests\n", -+ "-------------------------------------------------------------------\n\n"] -+ end -+ -+ def format_test_name(number, name) -+ formatted_name = "#{number.to_s.rjust(3)}. #{name}" -+ formatted_name.ljust(63, '.') -+ end -+ -+ def run_subtest(subtest, qemu_out, qemu_in) -+ @output << format_test_name(@cur_subtest, subtest.name) -+ -+ subtest.run(qemu_out, qemu_in) -+ -+ @output << "[ok]\n" -+ @cur_subtest += 1 -+ end -+ -+ def exec -+ error = false -+ -+ PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| -+ begin -+ @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } -+ rescue StandardError => e -+ error = e.message -+ end -+ -+ finish(error) -+ end -+ end -+end -+ -+# A wrapper around the bare QEMU invocation. -+class RawTest < Test -+ MAX_WAIT_SECS = 5 -+ -+ def initialize(binary, qemu_cmd, test_name) -+ super() -+ -+ @binary = binary -+ @qemu_cmd = qemu_cmd -+ @test_name = test_name -+ @output = [] -+ end -+ -+ def exec -+ error = 'Timed out waiting for test' -+ io = IO.popen(@qemu_cmd) -+ -+ while IO.select([io], nil, nil, MAX_WAIT_SECS) -+ begin -+ @output << io.read_nonblock(1024) -+ rescue EOFError -+ io.close -+ error = $CHILD_STATUS.to_i != 0 -+ break -+ end -+ end -+ -+ finish(error) -+ end -+end -+ -+##-------------------------------------------------------------------------------------------------- -+## Script entry point -+##-------------------------------------------------------------------------------------------------- -+binary = ARGV.last -+test_name = binary.gsub(modulor{.*deps/}, '').split('-')[0] -+console_test_file = "tests/#{test_name}.rb" -+qemu_cmd = ARGV.join(' ') -+ -+test_runner = if File.exist?(console_test_file) -+ load console_test_file -+ # subtest_collection is provided by console_test_file -+ ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) -+ else -+ RawTest.new(binary, qemu_cmd, test_name) -+ end -+ -+test_runner.exec - diff -uNr 11_exceptions_part1_groundwork/test-types/Cargo.toml 12_integrated_testing/test-types/Cargo.toml --- 11_exceptions_part1_groundwork/test-types/Cargo.toml +++ 12_integrated_testing/test-types/Cargo.toml diff --git a/12_integrated_testing/src/lib.rs b/12_integrated_testing/src/lib.rs index 9890351f9..b8370a541 100644 --- a/12_integrated_testing/src/lib.rs +++ b/12_integrated_testing/src/lib.rs @@ -160,8 +160,9 @@ extern "Rust" { /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); diff --git a/12_integrated_testing/src/panic_wait.rs b/12_integrated_testing/src/panic_wait.rs index 20493a912..d272a1975 100644 --- a/12_integrated_testing/src/panic_wait.rs +++ b/12_integrated_testing/src/panic_wait.rs @@ -23,12 +23,12 @@ fn _panic_print(args: fmt::Arguments) { #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } diff --git a/12_integrated_testing/tests/00_console_sanity.rb b/12_integrated_testing/tests/00_console_sanity.rb index dfd6b16ea..16fb6c790 100644 --- a/12_integrated_testing/tests/00_console_sanity.rb +++ b/12_integrated_testing/tests/00_console_sanity.rb @@ -8,6 +8,13 @@ TIMEOUT_SECS = 3 +# Error class for when expect times out. +class ExpectTimeoutError < StandardError + def initialize + super('Timeout while expecting string') + end +end + # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -16,7 +23,7 @@ def name def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end @@ -27,7 +34,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? end end @@ -38,7 +45,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? end end diff --git a/12_integrated_testing/tests/00_console_sanity.rs b/12_integrated_testing/tests/00_console_sanity.rs index 84b744798..03058f5ef 100644 --- a/12_integrated_testing/tests/00_console_sanity.rs +++ b/12_integrated_testing/tests/00_console_sanity.rs @@ -31,12 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - // cpu::wait_forever(); - - // For some reason, in this test, rustc or the linker produces an empty binary when - // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time - // being, the following lines are just a workaround to fix this compiler/linker weirdness. - use libkernel::time::interface::TimeManager; - libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); - cpu::qemu_exit_success() + cpu::wait_forever(); } diff --git a/12_integrated_testing/tests/02_exception_sync_page_fault.rs b/12_integrated_testing/tests/02_exception_sync_page_fault.rs index f1535d34c..8febacd14 100644 --- a/12_integrated_testing/tests/02_exception_sync_page_fault.rs +++ b/12_integrated_testing/tests/02_exception_sync_page_fault.rs @@ -26,8 +26,8 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { println!("MMU: {}", string); diff --git a/12_integrated_testing/tests/boot_test_string.rb b/12_integrated_testing/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/12_integrated_testing/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/12_integrated_testing/tests/runner.rb b/12_integrated_testing/tests/runner.rb deleted file mode 100755 index 53116e088..000000000 --- a/12_integrated_testing/tests/runner.rb +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2019-2021 Andre Richter - -require 'English' -require 'pty' - -# Test base class. -class Test - INDENT = ' ' - - def print_border(status) - puts - puts "#{INDENT}-------------------------------------------------------------------" - puts status - puts "#{INDENT}-------------------------------------------------------------------\n\n\n" - end - - def print_error(error) - puts - print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") - end - - def print_success - print_border("#{INDENT}✅ Success: #{@test_name}") - end - - def print_output - puts "#{INDENT}-------------------------------------------------------------------" - print INDENT - print '🦀 ' - print @output.join.gsub("\n", "\n#{INDENT}") - end - - def finish(error) - print_output - - exit_code = if error - print_error(error) - false - else - print_success - true - end - - exit(exit_code) - end -end - -# Executes tests with console I/O. -class ConsoleTest < Test - def initialize(binary, qemu_cmd, test_name, console_subtests) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @console_subtests = console_subtests - @cur_subtest = 1 - @output = ["Running #{@console_subtests.length} console-based tests\n", - "-------------------------------------------------------------------\n\n"] - end - - def format_test_name(number, name) - formatted_name = "#{number.to_s.rjust(3)}. #{name}" - formatted_name.ljust(63, '.') - end - - def run_subtest(subtest, qemu_out, qemu_in) - @output << format_test_name(@cur_subtest, subtest.name) - - subtest.run(qemu_out, qemu_in) - - @output << "[ok]\n" - @cur_subtest += 1 - end - - def exec - error = false - - PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| - begin - @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } - rescue StandardError => e - error = e.message - end - - finish(error) - end - end -end - -# A wrapper around the bare QEMU invocation. -class RawTest < Test - MAX_WAIT_SECS = 5 - - def initialize(binary, qemu_cmd, test_name) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @output = [] - end - - def exec - error = 'Timed out waiting for test' - io = IO.popen(@qemu_cmd) - - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end - end - - finish(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Script entry point -##-------------------------------------------------------------------------------------------------- -binary = ARGV.last -test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] -console_test_file = "tests/#{test_name}.rb" -qemu_cmd = ARGV.join(' ') - -test_runner = if File.exist?(console_test_file) - load console_test_file - # subtest_collection is provided by console_test_file - ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) - else - RawTest.new(binary, qemu_cmd, test_name) - end - -test_runner.exec diff --git a/13_exceptions_part2_peripheral_IRQs/Makefile b/13_exceptions_part2_peripheral_IRQs/Makefile index 4c2fb069a..e860f00dd 100644 --- a/13_exceptions_part2_peripheral_IRQs/Makefile +++ b/13_exceptions_part2_peripheral_IRQs/Makefile @@ -2,18 +2,32 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) +# Optional integration test name. +ifdef TEST + TEST_ARG = --test $(TEST) +else + TEST_ARG = --test '*' +endif + + -# BSP-specific arguments +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -44,20 +58,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -# Testing-specific arguments -ifdef TEST - ifeq ($(TEST),unit) - TEST_ARG = --lib - else - TEST_ARG = --test $(TEST) - endif -endif +KERNEL_ELF = target/$(TARGET)/release/kernel + -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -75,105 +87,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ - clippy clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -define KERNEL_TEST_RUNNER - #!/usr/bin/env bash - - TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') - TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') - - $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY -endef - -export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - $(call colorecho, "\nCompiling test(s) - $(BSP)") - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -182,10 +199,108 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot test_unit test_integration + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + +##------------------------------------------------------------------------------ +## Run integration test(s) +##------------------------------------------------------------------------------ +test_integration: + $(call colorecho, "\nCompiling integration test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +test: test_boot test_unit test_integration + +endif diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 02341a12a..b8cf351cb 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -758,6 +758,19 @@ diff -uNr 12_integrated_testing/Cargo.toml 13_exceptions_part2_peripheral_IRQs/C edition = "2018" +diff -uNr 12_integrated_testing/Makefile 13_exceptions_part2_peripheral_IRQs/Makefile +--- 12_integrated_testing/Makefile ++++ 13_exceptions_part2_peripheral_IRQs/Makefile +@@ -291,7 +291,7 @@ + test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) +- RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib ++ @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + + ##------------------------------------------------------------------------------ + ## Run integration test(s) + diff -uNr 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs --- 12_integrated_testing/src/_arch/aarch64/cpu/smp.rs +++ 13_exceptions_part2_peripheral_IRQs/src/_arch/aarch64/cpu/smp.rs diff --git a/13_exceptions_part2_peripheral_IRQs/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/src/lib.rs index 4ca9f03ee..9c67f1ed6 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/lib.rs @@ -163,8 +163,9 @@ extern "Rust" { /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); diff --git a/13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs b/13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs index e3a9ed8ab..130e952b8 100644 --- a/13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs +++ b/13_exceptions_part2_peripheral_IRQs/src/panic_wait.rs @@ -23,12 +23,12 @@ fn _panic_print(args: fmt::Arguments) { #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } diff --git a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb index dfd6b16ea..16fb6c790 100644 --- a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb +++ b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rb @@ -8,6 +8,13 @@ TIMEOUT_SECS = 3 +# Error class for when expect times out. +class ExpectTimeoutError < StandardError + def initialize + super('Timeout while expecting string') + end +end + # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -16,7 +23,7 @@ def name def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end @@ -27,7 +34,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? end end @@ -38,7 +45,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? end end diff --git a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs index 84b744798..03058f5ef 100644 --- a/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/tests/00_console_sanity.rs @@ -31,12 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - // cpu::wait_forever(); - - // For some reason, in this test, rustc or the linker produces an empty binary when - // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time - // being, the following lines are just a workaround to fix this compiler/linker weirdness. - use libkernel::time::interface::TimeManager; - libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); - cpu::qemu_exit_success() + cpu::wait_forever(); } diff --git a/13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs b/13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs index f1535d34c..8febacd14 100644 --- a/13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs +++ b/13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault.rs @@ -26,8 +26,8 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { println!("MMU: {}", string); diff --git a/13_exceptions_part2_peripheral_IRQs/tests/boot_test_string.rb b/13_exceptions_part2_peripheral_IRQs/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/13_exceptions_part2_peripheral_IRQs/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/13_exceptions_part2_peripheral_IRQs/tests/runner.rb b/13_exceptions_part2_peripheral_IRQs/tests/runner.rb deleted file mode 100755 index 53116e088..000000000 --- a/13_exceptions_part2_peripheral_IRQs/tests/runner.rb +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2019-2021 Andre Richter - -require 'English' -require 'pty' - -# Test base class. -class Test - INDENT = ' ' - - def print_border(status) - puts - puts "#{INDENT}-------------------------------------------------------------------" - puts status - puts "#{INDENT}-------------------------------------------------------------------\n\n\n" - end - - def print_error(error) - puts - print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") - end - - def print_success - print_border("#{INDENT}✅ Success: #{@test_name}") - end - - def print_output - puts "#{INDENT}-------------------------------------------------------------------" - print INDENT - print '🦀 ' - print @output.join.gsub("\n", "\n#{INDENT}") - end - - def finish(error) - print_output - - exit_code = if error - print_error(error) - false - else - print_success - true - end - - exit(exit_code) - end -end - -# Executes tests with console I/O. -class ConsoleTest < Test - def initialize(binary, qemu_cmd, test_name, console_subtests) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @console_subtests = console_subtests - @cur_subtest = 1 - @output = ["Running #{@console_subtests.length} console-based tests\n", - "-------------------------------------------------------------------\n\n"] - end - - def format_test_name(number, name) - formatted_name = "#{number.to_s.rjust(3)}. #{name}" - formatted_name.ljust(63, '.') - end - - def run_subtest(subtest, qemu_out, qemu_in) - @output << format_test_name(@cur_subtest, subtest.name) - - subtest.run(qemu_out, qemu_in) - - @output << "[ok]\n" - @cur_subtest += 1 - end - - def exec - error = false - - PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| - begin - @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } - rescue StandardError => e - error = e.message - end - - finish(error) - end - end -end - -# A wrapper around the bare QEMU invocation. -class RawTest < Test - MAX_WAIT_SECS = 5 - - def initialize(binary, qemu_cmd, test_name) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @output = [] - end - - def exec - error = 'Timed out waiting for test' - io = IO.popen(@qemu_cmd) - - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end - end - - finish(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Script entry point -##-------------------------------------------------------------------------------------------------- -binary = ARGV.last -test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] -console_test_file = "tests/#{test_name}.rb" -qemu_cmd = ARGV.join(' ') - -test_runner = if File.exist?(console_test_file) - load console_test_file - # subtest_collection is provided by console_test_file - ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) - else - RawTest.new(binary, qemu_cmd, test_name) - end - -test_runner.exec diff --git a/14_virtual_mem_part2_mmio_remap/Makefile b/14_virtual_mem_part2_mmio_remap/Makefile index 4c2fb069a..e860f00dd 100644 --- a/14_virtual_mem_part2_mmio_remap/Makefile +++ b/14_virtual_mem_part2_mmio_remap/Makefile @@ -2,18 +2,32 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) +# Optional integration test name. +ifdef TEST + TEST_ARG = --test $(TEST) +else + TEST_ARG = --test '*' +endif + + -# BSP-specific arguments +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -44,20 +58,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -# Testing-specific arguments -ifdef TEST - ifeq ($(TEST),unit) - TEST_ARG = --lib - else - TEST_ARG = --test $(TEST) - endif -endif +KERNEL_ELF = target/$(TARGET)/release/kernel + -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -75,105 +87,110 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ - clippy clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -define KERNEL_TEST_RUNNER - #!/usr/bin/env bash - - TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') - TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') - - $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY -endef - -export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - $(call colorecho, "\nCompiling test(s) - $(BSP)") - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -182,10 +199,108 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot test_unit test_integration + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + +##------------------------------------------------------------------------------ +## Run integration test(s) +##------------------------------------------------------------------------------ +test_integration: + $(call colorecho, "\nCompiling integration test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +test: test_boot test_unit test_integration + +endif diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index 6cb7e16f1..d178e9c1e 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -3296,8 +3296,8 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/tests/02_exception_sync_page_fault exception::handling_init(); bsp::console::qemu_bring_up_console(); @@ -29,10 +29,30 @@ + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); - if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { - println!("MMU: {}", string); diff --git a/14_virtual_mem_part2_mmio_remap/src/lib.rs b/14_virtual_mem_part2_mmio_remap/src/lib.rs index 8a0297385..002e14b97 100644 --- a/14_virtual_mem_part2_mmio_remap/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/src/lib.rs @@ -165,8 +165,9 @@ extern "Rust" { /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); diff --git a/14_virtual_mem_part2_mmio_remap/src/panic_wait.rs b/14_virtual_mem_part2_mmio_remap/src/panic_wait.rs index e3a9ed8ab..130e952b8 100644 --- a/14_virtual_mem_part2_mmio_remap/src/panic_wait.rs +++ b/14_virtual_mem_part2_mmio_remap/src/panic_wait.rs @@ -23,12 +23,12 @@ fn _panic_print(args: fmt::Arguments) { #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } diff --git a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb index dfd6b16ea..16fb6c790 100644 --- a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb +++ b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rb @@ -8,6 +8,13 @@ TIMEOUT_SECS = 3 +# Error class for when expect times out. +class ExpectTimeoutError < StandardError + def initialize + super('Timeout while expecting string') + end +end + # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -16,7 +23,7 @@ def name def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end @@ -27,7 +34,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? end end @@ -38,7 +45,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? end end diff --git a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs index 84b744798..03058f5ef 100644 --- a/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/tests/00_console_sanity.rs @@ -31,12 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - // cpu::wait_forever(); - - // For some reason, in this test, rustc or the linker produces an empty binary when - // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time - // being, the following lines are just a workaround to fix this compiler/linker weirdness. - use libkernel::time::interface::TimeManager; - libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); - cpu::qemu_exit_success() + cpu::wait_forever(); } diff --git a/14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs b/14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs index 940866a03..7b8bba35a 100644 --- a/14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs +++ b/14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs @@ -26,8 +26,8 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { Err(string) => { diff --git a/14_virtual_mem_part2_mmio_remap/tests/boot_test_string.rb b/14_virtual_mem_part2_mmio_remap/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/14_virtual_mem_part2_mmio_remap/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/14_virtual_mem_part2_mmio_remap/tests/runner.rb b/14_virtual_mem_part2_mmio_remap/tests/runner.rb deleted file mode 100755 index 53116e088..000000000 --- a/14_virtual_mem_part2_mmio_remap/tests/runner.rb +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2019-2021 Andre Richter - -require 'English' -require 'pty' - -# Test base class. -class Test - INDENT = ' ' - - def print_border(status) - puts - puts "#{INDENT}-------------------------------------------------------------------" - puts status - puts "#{INDENT}-------------------------------------------------------------------\n\n\n" - end - - def print_error(error) - puts - print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") - end - - def print_success - print_border("#{INDENT}✅ Success: #{@test_name}") - end - - def print_output - puts "#{INDENT}-------------------------------------------------------------------" - print INDENT - print '🦀 ' - print @output.join.gsub("\n", "\n#{INDENT}") - end - - def finish(error) - print_output - - exit_code = if error - print_error(error) - false - else - print_success - true - end - - exit(exit_code) - end -end - -# Executes tests with console I/O. -class ConsoleTest < Test - def initialize(binary, qemu_cmd, test_name, console_subtests) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @console_subtests = console_subtests - @cur_subtest = 1 - @output = ["Running #{@console_subtests.length} console-based tests\n", - "-------------------------------------------------------------------\n\n"] - end - - def format_test_name(number, name) - formatted_name = "#{number.to_s.rjust(3)}. #{name}" - formatted_name.ljust(63, '.') - end - - def run_subtest(subtest, qemu_out, qemu_in) - @output << format_test_name(@cur_subtest, subtest.name) - - subtest.run(qemu_out, qemu_in) - - @output << "[ok]\n" - @cur_subtest += 1 - end - - def exec - error = false - - PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| - begin - @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } - rescue StandardError => e - error = e.message - end - - finish(error) - end - end -end - -# A wrapper around the bare QEMU invocation. -class RawTest < Test - MAX_WAIT_SECS = 5 - - def initialize(binary, qemu_cmd, test_name) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @output = [] - end - - def exec - error = 'Timed out waiting for test' - io = IO.popen(@qemu_cmd) - - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end - end - - finish(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Script entry point -##-------------------------------------------------------------------------------------------------- -binary = ARGV.last -test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] -console_test_file = "tests/#{test_name}.rb" -qemu_cmd = ARGV.join(' ') - -test_runner = if File.exist?(console_test_file) - load console_test_file - # subtest_collection is provided by console_test_file - ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) - else - RawTest.new(binary, qemu_cmd, test_name) - end - -test_runner.exec diff --git a/15_virtual_mem_part3_precomputed_tables/Makefile b/15_virtual_mem_part3_precomputed_tables/Makefile index feb65cef7..86a64ee5a 100644 --- a/15_virtual_mem_part3_precomputed_tables/Makefile +++ b/15_virtual_mem_part3_precomputed_tables/Makefile @@ -2,18 +2,32 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) +# Optional integration test name. +ifdef TEST + TEST_ARG = --test $(TEST) +else + TEST_ARG = --test '*' +endif + + -# BSP-specific arguments +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -44,20 +58,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -# Testing-specific arguments -ifdef TEST - ifeq ($(TEST),unit) - TEST_ARG = --lib - else - TEST_ARG = --test $(TEST) - endif -endif +KERNEL_ELF = target/$(TARGET)/release/kernel + -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -75,107 +87,112 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TT_TOOL = ruby translation_table_tool/main.rb +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ - clippy clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) - @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + @$(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -define KERNEL_TEST_RUNNER - #!/usr/bin/env bash - - TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') - TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') - - $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null - $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY -endef - -export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - $(call colorecho, "\nCompiling test(s) - $(BSP)") - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -184,10 +201,109 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot test_unit test_integration + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $$TEST_ELF > /dev/null + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + +##------------------------------------------------------------------------------ +## Run integration test(s) +##------------------------------------------------------------------------------ +test_integration: + $(call colorecho, "\nCompiling integration test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +test: test_boot test_unit test_integration + +endif diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index 581dcefca..b6a2b5b65 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -776,21 +776,29 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/Cargo.toml 15_virtual_mem_part3_precom diff -uNr 14_virtual_mem_part2_mmio_remap/Makefile 15_virtual_mem_part3_precomputed_tables/Makefile --- 14_virtual_mem_part2_mmio_remap/Makefile +++ 15_virtual_mem_part3_precomputed_tables/Makefile -@@ -112,6 +112,7 @@ +@@ -88,6 +88,7 @@ + -O binary + + EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) ++EXEC_TT_TOOL = ruby translation_table_tool/main.rb + EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb + EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +@@ -133,6 +134,7 @@ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) -+ @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) ++ @$(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $(KERNEL_ELF) - $(KERNEL_BIN): $(KERNEL_ELF) - @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) -@@ -134,6 +135,7 @@ + ##------------------------------------------------------------------------------ + ## Build the stripped kernel binary +@@ -271,6 +273,7 @@ TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') -+ $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null ++ $(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $$TEST_ELF > /dev/null $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY endef diff -uNr 14_virtual_mem_part2_mmio_remap/src/_arch/aarch64/cpu/boot.rs 15_virtual_mem_part3_precomputed_tables/src/_arch/aarch64/cpu/boot.rs @@ -1555,8 +1563,8 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/tests/02_exception_sync_page_fault.rs exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); - let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { - Err(string) => { diff --git a/15_virtual_mem_part3_precomputed_tables/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/src/lib.rs index 8a0297385..002e14b97 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/lib.rs @@ -165,8 +165,9 @@ extern "Rust" { /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); diff --git a/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs b/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs index e3a9ed8ab..130e952b8 100644 --- a/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs +++ b/15_virtual_mem_part3_precomputed_tables/src/panic_wait.rs @@ -23,12 +23,12 @@ fn _panic_print(args: fmt::Arguments) { #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } diff --git a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb index dfd6b16ea..16fb6c790 100644 --- a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb +++ b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rb @@ -8,6 +8,13 @@ TIMEOUT_SECS = 3 +# Error class for when expect times out. +class ExpectTimeoutError < StandardError + def initialize + super('Timeout while expecting string') + end +end + # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -16,7 +23,7 @@ def name def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end @@ -27,7 +34,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? end end @@ -38,7 +45,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? end end diff --git a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs index 84b744798..03058f5ef 100644 --- a/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/tests/00_console_sanity.rs @@ -31,12 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - // cpu::wait_forever(); - - // For some reason, in this test, rustc or the linker produces an empty binary when - // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time - // being, the following lines are just a workaround to fix this compiler/linker weirdness. - use libkernel::time::interface::TimeManager; - libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); - cpu::qemu_exit_success() + cpu::wait_forever(); } diff --git a/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs index 6a0c11f3c..e7fa8800c 100644 --- a/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs +++ b/15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs @@ -24,8 +24,8 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); println!("Writing beyond mapped area to address 9 GiB..."); let big_addr: u64 = 9 * 1024 * 1024 * 1024; diff --git a/15_virtual_mem_part3_precomputed_tables/tests/boot_test_string.rb b/15_virtual_mem_part3_precomputed_tables/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/15_virtual_mem_part3_precomputed_tables/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/15_virtual_mem_part3_precomputed_tables/tests/runner.rb b/15_virtual_mem_part3_precomputed_tables/tests/runner.rb deleted file mode 100755 index 53116e088..000000000 --- a/15_virtual_mem_part3_precomputed_tables/tests/runner.rb +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2019-2021 Andre Richter - -require 'English' -require 'pty' - -# Test base class. -class Test - INDENT = ' ' - - def print_border(status) - puts - puts "#{INDENT}-------------------------------------------------------------------" - puts status - puts "#{INDENT}-------------------------------------------------------------------\n\n\n" - end - - def print_error(error) - puts - print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") - end - - def print_success - print_border("#{INDENT}✅ Success: #{@test_name}") - end - - def print_output - puts "#{INDENT}-------------------------------------------------------------------" - print INDENT - print '🦀 ' - print @output.join.gsub("\n", "\n#{INDENT}") - end - - def finish(error) - print_output - - exit_code = if error - print_error(error) - false - else - print_success - true - end - - exit(exit_code) - end -end - -# Executes tests with console I/O. -class ConsoleTest < Test - def initialize(binary, qemu_cmd, test_name, console_subtests) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @console_subtests = console_subtests - @cur_subtest = 1 - @output = ["Running #{@console_subtests.length} console-based tests\n", - "-------------------------------------------------------------------\n\n"] - end - - def format_test_name(number, name) - formatted_name = "#{number.to_s.rjust(3)}. #{name}" - formatted_name.ljust(63, '.') - end - - def run_subtest(subtest, qemu_out, qemu_in) - @output << format_test_name(@cur_subtest, subtest.name) - - subtest.run(qemu_out, qemu_in) - - @output << "[ok]\n" - @cur_subtest += 1 - end - - def exec - error = false - - PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| - begin - @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } - rescue StandardError => e - error = e.message - end - - finish(error) - end - end -end - -# A wrapper around the bare QEMU invocation. -class RawTest < Test - MAX_WAIT_SECS = 5 - - def initialize(binary, qemu_cmd, test_name) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @output = [] - end - - def exec - error = 'Timed out waiting for test' - io = IO.popen(@qemu_cmd) - - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end - end - - finish(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Script entry point -##-------------------------------------------------------------------------------------------------- -binary = ARGV.last -test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] -console_test_file = "tests/#{test_name}.rb" -qemu_cmd = ARGV.join(' ') - -test_runner = if File.exist?(console_test_file) - load console_test_file - # subtest_collection is provided by console_test_file - ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) - else - RawTest.new(binary, qemu_cmd, test_name) - end - -test_runner.exec diff --git a/16_virtual_mem_part4_higher_half_kernel/Makefile b/16_virtual_mem_part4_higher_half_kernel/Makefile index feb65cef7..86a64ee5a 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Makefile +++ b/16_virtual_mem_part4_higher_half_kernel/Makefile @@ -2,18 +2,32 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) +# Optional integration test name. +ifdef TEST + TEST_ARG = --test $(TEST) +else + TEST_ARG = --test '*' +endif + + -# BSP-specific arguments +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -44,20 +58,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -# Testing-specific arguments -ifdef TEST - ifeq ($(TEST),unit) - TEST_ARG = --lib - else - TEST_ARG = --test $(TEST) - endif -endif +KERNEL_ELF = target/$(TARGET)/release/kernel + -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -75,107 +87,112 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel - -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot -DOCKER_ARG_DEV = --privileged -v /dev:/dev -DOCKER_ARG_NET = --network host +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TT_TOOL = ruby translation_table_tool/main.rb +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb + +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DIR_JTAG = -v $(shell pwd)/../X1_JTAG_boot:/work/X1_JTAG_boot +DOCKER_ARG_DEV = --privileged -v /dev:/dev +DOCKER_ARG_NET = --network host DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) -DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) +DOCKER_GDB = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) - DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) + DOCKER_JTAGBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_ARG_DIR_JTAG) $(DOCKER_IMAGE) DOCKER_OPENOCD = $(DOCKER_CMD_DEV) $(DOCKER_ARG_NET) $(DOCKER_IMAGE) else DOCKER_OPENOCD = echo "Not yet supported on non-Linux systems."; \# endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb -.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu test chainboot jtagboot openocd gdb gdb-opt0 \ - clippy clean readelf objdump nm check + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- +.PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) - @$(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $(KERNEL_ELF) + @$(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) -qemu test: +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) -define KERNEL_TEST_RUNNER - #!/usr/bin/env bash - - TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') - TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') - - $(DOCKER_TOOLS) ruby translation_table_tool/main.rb $(TARGET) $(BSP) $$TEST_ELF > /dev/null - $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY - $(DOCKER_TEST) ruby tests/runner.rb $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY -endef - -export KERNEL_TEST_RUNNER -test: FEATURES += --features test_build -test: - $(call colorecho, "\nCompiling test(s) - $(BSP)") - @mkdir -p target - @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh - @chmod +x target/kernel_test_runner.sh - @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) -jtagboot: - @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) - -openocd: - $(call colorecho, "\nLaunching OpenOCD") - @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) - -gdb: RUSTC_MISC_ARGS += -C debuginfo=2 -gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 -gdb gdb-opt0: $(KERNEL_ELF) - $(call colorecho, "\nLaunching GDB") - @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) - +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -184,10 +201,109 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Debugging targets +##-------------------------------------------------------------------------------------------------- +.PHONY: jtagboot openocd gdb gdb-opt0 + +##------------------------------------------------------------------------------ +## Push the JTAG boot image to the real HW target +##------------------------------------------------------------------------------ +jtagboot: + @$(DOCKER_JTAGBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(JTAG_BOOT_IMAGE) + +##------------------------------------------------------------------------------ +## Start OpenOCD session +##------------------------------------------------------------------------------ +openocd: + $(call colorecho, "\nLaunching OpenOCD") + @$(DOCKER_OPENOCD) openocd $(OPENOCD_ARG) + +##------------------------------------------------------------------------------ +## Start GDB session +##------------------------------------------------------------------------------ +gdb: RUSTC_MISC_ARGS += -C debuginfo=2 +gdb-opt0: RUSTC_MISC_ARGS += -C debuginfo=2 -C opt-level=0 +gdb gdb-opt0: $(KERNEL_ELF) + $(call colorecho, "\nLaunching GDB") + @$(DOCKER_GDB) gdb-multiarch -q $(KERNEL_ELF) + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot test_unit test_integration + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test_unit test_integration test: + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +##------------------------------------------------------------------------------ +## Helpers for unit and integration test targets +##------------------------------------------------------------------------------ +define KERNEL_TEST_RUNNER + #!/usr/bin/env bash + + TEST_ELF=$$(echo $$1 | sed -e 's/.*target/target/g') + TEST_BINARY=$$(echo $$1.img | sed -e 's/.*target/target/g') + + $(DOCKER_TOOLS) $(EXEC_TT_TOOL) $(TARGET) $(BSP) $$TEST_ELF > /dev/null + $(OBJCOPY_CMD) $$TEST_ELF $$TEST_BINARY + $(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_TEST_ARGS) -kernel $$TEST_BINARY +endef + +export KERNEL_TEST_RUNNER + +define test_prepare + @mkdir -p target + @echo "$$KERNEL_TEST_RUNNER" > target/kernel_test_runner.sh + @chmod +x target/kernel_test_runner.sh +endef + +test_unit test_integration: FEATURES += --features test_build + +##------------------------------------------------------------------------------ +## Run unit test(s) +##------------------------------------------------------------------------------ +test_unit: + $(call colorecho, "\nCompiling unit test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) --lib + +##------------------------------------------------------------------------------ +## Run integration test(s) +##------------------------------------------------------------------------------ +test_integration: + $(call colorecho, "\nCompiling integration test(s) - $(BSP)") + $(call test_prepare) + @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(TEST_CMD) $(TEST_ARG) + +test: test_boot test_unit test_integration + +endif diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index fa594703a..15629a2ad 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -617,8 +617,8 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_f --- 15_virtual_mem_part3_precomputed_tables/tests/02_exception_sync_page_fault.rs +++ 16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs @@ -27,8 +27,8 @@ + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); - println!("Writing beyond mapped area to address 9 GiB..."); - let big_addr: u64 = 9 * 1024 * 1024 * 1024; diff --git a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs index 7b5cdb09b..c75a96eaf 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/lib.rs @@ -160,8 +160,9 @@ pub fn version() -> &'static str { /// The default runner for unit tests. pub fn test_runner(tests: &[&test_types::UnitTest]) { + // This line will be printed as the test header. println!("Running {} tests", tests.len()); - println!("-------------------------------------------------------------------\n"); + for (i, test) in tests.iter().enumerate() { print!("{:>3}. {:.<58}", i + 1, test.name); diff --git a/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs b/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs index e3a9ed8ab..130e952b8 100644 --- a/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs +++ b/16_virtual_mem_part4_higher_half_kernel/src/panic_wait.rs @@ -23,12 +23,12 @@ fn _panic_print(args: fmt::Arguments) { #[linkage = "weak"] #[no_mangle] fn _panic_exit() -> ! { - #[cfg(not(test_build))] + #[cfg(not(feature = "test_build"))] { cpu::wait_forever() } - #[cfg(test_build)] + #[cfg(feature = "test_build")] { cpu::qemu_exit_failure() } diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb index dfd6b16ea..16fb6c790 100644 --- a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb +++ b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rb @@ -8,6 +8,13 @@ TIMEOUT_SECS = 3 +# Error class for when expect times out. +class ExpectTimeoutError < StandardError + def initialize + super('Timeout while expecting string') + end +end + # Verify sending and receiving works as expected. class TxRxHandshake def name @@ -16,7 +23,7 @@ def name def run(qemu_out, qemu_in) qemu_in.write_nonblock('ABC') - raise('TX/RX test failed') if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('OK1234', TIMEOUT_SECS).nil? end end @@ -27,7 +34,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_written reported wrong') if qemu_out.expect('6', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('6', TIMEOUT_SECS).nil? end end @@ -38,7 +45,7 @@ def name end def run(qemu_out, _qemu_in) - raise('chars_read reported wrong') if qemu_out.expect('3', TIMEOUT_SECS).nil? + raise ExpectTimeoutError if qemu_out.expect('3', TIMEOUT_SECS).nil? end end diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs index 84b744798..03058f5ef 100644 --- a/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/tests/00_console_sanity.rs @@ -31,12 +31,5 @@ unsafe fn kernel_init() -> ! { print!("{}", console().chars_read()); // The QEMU process running this test will be closed by the I/O test harness. - // cpu::wait_forever(); - - // For some reason, in this test, rustc or the linker produces an empty binary when - // wait_forever() is used. Calling qemu_exit_success() fixes this behavior. So for the time - // being, the following lines are just a workaround to fix this compiler/linker weirdness. - use libkernel::time::interface::TimeManager; - libkernel::time::time_manager().spin_for(core::time::Duration::from_secs(3600)); - cpu::qemu_exit_success() + cpu::wait_forever(); } diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs b/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs index 30d420a75..47ef31fa5 100644 --- a/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs +++ b/16_virtual_mem_part4_higher_half_kernel/tests/02_exception_sync_page_fault.rs @@ -24,8 +24,8 @@ unsafe fn kernel_init() -> ! { exception::handling_init(); bsp::console::qemu_bring_up_console(); + // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); - println!("-------------------------------------------------------------------\n"); println!("Writing to bottom of address space to address 1 GiB..."); let big_addr: u64 = 1 * 1024 * 1024 * 1024; diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/boot_test_string.rb b/16_virtual_mem_part4_higher_half_kernel/tests/boot_test_string.rb new file mode 100644 index 000000000..f778b3d8c --- /dev/null +++ b/16_virtual_mem_part4_higher_half_kernel/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Echoing input now' diff --git a/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb b/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb deleted file mode 100755 index 53116e088..000000000 --- a/16_virtual_mem_part4_higher_half_kernel/tests/runner.rb +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# SPDX-License-Identifier: MIT OR Apache-2.0 -# -# Copyright (c) 2019-2021 Andre Richter - -require 'English' -require 'pty' - -# Test base class. -class Test - INDENT = ' ' - - def print_border(status) - puts - puts "#{INDENT}-------------------------------------------------------------------" - puts status - puts "#{INDENT}-------------------------------------------------------------------\n\n\n" - end - - def print_error(error) - puts - print_border("#{INDENT}❌ Failure: #{error}: #{@test_name}") - end - - def print_success - print_border("#{INDENT}✅ Success: #{@test_name}") - end - - def print_output - puts "#{INDENT}-------------------------------------------------------------------" - print INDENT - print '🦀 ' - print @output.join.gsub("\n", "\n#{INDENT}") - end - - def finish(error) - print_output - - exit_code = if error - print_error(error) - false - else - print_success - true - end - - exit(exit_code) - end -end - -# Executes tests with console I/O. -class ConsoleTest < Test - def initialize(binary, qemu_cmd, test_name, console_subtests) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @console_subtests = console_subtests - @cur_subtest = 1 - @output = ["Running #{@console_subtests.length} console-based tests\n", - "-------------------------------------------------------------------\n\n"] - end - - def format_test_name(number, name) - formatted_name = "#{number.to_s.rjust(3)}. #{name}" - formatted_name.ljust(63, '.') - end - - def run_subtest(subtest, qemu_out, qemu_in) - @output << format_test_name(@cur_subtest, subtest.name) - - subtest.run(qemu_out, qemu_in) - - @output << "[ok]\n" - @cur_subtest += 1 - end - - def exec - error = false - - PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| - begin - @console_subtests.each { |t| run_subtest(t, qemu_out, qemu_in) } - rescue StandardError => e - error = e.message - end - - finish(error) - end - end -end - -# A wrapper around the bare QEMU invocation. -class RawTest < Test - MAX_WAIT_SECS = 5 - - def initialize(binary, qemu_cmd, test_name) - super() - - @binary = binary - @qemu_cmd = qemu_cmd - @test_name = test_name - @output = [] - end - - def exec - error = 'Timed out waiting for test' - io = IO.popen(@qemu_cmd) - - while IO.select([io], nil, nil, MAX_WAIT_SECS) - begin - @output << io.read_nonblock(1024) - rescue EOFError - io.close - error = $CHILD_STATUS.to_i != 0 - break - end - end - - finish(error) - end -end - -##-------------------------------------------------------------------------------------------------- -## Script entry point -##-------------------------------------------------------------------------------------------------- -binary = ARGV.last -test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] -console_test_file = "tests/#{test_name}.rb" -qemu_cmd = ARGV.join(' ') - -test_runner = if File.exist?(console_test_file) - load console_test_file - # subtest_collection is provided by console_test_file - ConsoleTest.new(binary, qemu_cmd, test_name, subtest_collection) - else - RawTest.new(binary, qemu_cmd, test_name) - end - -test_runner.exec diff --git a/X1_JTAG_boot/Makefile b/X1_JTAG_boot/Makefile index b5a56d075..8336ccb72 100644 --- a/X1_JTAG_boot/Makefile +++ b/X1_JTAG_boot/Makefile @@ -2,18 +2,25 @@ ## ## Copyright (c) 2018-2021 Andre Richter -include ../utils/color.mk.in +include ../common/color.mk.in -# Default to the RPi3 +##-------------------------------------------------------------------------------------------------- +## Optional, user-provided configuration values +##-------------------------------------------------------------------------------------------------- + +# Default to the RPi3. BSP ?= rpi3 # Default to a serial device name that is common in Linux. DEV_SERIAL ?= /dev/ttyUSB0 -# Query the host system's kernel name -UNAME_S = $(shell uname -s) -# BSP-specific arguments + +##-------------------------------------------------------------------------------------------------- +## Hardcoded configuration values +##-------------------------------------------------------------------------------------------------- + +# BSP-specific arguments. ifeq ($(BSP),rpi3) TARGET = aarch64-unknown-none-softfloat KERNEL_BIN = kernel8.img @@ -38,11 +45,18 @@ else ifeq ($(BSP),rpi4) RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif -# Export for build.rs +QEMU_MISSING_STRING = "This board is not yet supported for QEMU." + +# Export for build.rs. export LINKER_FILE -QEMU_MISSING_STRING = "This board is not yet supported for QEMU." +KERNEL_ELF = target/$(TARGET)/release/kernel + + +##-------------------------------------------------------------------------------------------------- +## Command building blocks +##-------------------------------------------------------------------------------------------------- RUSTFLAGS = -C link-arg=-T$(LINKER_FILE) $(RUSTC_MISC_ARGS) RUSTFLAGS_PEDANTIC = $(RUSTFLAGS) -D warnings -D missing_docs @@ -59,64 +73,103 @@ OBJCOPY_CMD = rust-objcopy \ --strip-all \ -O binary -KERNEL_ELF = target/$(TARGET)/release/kernel +EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) +EXEC_TEST_DISPATCH = ruby ../common/tests/dispatch.rb +EXEC_MINIPUSH = ruby ../common/serial/minipush.rb -DOCKER_IMAGE = rustembedded/osdev-utils -DOCKER_CMD = docker run --rm -v $(shell pwd):/work/tutorial -w /work/tutorial -DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i -t -DOCKER_ARG_DIR_UTILS = -v $(shell pwd)/../utils:/work/utils -DOCKER_ARG_DEV = --privileged -v /dev:/dev +##------------------------------------------------------------------------------ +## Dockerization +##------------------------------------------------------------------------------ +DOCKER_IMAGE = rustembedded/osdev-utils +DOCKER_CMD = docker run -t --rm -v $(shell pwd):/work/tutorial -w /work/tutorial +DOCKER_CMD_INTERACT = $(DOCKER_CMD) -i +DOCKER_ARG_DIR_COMMON = -v $(shell pwd)/../common:/work/common +DOCKER_ARG_DEV = --privileged -v /dev:/dev DOCKER_QEMU = $(DOCKER_CMD_INTERACT) $(DOCKER_IMAGE) DOCKER_TOOLS = $(DOCKER_CMD) $(DOCKER_IMAGE) +DOCKER_TEST = $(DOCKER_CMD) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) -# Dockerize commands that require USB device passthrough only on Linux -ifeq ($(UNAME_S),Linux) +# Dockerize commands, which require USB device passthrough, only on Linux. +ifeq ($(shell uname -s),Linux) DOCKER_CMD_DEV = $(DOCKER_CMD_INTERACT) $(DOCKER_ARG_DEV) - DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_UTILS) $(DOCKER_IMAGE) + DOCKER_CHAINBOOT = $(DOCKER_CMD_DEV) $(DOCKER_ARG_DIR_COMMON) $(DOCKER_IMAGE) endif -EXEC_QEMU = $(QEMU_BINARY) -M $(QEMU_MACHINE_TYPE) -EXEC_MINIPUSH = ruby ../utils/minipush.rb + +##-------------------------------------------------------------------------------------------------- +## Targets +##-------------------------------------------------------------------------------------------------- .PHONY: all $(KERNEL_ELF) $(KERNEL_BIN) doc qemu chainboot clippy clean readelf objdump nm check all: $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the kernel ELF +##------------------------------------------------------------------------------ $(KERNEL_ELF): $(call colorecho, "\nCompiling kernel - $(BSP)") @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(RUSTC_CMD) +##------------------------------------------------------------------------------ +## Build the stripped kernel binary +##------------------------------------------------------------------------------ $(KERNEL_BIN): $(KERNEL_ELF) @$(OBJCOPY_CMD) $(KERNEL_ELF) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Build the documentation +##------------------------------------------------------------------------------ doc: $(call colorecho, "\nGenerating docs") @$(DOC_CMD) --document-private-items --open -ifeq ($(QEMU_MACHINE_TYPE),) +##------------------------------------------------------------------------------ +## Run the kernel in QEMU +##------------------------------------------------------------------------------ +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + qemu: $(call colorecho, "\n$(QEMU_MISSING_STRING)") -else + +else # QEMU is supported. + qemu: $(KERNEL_BIN) $(call colorecho, "\nLaunching QEMU") @$(DOCKER_QEMU) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + endif +##------------------------------------------------------------------------------ +## Push the kernel to the real HW target +##------------------------------------------------------------------------------ chainboot: $(KERNEL_BIN) @$(DOCKER_CHAINBOOT) $(EXEC_MINIPUSH) $(DEV_SERIAL) $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run clippy +##------------------------------------------------------------------------------ clippy: @RUSTFLAGS="$(RUSTFLAGS_PEDANTIC)" $(CLIPPY_CMD) +##------------------------------------------------------------------------------ +## Clean +##------------------------------------------------------------------------------ clean: rm -rf target $(KERNEL_BIN) +##------------------------------------------------------------------------------ +## Run readelf +##------------------------------------------------------------------------------ readelf: $(KERNEL_ELF) $(call colorecho, "\nLaunching readelf") @$(DOCKER_TOOLS) $(READELF_BINARY) --headers $(KERNEL_ELF) +##------------------------------------------------------------------------------ +## Run objdump +##------------------------------------------------------------------------------ objdump: $(KERNEL_ELF) $(call colorecho, "\nLaunching objdump") @$(DOCKER_TOOLS) $(OBJDUMP_BINARY) --disassemble --demangle \ @@ -125,10 +178,40 @@ objdump: $(KERNEL_ELF) --section .got \ $(KERNEL_ELF) | rustfilt +##------------------------------------------------------------------------------ +## Run nm +##------------------------------------------------------------------------------ nm: $(KERNEL_ELF) $(call colorecho, "\nLaunching nm") @$(DOCKER_TOOLS) $(NM_BINARY) --demangle --print-size $(KERNEL_ELF) | sort | rustfilt -# For rust-analyzer +##------------------------------------------------------------------------------ +## Helper target for rust-analyzer +##------------------------------------------------------------------------------ check: @RUSTFLAGS="$(RUSTFLAGS)" $(CHECK_CMD) --message-format=json + + + +##-------------------------------------------------------------------------------------------------- +## Testing targets +##-------------------------------------------------------------------------------------------------- +.PHONY: test test_boot + +ifeq ($(QEMU_MACHINE_TYPE),) # QEMU is not supported for the board. + +test_boot test : + $(call colorecho, "\n$(QEMU_MISSING_STRING)") + +else # QEMU is supported. + +##------------------------------------------------------------------------------ +## Run boot test +##------------------------------------------------------------------------------ +test_boot: $(KERNEL_BIN) + $(call colorecho, "\nBoot test - $(BSP)") + @$(DOCKER_TEST) $(EXEC_TEST_DISPATCH) $(EXEC_QEMU) $(QEMU_RELEASE_ARGS) -kernel $(KERNEL_BIN) + +test: test_boot + +endif diff --git a/X1_JTAG_boot/tests/boot_test_string.rb b/X1_JTAG_boot/tests/boot_test_string.rb new file mode 100644 index 000000000..029dbd068 --- /dev/null +++ b/X1_JTAG_boot/tests/boot_test_string.rb @@ -0,0 +1,3 @@ +# frozen_string_literal: true + +EXPECTED_PRINT = 'Please connect over JTAG now' diff --git a/utils/color.mk.in b/common/color.mk.in similarity index 100% rename from utils/color.mk.in rename to common/color.mk.in diff --git a/utils/minipush.rb b/common/serial/minipush.rb similarity index 83% rename from utils/minipush.rb rename to common/serial/minipush.rb index a9ee9bb2a..b18c49832 100755 --- a/utils/minipush.rb +++ b/common/serial/minipush.rb @@ -14,19 +14,19 @@ class ProtocolError < StandardError; end # The main class class MiniPush < MiniTerm - def initialize(serial_name, binary_image_path) + def initialize(serial_name, payload_path) super(serial_name) @name_short = 'MP' # override - @binary_image_path = binary_image_path - @binary_size = nil - @binary_image = nil + @payload_path = payload_path + @payload_size = nil + @payload_data = nil end private # The three characters signaling the request token form the consecutive sequence "\x03\x03\x03". - def wait_for_binary_request + def wait_for_payload_request puts "[#{@name_short}] 🔌 Please power the target now" # Timeout for the request token starts after the first sign of life was received. @@ -54,27 +54,28 @@ def wait_for_binary_request end end - def load_binary - @binary_size = File.size(@binary_image_path) - @binary_image = File.binread(@binary_image_path) + def load_payload + @payload_size = File.size(@payload_path) + @payload_data = File.binread(@payload_path) end def send_size - @target_serial.print([@binary_size].pack('L<')) + @target_serial.print([@payload_size].pack('L<')) raise ProtocolError if @target_serial.read(2) != 'OK' end - def send_binary + def send_payload pb = ProgressBar.create( - total: @binary_size, + total: @payload_size, format: "[#{@name_short}] ⏩ Pushing %k KiB %b🦀%i %p%% %r KiB/s %a", rate_scale: ->(rate) { rate / 1024 }, - length: 92 + length: 92, + output: $stdout ) # Send in 512 byte chunks. while pb.progress < pb.total - part = @binary_image.slice(pb.progress, 512) + part = @payload_data.slice(pb.progress, 512) pb.progress += @target_serial.write(part) end end @@ -95,10 +96,10 @@ def handle_reconnect(_error) # override def run open_serial - wait_for_binary_request - load_binary + wait_for_payload_request + load_payload send_size - send_binary + send_payload terminal rescue ConnectionError, EOFError, Errno::EIO, ProtocolError, Timeout::Error => e handle_reconnect(e) diff --git a/utils/minipush/progressbar_patch.rb b/common/serial/minipush/progressbar_patch.rb similarity index 100% rename from utils/minipush/progressbar_patch.rb rename to common/serial/minipush/progressbar_patch.rb diff --git a/utils/miniterm.rb b/common/serial/miniterm.rb similarity index 99% rename from utils/miniterm.rb rename to common/serial/miniterm.rb index 10cc4bc5d..06e7efd1d 100755 --- a/utils/miniterm.rb +++ b/common/serial/miniterm.rb @@ -7,8 +7,9 @@ require 'rubygems' require 'bundler/setup' -require 'io/console' + require 'colorize' +require 'io/console' require 'serialport' SERIAL_BAUD = 921_600 diff --git a/common/tests/boot_test.rb b/common/tests/boot_test.rb new file mode 100644 index 000000000..1f5301f96 --- /dev/null +++ b/common/tests/boot_test.rb @@ -0,0 +1,75 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2021 Andre Richter + +require_relative 'test' +require 'timeout' + +# Check for an expected string when booting the kernel in QEMU. +class BootTest < Test + MAX_WAIT_SECS = 5 + + def initialize(qemu_cmd, expected_print) + super() + + @qemu_cmd = qemu_cmd + @expected_print = expected_print + + @test_name = 'Boot test' + @test_description = "Checking for the string: '#{@expected_print}'" + @test_output = [] + @test_error = nil + end + + private + + def expected_string_observed?(qemu_output) + qemu_output.join.include?(@expected_print) + end + + # Convert the recorded output to an array of lines. + def post_process_and_add_output(qemu_output) + @test_output += qemu_output.join.split("\n") + end + + # override + def setup + @qemu_serial = IO.popen(@qemu_cmd, err: '/dev/null') + @qemu_pid = @qemu_serial.pid + end + + # override + def cleanup + Timeout.timeout(MAX_WAIT_SECS) do + Process.kill('TERM', @qemu_pid) + Process.wait + end + rescue StandardError => e + puts 'QEMU graceful shutdown didn\'t work. Skipping it.' + puts e + end + + def run_concrete_test + qemu_output = [] + Timeout.timeout(MAX_WAIT_SECS) do + while IO.select([@qemu_serial]) + qemu_output << @qemu_serial.read_nonblock(1024) + + if expected_string_observed?(qemu_output) + @test_error = false + break + end + end + end + rescue EOFError + @test_error = 'QEMU quit unexpectedly' + rescue Timeout::Error + @test_error = 'Timed out waiting for magic string' + rescue StandardError => e + @test_error = e.message + ensure + post_process_and_add_output(qemu_output) + end +end diff --git a/common/tests/console_io_test.rb b/common/tests/console_io_test.rb new file mode 100644 index 000000000..66822a4e9 --- /dev/null +++ b/common/tests/console_io_test.rb @@ -0,0 +1,48 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'pty' +require_relative 'test' + +# A test doing console I/O with the QEMU binary. +class ConsoleIOTest < Test + def initialize(qemu_cmd, test_name, console_subtests) + super() + + @qemu_cmd = qemu_cmd + @console_subtests = console_subtests + + @test_name = test_name + @test_description = "Running #{@console_subtests.length} console I/O tests" + @test_output = [] + @test_error = nil + end + + private + + def format_test_name(number, name) + formatted_name = "#{number.to_s.rjust(3)}. #{name}" + formatted_name.ljust(63, '.') + end + + def run_subtest(subtest, test_id, qemu_out, qemu_in) + @test_output << format_test_name(test_id, subtest.name) + subtest.run(qemu_out, qemu_in) + @test_output.last.concat('[ok]') + end + + def run_concrete_test + @test_error = false + + PTY.spawn(@qemu_cmd) do |qemu_out, qemu_in| + @console_subtests.each_with_index do |t, i| + run_subtest(t, i + 1, qemu_out, qemu_in) + end + rescue StandardError => e + @test_error = e.message + end + end +end diff --git a/common/tests/dispatch.rb b/common/tests/dispatch.rb new file mode 100755 index 000000000..148272657 --- /dev/null +++ b/common/tests/dispatch.rb @@ -0,0 +1,35 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require_relative 'boot_test' +require_relative 'console_io_test' +require_relative 'exit_code_test' + +qemu_cmd = ARGV.join(' ') +binary = ARGV.last +test_name = binary.gsub(%r{.*deps/}, '').split('-')[0] + +case test_name +when 'kernel8.img' + load 'tests/boot_test_string.rb' # provides 'EXPECTED_PRINT' + BootTest.new(qemu_cmd, EXPECTED_PRINT).run # Doesn't return + +when 'libkernel' + ExitCodeTest.new(qemu_cmd, 'Kernel library unit tests').run # Doesn't return + +else + console_test_file = "tests/#{test_name}.rb" + test_name.concat('.rs') + test = if File.exist?(console_test_file) + load console_test_file # provides 'subtest_collection' + ConsoleIOTest.new(qemu_cmd, test_name, subtest_collection) + else + ExitCodeTest.new(qemu_cmd, test_name) + end + + test.run # Doesn't return +end diff --git a/common/tests/exit_code_test.rb b/common/tests/exit_code_test.rb new file mode 100644 index 000000000..2f14ab782 --- /dev/null +++ b/common/tests/exit_code_test.rb @@ -0,0 +1,52 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +require 'English' +require_relative 'test' + +# A test that only inspects the exit code of the QEMU binary. +class ExitCodeTest < Test + MAX_WAIT_SECS = 5 + + def initialize(qemu_cmd, test_name) + super() + + @qemu_cmd = qemu_cmd + + @test_name = test_name + @test_description = nil + @test_output = [] + @test_error = nil + end + + private + + # Convert the recorded output to an array of lines, and extract the test description. + def post_process_output + @test_output = @test_output.join.split("\n") + @test_description = @test_output.shift + end + + # override + def setup + @qemu_serial = IO.popen(@qemu_cmd) + end + + def run_concrete_test + Timeout.timeout(MAX_WAIT_SECS) do + @test_output << @qemu_serial.read_nonblock(1024) while IO.select([@qemu_serial]) + end + rescue EOFError + @qemu_serial.close + @test_error = $CHILD_STATUS.to_i.zero? ? false : 'QEMU exit status != 0' + rescue Timeout::Error + @test_error = 'Timed out waiting for test' + rescue StandardError => e + @test_error = e.message + ensure + post_process_output + end +end diff --git a/common/tests/test.rb b/common/tests/test.rb new file mode 100644 index 000000000..b0f67f3a1 --- /dev/null +++ b/common/tests/test.rb @@ -0,0 +1,82 @@ +# frozen_string_literal: true + +# SPDX-License-Identifier: MIT OR Apache-2.0 +# +# Copyright (c) 2019-2021 Andre Richter + +# Test base class. +class Test + INDENT = ' ' + + def initialize + # Template instance variables. + # @test_name + # @test_description + # @test_output + # @test_error + end + + private + + def print_border(content) + puts "#{INDENT}-------------------------------------------------------------------" + puts content + puts "#{INDENT}-------------------------------------------------------------------" + end + + def print_header + print_border("#{INDENT}🦀 #{@test_description}") + puts + end + + def print_footer_error(error) + puts + print_border("#{INDENT}❌ Failure: #{@test_name}: #{error}") + puts + puts + end + + def print_footer_success + puts + print_border("#{INDENT}✅ Success: #{@test_name}") + puts + puts + end + + # Expects @test_output the be an array of lines, without '\n' + def print_output + @test_output.each { |x| print "#{INDENT}#{x}\n" } + end + + # Template method. + def setup; end + + # Template method. + def cleanup; end + + # Template method. + def run_concrete_test + raise('Not implemented') + end + + public + + def run + setup + run_concrete_test + cleanup + + print_header + print_output + + exit_code = if @test_error + print_footer_error(@test_error) + false + else + print_footer_success + true + end + + exit(exit_code) + end +end diff --git a/devtool_completion.bash b/devtool_completion.bash index 3831425cd..ce4c67b00 100755 --- a/devtool_completion.bash +++ b/devtool_completion.bash @@ -1,3 +1,3 @@ #!/usr/bin/env bash -complete -W "clean clippy copyright diff fmt fmt_check make make_xtra misspell ready_for_publish ready_for_publish_no_rust rubocop test_integration test_unit test_xtra update" devtool +complete -W "clean clippy copyright diff fmt fmt_check make make_xtra misspell ready_for_publish ready_for_publish_no_rust rubocop test test_boot test_integration test_unit test_xtra update" devtool diff --git a/doc/12_demo.gif b/doc/12_demo.gif new file mode 100644 index 0000000000000000000000000000000000000000..8afa23c12f22988cedfbcae2712baf481f841221 GIT binary patch literal 224703 zcmeFYbx_<pwh-s`Ha{^Rud$vS)Oea^16WaVY}_)Yx~K%i#;I{*rRk934j{05(z3ZIsN z;HZk2nt|eEh!U5CuFQ*`l!2|LgzNc@M#qWPGmAk?g^`+$S??PQ8aWFaA1g8~t8fIX zlnWa&H5&>i8>$E!nhF~x9~-6)8>TfIwj3J)EgP{78>uiGn-)7V4Lhk8JDE2--6wYD zuk0K#?7UVS$kZGp${cLi974$)>Y1EqG@P<%oMr`F5?NgOsa&SPTxOYE7DZf^1>6F0 z+;T-c_<}s7q`dT2yxMkrsEmA~`uwUD0%(*1lq!NEv4R?y!pPJjl7u2gEutt?qU_|N z_9$Yi!eUmf66n+tN-2`U_L7nC(vHKj6nt`+RPv&v3aHcy__PXc*ovrBipCX6-ZM&3 z=t^k_DyhgS8So#z%zc1>RckcW1SHjU!nA#-bnI(&!q#-rs6IyhFd(Bds5dc6w>B{f zGfCJp!=*O!s4{Q2w!omaz$Lf+Hfo!;WhW?V=bY}K=jc#);l#o2{3+C})5E>b+l!Fe zOV`|+p3diM;pcA+UxvPZZF>&zjR~~%3Gqt{bqWmEkd0_KjAW#Z8vYf{fEzvd7?ao> zlj0XE#2PEj2QkrxS$xbL#umoVVoS&$p$n*p*oQm6^rW)z$S7#r46CjjiL&rJ1ev#I3XE?P#^# zUvCE~`Uk~sN0U9r7mp|Rzfb-^&qUBKTm7$wvagG+t~aKy&yMa_JMNEv+`qm(7{fg5 zFFiN8Kezilk3>E{Jv_hs{@L#Hb0FyD?d{iC_^n|`V9SZ27UVpeftfC{((ZF6i^fuX-y4DaaCy!7B*NQ005wP0l^{v z{bl<51^gRL!v7aZ{ufF9XC?_E6u^o|ELWG;AB^~p(O|4De=r=3q_7q*H4qVj&-v-c zSbgDW9Jyqqy3f={g&&SUz^Dkr-0##Yk+$m8u_k zT=z3wldZMey#cT&q)KgdyMqz$-g}l0EG=o2P&3IWghZv*h|?>fpZ15M5pmOY@pTfg zMH1!irpThm z4w2)JX1@-!{dhcI#$gy4filYlhQQ%*f10WzHxrUYW)}Hcaw{Gxhb$mhRHtgp?*IB{ zeRTn`wD^cr270+lriuh#2KK#M7t0M8ppnWn!m%pb4I}nE*$pQTqc2b&&fddDN%5GM z#`2_T3dTxt@gl(21UGM58W~DUpx(=i?TS%*({7_lZHWznL_dWDLTdb+$z{pc0zRS% z5{K(1X=vLnBHfk47boS}B4MYPe)O43z+BlWioiSK-ph0zVK~lmU$H&TR!;V&#A9eE zn?qL}qTjbPR5dO6z&4a-u7RD_SD3{0Q%o3cFz3G>0P;X~6y3Qj|&q;247riZ5q8OYfUEF>uC)^1^a1DK1q3QoefK2 zXw56iFc%ps0NQnn;-bZ~KMr#VZo!miJK))HAz}uuS)~5hYz!xw&ywnC#is2|M<4~n*!6jDi6%Ox^5rtq zQ4Ho%BIz*?p)+dEHwr-wJaD%wPZneWXPPH?Nf~kz=#~h>8g#=8cIvuq!6*{CpP@St zz6aN#yqD@lsCB=sr*fM6&5)KO!3Ku8`DOz5y-@XSrnUEx0>g6@HWvyG>b$<4v4LTb ze<%rOZ;FzIX1&14{A}Nl}t2^{m8M0uaI{*Kw2{< z@{Mxo2ui)jYZvbn8rTqR*&-G~-lXg-yhL&Ir=*$y_dwb=VIDVuJg-Nuo_rmP65eSP z=vq9%%*-hVPrkzx7>%M;aGEE<$5!+q=9*l<4P`Roxh60VNoku<9gTw~uLP%Dj!TZ* zM`;IUzP$n-H?41s9zcpGG!fq@pkl2((mXUl6)+c1Hb zo|L24lTlDkSGO)xDxL)`Fso#|_sdzDVuCGiQktJnhIAB)-_cL(;!Jr(yBFC}%JX6P z*YVI#mGHEVPdk2or(*bB9QO-fA^A;pSmNXw9xH&Jc%lqtnh=OF{-VNSl;>#F7R2l& z1`T;nJ#Qv?K`h(~o(*hD%(-!25kmOfK;ALOLHQ0*;N34`_n`i7p!GiDA_{b4E+7Ry z1Q;LtFv5j9lmh2dn6wuW&yiwhn<${nDa%h0XhM#_xmlP--$$dVzyM!cpKuys!B8-O z3}SnaS7c-ok4daiUe?H71Kk^69i-TbVb(0O=Wo|1~nc+>YjR^2C8=3=g z#?z?+9ho@Kn)1bN3WR3PtcD!VjB1Lf(lm^7Rb$z2Q<7?r>N9tSi;>?Pnowif>URV|4d#k z8p-V!(p0H8dQP}%2!P%Ccy3%BvUL4qLQbV>s}{3FcKz7{l7YplHjYAgzoAne;4${~ zHqsd9gxB~tb8UdcFKr}`RQYFa+V+cx%~nTVgA+IlHRHqDP!j;gu1zS8z=bS0Ds6~> z|H52nY7e*b=P=uh9G}J)I2tBd0fRP}m$`Q4@BV68>}NqSsr$3OJz`>P;P0RAM-F~P zXhh^!A0v83Q4#m5=HdZ4^23-&AV9qF;Ef=xq`qWU1)gx8t;m+Fli}WJ2WVDU1~zFV z$~&jLCi24k04sw}3!%<9S~*LIAWNM<*$qYV^=fetlQXGs$*czoHmS`|y#BXdyFmNLxzXZUO=U~W#FrB9t?L|KSrCi-RjP-z}NRpcIbYA zQo$PuFOg?(LNSc0QG|2mQizBKw-{fh%<+BFuHcRgufH7srdf@y??X_z2_otWiGTc# z5$8MfrrXleqLt)D{VYkwC#%7PzXzAa8HA@QgoJzC#_aVyl>CSGM+0b9_3(lfnOOJS z2QnS;G4`6<9=LVYjE^S;nnqzTf>3McNKbZo>z__KFwR6!ebY|m9H*q7Lxu+t{b zSrapPR9RIAe|_ICRG5dUV(=#7tGe95nLdb%ClU7I5^%5*7#X-gNM`&56@Fa0oKJD z$y8_V3C4;t2!_Jpkeih5r$l-m*Fyk{lj$q z-|$>E_CCBd!Kv24^&Y}vkpuTbEZ@7kg$H@P&qGLoz|&hPT*7(4$D7{e>yT2sueS#4 z@&(FQ)2X zjn9JJ7yD~KBMubYO-HjBLH_v(6ibcg8 zI;9fgrjA%5MWP73flE1%Nh9F;f;%7O-0tG4XOH26`S}RwNse*}c=u-xE#8E-Ko6_f zCj$z}sQiXW;gitfli8M@+2I5JQkHqMcXgVQ~5DWVTl$eM1=!yCZw~+H= zp|Tgaxjj?lSD_eT5qxR_Rz97sBX$UXv!$qVcwe{jHnvCniK*TD{2Me8wfFl8H+kPD!Vo+>6bF4GqPkS zROBa0NC)+dRKJGYyWAHfO+}W-_a*Xt3t%iQLbL&mRaA+{&@Wab6?rE~(^oU{ej^tT zrTqlHc=E1lsi;H9)!|D4eJen$D0?`KL79)5c!&Zp=IzVmRSr@6h5%2IIPVq82|q_& z@aJ}z2^N3yAr!Agd|yg147(R*!X%vvPZW0fvGn%L07c%osJ|4g9g{BujXfS?5CZ=} zy@4*XA)Eo+epbmL*vO^dI0BU@3h@RnD2Tk<;lMtJ1ZY0cBO%3YOhC z-!zNNNwAxO_VmifNR;1$(UYkX_Th9o!TbCq?5+Xi>;;HtG6z8dSVS4-t+ZW;wJ>*S z#~Z1Gi41lT9%m`wixmME)BH=_Pa&UD^@ph|WeD#&6mKP32`jnEy##&qgGw}O6VAZl zoeak1ZMH<;Sm&5+xKeW?+klNNA1HI3zjMTNwo2l+qvgYT0#MEX@7};>RJe>4lN~iE zoo{DgcDu%o&d#pyokTEI{C58EHZ&z=dZhcLA~4c;+@$*qEr=0VWTS2?;?g+c()xA0 zhZTk!i^ciui3 ze+!%f{Wj5{-l!-bZeUzq0ttQq)~z&4CuPVBM~tU+3}QU+@r3hD5e)c41(Vf>qMvgd z+IJTk1^L|o@6un%)t9Q-xA_halvMlOOUFqMOno6c8`}HJPUFvc3&BAzS(TEnUEr-E zsSr5+Qq|x$7%jrYB>YZLLTN}QdWa~#vxjI{U1(TyihEcb{NX4Qkv+%Igr5vTJp3!c zz~r0ej`Mte_PlMkOCqrp;KLor8poXd@d2i?S$e!*Pd`AfL~g`t%I#-r_Qy~Bf_1Q4 zjB4e@{K3AX`p_g2eIgEfMlfvv^5atgx>`RdFpeMtYYL9j2@|Hhk@I!F*ZY$K=wiIx z9Ehn(<}6@Kkq1lUER5s)9&E`7YSruj9Bg_KWAhB6(2sFe_vp`;6@tZHI?c%lY% zvVkeJJ8SAbssjacs=NtUnl%CMIW@F6QCr2?8Wd976qkD-oC%*_tTa7jKb@$qp8aQ> zn`YvXe{!+Nwv}n%_Xv`tfx#Z6;WI8HxykI%GsX*QRBKcvP-ht4Zw@hgP8Zmg0;2tpX)7b$9c4TbZKacR|%^$ zw8Jfoe%fd!@AbiMh!-m?H{!P1gq!GQ_sY^P|4vpsqFL^hvVZ7OP!V2H>$WvBT!z6i zguF0+eU&-jJq_eeb#Hx8~rN;_b1u{;Ied1B<8d89{whaf9vrs;m0Ps}OG;)8JSMw1Pq8Cgp= zN@p*VPac=cg1#)i8aV4~wzayDn&f#mf5L8nNs-#AXk7*_5?rFK^r9Yya>Hmiu5Ftg zUI}j3FJXR|;ZRdK(nLR$l>yqPYLthtIQSo-UIjL+D2bIVkr1`&z8#WBkh7kGN1^?~ zm^tfsTPLw^C&*|UXf9RK`Xu&l3uFP+`Dv79E6oVEl*~(TN=^LM^&fS4=7_v7=j-8? zOi30a14cVeaTiZz8qW&W&N3E7LM1^Bj;BR6jqDONi1{!Cb!sm=YHtPG+Xak1q?^~m zp#mDPV$C-1RCSeA+J~YYBU|FGZeiBZ70a=JKYGGQ$px2bK}rLLKhU<6bx4odVHav_ zFrCF)6E7X>FS{kdTW?pTJXbfxEI+z9&eR;p&M>PsY)+}JUPQpZuV8jWV39>hi6Ai! zF4s%3;s~{um>4j^W%x-gU=dGTXOno+>sli96 z^PxeT;|bg188MC8l=Vq^795K46DUbo^2~&-?hpF*gq8Q?5QoEfeV3N|lEIq5_<-kP zAl>yi$TDy#if71!eL@{*bRyQB9v?CML-_IKse zQ?x-70mDBd#(&zp@hUkTQq*@s#Q%(vy#}ZL!D;w2)^yZ5d(zeW`UATC%5?PGKOcrw zLZ$bT4Yni~#TU<4QR`b$U6{+8S@IiB!<)kLn_ez<+x6?OUg)1(Zh(uzP9F*!8mUy8 z>aIi>8i`03`7Rs^jZT~Qx~sH&0wxu?KZrp6KrVrX$PD5#IHHikY`Bqk+g~W3!Hs6- ze4B6>lg4WQ1|q~NR>+n%pFz#gw3W}2cDVVo<950f4y`g8&Uj?hsxj>Lha=Ly&~31s z&XLa4zWmtYu-Owzq;qA^;gO6^ZL?Y=i^ND-aT1|MNc=ozao$s{qep#oX(ZK=Kfj!aJktV zPGi^cZ`i#IP21KB)6NAN&ViSkMv#1kqeRE@zK%y(ElR#k|Vv|JU zJ%ETJ3WC_BFlDLhrQUszVwc9*{_QM{XB@&VLttH0FGJ)cRVPjCbxZ^_Umui`x6Q4!AEN!|ZL{Wy{GFb=y#v;WyjNdAK(9Opx(`khNc=^*x zyLIkY_|Hr^!L%JVg>i2-Hbtpc9k#j2o&ulBOIZk@pDJoc1nk1AS32x!dwWvt>PBfh z?ftvC1s$4KM|A94w!;Lk^fpsE9Xqd|k{r99k2##IS*jS3f)@Ws&~1J zl0g@p$LLDCT#0B}h1{lj$#5KIL=l7^$k_3^-4{PR&$%t>Sa*A%%6kcWu3M9#du}+5 z3O{FSt#*5De?^$}+zF@a@vaHs74bPttxfeh$_*F!)c{HD`F!qI`u6#<)l2xxRWAaW z`}H7QukY9>uc+T6oa|S>=k3%^zn{mYx%MyTqoQ9cu2*}%LSv4$p&&l7EI3F2f>hoF zYMK~455yeB-2_C_#(k_j7lda|2x4y&N1suI29tnH5oaI8F;nJ3)+u+9wAuzNiy@(` zA-lB(_>zPxY}%Ynrsy8JjKo)S;i6rd7=e6Jlq?Dn(o$wvR661`JoDcM1@_(*^GS0` z$4BZ6QI4dgNwav~YG^;0;Scjsv&GIwTX(4u%(uz#)X2p+gqRacKFIK|%*Wys?2|mU zje-v2bhl)(Bq(Q|Sx9NUKxcAkS2Q?FNbQX-W)0+5(nL;7 z8~szn9^bB{#F&^q>s!Q`&#x@0keIOw{Zq(Y->xk9DKT@~zL2+{UxhO=G3)rEfPcPS zg|Q+r``W%haF_oBS!|V@%0-Ezk$z`&eS!(eeniCAkN~VL zZIdgF{W5j>1+*K#C0AL$X6Vg#XjhddS3AXI=X6pbg>c%fw&-gf9@pC;FS%}zH4 z3hJgHrPPI2e`Ln&)V(7}sgGMugPKzau78tHX-Kt9vt;jF53)^Z%zaF?78l&8V@PQ# z?asDQ>-^|Zp3+=Pl4@rvsP8zQ($d+mdYNn+-)YcG zklH>Qn(US@xTz_h+Odk0>`~vjsbZVjx!s)P)i1au`z^KWSS!hAzH>{wJhl7!F!9T- z;I`m+YR_|MqTg-jHuq_2FO(+nD^O@>_%O5&fhErsvug)}l-7^onGj4Nv`a0YHh_nl z5X#=QOKO`oNY;}TBrdc!7nwFh$HKg?+O-$#lQztHl@)0!WbW6VHp1(l{mrGz++jU! zRMZg?7btY#IG#2p`<4QUhjv+rpQVj|NXbvk7qT>9NT1LNi%YKWvecGOpEMRpPU#mq zx{gSnvSx`*pYJ-_DodYsdWuWk6*@K>PoME(Ny@zKvK}}|pZyA7oDCGV5qwFX3pa|& z$LzLgq|ca-Lysz?5I)^U%2-IRKy{*1xQxglW9+GX|pIjBSI@+~)Q4~Oh_nElN(#>sb( z%2Gv@w^{=e^cytv5WDwbeQ)|IX?GKI!o>gtH;~D4x!&^Zql!1veUwnBw(bD=(0xzOu} zUZ{R40K3nPvJZy16jrLwNxcv5M+v-ppIuNNVpa)KQJ-}~ABsT<>QbNiULQJf2?nU& z7^@%iM==&xzrI92c2+TtMZXD0GoC#^UXr+WK|di=3I0$&q_dxdxR~^z-(sPkyo!&U zT0)s)fXcpvLUSNQaey|fh%RKn#%zG$f|sF5LV93;sj7tWV8HwP04r({8}^|6-2ev@ zFNc(*u=*gB`=W%?ebC-+kT1HBzi7}oXi(6eM{r4!Yj045sZ=(xTPM zeZ(tx#3rfLwrJ#YL#q>LG~nH+3w4_-*J!ZhsJmvHhs9`^+o)Ga zn|IRax5ClSO>JL>Mxz%;{SMmvA4cO~#{#h11F6RnILCse+CwzQlFi4$+}p!L#?lhU zz7@4YBb&xD2gjn9+G7sJa_-0CKphb5@qDWB1g?%msqrF>@nnmR6!-Dc;PJGij`X7O zipKHGp^mJj@#_8YoQIBF&_vz4iG1qL0CJ{%f7>>XF%z0@p7bN11q@5y}@A!PPvQr}C_EOOKA@1edwOS5PPvu_W5P|zG0dk#p` z55qn8PHGOM)emPmhwDCv5GqcUI2T_uhs-^WGCY^GG>2w6j{Z283YrHek7LrzXK>BG z8y?5jn$NbF$9){f3!TqPnkS%{AZ(s59GWNAnjkrxFL{_J3!NaxStzGoplqI?l3u9N zT%b9ep!Ha&4OyVanPezlXlPn^FFnb$ywH5Gz~V8<3b)vXy~tiX$-%wYDYXdYTAt*# zT^+3iDV~3|)&@UV?rrUQ=mai-KE@ zUS3l>T#Myij)Pm*#95EGTu$I#*O6XNN?uO3T>t2?o;tjomb`9Qyq@v6oH@L1vb>&6 zvy$_;ZVtDRr?rw#vth-(Q5d>Xq_tsdxlz)*QX0BppS)3ixKh!);WWHag|k|HxZ(P^ zQ7gS#hx6To=6i$3YNPabZ>{gm#j7nI-@k-@Z(Ck%FaGY|{Jj%yt!w#v;NkZkZrxpg zK|thC|8zAX_m69aA4AD0!Y|2F~LqpTemFkB1(9K<+&Gl@e^=8A1{>>lD*&BzOP@UV&ZNuapoUOBs z&An`rJ!vBiwXMVHnthM072B&m@%;aCmW*i||1J(` zBCh8yTi`A}3tNA&DPz$#(OWLT$}ZFHE@@8!8T=k2<{m|C0wvELo%kNLV*-uU9*xT$ zT@4#f%3jsCJx1Du_al3#g?r37gv?KS^QU`ko`h_)`*8&OP);L4PVIdzg?%3QJl-(# z0@Ho|oOpqjef<7?A^&*cBXiz_ebK8*G28>ttsi=&_JGBF8+ISgH3(UX+Irw<_ z5{pEN14Wi9r4k6wE`lk-`r=mhlXSnkf)1D@z2ILmexKZi*#b{^3WK zhTh>r5}8B8mNfk*OBmph2`I$N<1bHnzAMMx z^~e4*G=F>Io-fCNDx85lC+-X$Ss-V&A3aw@iXk`$QdX35Ig-)>n=3Qi5YUH;2i zJ^WS!&-t$Oc@xi8mDTyH-g&FlR$I#XvB!A_i+RV$`C0dQw~=}G)A{A)d0*^)Kdrq0 z{KX)8?U1&;0_OS171v1E#S{0%IO)Mei@oH^;gm|&=+Q;|&Bd%E=@jl|8q(!LSj3`? zL%8bYvQ@;2r$cDK<(eb+c*!NqU#Xak$-;^QG$H+R>&;;k{pvfZ;|9ysnuz17%GI)w zgo~(97Q%lWtQ;$>Q%ys>iQ{B~d&6`s-`b{ONa|O#ynTT_#%1yD6bCKgsfxmNp z>`iWtb56}oR*!S$%uV{1bK2ES%A0dC`fVbqO9IO+M8qXde(%}`0C zOIXcqNRLbK%x&P7OTg8w|C@^+`rQ{&*Uv0>-XgAEDt8`6uI`R^uKunrv3E{6u8uW# z_C2n4Gk3OIt~OV9R&TDB==bKNZe}d^e@sN&j8*Opjob_z??3vx>BZjbe@^9m>>IiimUmi692Li|rburC5E2nhJA5k>?7fFRJnFc1L)0AK+q|IT3iZ%8Pq z6td*%3x=cqQ9>Dw5B|^V{^N8g(toYM{FkOv|NGtlcLC>4-`9T=aHh@W4F;i7vG~f} z1`S0)8AXQZ)e0qJF@*_1LT}Tb=g0qY%H!;(NBNHo>PiB8pmuddr{|~fG5wT#L;QonDnv@+3>5^ zj5gt@G3IRIT5cd=hZ*DZZ$C(BJDk-AfAMKE94}}Af&Ynp#^0s;9|kX7Z3kJbDaW!hwT8G}L0X2mh`(-#86cMka}z#; z7Md!>8od*6mcgQ$CqbZLU7Ov^plXUtEWQf|hb+NrLSKj`)}v^uC*mhZF_`!Hm+P=} zxkO=#=WE7ev7XxwB8Ar#_ML{TqVShR_{ED#Pt+>+%OuK2JfD&^6oQkc#!`L;>v1-% zzu2Sq^_Y4T{hXr+i|t;_(F-yHiEXo!G5*SlSwQDC#nHU%X6B=e6xf&I_H{L!AWz)O zjx+Cr%_D8)a${9hCPz2CKqOeYSbryLzf@8+Qdek2kR^d@QIuR^oFRM$hr@dqg7#sL zZ!n79rwc-)0GExEC1(9vx)Ve_E=3`Siw z$)j2}8OCr?7HAhJu}{@UdYI|865K2`;?>4&Cg~S`w9}k5=HZ-W*$T%Q{Pz-t}vm zQ@keH$sRmCXuapybG+Y#(Ta8sn%WXE5UD!m2^640}AafQ^G- zi(Ukg&JYI3y*{4fU17|nv%#5>bI#=SI(2Z>PaMS=3-OBCt|W<18};ajutevXz$ABn zp4Kb_f1!}1`tsYX7>Cacgl+-Y8xEYfgF)`Qs*9#iGsO(>jWmiN=`SJ)a17`_m!RcCXe8IwNRBbe!mBoA(H7UtI4pXD z>}g>C6}se+uJuQ9oh@-Oly$PwZKhKDE=1ZU! zCs+a$8Q9hx4yCeNH#Dt-OO=uZeXJhJ8%yS*@!^5G6HTY;Ub;-AB=O?qijug5EZ`?&crcLe{lk^&dDca^M(+j&a&m{j1z91M+)KW>pN&I?5P<$0^tt zmIS3+^{*hcn@zC~a5TA!?&F#ip{Nz^`C$YZ)Y@YIN_4`*{=ZRVU3TDks-nv)Q&qAW zH6wz$iufT?bYGqT^wP+Exx6@xRs+~+rm71*qT-?I#hj~A`5LTwV(?;HWH?lRE3dO@ zXmYK(9x|u^EufXs9zh}oS^Fv!cj;nWqbY@QI)jwZkR*lfQnmm_-2LOk^~XxCPz;Hl zJhR|zlQRH$HZRjON>4aa&G5}oe3Pfbt~7E2kB2x8E4>LmtRjPmw4a(fu{ecGxBqld z;?mzD+x>YtZ-a-TB33o%Jt6SKi>cz0YX2-*Y(E(}iB@~TELST&116(LVShB$Z}Z7f zwzICdG1ak)W-%jlI347h&pm+R40n>M*uvhHu7D=-Gfy5JRLH(bY^8Ykr z{(m~a@OSNK{(Z}y0RS4{|Mgsn2SKs^dahLS`+~tJJXvEmrb7|foNtZZw|O#=#DWTc z*4T>10;v^q9@1R%#}kMy^oXUlAw1zx;dfFl)VBLSZfgLRL#}e+*FZ`NHhPW#SYD4-;B(%;X5WbZM4v zB%=YgM;Y0s=Hc^NE;njhYl?3y61Y5G7rwb!O{iz2D#rD&RhWU`a6U_SCe^5)+?{d- zk6};X|2Ds7f`$3dqsBi4OXWYndifvNVB8KM3uo8}PinJn@*4UCMf1ts!@x0LUI2AF zmU4UjCLpNUNQ?C9fcu(Tb!o*6b-#gf*>|R?9R&TkY8KW*f#Z(_fBC(jrO(E;(f<|YUf96B%B(*I z=^d94uj}4;)cbc+8Mn6r)c#(C_Tp#&L8g2|LZ-ke(r!K}UTbE)pg}soZ|4_qgt!to zsu4xqMlx}m+o-8%I12U$`xGX*$Rn(C1MR-!9Q#*DUb=+hfU z=i9TQ6t8apc>0|H5U$P6Cn!nTP7o^3$xblXGub2rTia$g6yK`MRO+4Me{s$3MVxua z?R}#!Dcg%=DzP$)VjH2~kLFq_TZ!U1D%+10dOD#}=6j<*fJoum7D2>G%MTKicup-s zsP9pW=9STACO3*|4-A z<b9eb^O~-wv-AIS&DM|6R$eqr@tj{Y&dI#L zY+BN`yKG*ws=RFZ;dy@9x)b*Ps_h`f?yCKyr1GlcqUHRm^Je7zb=Sj+-F5fRqsr@^ zKTqe^y#NHJn?4Yp{Y^h2-G8`dWtnb=uyyQjhw-heZbyi{E^bH3!Vn?q`IaFYaf>5SSn4r0^Ub=H=+B9~P8&FCP|FWtktB z{=+r9V&HZ8xM~v4{Is@dHYWzewg~~kB9)0feK-Ho2Y^8kgFJ~7)0+T*zW2}=9!b|l z2QS0GRX<*Hd?~R$0Fs52I0zB7E&$|(Bf|G1EmuGS5##SG!2T7AdT+x~HwA6^{r2ON zDEw`xa}G+n0>hD2ZDC%vtcD0m+1)eufv^rpG2w7;gaaC|VPTp4#h&3}AmXqnd9~gb3m6Z6KVhS{d-SfOKKM&ePrbIvzEIWm zD!=|Flf)_&Mg%$eqvZo`QB43KYCxc6x$&PX9q_qnT;MKM0Du-j{LIH$naHGXtKnD> z*3+3bZgK~<_?QQv`@sW6o?=^@503yVAPkv35E$z<5OD$t=X=QqK=IUt&^D9<@d*N9 zqUZx!n!zkQL#8iE5OFM++hE(=9wA6f`S!3Pg;!D?TSb)zgc0j+?yHYMdNBgvvIT;j9UY_W^l|CNY4=`K z5|B~1;A7>mC$iaMK1)T=-Et^|{g|gJZIWlikq{gAz<{8X=VI@xPlSYs1!8*_@5RcF z8|9q33nmqbaePz6Goyj~f923AD!=Mt*zYz9^tBip8ObxA| zFFr_v1{>TDqe9qTmatT0NMa)kx1b)P5CTFb@8caF*JkdHbHQN$zQw_}ihxcX3ZwzR zWA?R@a9lZ3=^CG*LRM&N(9_VAZ8(awGTtji8?d8wqNi5FCi5MHJNIWg0>u z)q8t8bi*SVkK~Vy4BIIaf;Z)!&CiWFFqDE4pP(BmjtJfM7l;WTmm5}KKck3U9)*9u zuypo;SEKy6{826!0sw~KT-sbB2lORC>IjQ!#FS7ZBcl6YeC!?dM<=OwIJ_#5Y&QWQ zz?_LXd5*Vm9}M|%=Y!#52!Os6o=$+t`)G`FX ziM7vr-57Z81e{%aIc^~c!o8Pc5nOc;L6s8nAGe-eed=G@RqNlw)aPWIV&2iH9t)y_h*f{`W*8 zh|jCt08*DgA`Tc#F|P?7n%WFYE|_;k9?)tOg8o1>MgB&dm5{hzR~sjJA6~fpcf! zK#;MB$Ut0F{kEh9<@CmYqz85=gWQ(Oo`sn{i;% zLjW^4Vp&h5J_J(br^5#HXMW4(L5jef3mUP^SNu64`#xhC@#R!h`ea|P=4De_;s^(? z0f-=_zO0{QoO4MFR1S6r`FyVyt9X?K$eBdhOm4Q%emfs7U|)cx@Mh-@0KgGg>IPgY5=~;t0dGh) z3V4kTF*?(g3YffngP-=#5FMw2Ygu>U_4q)XfX$v${^^^U&yhX)?uZH+)$fI~F$IM^ z#rZN#A)L$m_PG?> zujs46JK6K-vX%#h?Y#OJq~#)`jhT<*@`yYcJ_LZ=oyAU{@3_R2HrdY~WSk%DI#sVX zxV?XTV1bNMQ9mRYvLZCk{3cs}BTtdyJ%XY;a{bm%u;m5Z!T1g0Rb5q28@(L3@%fFR zy%Txh%l(PV=-*IX4!zGBk3Djs`Qy9Kf1`!$+- z=ivBLcuAfNyQ_r<1p^>J`fN4+By^j3?>_O(Xk0upmuPR`9k3x&WvSW!y8f`?&q8=F~V!g!lzWnt37q( ztt1ET4UafW_5K3&;)}J>$+8sOJ~opWGp#N4h9G_!XpC_A8HlhO68`*gB+&xj1n67* zzX54LmcLg%)_?;ciUVOj_SS_1go=H&ihC#!1bB-Kk$l-Bixt6o5NLoERER=HYK&-u z71TNfpo@6ej2l7$z~y6hSV7E}00!7PjRS}##Dd7DcB7Vk@AZSBn24=NjyKqd)fkRB zxOXCzeTL|1mgsSZM}*|3jzwsV6BSJ)2gB_FtY5#?c1zBw}nTbYNgbz87K$t+r_=8Wxlbh&vNvMy|w0?QiKHZmL zS%p1g2a+w~wqTjPU9=D2c*DVBprjBTiRF62$sREZN2 zl}1U3ni!ev7u5Oaut!Q^Zgi9n=Ak=3O@s@ajrRXjDhdTW?- zkU0>AnN5c|K_2CPtSFbni6K;%lWgcd_V#N?wM)ymXK03AFbQr(xRD~oSNc{EVn>Du z@sZ zwR^9apQnj|Fb7NI6g~ioS_e6eJ3#>!8G#B>kK;*S*+~%d84=KDmc_ZD5uySIc2oG5 zRSR%9nwDKiBv;vmebKo((^*6T5Ir_lSB_a-S~y!RMgbu(qX6Im71u|@l%vU2qZf#y z#)g*bh>r$moAF~e8}LP?D3K4rpQ&beMtYAQ5Tsi;q(E8_$4OZNfdLbc5nYNyV5*37 zWjKceriaBy1`$HRBmveb5Mz2&Wy+;|#Gu>7rUl`qj02~8bY4S|n+oBgFuFjIG^6H| zr8l~z0RJEYFxpSdhmagPsSrW|b_RwFuv2)pMH+Mfb5mE3#*%-Di-GBgzT>G>8J4*- z0+6*h0YFdO=~-9isqR!rtjc^iHaBY(i5*3r?!$b!Cs}I=5y97~5Y>Ug#H&g9ljYN@ zxmv5S2CNnatq>`#Sw^ikmO2)utM}t)8kIhu3U;}p0@ezv6ck$S)`mOut@MPKIX6-| z$`inNQyQw7R)?yj(_FU-Kd;KIX=7az6=9w#tjS5K2CE>Jij%pQK|cDW4ZuJUdrbh^ zm&G&?(+R8tu#m-GaRG%vre_cyD_JM|o=1A6cSVb-BT({ai5WXW zk^e=HENi6*k+Q~fvIL=;8tba~R}nz_u_v3bX_c|Y(`ERTLP5JYCB%kBtFl82Y0+digwpCI9&qgk*vl(idsZ9FTq3iMV`sA~t?wh=;TK-Es`C0~ZfYwUyoWmK7> zTCt?6eCr6Vd}U9MYEe3XQ9LDHDu7SzG*k!BQwK2ug)6s*dpA|;s{p1z4~0IUBZLtF zUkbT65azfsxvYKoijf;Rlbc``pkso|w-gaztJ{ICJ5_`hyMaql+V@2uMgWZ4Q{n}? zta|`PgTZP2lT(`g}?IKKK09Kpcui#^mbxNIjova6?|9HHa`@+Yp)26X+;1D8p0sV z!R|{{E=s^Nk-^Fvp`3_Bb7;VRl)(?Izb6d8-TS>f93UB-UKj~{>Bz!VH4^vvM;s8i zcuEv?M8g;1#4ZO&R>-PzR=P1XzZK+uk{T2<9799=ft0v9KitD)Od}`chF!5q=oJDq z`J6jk8evWyQmDV=MH=y8pAAd)&xx zBB5UKQBY`lid-8?lsM}(teu;9Ub+>K?8uvU4^Gr6wo9l>J(B}@?qM`QBa z6{8%=v^*yJTgf^>IrlTeK};Yc0Hs)Q%e6er-g3vpe9Xw4%*wpX%-qb*{LIiC&Aqe% zR)EV~L(SGK&Dz`*6p#hR&<@i8%;tdsc<>MV5DQPl1B9RouaM0YLC)rk&Q_t$?Cj3; zQ_kls&Nk7_-yF_d`~-%e3Y8E9+bkv{0MG(W&>wOH?Z6Mh;6hr^(6t~GC2$M>aM2iT z4n@EaJirOq&<%eek`$l?_TUfx3?Cri3jZJveejAAZ4W2?6DhsYE&m-ABkz|j1V3(gnO`Y_QP zK>LmR{gbuUD$^m*dO2n zVnI8`og2vgI)hCJo^S|=&;(rl7Zf1dstw21P1`h4-HtumtN)D@U`^G&{SzAC2*rR6 z>R=E3kPhEK43aPflMMvg0M+WtTZ3Q^`%nr6!2^>p4jAnZ_D~PyU<+0NUwF_CPo3cZ z;1AWHz8EkF3O>~F5C`r9;rSgADQyq`fDeigOWvH`>h0L@&LK;jg!0)(&$Kt2fwzS}{%2pawm?tlfqt)8~Mw4Z0u+R*>ghLFPw( z=5UeftKRCb9_zE-!gGG>b`I=xj_XYk-hoc*CIJGs5ak$s^6%XezRP!T` z5i9QzM@8`#Z|&*{=z@L{46oq!@bD2o5i&px|6u7~G4)kH>9&FOR?ig_UVE@PPRucj*7A>cy#*r&m9(q7mY&1 zFi3Byf+N(*>q8u}3Kbiygr*Q1fC@AA>K)q{zIuj1jj2Zzync|se5J%!Jb|p)e!Gkma1i-qmu+wh5?!J2p1!U?AF9&Myx#T{R4%)21@J8ScfDFVa zaj2qj^5_K#+5(Cs-)4kMy#vcTFFg-w?1%z9?9nF|1`R?eC>|;D5lA7AbSuW*n1u2n z7eD+l$RQzG@hFa_M9ay@viwjf-cSS49uX-Zk)RV(OldB*qO{A+y5OV}PC4sr@<)6W zv9X}|s-qwTD-h%k13EYx3;#9M;u&s)5muOEDt(T5VNN`4%Ioms&O$|>OX!G*YoMtC8#5qkYK*Z|F%Y){DY!>5m@KyXE`#~?z`EHHiZ@>F{# z642Nu@2!plRO~U=p?d2a!C!#+1-X=IX5Hp^s`d&J`(?1pW~(TG|Ii%~@>v$??+d|<++Jy!-AWlpQxx#xd=4m#<` zQoWc*3V;)iO$=J*C;yk`Cf2ecU7aP?9)Fs-vPUy~Jr<%7*mIAgv**LmBMlZ&4hs%G z`R5%s#1rWsbeO9s-><`Ns{>b}9)Z=IQ_6d9QQgW|GRl&JoZ^U9&_feip2?=3biUCg zk02}^7yt^Q*bE*>d|CmJ%QH`>9AYXdVsQX4c)}1?ior%5&Bbx16we6jE6=S*ku>JCVSYa z;-@05u|pPY>Jf(+M6z}$`su6Z9rm=>5Xyz37}8LOKEUA&m#|M@?t>q)xQ9Ccf^Q?K zTb<@yx4ww@kNtAbKmGAfgn(G#0xdCxC(+J!6#Sm=gkr<) zRZxcju+Y&I0iBkO2>>HRpaL0~22rU7a>9_nF4(IgHF4n#?^p18;4Ea!z8Y~AZ(7ZD!f(0V^? z2=IhQJnJD(c@Rlm44pB<9kz#$d-UT_C@{YAolhL;l@`9eujGLc8Hi9G(`loJHwAMhArN-+4v zK!!4AX#a~EN`h!Nn7L&#{)opJHll*5`NJR82wz8PVx)+Oglh$vChImh%ZKpNhbihJ zAj9}MEIv+)E+k;(^w~Ln1}J#LE8g*v2N4feFodW1W;n-L&U262>OSALzK-Lz)R$KK_bL8Hf~V?ZFxIaLhLOD(gf@(23PT zuD1U02ChaMOw$U51e0)$ZC5MScJ_xKt|d}C{y|#E>7%drnQd*8bKBf9iSb@RoKV1knK9^ic~@&CNdqaqk}C+fP1j5lP-{+7-#WM++wxyOO1>Wyxre%qpv! zAJMQwJRlX;uC^YWDe7GcyI{onWNpEG;xGZw*?*Y#J_X+6fK8NI;V!ox$i*yvA^)de z{`$8;Xv1+Iu`rb126wu{tzv|qIN|QvLzM#C*j+hNW#+E-eS<|UMq2!jdDx-}5CQTN zdwb(3$4y}O&F_Bu3oAo;jU9ZOO(XPzi=pt?#}7%cJ-UL&iHzU~Q|WMAA{vK>r~n7l zum_{W43SIQqq^-x!@(G2Xwd!`z`Q-M=n(Q@e@wQr8~$fVR=r0MzuFs#8Y zx5u9z&7>>cX-%7C1ZeQb6YVjNY(N%B%^>NHXWCebq_@cI<+XH|yIx>-Hqwu!b*n@C z5M29u*FIUXYs~!Oe-t&%{#bLH-wY8lD;vw`?e(=k{j^p^Ej|5-EdYREEB_aI6}-sM zi!Zp_&gMPOAva!YquE2BWl6DEvnmK!_`$Fds3u+F!Vkjtp>X_w0u-h&Im%C-a+5>i zMb$BgTRKjekAF$A5b*#ZD6wV3p$b|kSs%g>@kp#&Vho*_A_YUmHFRO} zP*Bev?Xboa9uh>Ve4`hvDS9nlK07+JgEQh7IxUJZ6(gR2dhVP3#oPlF1 zK!F+REy*EV*N4}^E)B6e#002-+&qM?ie6B!85eW#o zS4H24ls7o4g6P>;u09?QdAdnHB7he>X2L#ps7sydR>!*5Igj_x^Z#8Pepj_}OMm** zuRgh+wS4F^zxiB|VKe^VoF-V24S(<>9&Qi{?7uF1!jBz@$Vg7$w@>u$8>Bzj{yX@K zO?qdTo))Noc8YSob4v6)(h-TgBJh}E0L-7~kajGfQ@pe@I;2aa5yLFavJa+-<*+l&w#K}sk=pE$h4TfD|ohsQg<;d{U0Yp+zY z8ZcW5j&ZxVdpo%syCh7G0c1SK+XzEp41ch^qVpPkPzL_dzW>21Jkbe4-~vGr9Kj=O zzUSjX>a#xVYlyk22Q@qj3Lt|N8Z2W&!KqV0b+|efbO;9{F1#25XWA^$5H>`kn_1bw ziTI{()06CIkVCvZMa;bx48PbTzc(bk#Y01?Fu>W+x&QOI01-e{EI>;!6tnvWv?Dqj zq@%m@JiYTh;G@J_04GYbME}UdkO;M^@B~M|w8}^YM?k%NPy~YTKOv082kgAO13l5Z zwQcOZ(EGh1xkmd7K;ffB&Nw<-oWWeILAJBNkAOhvn?MArz%itQGCV{4@&`bC2rmq} zp<_ZPEC^rZ#$X(hs`9tFf*WUH5a+lRr!cr&X`ZUcga2r=IDJ8u_L#(tOdF_xo31Db zgAvJwpczGT1CPLfij0PfoQNYZyEW4_b*zQcpc#MYie!9>6dM5>Q33vokvUm0MJ%@9 zGZG9a18q2?Na#r$qkwKflUzwk{~*P8NCODdmhp%g=NP#vC=pfy0GOP~Z6nJCGYi;q zxb$bc5JfI7QNbms_cmbpg3&)7c z?^{c4aZ9*7Kwn8U3Rr+jJf~|Ga0D{p@73P5C7vU2)X>sf^Z7rq)X6rh@v>0f?xu( z^M)dT43T)v%Dl{F>B-ReNe7gRumDV=;7&l%PM};Ep-cp#9FF8V%A{P%rc9%3EF%IL zDF_<_#EUVzxXGN9IU8}4ixJHckxJ97%DJR~y4*^e=*q7QvxeBs0wJlM1kQR`z3j}m zuo#O?a~rs+jJynw(BMn(aLviIPT3S4DqN)n4a=A$%ds@eg1E~K)k_cU%Msv8TR;Le znFEZ#kU2rl0##1m42}Lw3rurR-;7Y<1b}9$j93Vd763o&7>fT~%+jPvho~xe7zIr5 zgF3LNhKQa+&;(fskyl_eZz4vCswb++h5wTf9D7&}2-5*67@%ay87U>rsl+(7^vvWm z&d5{<^F&XApwRIk&yfJsOiRxO;YwfdN(e(y#!St&TusP?s52F(H0_Kv{g6 zeTW6(V1jYz2PgDN{)1Gu6rD-Ek|2uIRXrj~9n|fFPd}Z8LoG@gMNmdn&@05y03n^q zm`~BE&l_Y2U2O~-#Zeu#kGgnK?TXQY_|S%MRaJdV2H`iWvMyx$hr@x$f|HK5>W)w8 zjBY>#(CC}Gi-;T3wkJvy`Jgzb8-e_gPyiW$sWFq&Y>0sqiyQ5Tah-~CJy#ETv^wjI z!x{~Keb>7{)#VIN^<)^Ta)pmH%KwGHfYd@G-C&PD)KqE(zc;0byZjZ5EKy%Ah=Q#% zen41FQrC!Zj_4GRkWovJ#g|@D77S>Rn870o2(_#NoPp`dL9ClIsEe1CPKS`$icOCU zz=MAHhes5cp4BSOjGJE-n2ik*hcMch{hbjL9j0|!u>x8_RIWuc5kUi0jC2$nP(NcF z&W{B&iU=Z_orpHwtf|RV>+)I(0NbZ6#DZwpsP&quwW5c`62lE0!J$|ux!BSB$iQxOjvMg`KxJvCx^o3#{$fx2@Wcg`1JJLL(v!y@&<19RarOR<~u)&b3-3 zvDx~-**TFJTV2|w^;`4vvi~aVG9yCSC{tMg86D-a-JGosODU|4lNzPyU5_2zju6s_ zNZqv6S=Vh&3K&&;P$mbPhgi_qLEzYj%MUX%vTtYw-NQe+sja4yjY@Qg7Z8)Q?1)6O z8SBsnP<7Eya8u@MC(tEDy8St&v(kx;IESrOh!qOOT`Q5D29gC!fwKviMcOWvv@?a> zf}mfGL<&y51VSo!Qet=r-E?cXMtb|e$L+BOT+AW3K)!Bz=VdKOVg&Yna2!T781k~f-eYKUY3fE^@RVR=I6%B`0n5uRn zEBnnAOA4liogKBp;{TeJmWm6@ambN|C|Gm&VlR~f9S8y)(PA$4;*-E&w}|68hKs-j zT*UR-*`XSai4mrKv3TeM2J{KnBuLREU{1rd)I43xm54t62(;XnlavQOak{yUTk9>| zW*`w9zC?XRp&0H-cBxVT@MKU%l0ud?A(9=*%~PtKWRQU5jfmwwDc~3`2#?c0rHvU1 zaOFG|Wl8$2*OhRHwObsSqh+y#yZTvp}6<>rTg;fTfCxaP~=Q& z)I%J0EG)tCfU#?_XrV;r~q@xC#P0d&0xCq`D0T7^!h!khTh2^jY z=9!^uM+Shbc8IIy>Pr6SlD*H4aObdy=h)~;dd}!4-RNv~J&Rswiuh%i$yQ}k#Kyn{ zlKocbxZ;Q74hndJ#c;81NCQANh?dFm!|ME~;;ZPJbm4&a1kAnxK$hD(KGf)fov&TB$$=;|07w^C1QXoPz3hi*toiV*JN zHg4oL8zH!ZJ1B%L)q)p`SZS6BuD^_BsBp8HBAYN<#-!N>7jXvL=KFgH3Fipr z*6WywZh`~~>b7p|4&;aZ7UkNa2}X(c=5G0R2-*hnl<8c-#_;j3gYveA^M+gh4Ayn< z*Z;3>kQDER7H9DnpN_COXzdrkJy(B4;>hTmNqBNdlCx303nbb zoA3;Jn({fXa_^QlaFTV=iSCF+6{Xmd7kE*d`KGgexGvZ5Fwee32l3}_kRyMRB8TpT za6S~3#c7r_mWXfihA%EGuP%x|3lM59-iV+0iVq8%bb~T=3&q`| zY-)&;|J%19a_1K2dXcZyvKx+|`6sgZy&eNtNC%*fEHR&6hVvS9lJ-9r`iZUiLFWBguj>Z4+_e{$v)5&ihK{38 zdOi~7w&!}U2mAYw`|~r^LRNdXVx9oIGB2xzji~Xv7c&QYr{k4w2HqCJ?+xbm_DBZj zEFa`|FJzwwIx`FUCSPEtuBfQL_lFpI$SL~fj$Y0cX^LPYq*wYSWBM5{?f<9$odr#N z95XYOb&5roh}KVz*FVX!XXtOAd+FwL)klfsuXUs5{m@Jbx|eahj|k!~YsKF-#_tUa z;SBgic&aCQtJfO6zkScb^-6nzED$;Im55*okIb5m)jwc9dVR9?U7rOA00IXREI4pP zly>_{sW^e`-@km9UbLXbuhxix7{E zGKaQ*=8CWcKr<%6pEfC&vFERv3Yk9J*y|@uDTq!G?eV+TDh&k+J-2S{;3_0UdU2GN zxaBWi9D@{?`aAZ5YflPj>`5#MM4M9;6w2B&m$V>GojkWjom%y3R{xfT|53GyRqNKk zJLA3t+OqP{qDTL>WSSRl+_{^Tu5J3%r)H!|N30dutYk%d@y;may0|f7$TRuk6Lsj2 zBWh>+=oScMT;IV2F$}V|aI0PlLy$FX7S~L>e#dGH9FbvA#8SmsdLQI3pLozm#`%kO z2-EvRl%(pbHSpv=w6$Abe9JQWbk%`2Ab|sU1=c@eDTNoB(9w=P2ts6;K=`G^U1vr$<6n)RJy_ybCyuohQQ2(= z-3E39^PxTV%ri_HFVrTZN?gex+&gFNsKF<6O~lfL7K*kZ0RJi!c_d>>GU?=HHSIT7 zKVS}orIt7v1SO3GsmP*>BDpE1kw}`^q)c4~#pY0$t*N12OywC;fA^sjC0ulZ79>Ba zF&Af=FwR6McN7i?r$lo0<{~Uk{NysiOaI}(^3RRyp$A#Ac`|!1SL6D$ zs+$ARYGJA^@mvzT^5(d%Kzx#!rVRuafI(`U*Qwmo7M zK>-GH*;7pr#la20cLbNh6>AhQfn1d&Qt(AnL$bu2<^!Vs99&o@+2ZRC`{T`D2TG(C z9#qvL>bzNfZcpd{oB#?arVwv@@y8Pni3Csgbg)3SU*i0Lm)}%2mqE`o^h}jiLx(y> zaf1c#a6u2i2%i%?OdV4Apo186(EaO-K#HbX-~YJ^G<@>?Bab}uKyy9|kxo1K$_fUa z;0S4aeM8-^lv#rGD9d3vfSMX!OSm~^8e47d66v#k`JTQVBl3U+g!yOG?a4XH~ zN&3dIzV^K@e)DTd2p8DE5^Ai2t%(=Wv^SGy=|wYcfzE;^m_a3S%Ujmak8XNl_NutE8+H+UVHD>W>BhCu z7-O1k{i$0f5;QEkdt+CE6Ip88FjMHno}RS!-7dT=0xGeIEfMXl6SnCW5tO z`-p`KaPrx>v6i^5E$dm|<=JV()@NzWE?fem)^JX=tuJ|NT<6-4y599c;sx)#m;e*A zR??kh)kt&1df$N%iycjZ623maOv-V<)~TCPAZbYMU9Q> zyRO4-Rk1B%?1WXzxhQ;Sy#ID0ZY^Y@T$VtXM#hb(>e^5bdsu@G5J87spBCT&j{I72^SnZWP|%9NYzF*5F&eQfS@s|J-l&}%AQ0bxvNq}c6?i(6!@FW1sYq)>D^+% zm%47Es7(+uUj<{-zDeUPYAJl=fh5@=OxCI1=HgEigu)b&XddH`E0FgNsI-!evxo~~ z=+ja*(d8ubTFXl}HT#3G3*M1&D~$vVmjodtE;Hs(T#^v)XwkiuGvn|)g1h{K9aumz zGOpQ&%svqxeZZGb4n&V^zV@*OS>5J918mS#M3(SK9ewiR}5->gST3gnMdo>|Wc*Q6X@PP}rHX!(lyX~~_V7sv318Mj4 z-mMN@_HN>1Eo09vuFQ(h>){l@k38y4pcPASriIRdOwZATsM?g}1W|d$3G!`MgC?&G zf3Qs1(QRTUJ=_u<_xvtoZgl%Y-CS9L9xQ<<^`hpgARTSh&Q|-Ko7Z^$9Cy_FRD>`=lgP*-3N$=7)N-od0>N5 zcU-qbu1eK_EC2bS3#>JBs(|>f)|q7fI11;3nmKpO;UDLi zLSB_oTW?ej0G>Yx+mq<_NJE|ELw7ouC2g5`*IVuxC6B}#A@Gxrz4aQw!|SUN`<&_d z5-J#hoKi~Af*jvL0GB?+^BDc<_oV!N7ux)oZ|}MD6YO?x^SWTZ_D|=`^wJsdon!giy!;eV<f0`kZ11Qi9Y-hWV@9F2fc!9-n2Kzj*`LjViql>l{gi192$(RNJ5qv*>8~s9Lgcc^x)KpAu^ERfaIWlfCdj*)%$Hv5Tei!7GY1A z04gv>%z*|eexg9+RXTVAY+04U&~+ot;7Cp+WGW@I~G+W{DF*p%v*! zUM*rH7DzG9njCo{7#`3YeuOW5n=$s{;jDx#HE9N5zVA<3Ph4`$-rSVj{*VHAea6t3MLn&U7rs!H`c5&`#1wAd*o^LgNtqLrWH2ZRO%FUK-}6&Qr1!M#Ui2uR) z$e>H@11?PEX9j`KsUoYSN=dY$K4>35<`LHXLp^ZA)@_eCb>ZphLp0Pu6X?JbOhPs& zWgIc4PxM=zn1Cp(WkIZfEBP8E4qs1{U_`V63epn>NI~zVNiL!$@yWyrOhQiVqd+Jp za}tDeDy8YPN;pJT#ZW+C0H$OT#0H4LQ+_1nJz{tK&NmEaGe8`HzGr-rCU=M?!IY+fiUEL9 zk$tM6X6^%LdM0SrhKZ71vZuaJH2Ip{o25UrU zU`{3gROp2g1eYpD4u_miMEdOh>lGIJA1z(Kc zL6o44rYL~2D|))?x-RF60)T+NYmJ8MY33-Jk)ye;X}T^Y=lEq;J?Ww*O{HFHNo4A# zmI6Id=cE3qfd*=M5^TGwC(#sOR~l%^=z}m21PS!%J~)FZgu*V!P#iuI)(BiV@Pa9r zf-3An1H~LsUZIO=pagc{KInocY=YB1t<(NlRm}#CT%%T~00xi&i}-^+T!Suz0xb+@ zqwH)y^z6?D?a8!#D7PD$s&7tQ_?iVplnXD)fRh)M1VM%Qf%<-C7N~ zZit|KpA%3=zsgEXC5bo?!zgS5D1?GBY^!t{1lBf8Ke&j*?*G>X*2#L*L(irHC};vJ z)U5+GABLJwD}qSi&h0K_Ln#gfG&u)Dl;KCrgEPo&?0!UuR7mKKZt0#b38`+Yh(yoQ zf-!)^_|!)2E(EUOUE|ur<3{e}RxXSLFE5a;>7s7jozROU@A5)K^HR}2=q~SOgYRx$ zz+^4X@*%g3@IvgWt8upN+r}^5+U?T_ zBHYSt*jml>RqgVU*is%j;p7-aqK3E<{{+>gwi}*gDO;V3eW3n z#HJmEOg`ws3xSa^ULnKnU~jdavpN05*4T z@eU9*Pw*~R@cj5P5Q9bMLI(*mjozX#827Rd`~Sll(`_5WR}?Et*HZE9b~7=5FIOJS z3kb3ekAMllLIZ^aWd1>_CIZfu31ay}IQ#))>)8xR0F+9y1e`D&)|P=Z zY*n?apa(ECNW_jvL^MRME#_iC79U=LuJvuZ!4q>4C39 zD@#7SwW1-lOlNdznQ4Z;qsX<0&Ye?T-~aV4@w88KRVxofUgvRv*mOwXbRX!n#q{(~ zD+`4QHOgTKuTDh5DI3lzAZVC&O6PP*VC@_%^HexYIu8@p#`Hml1Wi*92Xq2bF$RU{ zMN&8SL$@6d&r5znHBwD=Rfh>=AGBmw7L6>obe}a8gSA*AbU&E2Y8@|vAa-&1c1{4Z zGlz+JkM)bBw?E`1$0Eo)_`;@jb#+fCbKkU_L3d>PL!+&P6EU0G0sxa0Yk@%bUkA6I zkmF$Y15))jDSO8vD2rr6#G~#hygm_7gG4^O0$(|g?WD()4x+|LxO4|LW#(Xj} zHc-EIjUV!LPq=pLNL&;4Tr;L-mH#-1qxgLHllS54Kta!qw|1j#N^`%`kU#U3gG6Ch zjAui6Q1AEx_IP$v^>9BA2K=_Nl(;+XH(2!de+Nu`qqllXxm=H{${x%CBBVa#f`4o* z8_2>lgy7>!!iZQ14;TU~JOgs1!#C(cC6F?g7_Co%N3U`qcu>noL~<(CNl4Yk3IKvE zV8gtMgEOQ8B7heK6#Ahj`l2`be6MXucz_)ox;mtTsILNlK8?kYfFl^gtd|2Z2>SD^ zKp^mXI{3OI>IKnPqLps<)vGan~@&67SD^R+JAUHsSE(m%Qw(9H@tU+L1&|-r+)B`wNgRPUu zMxeX8vwP3JJ9SwDAjEDxJZ>e_>&~zT?Rz2E#4 z)V2Z{Lpr3xHKc-IvV^DlMCF6}7=>vD|Xc`JRM?7(4EM2;5sqzf=C>gP0Xu2d#S=#a?+p=fV=2W5i;OLBx z9|IV%daHyH9|2$-;hJ#I)f8ekrQN&fZl=0}A5>u*w{G6Vp+_gad-g=>Oo8V{$XX}& z>pVxiz9H4ZEYNN+hX$JA2|L>yIwPd@tW3$n(h#59OJFqL~z$QG9>(xVqpG!suX*_6#s z_73f-&Vuf2Q@Pd_)yX(8fkO|?^;javxCde6v_DP`Vu|Y&ItJgQ)yWXI^l3I_+hgrS36afJ z+ikh+*4uBv4Od(|OSlRjMaE5c&~pEwrw?^Ax@(}`*mW1)?^fLF-f$^kVjO*rVJ+Wz zVgIf7-F*@6mdu11-nLwO&TZJqfvJ_);)^lPSmTYMdpKf_K@QnfkI~h3VUbZzS>=^k z9(RR01|y*pd+a%rDJD#3o_$U^Xq=UF0vUVWaf8gGLmk?mo_|K# z>Z`+jIp&#bz8UKZ_5E7xvB@r*ORaN~nP!{KZaZDH4^q2px9P6i?z=TUVjOXtQNHF}<=eT^bDfq&o>HpyzgPzuR@_g~T&);b2 zskiG2*VGbl9tbRFxp$u16!>>`G408F#D)R6%9hT>YGj!n* z$wxvHqELk_d?3b#h(sw)abpxv;t5mukQKI&ig&YO39;D37RqOXW#ph%x|l{a<_|u% z`p6U47)QiS>Q-{JBm2^Go9f(=kA1Xb8$Y#2Kbj4YdK?TL2boAkE|QUrbpPZdAsIPrIl%+J~DN&h9Rj!hit#supVHrzV&XSh3wB;>v znM+;nl9y@RfE7m9OJNR^n8h^aF_D=}5RM=Pnqb8%DAYU(WWgBiAeb_-iA^35LKUwF zW;Vfzux)npn<0ygAyiQb4~DZRM@Wej({P76_C8$9Ws>yq@2#$5*0VlB0jX#tVX$!sRL;v}$ zjb5muKKW=!M{2%XY;dB{iJm?9A&i!?ETS2#>GMY5PL;f1MT?Up1^+(agN7khsV!qF z;-31vD8g>4DaFqQ<}eO_WYBt?lTuWhf>o_@H7cF!=@}UYR+Wi$tMKb;=g61T?o~B* z6u<+`+To2Lq%R{v&;ubnaR@{-fusmyD_Ol7*QOd!t@~;$e(0*CyV@hKd)>)dr}tCh z+%%^m}thzbG zu0Na$U2I5KfOsnJZY|=^{EZ+3ix}KL+<^t|xFm4>;0Vp~;s1*%6jogtVJ>>Lt6ugR zmnrYfpM&8AVaW0}YW2Nuep%yP@QPQnR%(Mi`k__WpKwk8Nm>i=8tnlGp!U(?w~mEiGT1z83P^- z2BctFgY?+C0EqIGtBjkLWSLY5$$>Ubd@nAq?8{WPazU`HtL!J4) z8KfCEupqKFhxy864i}yChiG^ydcyH#ut)Ih4?PR4L1#|$Wn zds`H5n8jO$aktUc1Bv)wKE~auo14(3R!<7w^cYWvfrUT69C1pfngsE0qgk?fr>oj$E-L{y5aa;&qw zJUu2%z70=sG=F+rZ*RQJeI9pd4jze!G)N0%{3E#EIPy2%dFdRVpU?+AS7G<}!8g4! z(=)FFuAs+sM_?7rQ;G+Mki`kpA&$>fq6n^80U%76je5+5_vym`30YALY^Y=Sd}{@? zQZSje@NE>5u!4$sApN2M(Me=9BmDBKli$k%8}%rtC8ofd4X}X~3~4?_*V&Q-eSdo4 z$C&r&Z~ub)KKR3bNbz^#_zWTWmhb(ZPfMbYGH8POtZygAul&w0{nTRq@-NWJga4jE z_=vCf#1Hr~(Eo@50ChqD36S}6&lwIdC;up*_HM5sbWaxmFkiZX6>zT{bRiOq3<$Kr zAKoDiKCfT?MH$QuVie)te1fx9!Xr@6AXY#RG@<{}!6@=T&cMc>49p(j0TP(d387F5 zsW2n5k21C|4|3wn?5sjw>?gSJ1EWwErttH6q6wdn4ZSdZR#5wXhz!MV4K=V1dEx=h z&l%9KrPi97x9<oF8TPy{ED{_4*HaZw4qZZN{J08cOl55gW{ zQ75F)7mx8f#}+ad#`8 z<$ZT|_QTHX>@UcdOeXU@*L@ykWPV`s%z6s)E0UfNYL__*uTMCuPB`pIc*Kna#45DL zVA9*cjrMKSKnWQBAVPa|5m^%qEd>9MAdS(KkJXouS195f8rsifc+ZL<&!%KL6flid z)Ab`HC)=lhKxU7Gpm$Re`$CqwkY2@xScr;qR!QOXYa!_~Inld`UA?t;z4aHH5LSV2 z6!^pJZ(wY(81JEP(qlJFhpNnI!OTrb!SA#zEmSDb^RRppRtiv65>3#&|4&FWW$N!syR`csq7Onm0FaQ;3*&Jt%)lg>d7- zBpFg=O_~CQWB~J}Ol}yXTAcHizv-6mXfB{0f%P5(p`xFpdt1A}?04u8{~MG*3no4z zh2Wu!pi7b9IfjtXj_`i6a6Swh+$|@NLuBuOr?OnAGFU`xUIbr?>t{SRbrZCuggCh> zhT8xw@txqU8m+3jNhp##ZM2s}=n>2<7JA$=rB?mhP9jUz-bt$C`dm=~JkZh^zn%XD+ z;OLGOu&J3VHkT`fY?G+pg4EABeyKZgIgJp|EKW%wP9*x&Q33z2ejVFFybJ}6 z^ou{Q{Y>qtly_phiHT@*K^XOOBm@1J2VBO~#eg^0%iP=QqV1?sgk!|Z|esubv^d!Xe z)`Rr)VJxz>EaaB-;?eZgxvjeH^-(BvB3g9fxpd`V)!v}o)>4Ic95|)>hNnyqyn|)V z#7NNR2hz^?oU5c+Q$Iuel68K!aNdjIwI&C4&hwI%eErpG7_Z^97>vGP{QROX*)Uu21tTEq?m9Cfx(vr#e zpeQ`8DQ>IjGNoZ9!nZ$~Zn?qMv`f~vZpfw~p6836%uA7tx$uPGw+6)}_@_KtBYkwz z9s_qhSg>_WxiVss@dAo*V}A_tG^Lx;KT%@lP1y!eqF9t^wcEqN0TjW1P4yN-?X9N-R#s+yNsmCjDnt;n>67!ZpRa88PW8A{WuAMWMlKyy@{$M zB_rLG!$mvTkdzbdU~Cq{$AI`a$FF<>T4eMoeB2qUIcd5F-Mid+Ph%#0{w$_xcB}M` znK8lmlMg}Ze!93&eVMmVS(YKtWgR54?Gq=uIbH*~n{zA;2rV}qIooGBmR_mgPHHUv zT&0JVr6)n@5C-Xv6o>5;bIMe6N@JQhGC=_FG04^=8F~UR6y6Vx7JEdl!oHiMOk)&_ zUG2?kiuqd{<$$#fKg`ANXc16iPdr)J0);|ZsOr8(=DkMZpQRd_nGxrO?heY-pC)0O zpm!x3e^$NamM8P%2m5+=it_Y!K3BUHSH^g;6t0mbbRduDRrYt1o#E$~;$v6ftGErU z)_c)InlrOA3G#9YYKxzF-=4Bj#1-KQ3#h4czQ$5h{37GN&IRxNh=1Zu?I<+nOCx`- z$0f|^!Eb;vQQhIsy3ee8^3rB{DeS_p6Vius?hM$UBcAU|V|*qP7s&tDk#+sdcK^)Y zx>^VQJg)|A=y)G0nUm#%GsVTNBf2GeRum|l6?K}^+I{=)vM z3+LNNwsZ;Ngz~uH%N&$y(vN9aE^a;StV`T&W*2CIcr}HTw47&rPS!6n&(5{>8DF~1 zlX_{5km&Guq0L%X!1n3!8R(HA?EUvbdef;05@cTRS*C+)56f;J?`u!&`W~P=)GN>p zK{3NbWAZOp6&n`SL0dJeS^wFf-;(duC)bq)45lW|>Dwjbo$%_-KF@N?9`MO-g)yij z?QWL9Z*(8#Jsja;x)ee!A-N;Rwd$zy*(sX;)^PJ89g|F|hz%orM%t-%r?nmoVG+Zz z<8j(ru>t~IGr!z~3LpxMjQNTc`Hoc^{Pcw$FBcn^AfgNhO`co8E^WvJQqdl4glCMC z?$fW(ZB11Y36Kd*|B;{^Qt-DhpMlKJKEl8nK;_Pht6)c3q23CQm6Cdg^PZnNUc>!_ z-Cf`Le>=0+1*CVxA~_B_eD8Z4Wc%9hiGvGdy{J9} zpDo^Vc^P}RejCnVl1#t#Y~Q|Y8^4VR%?ZJ8?gaFb$qN7R_;UC&R~Ru*b@yjY3&lJZ~FW&8KO65g995LV%}h0Te| zyyJ#<#FszEv#3*rd6#g+m)Y-T(M_aTVhMG{6nOazN1H+mE{*{UEnILvu4%ca;d=1% zZasr9ekZ5cb&|1tb;Ry>u|dDlqPZ@Cxk=?JO5UVpAkEIj+m;Lvk^9RRMRYe_++*@n zCD{1E&}d68^O_j4|8}f&V!EBZxWocuY@Fmx)IohEIuMY;=LLx~G{mJZzbI2I-={th z<03gxf}=-$6;R;jgz&BG8)$4P4UEtin8^wNu*FH zCn$wQw<3tBN+Feo$l-VPshB8A7<=VM zrfik!4>La$p1O@HwSXcGTbw|Jj8A`yH1(6~*0ZqIR(8|vm;(nMo--vqabtF}5^pc! z_%Hz<6I8!3zZYi{u~167Kqt6cjDbl3pU-H--)e>%^KGT`6fqx;Vk{Ep7RDACwa_cp z-d?PyHwE?8zX$}pt6$BJakBpV^^Dtgmmdpev^vwdKCfKR?;Y|@aQoQqwyiV8&v;wi z*KK!N$E*BXn=4l*J+7fzL*D~yf+X&){3!E3ZyW57~14>)2l|3H~5tlvZpD@Px(6S&>H+m`w|pS{#T`KD=^F zvw7Lus;6^(>-+u5`yz7!UuLI+Hs4=*8OMPC=7=&u90MsZ+?cw?m#F7!^6{ISMd8B7Gm+*?31Ko($D#Q0PO zr`bQWNXAG+D1xI;e)1=|fuDuT^3$zQv6$?aIMpEWB7fTZbE~7{oN7e^#ptP1wak9U zr^Pzo1;ewxVJZ^=EG#Y*7n8Ic>&FjP&zhj4?U-AEQXfb)r(tHP4D794eW=Mh1U$o)ltVB6VU9>)_EjgEiyp3)`;snQF#Pu;>VXrMiMq7>3%shL68NRYa)3S4 zQ9i&$kZwHJpUe4Ix>eToWMp(_g}V@@l}_`M5@TMXvZj2)FI>B^vTUa@Ia=2j=wEDR zh<<{tqubHky6`$D<{Yk77!OXqF3ZF_<4&r9K_L73Mr@HqEY(&6(bH9Sr5FKy&S@Fe zjOJyO7KDEl&dAY@c=dM!~b(TJ- zvOdOTCw1%Xx`Ok{Vf}dZCFUE>4&o=ZPRzty0=NoFVNAc zZzP^Mdqib8c2Hup6r=HW%@k<&>{*BQ3s4K92_(+EYKY`V1sE9e9^6rXUr+vtzGBM- zDB~AE0CdbdXrQo%WRor$7=efO1b?~|Q}I0+NkoEcN1oqNFIXA*-OS-8lx)aI>YTy2 zM5aU!o+J3^jp^Q*BP2(Kzma4r#AhM84xW>{e|p73a`w7m48C%hwb<*3vJ$fe0V0#K zG@PKLN)F(7ZPN$0xY;C)_F{ol+a4vJBk%i(oIbFzgde06$*^;9v}ws3kuD~RnMw$e zFPOwxRu1`^-GkJ?lu?K`qY^KkElHAEg`p#jeBjDImR}aMy8C-Ku#u);ar@mQU5?~A zA(&EZhb7)z()bfwgq;EE63f8QE3ZoOc+D1EVsby4`;5I>4hZ^!Afl^aU)kpfksKTem2j&~Ne2@1BKb=<~4@_7(&JxC#!vQQ5zrOgW5ohK!)eA_2%` zK?IsY*;JTeeBU(`Meb4IaSq{vG=YbZI+6`l?zsJ(9g zYPSePSDV|z;CSxv-9N~RRjnrRSOZ2a!DoigR=usIQefNnq4%rE5bdUf`we{a6XPOR z4YOTXR%)WQ2&=+S4_j&(tHNI+Oc28}U*6rqwnX|htusC*f$zW6ph-=vR(`Ws-&&Q2 z6&YOPZZ3s53k%yeAVZ1@4eM~UTQnMs%MP?t{DM4P?hsmHca0$P^Id{;EF)MJV!p9# zw8Z9EN@XBZj6D=9O12FN{3Sxy^~*P9fYwpakrgFdRv^oxGkMX-w$5kuXc)dJ^AD5w zi~?rQ&Ek*WP&FogdVll>(wiQ1nZ$RK)Mw#z#If+|jR@?tqB`&R9;xY&5#&wcmRfpS zMkj12nYeI}my61{h&{ONwO^n`fBLeA-1W-Uwz~Gx+z~oj*KvlZ zy9BeT&4Qe_M*17Z#H6+Z-!dEG4+#Y^aDl%Y_c(5Ma#ku7c2qL=ldoGs8D3uc=mTlGH!L zr&Ds(quJi4!tPkoJz`=iDhK)+DTa$l+Lr=?}LuqcFfTifCR zHdJ01rv<>#DQ-JB1qf_xhKIu948mF^uPsz)wG1tspSjDtG%FuB6AI@NK3xoJwRfKD z(e(u>6YaxMu^+>%s)n^4@lQ+?r}aufXMdRyBW~P*$Ei0o(j51$J>--nJD){O$70kn zdKDL-6W@#&wA^O++ur<dC=JCH6_((U%`|8XXfH&38wb&bymNaYlK@I z2c0($Mt?=Qt`G}~Xgcp|ZP|s?eIi@KLr9KaBTC?}-)E2F%Zm$#mOU;)e=3}09$76s!1KKqYsN6sDq1J-!7Z)5^Q zw*9tfA^x?R%HrWH?+O0P76J0<-Vi{bh%>C(4b5+`452KK$sNXDT>tdaz(6+%#D8p3f`u&Eh> zfdO|`&UPUaj_>Y@r4#5@;T6}4Z5B!8v=Vw*j!PODND&!Kt`!7*#9tp0{#7*?NhdPN zC9;`QHzz`s;gKHGC4jL!fMp>dx{rion9VhW@XKhFgRCbmIQr)xNeDu;OK~*!uL$QL z2n@!yZDb^AbYv8U4Q8Bdq-2>}Vg}*ce2B=R7jkoCzF73Db#$9-;C*_`RAQ|CQ9y!c zM3am~;i#gZY?N+8g!78E;))RZDAiBgz*ZYVb7_KU9l}0vOv9j$a(@i%Q4C{i)EIBv zh|PB{=#YmiKR4Vk?MM2rI{qrn_*Yj5twT<9>{yNLPPj=*SOp0PNW`VHH1`QOY-UkU z?a@b%95&Di?8WFXeAsYa36Nne@CR$zES1TP38^3;ixOeMPaw|>wzI~3S&6S+411Ko zcymJCy>b3731#ktC}o=>C5MF9gq#tGD3IofxT}Gh9txsm!PC zY$C^`+zP#5Lx4D;tJmX>U12DteX}|W%kNK3JxY~=N_(FBMrRv?^_v-YHP!4o6;0Re zE4!y=T0~B;lRG(|;m{L)wQUyOO~Op3CzeWLO~;pPudI6wDHB`p7!n8SvLseTHl}CB zAUznqBd%YLq!B4moIEjK75n%p=f=x^_+vOdtlndc=4)K~o15v6}<$ufE`s z7x{@6Jep!rK4T(R(V%q(Ha-RC&){L|6=KZ^`j{5cG2n6_7e5BcbB`5pZWnYFmhe6o zd~E_FB1<_}LNn^dymFwwZ=oq|;REiVO$o}pw`tf-LG)Q93S!0CJlUTP9bdjd_p3M` zu9p2}cVW#agzYax2!bNX%AUUm?vtUd-Gd*kX#&ncjd$@CgW$(l-b2|-Qi8kLg# z*vexiy}Z@;Wo0sBWnT_+Xg`&tpM*266}iz!`K?vCc5-EA$vC<)L;5^}%Zul}5H0Fq zjhX<%h9ez!e43*pq;Mkit8r&b+GndpX32`>^Cpp`+P>7N1lJ7w!#Q6IrS`7N z@cxlp_yc!9VB$+269&#+d+^3dEthlE{WmI~S>WPUX*+G*WoF`mXTtpaGtASg8uEEXFyl`3$fPweSBazL-NyAn97f}r%# zeq%#gRUwZrgzkWIYgGv(WJAy|r0L_sW*7&KZQ&4-!E35P2>ND$vk3?Z(Jh!hAR01c zHxMjm12q0ga=TcIRt!kzG>%Rm)4``jcU=I*xry(EQ~)K5Yp+R^v02itSy>#);Izer z5#EeY2wJSrQ}coufgE9!xHP z-BZfMSJe34hsHu!c+Vy8!jD5hSF<+g;TCHMQzq}+DjdacH_oy=Z{QBMQ=mT=< z>ZQaKw@pIf%m$eSO3ZdOE4?%?`Lz9{=-@w%aeaZbx-+)bX}IcF_e8AsM4e)C#k4_n zx4~@men-LjLD&WQmzgE-X;q&b)u1n$u-DoLY^zVc|3YNl1f5)u{b>|~Qm~gcrvd^( zPwXqiQ-NPgplLU$Az~`!@*U*s9xN?tdk(-5(8VKM4el``dlCBNxFL? z1sXu8ed!7v_WJ#BHEfiD{f#m1>wGmS7VR5^17PSuYQM4lZh=%tX0dH`#3$@ZJuL7xh9dB^6Z4x zU4YzNkFvT|k5Pp6n;dMaO-SwVW*o`kkL|;l*yCIW*UMI&(QJrO>n90BfRb$#WH1qABYS*5e9Y|}HU)Bkd&UpA(nViuix7Pkd;3dE+!a%Tpjuwko# zQw7L9WN1i#mS7SWohKKaE9OS+=YZ!i^*6K7Ju9%aZeEd-=dsINwaY|rAgal^yxhzJ zhnb?UwN>TD#C$#q5_0dA$7xETZ0?D4-wd?M+ z;FeYD^BFqg^(dwF<54NpTF8Q;p?ZO!ugBX|723w!+w$;ao)gnL6!8SP(qAOUzbL}! z4}u$LBr8n^e=&vEoGLeN`!+h>HpT!PL{hFe90pr2=xzMYp+UFs74#ay;Btq+XKFToeb>Z`(bhm z3}zO9?_0q6QDxn}T7`QWa1R;R3%s?X$V(ua(x09;6tUYalwD8STv5H>H|`(x+|mfq z_n9bAo4Ve;i%36|cdv$((wa!|5pd{`@!-&LQ^NLrPNl>GZ|2}T+~GGCZ-4Y7GvOod z#3P@nqfy>tAID?+T*!${T_H7&xxvLgJJx~uMU=H)d6E%4UxVDZTaib>!&&S(ReRk04_$Do#&3>bPbZV>S^W_ZL9U3qQw$s`rx;qf4;sW&W$e z0lGv{-bKFfRb6jIY41^6@6@)V2%K;RYJkYm6gYIs>cQm*jzA2fxv`ZkFRV_E^6%%Y zA)xeQkC=Pya-2A8K&UZ0G}#r>Voea2EYv8p(ot_?E!h-uyekpuT`|is7s5p4VQU zK`u5yvJFHyS5d;4vl~u8KXyW^OhC|MA;UL8XbePq)j{u6K{6A7hTn#Aub)0q0p5~5 zm&8YZR8n*E(H|1fH@aVz|J{!NeSN-qoh7}c^Ld}Re4Y8xUY`G4{rCCDG)jZWi`Gv( zN&tvt1!Dh+YfOzm@ceOK4XSTDGMwbBKB=s3Q}91)PUn>d$}m)S3)80ZA31gMN_3tU ztBs}6*nu`egIcV)yw=^4`(E4NGQ})z=aao-FkEB6MLnnMYSIAa2MO6SkHE4!35Tx#dG|X5+t$;+%Ur_d`2H`%*(Ds zez{+?VhOoyWQydnrF&W)4u0i9iNca47D>et@ST*!m?~=6AD@*nj+|~8rS*lS3oAji z|ND$rWtrfc#qVT#ZEwMRf1!IYr7W!WNbbt;l{jIX-~!RJZCMsPC9>eJ4a(nCDH>*9 z#gSL5m4$jUTatiGGeB4j<+*sTqhgN~Im*>Pp$Tj1Id%9MDwjUR+EAbu!_-rr z*3W+zxSV5a5C?o@GO;U}pbxV&=X4TBPD`E}x-PLKCJ02`Khl|P4A()j=~fz5qKsVw zY6>h-*Nv0mLharFixWvob_Rdd?V%qA=%21AK;aAekdUi{rimET&oN z1W9i;VWhz?s>J6hx{p1#U!O0-{r~SEa~U|k)>P~5=tyyOF#hflNSqQ z5rV37SjKSkvQ)P<*1u`;?@p2W*!*Ve%hr77MdXmr>3+WD#kt7Ddyamqs;ZmaCo|W? zZlh+mD?wQx!sgNtc`tYa{gITpryr7z@LnL-FQ4G*)|pTBFbG^#H#O;;p~#T>=)ojz zh0sc(Fr&gZSm&#`=C*DJLV5NNHD!s9ms?Sr$cMdhFHP=E4@`Pmrj%>0nv@A^dCrEj zm0g%3Q{=Uq$>vifm!^@bWy)qSE9op3f%8_&vjWN-P0IUA0{N064#urS={-&Nn(EO# zVUq>k*G{X8jl3))C(_oe12sXbeM!uY^yf`n2P-EnnA|h=olo+5R*;SmBlOEoluP`* zV0vCWr%qU0{f63YtsKShMXgwe^o;`o`wse{Rnbt1O#`Re_D1Wh*G3A$iLsYY7k3Mo ztwh(FBI!9!gDyXEAM$XGZEe$ufZBF-wA04|3UL};0$`nbdU2G#) zC5+L+)L2UeVIiRa^Pzr(uWCk|=wODLHFg`-&QpRIUNusJ$_xUI^OU5bQH>zWF}%pA zkff7VjaDx)!>*O>ql|^2tzrvfuwX~`d4(ZOu-U<$q^n?ZY=|*N-NhUCl;#~){q9m? zPJB)`K&xd??>TMS-C<6ShR=q2b{!vcW={6kQ&v{`Cd8p4R^$~dP4Esw zlBknQg(e3j|2~+Mfx1g0a4e@$rj}Z5VaX_yF;aF^^Qo!ClG&Ui8@YcjwX3<9$>C{S z|3)oiaBm9}l^xTqc~B&cH{>N!Zls*(7XP<_6$j~)qP>-R_WIrdw%ck_Bot=G0Uh)# z;;ND>27Asz@V$Qz`i|MgY9zzy|C6wO;Dlt!Zv z#nMKC_*o@VTB8WF)JBTgTQ$~7qnL2tMn>paH6dK1gp%4;PRUy>rA(ug(b87I@L4Tm zSfh-i)KU&aqI2i0497PEZ(^(i1)M4ioClF_V6PDT1QaWGtI}zgOyW=pZDLCN$bW` zp|6?m;P0}JqNaFsFy0S|F_nBeLzFbmQA)9@i1XCHnpLNCN3WJWa<*$}%bepJe9eh4 zU4F5wIv;}=IYI3^q)+XV-{X8OsAzS6gZBWmY1bBXw7aMlUp}Q23tD2U>vZibesc?W zO>knd_-#((k~!&X_24Qx+EOAzG+JV%-wIi^6|$0xck{xLe=f#fmUPK~d$kq~!JV>q zZ&H9)QsA?G`sv%yWJe#U3xwC3OD=OQ?eUS%V(%R9w7p9ooP%#XTb(PWajR4^G@h@6 zL0XWzuOi=Hb2=%;@fCKfb?~z@;Aj;-1DF}>1(LXvJuc>vKxx46?i$eQuZ~-B8!88; zcw%+3Qj3NQVfg6)%>vO)QQez*-W;4F*2^hXZihs3uu&uXttdSwMIM4;SQq6{+e7z} zX;fEzQz(XZ4m^Ioy?sskA`tuR6wcsH9%mzgya3r6;qnwi2_5NY56G4IoRAXt^F#t) zAKr6B=-oMib^*4cf*SBbHo6}RfQ95A$pkGD_C;xjhi4KsGV_vr%Q-YS6u_lvS#x97 zn;chiN>Y~zOf4~GKUNmB_MXpuzZ3s{>DT#``5=zqR%2~=s(F?fLZg_5I?X&*p-MxLp@WQ3s=g+ynNB7~MhBMf*&9`@tzU8dTpQyguXaSy_z|FKs zYu{aBWAk6jH`mGKzWaxSUV3^)x7ml?-`GXQry`8*ifPXe86~_I%8l+lx!y#yAl@q@ zy_L1;e&-GWFWGY>|GEzSE|sc${vsGZ4*sC~m>K)-(Ha*FYy02kLVWE%kv`9t``?q( z`<+@FzpR&2Jaj<(t|CYyHgO9Ao+bm{k&{T@POSr8?#uoEjTpb*B4E892L!wx{-k{P z_#J-x>Hq#B0!7q!jv@*zC<<#Y45KItZ{N46FNzQ=isUPVm?MhPEjm>#in<|+ZXt+v zCJI9B?|u;lGl^mP3SbC|VH@-}Dv06uis5bWfZHSS+idmhC zkr9bgaB`6|iBl(4K@D=bM&6D*%n&*}%01@-UW3De8 zi~#+=G|wu7-~Z7(+06fM&2x@Zwz*)YST>o#aICpo@DR} zr4TB6k`SF2OMHqKu8#BnT|$<)AhYP-hA<_n7~oVt{#OcJZT`(A|LZhJWG5@?EgQ-x z@CNyxn&-howyJQ0C@ff-X@x8V9iw3oWdRJ%#Wg%*gLWg)#WM^i^oBLG(SK~4i>zuG zR!3^<`xnnAHSsGqW;@OU3z%avKS^2|7h3#X@J3RM@ZpM5@f)D|ABpb`TDvj#%T^JE z3|_+Q_m5iBY;io_MZ0B?N%(&?&);)FuuoJ$7`Uxo! z(I?;u5v6Yowz`R*Aq2DjC$GuMOvYa{_Icm$LP(QH-J&bYe1@9`h{H{GBCr_%>$4xn zH|tE2GYsQ3O;-O?^Ay0yLbwkt&vqEdoN34VtWzU>WF=b|Is0V0=5}+@3{9TBCm<)l zI!w15dp$%>2*6S#oJVT3pxo?4iYqAbxG)K-l}7JEXc!WRS_4)h{w^Cp#0uRAVyu}$ zz!*U0>95zMV^svhyI!FHsjzAcW=AYlcH@O7N5@ouDM$;lE8tHg#tFSs<;Jrz1ZdpN z%DptX&Z7#&HP0!opPm0l^K{GHLN1}n-b;I-Mlmq(2%9D|iUD#huk2M+tZYnZdiMsr;7ZnN7?I7?8*8hn!tr@YWSWKob5 z!`fX50a9JU$3PuFoMDd3CPL;PciAixWCaww|K)M1tFR=Vw}-U(T%thbzlaRJyx-qU z@LYCVn8s7HtX47Gyxn8w)ed1xJ{G)6G1>op`-cH_g0V)m6Q}++0J5IDwO0(wzXn;(IoH;rQ4o78EG(-;A=7IqK$Z+Xi<># zT@``rOC*WyK+)26G_){YKUiKy?7POC8~OMi4XsXpsP9cu(Vhjp%jlR^=S`~a>h_(t zO?mjDH1x79X6_PpFLP)(U*Yk<#!kvO-{LX3B-24yU4ay!U{uKvi&Db9Ahg-g21U*Z z2Q20?+0`-o)hTv7c1vbtFszQvH*^%Bq$TPy_mwfk6%je%8i>sDiR$(hK^Q9tk_whA zB!1_O7(*-$M&%qJd2Zj3dPPzR3a=5QcSYt6iyv^n85QKqo>*PMXT+EscLQqf7R&=U!-ajMU5Sd zy-65**0Y9n2{icB6j8G_`s1{yg>pB?T#~L2Mh1_PcF zITB1r0oa!!JVC6VHU}9W3J7S3v96FH>I{@LElvDx*KH=B2M4sl9e~n+5CMf>?Co`4 z90~C;cOukk}dZ7RR*Op1CU(N*Ogu#|E(QxQQPs zPU)yj;r`sX)dau+ta4urn3zJLUFqi7(Gj&fzAV>yhK(kmucBgnqH6t{Su0Ld9o6*l zX8Z4q6p3glI7IKJC;N4TFJbF174d=_XmcwRvSWgp&M-izCF^&1qEWHQ^@CKYAzFK4 zCovxdR}&P*eq^KnI8pult1<1R-;$LTw(svX0{=z=BZlh^MD1@-lSsE%7o#=b zq{YRzFkB3sF>EEmcTZr>p(zR>c__4AUv7%=qh2Phqal2%dxMzk5DcUn<=>B)0-B$+iO< zXz(iu-Jva3LUQnCH#!BDE(!y%T^j($cuLdj zu_`kdmSMDNE4uLcih;h4e*ZBKN6lp8Q_wL`oYxT+Wn7q{43QkQruq!KFvKXp$lQUU zwyPAk#}WGXD2zq?qew4XjN>UeT9=S!qEt^o_`W40xDm+ACSY;&PWL>U+j*RxFJN&iVaz=a^vgHN{~V$BR7});#Wz=frzv@B z*IsqhfeK#xyQ&h1mQpZ8C69})O!a=9GztU=ibjsG`p=c>

vTBgv@j3O&h&QCegG zrC!5s(*AChgOi|y1mCZ-bTofg47wZQ{W;fEMyzEeb~A!)(~4L)61SL49HjNXez18x zr3XKSn;M=P{PPy*t4qi}r92e4e3}la+sx>ZA?iNzg0OQV+TAA=x~jIlE&O%#9AeD7 z{vP0Wi@kFfG2?ya&Lyz6=m-fp*u{8F2UmFbRv|c+CfPvPfXg6%Z(Q+#K*t+N7&GEy|#F&=Y46wvK*TF=A*#5Eh;uNn}tBtvgaifM8ny+;T9>WAM0(6CXQ^NEo?X zJvHGCv^%DvK%Y7f4nZ-P!hR1<*g}E-kz7cJ$zGC{AwiC05PC$?Vip`4${tDu2UwVg z(*OWS%!&K&Br({C`ajOjlbnq*wpX}t>V&dpyblS$AHTw|OrTpefng({`w@^6 zT|^DkCwCj5rwtn!r6d=ntK}9CMBDEU%RvxQ%EM>^2elI6f*9$M5kWX>4P=KF-QsyM zLBUF+ask*psUSxjdGrm3GqNE?+eEvrP~L_xlg(55!hDAJw28S5AZy~SyJc5oLw}nw zhLwo%c{J(>hKf;F4q%JQKa6n2iwqeJFEMBQ@hjXs5V?O@Cc6pT6bWi*kHRvEYzcyj zvSCaw2)Zi?w?7KeT8@I$1AA5A=94~euEeJ5M5M1U{Xqo$eFP;I13N_dp7tU`p`wy_ zb@V>eF;NMuVjf)0Sfp)_cQ&*IF~$vU58PRv_dawTHt5eF)O{6X)&i)QtM8=+V1nO? zBuGgVa!E8%Ao|rrQv4)J-J}<%Bt!^%f{|82DL4uWF+>zUMO=>TQzWKRnF7x!NC7WQ zB$8ZW*ur97DzQ1Gd&v3EV8EvXMRi05PYT6RD0L$zA_?knv3(dqTmo&UNW^V&2Md=w zDo)U>X#*QIVK58^9s(LvVvUEXi>-qD6Yx)&akCZ>Pzky?LX*0po(!h?CYMfml^&EC z^^J$~{2KVGV);7=O7aGHrUDm{W`mhYgp~m0IRnf&&P17w2N$L%7RTi9qQ8O?DElZ} zaOAw@=yhkJxzq{cj?+D&KpYg}-l#Ot&GDDs-1%tm3>kFZj7p9D@VoYQco)Vmke;q7L_kKGcAb@EJ01Q>CYfyzVf;$Kt!i60XNa6dX*3rMrCj$ z_|?;fm<69BFd)%%L?WG!g5rcGo1r);kZfB8snUoGUr{TJm3~H+(j?&Z z3k-SD^hXo(9it~9fRL9CWPlXpd3jAr=iF@oL+k;1C9O9lc4HNwe&VuugcS8s%+OK} zZ1DY3CV78e!Ev6QwXC$7X_iLd2Sj!m1oW#ndR9TfXRm%!t8UAx&TOeh1e>x!*F00z zkPlTOidAW6m0}ar&hgjAI~Q^%I-yugYBqy3RzY-gaLSdb{aAGjBhC!oEK+4o)(h(T zUJ9iA#By3h4iw*v=!l8TgdO=xBsicI`&j>|AOZ;M!<-OX-I8Ui(qB)0Koi!(uJcpK z*^1ZGh0W0EZGEZZ_d-Yw9Q=R;lj|W@KTrhf;a=*K=w((Plpj>`Q5%e2RZ!qotNI4) zyE*P2d6d6?mpKQhoOCAof$sdVKQsP`h?iWU1(E-LM$A2!%!o0e4r0+KlXu zECt*pJxq3ZNxn#CYCWb)dE(OXTB7B`%`tOzIt|&Sx*q~{41H<5(iBZ7T@CAL_}48d zX7Fk_*6DMNQv+GOrv~Y2R4oaa<^YuAKGrL67BUocZh^q76IyKo{ER%*uWH>9#%^Q& z%pOP#XELVV%RX?xvn<6b=KzJtzAKAIU;OKr0zn@tEEPEqWG1n$k1vMGH5r0WQO{2W z`_XFXL=FM->V$!Rd1Jr$F0jBqe|1lxkj$V|evM+^w(BbuZw1(Q_h)zKez8IChpL-F ze|-SWDfBGG^pro_kHMAV7q(%(X8LqlU(k^-h(do)Kn@^-k{`&&R~=Mx`p-*np^X9n zU`V`XQ1FaII2J@3E4b*>Gt=3jVem^U7CTn~GhYGx=cJGnqwcSK9bo1cx|mW;3^@f4 zVL4-%C?2e&Mi=QP9($xZ+29{t`u!4=43k!wEl%~9FGks`_RQ7p_*;Hi*n_w&Q0@>A zYRflzmu&d>qLJW@k-v70DdJQIFUegM2^UelnIpf?*87g0M)F?+0%mze@0;2(ej;we zM%)I-GWsK|=w4+4&Y6_KI2xC{IZ!j>brQ@+lgLDOi`OLqxzyEa=RB z>QB&=59MT>J#ewxT6<&q{dM|N;xCSdUzqHjMjOLrF zE>8v+08)ReSUOINamXjA_qyJlP+5Fcm1M@i^m9~ipwP#dOI}nrGC1J^7&8dSI8rcH zb{}zWtNRLslnvcSX*dV=2AdGf?Nm$d`}DG*j5eB&zmASyoHn=P<=w0|J;apL{Q;2$ z#tE7i(Z_Kzo`Y`b=5`4aEwUYdBgv{L*|PKi*_h`!9p(*D<}O=!@$8q{71 z{Z?C_S4-7alndqs&*#@rXKMbe)%oF?4ocW$rUJ!=<Ko6h!|b0Djy)lJUVe@Z5I)3kW&D)5#OhBQ z9fp*Wt*Q#+0@w>jA z+^-;0UKj2KR;I0#o6QKCRUSj|1yLa7)(%)0%txXp==e!wlTGa64?W4|^U%(Zo;}6Q zy}uaiO`Pi>&osSu>N%J^pOJ1x#WyO`N>R^SM{T$6(UQ#>Wxk5JH@bC0WMYsI0R08wm=U zYWI+@V;h6L0s;M}h7dHA&d*lwpD01c7cEg2h(H~A`iP78Cs<(Wb2IP)^zj_U^*hM# zeF_cc$i?9pU976c(UKtN1e5&~`~-KiZ%oE<)E}8}4ibGOI*CX=rk31=HQ(Jb01udr z!x=20Z0n>2_VC)#KLnn;3-vBfiSMHCx<;ObUF@B0UR)S~p?h!_T*5H74(ID*?1ix_fnCucY({~%lDu}0 z1rJPlQy&AnF-6YH3RcuIDe{fb*ePl2KqyA%i_EL{5Vl@Ak()^LUi4YnMh>(fXpYPN z#EU+tON;EB^w-0drxY1NkiY$>O&FgdT>LW9~ zs~!R|#V50{-lf+4$bUvcK1MsVS{liuBL8}eMg9P0hbHJi*PE;U^y6GJSyzelK|Vea zOn<&fbfl>O#plE;6L05y6pd;cF^cK1So;_&@{p<#T?+Ln##lMyYju`!i(mZ3rO|U4 zWbK*+6P6i_ueXmiS%k?vy-=)h&=_)~M>IzD&EEdU`#Y|wPDuYAoPyOQH}xUb^>xlA z&l?EM3qTeQuI~^3nR5l2AvQou zb|nNPzD5kaQ5?KETmLpY|DLb&_hCD#RD}>ACZ^C8uubs<001QL9UdOG)e==G0OJMiqXG&Dr2*MB8Kk$=HLi9*Ho_@-0%m#FNd3Ar#B1!Kv5A`ef-&|JHXxIhhy z$CObSEj%~=Y2zhNv$qm@6bTkhIcc)mrw-$Oct+~d#S%r(hHU17v$bj^6Y;pj&k;>v z;RPWl&I%Hov)2}Ct)(XW6q;iT$3qp~w5^lv z>=)J=4_EXA4NIXF{B*Q+HqCb{+wrsz_ z2kQRC8)7#wK;CZGm^^0pC+7vfV);D}UA*VO1hV9I4lUM-45TRuJo!G~pZ^`nH-`AX zzr8^a*hHZasOm*wfRb!taJUxrVhE(6paB$`l6r9z`dB9(q?D_oLB@)rcsLe*ss;&` zSkG-ZJlV5>h-|I1dZ{%@roi$s12xSVg;%ldYlx4 z#zE%1PGzMS@0_wMo63^@hve`#M!MN?b%q>CGBHxC4yH(#2vhrD7aPTf=I&Y}-c1wrzKu z9e3<>tWMIgI?0}VGY4}t>;I2yz3-~CT2;?|UspZ0446P+5iBv}z zJStD){mD_h%-Kdkp#k%k>OD750Ay254e`1lOI!b-p3ey-Xx}_R(`(;4FD>ZMzGl_y z(D^&#OuU;}(@z3Jgvu2_A-#<5L@CUYI!ZaV={JE29vt$Qy7NYjg3QuAPYRcMQw)CW z9}IgjRd-k4DE(vWqREbC5gDQ_T**q|A7#W9C{4U^ltRE8)sFk}U#$<))jv$c*`+SX z55`A+j2|vA2BrL#{{MyPho4s0o_^any=1dN&1Rqo|WbW_gF5N%(LW#GKar= zSH*E!i=aghG5226P6gMY=RsB&@H-8Q;@C)iAz@vcusl>%LQoZs9*9_oFoJMD%S{nU zGjEU-wGaB`$(@+jIq-v*Cjig=s@kzVdUFq$=(JFf-olp}Z&+^m0i)soMfZ}|B0L-> zK(j?kwAnk9jS%VZY`nFkIl)SoG*8Duf>XFT5i%FAfN_x{(Vgmvcv(2Jvm>IsEJQR3 zZ};F>bh4BwIOzu*aOP?dPCi(i)aOBl0j)k+41(wYDpPJ2ojjFta~L}s4cI3~m13|S zira-Yr53nnXA5sSEschbx^Aj|Q8gh{RE#L6Xc|sQ7D5?coamkwhSxuX$Lv6lY$&t^>3==fNDiMfgq z9nK{FP$_-a$IRV4s-QOSCPauIgtT7|r<7HSJ4?e3an0#}{(6u>Nik*X4Z98!%s zab~f5i&AxENO6jE%R6?*5?FanK|-Q1n1vhGf4x_WjZG_qeBCp4##w)_BFn=17-v|- zXTY{FL0JSwfmp*47Dg~Qo8!vs*qCeA*{-4T4yY>mj~QeZGgjLwr5wI%_3E#7WQcj> zZEbdx%6zC=*0*uX)1UjKJw!xwLNJs=skgQ6Z5{P2xyUoM5$SJ*=0NCgHg@%^>Jk1p(hGgh@~&n=`j7C%`z6Q!Fi7K;yp`ua=r9x zYGPX(u-&MJ28a5Ve74eKKrZhol4^K=J@yl?=?_4jeJzd4ix+AjUia=4VSF%lV7vZ@ z7rkMg;MfQ_ItWSTt^}_ySZXd65$iA5&PtyZ zPv^$0lku+PBHu4ig#;cKGL(Tyq-*1$7XYl#`OW>QP7iX zXpMe>vx74uL<-Dlt2L2QM}4AVkGOFR5#otxygA7*%SVJEdbp3e*WTrM>o*9-dk~uC z2=v|DW)C*W{2{h8vGYh~<#z{H_e8WKpZXqrWKo5{rka{U5%dA5cu?H`tEOT_hSumE z;)p)rn$fv^%o_D-hv+zwwYGi2TbLB&vAvvx8Vx}?K9Ber=T?Hdb0#C>yO=)UUctL_ zuJRRfDhGJf=W^;;hUc(!EjTw2TcZLSP>c69DsIYs$x=Q4xi z;E}_-lICjPex4xfhWz$}_1}YBe*TGQe94gXcA^oq`J2pcf2{?iv`3>5ej z0cwNA2#M>H_ks1G4Mk5agwiQsmtoojcZ`~WR5mFk3<3+h!AwbzG9ZAmOpzfy;)7tW z3Tu+biNpY1SmbK?B&45P!`l0KOajgRTe*70b8|$y0)B8XWoCwUh$oi`YDdHfw zMNPvV?6hbp6xPphI?+h}!S*7+pOFD{0=UP;><7W0OF5#3)uKv%*)aaXzMG=kxgC;z z>GZ&XrBpCFb|Okqp|)m19CGRhP69t0Sf@Rz_M%`^(e^gDVe;@~3NFHZji?}Clx{RT zl{+(C8!DM@Ji2>n)M1~q!OwqH1uj5hebX2Y2_*8oB$oGKKlqWXE@%*A2dF7!_^n7h zkz$aWYmmpip07=u!a@{Qh*F5D1FbJnh$%<(e27%21XU7=S(91x9ahYq@XJ&sQ3RzZ z8a{cKgpLNmlO?GD3>#(?13bjg{tA|vDD(k2Q1WC6^tiFv$W62br zAw-FlCZqvRtAI*fvXN;!+g$lS?JQa6@1`R5B)9V-y>gI$#GS)}Yd~YW@9pv{L*Z>fGnw|Tlo)H2qW_> z;86nk4RH`(LL${{X4*tM7&0yu22wJF@}1&Nm6G4cEO|By2wN5*9xHd}k`v`e{sNSE zKbms6=;RW_?9p;#64Y2{jPk+*{0Y(AsnE=*g&dVW%LY^Z-nMeiWfD4GXs+bwgQ-%H zw^E_WgfCiA;2x~!AjJA|0s>s*UW&>+d8-|i_{B`k6A|SNy3B+9?6Z(0wvY-0GYKSg z_tJLSA62DfS(O_}B+lUJl;r8RkzjwfXk9vvfg^MZwgtvuh=Ki3lKq(`IVv@ZZmRn) zVJq~QJj$UaB2fYwaRORyUEj>|3`?!*(t+Ae6G9LhtkaLu#bspFBFF$|Y_@C$Y9&O> zuE{^nGVk$dH2F&HO&GpsVR<>V6d%#7$V*8@O?OPnU+HJ1<`1K2ao#l|A#cvgv0no`95Lo-K1dLMoOzF`*HBo_ZunJX>UuHW=IIMuI zgby_M3U4GNZwOskdFVck#ytr%@BF~C$O&C=LFbY{lV9N`noRu!&X}5DVk_mkD>ODM zIteRp&gygynDp&Xf|j+svdfSD*#USkPWw12z%U{M3U2%rI4ll~EA+!fDooqdaFdWD z1%Sev_OV1t(o1j;*gd2CLdp;vPn2sy7jt&R5r^nikbD8_&7^g3IX`dR;tUQMU zT8Jgu)SsyDO_I%I>wNm_V#`s6ztp?-7u(PPpi3a^m;kPKb+l&|EOt*Zu3>V_)nbXs z%BdS4Sk{VqRoz@iwcdF2MqAcK+u0gz@5%+4=JK1~ z#|w>`Wx+S4=!7fXouT3wFmT<$2(w&7q3Fa`unlxoq#%kr0*f2XAOHHh{SE$G~ zP(ggU?NA){?GW%}7{>2JVBP@MuI+H(_OviHrdl~(Hv|wmcplkkU`PsBKSpe-*37np zF1E9>R{4QgH1J_>d z0W*eb#_0NB$2blhmy1TmPhnYyupBYO*w}Vh*-iz+AOz?k(gx-ug8cbn;RA%K7)ma*TBBC zhSMh}A)7#I<5k#A(Yz`Y@C_U?zQT#oSQrX0NLsaV??;)-N=pqK3I!(Gici~dRLIjo zj6!Njm_(Qj!pP25Xm^n!$cFeaRN50j%QZnXC({6)%tZ^}6f?|~q~pt)V1KNbb2reb zYV(&|9GRk6rge9z11u#xVYIGkN!XzDykX2RqKqi43_Ogrt4_=oBV>OyQ3gbD*({_I zMCqKPKx3Q3P?;x58pP;=bKzL=oKQ8kDS8J(d1suOS6GjJX`Q-gon`BUUZTrXS3}PB zIF-!SO9itxcak?({y`n#tw%29!lXTlv729829EJw1^P^(*r z@vCQAWM{|ihl1Cz=&P0_Q-?V$1=$&>pC6~eyEghK7D=B+7FM-ca~4cs;@@CSj*YA| zq>rru4f-Uu)-UE7``;eW&H)wQ4N*?aNxqMAo_}4EOOn6WH(VU6e@~x1X74KZ(1Wf^ zm|6pVPYpyBhKn#Wp~-QD8LJs;@;qs#8E!L^9v7#NsT!=XK+Y66?BcK|pYN zAH6+tZLO4*#S@N;F$-~3ae83#vchY$SR5JZ#Z?;Dg^~3480XvHAueOv3Ft*+J|-?` zg)X@bF7Ra%$TG0JK>FNmRb4HDiB0Z%(@tNoF-#Eggwbs#*{#%xWtL4?98eF;WYmxU zE)U_pNc0Z0xvoNDcGpA}BIoT&2ReQ{BjH2ELMe0tBjcrEYQ=d}yGh}b@!&&bJ@o3( z0^htr5iv!=<`iCwL$c12kb&57t=qBU<;jLMu;C*DOT&|iPxn@mQ6WP~a{mB(Tt@eh z>-;*pAA-6bA{pJ0pQALQ;Fqdhu~DN)*OCZ%JN@<_!n549SWC1?JIUtHm1gfsGj63; zZJSMgfBWWP3X0|+bxVeGVJ5w^xpjS2b>)Udx-RxOY4$K&b2xw+c2c=zOnRaSdax~o zGm3IG{$}a+Gt;@TfrVGB8EwF5tK&2NPg=4U<4=$4jX`#Tt9bWkii0aJjZQz1{XeBR zR9bydC>~`180mcUgcNB0;Kj>V=AQcRlTOfsu8ezv%&m`%mz9@y@-##q0r+ENCO{-i zq;aJkEF?R4FDChk^EYB?wS~e#oTsj54DeaL6E?6j9{rcQA0Oh*FiyOhH0|BJl&DF` zc|lC$ONq@ruU8>oW?b2~x6K>kN|oq3)%80-z}JUQTVEto2v|xQW+F)XO#0z`07M}Tr@Ta zAvs{PGho{(V8@z~1sowHEzSN0Hju43w~A3A>8FHRgz3db+S$k0OTZp(;I~dj?Vv#a zw9jY<@AbySi^h+wiNHz0Z@CZ=!E#fcO)xeP#Y1?j#(1*Y)zCp&(cz|;E@YWzV9_q1 zH+T*3Ju(=M!^_uy6$+TO_Mb0fZsurU&^&2$O8R(nOXJC@$wW-HsbB4!7pw#}A7?07 z%{BKB2<)jW0l%v{C3biX8l_x5k307mh>f2?Zv&5u*z(xSw(V3|Tp42sH$U!a=kPUcpDwSarHZM8(K)I<1lOP%ru)EzxLRocCdDGfUDo>-KRX+P5F9RV zr#)_q%>1B65@&n3>xIjQZ(kUh-Ry#MSU<59vG7iT!3jvl= zC|}0V`EAF3tQ)lJQZvbm={oL1sratevqY5+M+?L6dWe=Z;E8;h(nqv_mnt4aU%$L9cKNEY z%G=jOF>4QrJ$mRZl0!>nY0+Papt6^u_vAzrVw5ngOihcSR9%D zGUEMR@Z=l@?PVT-vuRVV2Pc|pG`C?P$IUR7reS*(t>(Je{zQ-`?<+@}g3nMx(f})m z7XCxc8-~El{}npRB%HqF*iD9MR2iER!ty|s-nH^lBuKboh<5zIk|yVSOZKt_?24=k z(RsqJA?Cox{nVya90T173#IRh<(WMe*xI{3ZT5{+AR-{}PEOE1a zuQvZQp691wv~~gTmt!QWqjg~dj@{r}x?_bnCaF&uz;O+|hoG@Rg~NN$GsbB9kYE^S z==V5WnNVR<#z+3YrrE;KN8y;8f3@Q3U?GeCGDhluSYT(oI)Y-g{@9LOHqD89yA2iErpM= zA{)Q#$7FJ43cvP=-@{?)TB-;Kz8Nr^FhBN(Vn1m)k{?D>YL~Vb6vFrd9ZQEQ&7%_n zRnHR{KSOPuO;x2MNeLsC1#R&phgQLacp%X5uMZWl8IXRiZ-;AOz8?91E9ky&RdDJ$ zr@*~N!9mtZ{AU1n`JH5%$( z2tXx~&9$IA)md2^}#Ei;;AK1K$e=lJ^JjO3D3S{LYOP zYX6F@M$tjO@E&^1)EHs}b+^Ip6HeyQhdF77C-hiD-1soT-IJ zu^R?z0GuS4g=n-SlhOi?Oy+p{5p)SA)iY6L|0Z^e2Q2uu>EOBq``tp4WuMXY%v!Gk zU7~x~rE$XmE`VUj$@`vjlAu5y- zu*LW_`E-^XhGp;`x;|UXxU)sq?Ol>avVNgIS5wItwzo_OwU+M^swl9rC}#Z!KWlnI zoloXI2j_o0{GWb{KT$mFPyS|-@nP!2=_BTaiOJN_Gt&KFC|tz#%20|Q3Dfo!@JZ1C zsZphhm$R^pL_IYWvxhJcr>)>`{CScdXqLy`Z=X+I$)9?%6(n`ljGCcKVMN#ga>PU$ zRyWLkmTDS$>KYSF?Oq%=l}&lV z%+sE2t1iS7tYcI^qq&ZFW6KOzd@`%4;5_9VAn$?~HGKEOvpnBvZ3!2dro)C$G!_eW zSi%X*v=HLbfdajC`y}V6+h)$9;`Nq16h_9_9CV%`xNJAo~6=TO*Z20(f4*Zkq+ zD#mAYs{UG{zRE6Mx6O17qa*wzHIs`zvPGxz%8sY%7W^T&_tx<`fVA+}^t+}>q6hwf zNrjWdG`-<70YP_kU}7+M-8K??=FGV*tIN;JGueIIUo1lz>WR&5Qwc@Lj%H59+``{* zXkUB1&t#I;kb>EDI%i_e9|BJe8D-%sC(|h*^>$rgIHhOC*fN^s*shKpmQVQ7oHva44@*F`-I{_dOzMErw_v6#J_9IBljXzOYhl#&r1y!S z^&Xm0_KaXQ%s_I|^)%Nyf=_PFm@&B~tY6nH0L9f(_>8P>4e-ycw63+>5?M(P8qGSM z=>K3^wHEk8upm$2UP(=S?ydK>WO%nxhdocFtm8WqIo_5QqTNP=E`YnK&4Kab*C?|h zN7W6PC8n|Lg5#$J-9Cv;S@aThXm)jgKAW%)otbkNL%!R?b>@KA(}~EyxzmtOS5#s- z>4$HJPkz{!TBzc(tv|X5B$I9b&oXUxn-IThhqq;1#8|jYMdjqm*K49ysbTTSOPGrI z$EqPEHkqD=o@~0OVBtAxEwze+5^@`OcVg0oR;c!qqNusxW&aMR9 zdk0nR-RA#pUwK^l{o%8F@R7gWzjJyV+c5d9-K0>#=67+Hu^^xE0ayVrd{;I_-DgAV zBxQ2P!+6VJis0soGRg`<`gcFU2T?1?kRb*! z91S{;e?Qz7bX6A_;}M=FBO$&agEehI59{keC8_Kcp#;hwKbX9Spc^1b5OaaUz=0Ds zL2$o}kX#fJc%lu%n;uJ=zSS9XwiR509RT1c$O$MY7!FX>OQ^J6nJgJY)1Msohz=0>aL z$r)mNhjVPD%a4jS2^m}vvzF~RHY;0ekMZk1IydR-$Jm7`Ss~VG#vMBddCWy_HYNt_ z;Ck*7@LN)k*IhY+V&7JR;a#h z$bB&}t~3Tm{Rqjdc2_)@&}W#?>Y>msr9qlFu#2(*uAr%U0*>oQLi$LOW=>v!N^&(s zauZT&(dbAD1DZj+0ghanO*L>MYOoG!_7Zi#L`CihjQb|6YG=Lc7#h}jm_w0Ti)I8r z`Vh{x?5=npE~`{Tr~pS#f^s=b@I~GrHC(|YGi}mxm_J5;{F_W5m29j9pdeiH^O!@od=$w- zOy5F~2y|k=LX(|!B*0=}OS3oAoiBr5VL(G;$WfuvbfmaSwF#bS3$y!BcC;N(bSX(A z8%YI7rNXQ!{qb_*q+a1XUZL1irUZboelw@SKed3Vq%1zducFXHJM|rla9^c#Jt=Ry zMaMdav_Eq&?b5HwMsnjUPH~QhK0N z@^^=SGpn$QUGbwiR_C$&8>34&fM+TiI@=#^&q2ExB1+o?&Azv*KdL({l;?>-D&B{u z%?1@|IbI{I9_FYJEnL!;AnKpE26tHo>@rm^I@70$z*?b0zoTkEDyP6Iw8Fmaw6ivuBL56A&-B>~J6`Et-~QcZ{wS z(=Vk=F54=N)K&9*CtijIZ(-Wu9oFX4-`##GQ=zf+6UY7DK`EGu1{sqVQEzAnh>Sd; z6ET!vEE_^EQ7=A`D!ra8mA)toMT6}U7X*uc*8Fak2+mX%)~Aa{pk3w+W(gK!-o=z0 zOYyy&O`2Ku%VZH1L;?eoTvJWKKydxNy5J&Whsuc}a&Jt+q-WPpzA)*;gvds2g>D+$ zs509$m$D;KF@~sWPFox0Hy;lT|Ke;N#VRBC_Zj^z-HN5VED*r^v4JGe46V4OIL8<-JY3^9DK$2hAQXDC^$ z%g?WCw6dY)s~JTp8k(~oUS$e8W`N&c`}zI&23S{j!PGSXHJL#&YIQM6fVViMSJZOq zd&l@&@oF%2Y%1l?EXidZ0~6?&N`KdGNWftZ$KfEY_y_sLfU803TVeatGOsBZLHr2m ztLT0R!hEdZXR)NPY~sCrF!A7U!60OEXd$qNVM}fB`d3`U>XV#~IQ$>_AG`?5z-N83a5q^u=vHB)9 zp6w|eT{dHhD(jLC_a`xLu>L-AH`oRDRP4I>1&e zlN4t#c~^%g_CFb_U{E$wv)H2!LjPtlTgQIRhfJyrD{8ifFO&Nsc-X`a$D7N=z zi`1(M%_{-aocp8tL(D|hoJ>YA9P&`Uxdkf)UBt!Vgi(9B9&`EzjWZm}59>(Z1+UZv z#*Phk4G%4|8SNkP=S8D1p0h4fBs8Qw_8JXXZ=-k%(^{xK0jPNoio>tCsjlYo39rRC zm<^aW2PFcVH0i?Jme??M`#yIICvihrnh%C~^NAt>dUAoEz97nlqV7!GCOY@@ZP{?W zDeQ1bO>ZH&zcuZecB8aXTj2m#9)NPq?M)#x}*7PjXTUn#OlV(>jqD8H@tX7 zJl`-CijL~Et3eSHb0Wb+mjx&2I{&r1{ah}!K?EvsJ%3#$1<;IVBrA+ih%k;gu`akO z1R`&1ewWx8^C522j1-~<}m|939(bDk7#D_)5A~& zO9qC=pqJZl+GBGQTmG}XrDPxd$BHU}j2Qk@GZE>6K1i2U`Vkt_gMg5mi?X@m%X!2$ zT0VHEHo{)JseeW=%EA+7!bS}VzlGZt_Dl@?F41B)We%jXIjA##wd#A0_3EAP~{Ef|n)DsoH!wlE+?;W=fx#}JdPs3-nw3ZDm z;j=F-5+>P`32Vm`2d|dFqg$AVdu^xnGxv>XNAc?8>%pU+-xZn$ua`S_APCakugC}F z>gnaMOyl~~CdKwtGR;g3h~^zs`i!W+qYk19B8BnmWNo109j&Uf5@DY6?C>s9Van_D zYJBvMdzKmvZ&la(BmUbH*Tn-H1Nwcv_nIWm4dmIn-9j4Y=m_E6AOv#_=R+fU+@9P-ljJTLF*|w6e?tQ!j-cdW)%!px0UXp@F?RJoDfuF zsiql|Ka<*5_5E>((&Q?Tyqo@0Ot0!GtLz`?K=;xOQ5F<#?T#|MUZdTliq5eZfcPTwTYY?tH=r79UKU}z(ut`3nfRAb<5V~c%M}13qkz&nKafJmM{;#tRku$oKeKl8TP2giT-_$|fuidM| z#zHE-karrfw^NKv&?Hu6dnfBmXnyO_No^3_Le@3AA4X@UP`qD^>0k8KM=S)`evq?# zuuB^$ndco35`~UQ$H%x6e|ZjRY$>Swq!&ugAR4NOYsL`V0NV&Rt*s%RHCNi`f;V4T z#yqD-QX%uR(tl72SiJ;qzWV;KUNkV-R4%~c+xiC@Wn8!+z7Mv6&EjlPLJQ^Fa@Zm9 z&&?Br`=sOhLu}{=0gV>)7mrlaC4c-_vj#!UV03=&Ydn}qMv8&9XJ5S>>tk??rC$Hy;oO^dT zlE!M6PCL|iHCvt8xEKAk-U{iAlHcg%eS>>_dHy2&QWa+kF_azz zELH5E71**HII|y7;$78WM~Qp8_O%Tk8NHZd4NujtsjjTLVC_AB*Ffs-3%^q)d2exJ z8^8t6f*;Hir?92^p$bxup^k=@ZVxMp%^k@%;(979{k1PPuV^e}l` zN#Gy^HIb(M-Bu!#g-9u-f$L?IBp{I$3=vfb`GqOv^*2bnHE74F%XA_5~ zE^xqJgloT`r2Q!3Qd7gqW;88LBoek~+?rX*U_E_rwnC6$%$+Rotx+>=aVJgBH1?w; zCXW(b%PwJAy#XE9&Ya@;$Hd?pXe0_u9b8TjQY|mXlsIm9;2K9DB;qm#9mGH$sGud< zn)ahqpjQ0Arf%ja8V(KreZf#x!ycl^XOtenaGJ`ik>Xd`GzKfeItv?_=u0d6t z@I3rvSkv`UZ8cce;yaER>Jps`U)|$dlnef=2)&}hMT-pStm%~0v#Pa=$o6Ftnv?l5 z`(L3|8zAM9`zH~DUC4~I!BzBTprB*(5H{VR7j=Z%Z(mG?_o9&vPr~a-D-C3$rizZb z`%T@Yv%=);R-vm*4xK*NR4wf+(?l_yHFFD5oNkUxdOxD?A&N3aL!FxueJyF2+R>KY zw5My&m(rjYzN6jW7nM0B)qxl#r{Ao$<2}6rVeKbFX~xfstr)u;E{_&0&Jp@iTUGVF zjtKS>F>C8QyYzYeF0XgieV0&9Dk+CcmsqZCWn$Ejiqla`U^kc_UDF&u&W(6cXGFhk zYb$mEa9x2#j8k1yzt;t0OK)e4pdtx>RyC9VP8j-3Q{O@o&fJi<{-&pjKr!79GFu($ zKeD$L{)Z18%F~9C0UMIRt|0ubmXMK#>^X@EykjuOfiNV=k_}=@y&x zi4g*NWNm!-Hz&cQ+B5J&ES|;I8Z@<|bHnp#uX+^H8%!{`XJzOw1k}OOG)ZZ%rtG-| zb-gj#Sfz-Gyj2i4@W^$#JsnueEXw5KzhhD2&s15wFX;tBj2GhJL18VVfFd)$U|)*a zy79UO8f(LzWLkevP>IBl{no!CSE_$+ICAhRjl+9Z#~e+Oa!|vKaRQOFSfR<7Oc4zD zgs22<#1*zx!DJtNJ*sUc7fY2+l8eyNk8Y^O*-0+Pt8xmU>q^Ybdc1<+;*~t_j7Pnz z#Rs5Zobp`pmd<3gyAkyIFI>e3;$;oTq_q-{#PPqOSs5`akTC(j=(qIzG*BBP2YV3= zn$2-;U+xe_sTO$9P?FtED9&OQiDEJwAjto4t_?U|>~<$y!KRO$2xvoK6OGJ=&RJ~x zN*VxA`){25qloSfD=lcFlpvr@v4j@+k3`ZQ z7bXrYyL2r%MqTrfHbw4kTx<#iB?>0*nOU6ES|Ev zc4VO5R7XmtApxxl|7;<_ouX6DR>BicmnqjD^+JJ_+Hof+k89La%$`%tw{8QqL;h8; ze=F%Uyo$T>m80h)_%&w{#U8Y-)%V+{3{joU8%lp^TmwMc$>YDfi9MWfQ3AGh=3uJK z7X?~!iLdw$1uGLipPec-cJ99LT??fT`n<$zD)T%GA1K}tS%dmtaSJq zTl41klf|CzGzk*QK33wsH}L(J%E$Wr2O9+3xG8B-$iajCuxyMOTgmOg9Ox4%&~upd zm9Svnxo-BGVq2L%n+63w- zIdPjU)KfIMjcLvpYAI`KvnFBQ`Fj2d4GIvl$%e-M6bdOYvLeq?aj*jUP3DF`- zFi2bZK1Av&&lCvpfVQx?M~<<}TEKOSVeLq!%8sp*fK6$QdJjd)5&}7Lh+Uh2DMEqB zV}`jizFWX7Ty>5d*}VA#PN>|7DZ-AG9f}1)596m>zzAZ;!KP3VhdD5zz{|YFjdNib zL??7ZfJ{JVCV|-m3x*XCBd$}rkY>lhy`=z@AW>bCiYLj_MRU(243LeX*~Q{uFZ4Y? z9ERHS0ZT|~FVN~8Nc_J9l@twd7C>({fAuM{#+wtaMd}v>baHiyR@NWCDDlLaH>I1g z$j{X1OmJ*nuz&xhB;}yUro3c2B;Zve#2%($9)YTW!uc1RV9~NEzDOd>PBiaARKI{b zuSVNxN7;kkyu6^Ms7|2KLhw3F>&8J>IF6~Jj#a&YRg6yms78-FOs#=VV+3TZ=U~!t zB`m(%Vi-Yd9m1fz#ixvBR$F9lkHi5%&eA8sF{7(7n}!k|a|pI!(4{n>lQuJHxdx_k zVB8PkC^pTH(~z?c#n-s9YAsSXHNKKuu<~Fs334(9aS(nXGc1Za=?+-j1&5)dmp9mBj%q|>^ zoS~by#D~qKC+y(R;{0lNggN)a7{kQ-=+xayq;0n(_b#MHb_m#b` zb>oXf#7idyu3@@mh|tTy127sC`nxIger20A6(`#O zaF6m=%U0o5EV__t*y(C64{DSv5!z&Gg{EqRTH4f)ctKb2zpw~5#YhK3G%H0V1k`D4 z(1v5l3E%eDDTdg1?q zBiq0BO{If>%LJQ^;o zhWy6DvintmhFzF?LP_xT^_*G zjTtA7bt#Tb#Df(mDb4-MPJD+%IG|~#WBOBC9sK7?v`L4ppSJxsGxgI)yL=v#pbkFi zLC4n)*EI{C_f#H&>;2&b5%dMpvtrc1ajl393+qQ}AuzNYP2v}D8Lu>sIvl$Q54&ET z0#$P20dPERaP+LnZGbBICyz7iGXX*-ZtbH^Ylg0vy_es!7q69%gO@IJ2OAsY7?G)7 zZaQ8;I|~0AYrAsYI2o2_i1Z`4%nx1*2W%blb{lM}lb2$A3wzNk9d%`Qaz&nyz2jqT zMEu)o6n6386t(^*v|wyAvFBt6Ja}CcKIx6K7P`IuK^yb5adGe4(3EyS!-Whr7zK+R z6jv69^h-Ku;5IB@CHxt8z!&ORvw=e%kF&=-G^m6{bCB@1Ce9`q=Ht({Vg`&+G`AUE zOPdX;89Bhl*(&%Y(poph0WYdBGW$g*wl>U`+FOYZ&zS;`vt?Z+T{?0|R-v~l?(Z|s zWR`x|dR!KC_;XV@@=-jwWpu86jLW#%=*yaoH+ex;cxFcQ{BX1bWz;vQXw0({`>sTO zz6AdD7)rP(HNMbI?}$g;lXh+cH-OH*65p1IMTzq1i1 zNiC^z46P20u=q;@=^_8or||VP?uiXib>wilqmcBNkuN15wbUKnw-~>tnE0)jT%d&7 zwE`oe?*&VG1qU(BC>psyHCnV>S=w>rD0I{&TuqfI4Ipt{(%FoB@@ zAAi-bZcTGfP5)c%kU-r}-*UABJpbUDF@bvWF}>wq8lqD5`DKRU}#mopDyWl@cP5ICUn7OY{F*P-X`4MLc*=4Q(gN(x)#!ZE$o7=+=4g2 zy}S&S7GA;zYEiU(CU{Z7HdMccQxo_tyAozAf*lx0o#xD;GsbsvE>*!!55G>It$YW< zPQP5lpkCgVoEEV+9LL3e^dE2n7B2*x+8r!VSN@{M5+v!xUo`5()I@cW3E&?xR3G^j<7T%glU zN@z^We@w4$%;{)3c2 zc#%3_k-mSC`D1bEZGk;tiNAkI_+v>-c$r*kNveNY>0?<7UwE@TV6(b^ zlaY9{UU;iLV5_@-Ym#WIUwC^gV0*HE+nQ*5R(NM6U}vL$CzEhzM|k%*VE4R#*M(sB zO8ECGP=8oRC~z?FFE%I?7&sKv|I$Rkz^K77z^K0#82<+v zS~L{)|AK~ckp1uX|L2eY&xKMjKJXT(WAOiAO2Ofd25qy4!Y1dc(16Y+Lt!wOEcRz; zW|9$DjQVL==wF4_X+Ep>F7!iS6q=MeRrcbMd>Wh)j|INlfkHBa^>Ck!2CZ^7lI`6k z9T$~q0<#~?j;$vBVvc0?iQ(H)wM?~4qjo~iqs3Y-bv;i0n|sw-li@;vQclK|ewEW~ zsD~h~qfs|tW!GVI)zP>o=tF&l5bv+aU=(!bF=6Jdaf?1YEl(fb-FjW-nAysEdz1Nm z2Jdq@5x$G%ObKDswa}~EpA}!ht~Z%po_;=U%e9V#-qPzGkmG3qvSF9gL2rWfdw$=u z>rr16B!i zp`U=@f1luZh2D@YyoG*OLzzYX+Y35-fuP4{lVA)M*`f#-p6I<01Xm!L4~iLFaWI8C zUP%;gNfvoD;rI38IMNd@ig^0Pu9A4>mY0%5_B&q6ByI?wFA(WLS80l{^b2LG7zck@ zsuYG#SsGGm)j>42v8`E}s`7?qs`h5|ajGVd-pLPB1b&)KTY>J1OoxJ3nk>E8Zt6Ub z8Xwwxi$L3oLYtHGlY*de__MqS1P0pTkSE)-(pZuyx{`zzeEPCP;~LwFJpE1jg2KD= z^Qxj$hVRu?G#kkX+8$JBwGBNW1iL!?S@}OT^;0=^G99~b7Y*$Y3Tll57=E?QZ4`vH zEj@3swXI0()3r_W;)ua*L;8Nq?F+8|YCG0Z{#vW=)Tmiz3#luVRY&} zpCe@LpBqrP?tfp6b83Ei+j1HJ$9(_W3=Qe;+yYP8chhqCo$+=QOUM3p49~jmcAUuT z@^*qOis=qWm1%!BNmp5SH^tO>c{j~A!E`^vwPt@m%XeCLKPU8jc|R`-%lz<564&A3 zzsvQDio913OR6%=kIR}m4v#Ck*7cA7!IVC(nM5%^ty^R|JZ;!i)<139cV0bhIZrS@ zZ@aHKJnwj))<5t1KZCBGe+R>|yzGVJI=<{j(>A;u#PeRi945=KydI_LIKCcdSvS0% z$KtDb>H*#zrTa9tnW9Y zxK8i4K-$LlJ5E?~Sjw_wiV4X8$$22DnC#&D)3(>&k5dLkF$DgjZHYOa_;|8kOvk^? z0hD6?tf2RYHI)~{__n{FJOP@`XwX{g<2>7^1K)sR5Liux5V*iWm;xo}c(S5btA(id z$$jX9KVe|hvWfK339xyC(!8I^qOdtP;dD#|L{w)3dNafR8p)!Vz}q-IB-PNL(ZAQ| zg1_pqrNp~%p>Lo$=@8VHU@IvhGlWdwD2;kN7E|8@PoN8`^k6L3&n|LA5LrkxxF}>I z9SqO1X;NU>A>nB9C{`q}72U!F{i*1`0NFq$zl}Sv;Ki9y;0cDH#WS)I#1dZY1hUA+ zI-0;nmjP0cgmj`J7s*J01fhJ>ke~o_$dDp1LW?-87zl3B4&cP`AK{3^3+kvc3ZNm5 z{-}n&?C1npjAJ*0EM&!GK#hOIGG?^2Wg;8N%U(M0kx`@}7FJaRDh88c6zIS$?2!*$ zfC3b%aAh3xafuiB%mabw#VXQ3G!M{0A92*jH+V4$OEiI$4awy#VW!Sq`tqId{N4-n zM?i8gf)P8gCIHfK|2=xTj~srq76nSMOyQ_c1w8149O?M87ie#29!SJLn7Pe`tTI;Z zlqJhD8qbb;RCyABA`VGWi(33*2NJLXIchw({jy)mby z7%Ioys9NII#<=;-f;9T;kACzc9Dh&>2|A%++GyAx-hdT}O?={EGGxXzo+v7FjLzs9 z`N)C`pa2z71PX+d9J#0k3sj&|mO?;#5=cXJtAK%bm6fJv&2Jn2(T`)a^{w=E?_2NN z*8Tot{~p)ig(_&V;(zEv7zB1RoaIdCaN?OE9v}oI20e*aI=CYResi4XT<1F%5(9KF zusHZ!j90R`Vk5}qIo!E13b29D7!HRg22E;7AOWppz_33Is|i+Q!yo*JhZ}5?fkDJ^ z9?S(rC{)4QT*7~lw!v*%oG}stXGRDI>`QA66 z6XEaPjDP|qR6!23@sC4iB;foexIO+okfEM8b z|7mBgXvxf19P!{LK!rW((TrK402%*~M~u~n@)cM?p9e3Y%42o&obMa}J^wkhm(Fy- zrQsLkn}rW-Sj86vL8a3m2NmvFK6~DiebwE@OUZG-fFhcpV4f8Pj==DLs~N8Ke*2}7 zzF&R!L^ttvZ{4SH_u1sVBMWbMV-dagzW?LzUt9#4r=dhK)0LF9UadcT2;*j3Z0H@y zKqKt2k61K5y*R+R+XW)pe|V$GxM~&VJ5u`72b5JsZo2GeKej(~z1)|W;w-S}&wt+D z+~$^|So7CGXSG`eJ_x}bCXtQ9$HLxlmn+BfTlj{gl^^)7K3egh)(%r7@D8U)|G;~4 z{SM=R8urk+^dDa_4#?cj*GR>vSs{&o_(RsQ2S7WCj`KNISn>Iw2>R3l=qC?~fO5{V zf3ZS<>34u;M|%($fp9ZJ=yL|_b3U^Gby2i|1XM!;#0XVj1%2=bau7qe@@2bW4`g(B zxX^e=M{nylcjMO(BVY#Y;14xNeK*K=G-!i3Xcgc$cn_z20#SoEn1dq_1+TOY>jr&u zH!(AX4~lR^2uA@i_YX9ee+rlpBftcm)D49YhGJLQi1*+^yJt*M^-#AmQ>zsYkYIN6{}p$RCxb*t zcRSc-l~`Ys$PhvJV3Rn59%m1!)GPMoecCbtICUF4H3Bbi1Q_-XXHZl~Wpar2aVF+N za8(O70Rjp&QT%WXt&j!?*b#8p5xQt&z4(j4h={QV6$PNkgOq4@^`(yPm=T*uc<7j4@W_Mj)eM$5 zillUk z%=nQY88s?(J{wd%8F&xVNR1L?UM|!JXFzUMw=3<|WEgNt_D~H${}538XpWW`kM_8Z zJ;{zg>5~mHk3(3GNOzC_b4sBYgQU1#%QTR-LIE7`3I8C4mNzj5nO`3VUs<^>^+f>~ z@C1$^4W0B4@DK@zl};FWf?`RQW=R`qxt1YWmv%`sK|n>M&;)1D2QFl8hNzPDM2PqF ziYh>Ev2uJ1|6y^(>0?D01^g9$JK2sx znUti-o^sWh6hVOH8Je8=o(u>NF=i9%IWcKfW<4F&_?{F|0t5;nJUW!@$eLy34GJb^WcHNq!ixX!4sPH_9ybm+ z)&vhIF*sGDvGQU5U=6orF-bUcs-zTIYNHbqru8tU9J;1#x-#IIm-}Nw(TJTLn1OvD z2exNEW#=p6$q?sf4^Ub{BLJEHzzrW7iI?gxt1V5Las1UQFy7Xz96pbWAI02?3%r*?3G*ALN<2QE0P zs45U9APRNhTmY~FjX-8*HWF6(s_ex6 zctgK11u*#w&eEdpmR5I0gG_*7_AqIxpbERte3B@di`t3u34Ll6XLL4tm)e;s0BQWy z53BWr5_52(MR}>P39hDA)ldYqS*|u{f8#V~*)V$lnraC-v9u7cs5lXs6&nW(@v$L0vL!2*C~K`Y|9i6<^GNmtL6Q`h)JP6gS3nYgm>$HZ zW@Rfcz;dH^Ef_Ed3^%Ze8lUgkuRh8U=>-CxAh0zyobdUM@0ng0@CF3B8~cC_qiQiK zKmlcdenJ*bUdk~XKx3Z7wbw!cDt4Ck=MT);R;OfQbLlE^8(|a0wWcJvyYaU;o4ATQ zF@F$M*+xFP2bkD73-{!gFeCvH#0XbfUybQ*FS>d9;A`%6N0T58>R?8jWeS$SO*ksG zJ8G|0yDBRX2)vaJ<&Z|QNVS+b0v^Byn_vp%lm>Xv4rxX)LB|6k*AL=A45IX~0+j)F z5L@=359Oc>mC&mrun-<#2lUrl^8gLH|DZ>23+H47=5tOyfCbgldr`Cv(^CX0ptIZ=qKtq*0^=&OL;)VD z0ta_dA~UTISB0s?dGe+)LuFA1_c1SEc_WNyuO}Mj*wsatTf{c8z~Uoz^#rsB z)DtSu1O~Lid$R&{AY(LU59d$?Pn^bTJUFv51=(X2-Dw49UEvOLSQT+6n6%eb7&y1dK0+{?cF%fKAW z!aU5xT+GIN%*dR~%Dl|X+|17W%+MUo(mc)7T+P;e&DfmH+Pux&+|Azn&EOo);ylje zT+Zfv&gh)Z>b%bE+|KTNIGHgTu5lWiAsO^6&#++{rJ)%6Od9d58mPe<_YBbWtk3`a z7y)h1{XEYLozMJy(4djf2AvuRUC<9r(GhLY{*2MDA<-8t&<*X;8x7JB-O(D&&?P<4 z6^+j#jnW*g(izRtC+*N9|EmDjEz>|v&_TV@LS597G9gG^B1x?! zOKm4jttU@CC{evAQ+?D*ebr2j)lRL|P|ej+?bTEb)>fU>WWCj8-PLFP)oC5pYdzLx z{nls=*J>@-Y)#j09oKk0*Lhvndwth>{nvaA*nS<@ggw}XjVUdH*r}2zp~Bdv;@FE_ zDUnSnlszeyUDb+xDw+-1n=RR$UD=quDUa>hoL$-N~)p%T3+U|Lxt=&E3xp-r^$Nc<2>Hu zKK|oC9^^tkqp5|)4=4{^PZvN(Q9_MmC=X74@c7Er0p67bL=X~Dhe*Wix9_WHT z=!9PAhJNUX|DNcIzUYkJ=#KvAkRIuhKIxQR>6U)!n4amDya86QxtiX~BY;P$Pzl1C zxU9KasA(~FYYgqsp0x5thUB__Fw(c{yUK_fe5y0;2 zKQmuUinFIa9jRV1`W3rd3P;l+>u|$EJ_JN`$uUVpHp5OE#SS&yel_DxE7<-+LFBx| zbMD%%?lAN1f_H*FgznouG|EoM%y{bXUNZ6l!>u&NB!RKdJqgx`id-b z0>RMg_&zdjMD728pdNEbNd-15KnR=A@sqHdLMSyD&uLLJ@^v`zpJef#v&nkZN3=2m zs+v=Z|6mgx%&G&CtjZvSD8Eo*5k_M)b7!Q!teY|$-|?I9@$nujJ74h?YBc@1xTpSa z*^6|H%&1DwDnXY_vtjKiGXg{K@Q%7Og=>Fm+A&5IHVCM23>WTAbM{i>_K4QEP1!jV z&`PgV8yQa#UQeL@fR$)vYc{_Dcp$K@C|7b{_CLyCDfmhG;BqOG_D1<&ho8wX^Yn^) z>JS3$1n(>(pc`C&GGWWMVZSrjq)kI__Oa;hSrhqD1NTtl`WaFB+vGXk1WtA~PUVD~ zu3`cYPa6TaQ#D@-@AyjaAb_!-O|$PRIF%0xt83Ead@jTKaftk<@AT8BznDMV1CNyJ z|F!iDZ!*dW3lS+{uI@7q6*90NHq!qz>EHIL4=v~4IT1Bc`A|{&0=S>h4)b8P_;3jT zK>@%(41?_LD>V?n2q(x6#(FVA;Y0v0Bv7PS;?}=-aZ~_Mp@K)JfB)W{k%&>{N|r4L z=Fs-fToIQFE!M=DQ|Hc!6x!J9C+s6nqD74!MVeIUQl?FvBGvHeA3cvvDQIcuZ`x3z z9bB0_^OU6tSnBNk3peSXR<2^*zMUw8+P`&8Qo_ZH7p6>_dHvRuOV{pS!Yl*Vy<1rE zV#X>l;Hl@YTOSIESz@4buvH^_`4n+2crf8W6R7_2Q-;LyPL2LLP8@M!BR)lq{{ifs zX4~}T_EAX5oyx7<`o}QS^*GQ zu&D_RXQPXLd2Dv7 zSSV$<<7N};^jP{j?0@Vo`|KRC^0_2`{Mjcp%jmXSX}tBuHz&u}n!0L*Le3n$lK19~ zXP^D~duXJB&8L*gatiq$ZqC@QGevI2&mdS+uE(Bt+=yr*$Dein7T4nFX={1<7}fNS z^^=dZCL@sDc*KsKrHi!-f?OEF7LtVGt#OgrZOY4L6I1CmiuXVWq+m z*$~!e2ttN|`J+#oU`00k!H;;j;Uh*wq7s>y4<|y=Gycm7_%Jd85?(_Wpa4ZEq-2kF zSYrw~YQtJcH<1^1BuR8o$6Ct4iyLMPRV5lykk}KlCAt#9-2xnyg_bMd-3NzA=t-|E%L3`6wf0%8-UO#GwxH6G|)U z0*hMYq8E)ag20?(FgWo5XXG)o;IYb8#w*1J0Qrx3HqU-3>|qNtBE8q-NMLyb)KS=o z3OB-0j%LZ`_hPck69N>Ho)fb=nvL`mdLQBxKKosSHFmu#pfHmhL8qkqS0u1RF%7TaJUZ#Ef7s zClLq*s-^-pV&sLP`r^q3LNu)opyAI*z%f)g@hp4Fz7H)s$eM3^6gG|3dasP=zpoquwZjKsptZeb@pCABmhl zBq)(Yu8olkDJuZZYFM1SaGqi<>k6zOLb-GXuN2U0J@{%I?g*#a>~z6AG9CD8q0>F;rxsR#+}y0uwBB7a=_2y{Z&w;Ap{CgWRzyw@n276$Yd* zhDmTV9+W{!co|EO@~NbFhA9#Fc=0hnX8|3R=wwACd|Wz18t!e++9}rTBFOof?PQM$j5>1c3sodWg;zG6EdP=rv|WvzpoL=A<~7C2t-?J@k=_ ziFli%V zfIB6VGjG_Y;wdYn*Q0^1sAECu|7V}tYRWvutJ^w)Jns0%4~KZf>n)XgL5@4Mh8+fQ zjW}Iz1p{*!Siy1v<;6&O*NO?u^<3Qn)6gbReg?n`m5r0LD(8s9I+>@CGy)K8#2g`! zITkE~yW#mY=i1nL&t)>-qQIObSSc))Im*GU(P$yA#oj^&?!RnRt0OWBVcXKaROqjTgE&t?|55m#19++fb8-xJGtQkb-HFZ?fwh;(7EFo zF4u&FD7{81fQYWOyp$k{el52L%=m9d4IO6?{Dyw*G-1x_Tc#w}Sh={wZjh&FYngFruu!jn)5tYe6(lRR8FoUdE zJrXn~6U?3rE5HLxzy-92+mpbRNWo%a!4`bM{40w8qm2gvE>zMjuE+su;D>~RsR6Sm zf-neuAdr&i203WF|A{cSjerMHID|vkgiwHmdQhfnXaeff2?8`g1XMr<1i#mlH9~Z? z8mqvsqzfy<@RfC=sV=d(4#7+bbIJ?5q9z zh+yn1V*I&HEH^{Ez%hV@bSNh7xxKCNH5wDbMzSw|sD}A!t2}zdFY-DM>^p_%#ialN zl9(KRxQ146mBcfD9Y4pUYq)NX4 z#e5-1awNWUoUX|ko8{pNS1b%zluD+E5fI9{)}ROB6Fs0bykx_I@e{kH_{IMFjsKg8 z-dLQ|s0IByzmKrXkHE_Sq)1zw3~Uv-02K?;iOmd3ijqvq1f$olGVGk53I`m^iRUL%0+rl zil|K6T#kRRDO8yPKBNe-Bn+~&jhKQrlN-l?%*B{1PQVC3^TRn{L^@%FMq|vFV`xJZ zV@3~EzYr};H*C*^FqP&ShB}Y}AJ73Y$c&Q%0NFUT7kCcJ8p?vq7jsO=eTzzrfK8;( zs`XHiY=nnMNT{JCQI^;nvRfVo&9n`jyY~E1rrXMdM9Ng7&wQZ(C6xvybkjb9PA zs8~stc~Lf%Q5v<;Fm1~Vl`JhiBB;#Fq&ztq3sWd9(J#u%BK^-(MOC~A&>Sh%AVtV8 z`j=_|q>Mq-k+VM6e8udu4e_W4cfuWF3CK}IJGhifc0en-JW&$}1yhKGix^A*P(e)P zjbvTcW(9y}?X#E=*4{%nA3!m0#8Dd&ESS&=k&J>s+E#8;8LT1IjM2~djElK2))jRL z6<92aED5WNL}`7nt>cNnnAa?&3kxLBa~;_7Syzu#NZiEHvAS1l%-0z#ieSM>|8M;w zaE()96|!h0%zBv)M4Jc_^4Bz3oI%AW)M|+C(8JK%i*&^mf{HenQW?mcSnjCUZ`D=y zJXjmlR)U2Mh$+8LmCrAUf-v3Jj&(K5@`pqe+EjJgr&WqoT`q%lS;b-3>1x6efFN1{ zJ)CgRzkpDe*p`N|(w%rb+F;G^V+r762R0ejnX8G6L{erP$(~SA?Ls4?tJ@q=(WC%a zj>Cw@^RTH^2OQ<9xp1c}quIB-9=n?jZ<8_rY}&VkTeoPdc&dlF`VzdY2}i|AGzBw_ ztqHbu+t6*?=JU3NozI3%EX>thqYzwRJ;cJz*vicp&V33S@eYGL0f)F3{|Xp`ITT*u zr3Cz21x!E$(=@OzDV2fQ2VB4x(q$Hy#fam;hf65iz>TlL)gapSlE1|X=~W4`gNLEn zQzG@$#VlHYp#a?dvll8~Q^km}br_tW-?F`5`_%)o)nERtiTwTF{jJ{s_Fu6TVEjGc z{vBWgF5s2`U@LdcJUE6({jZisc{YwgJzuX8Gf3PF}`9-GsVWj09X!DZe@U@9h zTA%cQ#gl|tsnUtifJRu0X~;LsIE%I&3gFw}{anX^Fpn_Ci>KNj{~!(GA$Exa#Ro{Z z)9K}>J}{jcUScfPiY?9zNsG&&1yj~7*ffsg&fPwMtE_(0MbTY}HQp4bPyz8U3GW%m zambrJ94mW$flO{Y&Nw5GC?3&J2o=z{9wDg^ zmR3mWyO1zGW9rjThKj!EHR8sFCJ2WftDiQTj}U6kD(dt#lJ(`Fsjg~@!0PV!j{lHp znpPc~Mm8LFHII%7kQV8QAnEK?iN_Pil1|wswz%gs<>h5+bn{{YeQSo6>j#@2UP{^5 zk;<4>w3)7H6ke{CZfP_T1ExFb=M`;4p@5QHw>oG7{~dUONq{1e0_Hz%4hL-Pu7>Qd z)-2}I2Wv=bF$NDK3h2p<6M&u^fv)c9CSmEmZtDIs>jnz%{%7sZ?(W`h@W$@%CU5mN z@A3}s^Iq@qZg2K}Z}(1b`L6GYgJ&tF>!2QLgh+{n{Rr04hiY&Xmf&Z%-~mF|ga+>f zRc!E0kOZsd4LtCfJz5IY<{!SR3B8oZph`)(jguf;i*G=tY-nY*I#DB*+;JL@(=c%x z+J$Tg$z|+e7Ux&>*kgTghE|e>f?}r+vIlV>hEdprYeM8umJYkYJ7L3&tmp?|>Kjz? zS)3knR#Jr-FTS22@tLwA6pvY+;MGA&1yH~u|7j?R`27e`rlOMIhZv`Y8L#oNVjUPV zsW=Do>~LW(rv)%4l}P*W8!9MK-YD9Ua+FF9V)|6R=87@rg)%?$G&kS>sI4I{@*|g{ zBo}dtVDlG$b7Ye9H~>zUp#V=An{Y^lO~|6tFmN;t9?ru~%pM4`d*0{<^i>jcRBH`* z7zIqALpd6mDyDQHCvqc4a@OTyW$N)>`0+5mw9~-#W*@632MUr|MD?}QPIq%S4|U)a z>^t6jV{JDUzjpYw}}0soj>WG zR|-^;8R#_4zRCiJzuy!x8`PRvKowv__Xwrvf zdZ5=FqoED1_=u6PDb4AJ&5`+#z4@ePk;80 zfBnv7mbm)B*ZMsTet|hZe*y>q0ta?*_3z!8feIHgZ0L}o3~K)-QmnYH2}2bb=19xu zs|kgNB1K{tq^@5x6(a?zSP{sWx^(4;Ny0b~1+stcu&7)qkU|DKvas3HS58+YkUkeu zKw~f8rJf5r+_Gn`h`tDE7QdW$hZa>~gfz*j5dFZ{NRx2Nza|Qzy@nxOIOGYy7u+Sy4a% z4~`U*O+Vrg6Uht6t(3tcPbtz~N%rhhN<$#HQj{>4we(U<)7iwqB*kFUjy!M8XoQNlS@w&a7Ug(;f?p*K>Kyo zUvFhK!~-D05qFI)|DipnkVYJNlp2p4b@b7g4VAejnr^CjW}6w!$)-qe(wXO-aoTAn zo)766XqtffxzM18lG$gSh%(Bjp^YvYDWq^pDv$<=(G$mu)kQX0WsN!69hG@ufX77d zxN&Jg2PG6Btg#Lx!GHtG>X3u7b_#$BD)s8#3pN5U!liHhnq08}*qRrQFD`p4vb?h7 zECt8fs)DczdCyW%vvda>< z_d@F~yd&fbEoRn+ge{`JiW@Dn1V@RRx%+0caKcNyD1iiIsx%1&S~V2$A0^*7!bJ$@4fr}dnM#1uKVxB124SVL+}Dh zD6*iKPbpJJPQ3Be2Ol@?lT%-P@z)Q(JNL(9kNfuF_wIeGLd2mkg&3Tm2_4eH6s|CF6llXl{NWGzrRX2V1TP!s22w+M3%aArB9;gL)8XT70NWHKK&{Fss;~B7|;)OKpsVJ zinRj%?+$thhO6wxgGpqg9{;chismtlGi1zjPqZg~1o@B?$RZmTQK=?ZK#1N=|Dqij z?CAo@YSt#8wPi{LYTDGwR=462uYUdO=&I%|+*C`f%DU^;dPS@O*=}x^Lq6UID2t+SdAsVx(maecgEozBa-MYGGR0zE- z*|1An@RGN@b<6G8yo*!S{8nxtfCVnJID%7I6(RrJ+Y7o%U5TvLmEwh~{x35;-TxoILu#T{a5%l?c`@f33^f?=Bct{~s5* zP0`N5t<L)221wZU}I7gJM76))4iDmWex^<47=|M^=~!ES_n_uk3iioxo}YT-wDm z)FBRCK%zj>n9M6`s*n``Vkq?hhchfO1*tf>OHrP3l^t>AYh+n$CvFIr`BGK8RH6uz zypN3+a;ksyqeD?D&~?Bu#ylja4fa?Pcnsa=o+#OZOipTdy1|Ml%S0EEa7dh=G3S}o zS*06b11n%6A7`Wj5ipF^3iO}}IsfGwy2wLH2M6eM2%3!Popr4Vtq>gRm?1pAh+4Sp zWiX5RMM5?Nk+EUqz##cDZ)QlQH%*LAFC^8WD8^Ta;|%Gd_0yqV|2C;jt>SSzv%&Mi z^`MjU;PAu`9+3E&LGYyQ%!L$Vd=$9K#XTr57i zw`lN3MeH$;Y$UNL&CoIhvLFpc`lBDj{X$P+Rsnqq6-_|UG79tiu+K8omYMf3;M(5bwXVa zC(=WIc*C*1KL{a#Q}5=5`3%|9Sq8wD0!Ccb5qCk`?2- zQ9yn6UusJ2M`Xk1ieNMu;^z-nlQSFs;72^%urcaeAN%sl z$M(CQkM9#~@L7R9m_Wfim|)c!zq#RNNuGV=gtssn9jL=*sYmgRhdw|Z+!-385uH6O z+D|ASCFGtxG~e^xUi8JvsB9MdETDzt0tMvC2;iSC=wB%CUpx5Ut_?!Zks9-;0!-ur zW{t!L&cikE0`YB}?+L>sVZZ?hIpT(g|aK&IM z%wP@PVCDVcE?pi&EFlpZAritJVr3lYMUUvkn?~VH%`L{=#N5o?26ow(vc#4e0>FK| zLmF(1x$w)dEK7AYS5rL2B@l~KEQYzX4NCMxL_9^C5sL|+f<4qjN~pjoLWL??2xE+Z zE2c#K%@%+W%L@#MgJ3`zoE8=i1a^7LLNEb}cta6{#0UVvY3+k8Ac0DlUFac4&_&<_ zPN30|Vu%%(cXigUcmN*ogXSFGC>~=83WPE;|Ksf4L{7*9$bn->tbi5x;X|lbu6RHq zsDnKeU{+wGVr*kK&VkN-W0NVQHgY2Xcw_V+2PIb6C7!`^#9}_oA_dgqE#@LV9>gaq zODNV0L#UxKA|pL6qwFDKI|SawtiU8NWAv1yK8B*L_#;39B2dbC zToxMIKcL=EK;Y1gBLyx*S&kTF@`{GY|K1G(g>V_BLMS9Pwn?uT)>0CLL0kyVEy6!M zCc0pNWKt%qIA&)uWo@Qj&~4?Ikkm{bgKcC$TrS33jzC>@%eR!|SiXf|CI?}XCLDej zP~bs7OlIlO`VVe|3H2A!F}eZi3)^*GN|{phl0iphBCyAHpznlKtLtw z6y9i#f~T0~l6Vfwk6u!jKB;!%+sSwLXI*6c-Kop?JI!K?MaS=aMVmMxAWvVDdgjjm5n;Fzywt(s#h$^X?s#I)C z&kaK!+}m5MW>%O%cL*b^x+<)$hOAa%Z?&pDyy_rUtApfeER^PH^{Gk#qgy?yq(UmC zT1Y8SX@BO&QhaKtj%pp0|EfKd3bP8t!pWS280+RCtFn$Mz*-)QbxO2m>$GMew;on^ zSVJS!gFkeG8?cun<_!j*!8jykDfj?j-Ubx>bmXd82oI|Rs@NGKqqJ&KU_mAq(PW2#l0@2(sI_l5DN}HZOink+NuTH zZdTHIEXXpY$Xct?%4~|zEOWZ+g#zkB_$<&u5JKJSu(U0W0)W?ot=N+7#s;k9vQe3) zEz|0Y;%*GnA(5&;|JcOdO?=v1EP&A7-~<&SM_bYZoRX~II)tG9C(0J8%_6FwEowDN zrVGvLgQ_g1QlRY)g+PUE^ky>%OD_hVvP3Or`k7|NuGM

`R1gLZhGjCqaL}}sk6TN$F0Nu zdc(2PK0AiDdzky$o~z!w>wKFnyjLbkR$LPiArF!A4>9kM^9@1Ikn{^tuaNZ#VULjZ z2XSwZ_XUAZkoW_UFOc~@ps#-N>@VMb^Y1?&fAsTDUw`%YU!Q;W`)}WW_y7A|0K+%H z@fEOq2RvT_)3?C&x$lASgP{B*NIweN&w}_Ho(9{3whemlgCGo{2uDal|CXpnKqX9} z3RlR&7P|029ekk-XGp^u+VFiA!r+AUssD+AH%%T>z$i*Kvv5Q~~qZr3XMgo1YjA%@w8rR6i=?qZ{-f5#8=Saso z+L1LUyrUlX$j3hVQ7~uh4OFB6fI<3zkcLF$ArD!|MJjTT0QjRMCrQalUPp%x0)Qsh zu*p3DfRlUR@T$x()ql%v#SBrVCxR=V<)w_y-)2&u|i+VYmT%%v?+2+Lmj@|VD5 z46jV`tKaDIn8-wCFqg^9W;(Mjfy4~}_>jzMTJxFM%%(QCnU&&Q|IwJ%4CjLckxOox z^PK2R=T4$|8*zqH8~}I*F4alTdfM}z9?2tc!~g(ty7QRY0DwMj$xD11^q>emC_8O) zOMg1FpAaRdL?c?udNfp%_XI#8G*O9!di109#3VSWIZc25)0Xv62SJkv0$GSl85?Qo zOB+H0M=WC)X;4TRiopzAoTjEX%_&HSO4R7Qu|dAkBsIxN(vzkm9m<4=5CCw8Llgo4 zLM30)tP&$z6_-~TG6!P8G$C0SmMw~4nz9b-tPUZhNaT7Gx)$}Wc>T#jV*`La zc=DvSgsM8AdP`5fz#&3##4d=njA1#$5?@6P0O$Y>Xh6as|0LiynPGtx@9mW(sf?n9J zcOhgTmM@5FkhC(x7C|sge1|n(hO}3`>`m``XY$_xEBC+%HX(F?vxL9)m705iu!J8> zVFzE6As)ajF(gqE06f7Ew7@TGzI$1M*w-0<$Vz*afj)<<*n2v#!zm!EQ=ZmVG%=pB zc@Hv##U9rqJ!Zy_3G8F~Ciuum&gX)E^IQj8Se1N;|8Rw`3jo5_5)RYE?m}>|*kt6v z2lUm!jzzNp*IG8e0tvD+D$!Z2{I|sodGki>dt==cIKb|$$(@tT=RRZPjd4usgP#oL z2t)V5ShgiLz@f@RPt%jlHL+Ta{F=#Dw!}W*!L)1vXQ8|p&n(`^PVb9mJ1 z=tVJ>0SsP<0uj^$0JNn|ZEIsYMtV@iE~u>wNsJi)V+aK;8oP{M=t2~AkRH0#&2D$Y z+j; zosiFpo80DBh@63(UhB#9S*$IHeSN`(j{_qW#wG~E58-f#GehDA$2hf9@$G}myyg*4 z_{4p1g2kfvAZ~p{MVws_rVFIv7hiYAHx9j%D=Xy#$pCJ}r7UZ7TI<{8x{&34aFCxn z-Rr)&y*Vxru#25oUe6KQ>-}!F%lhtj-79RCu!bnB5*xW@G_iM`%wkUyx@z#ZiFq1# zhL8ao%!maAnhyC_9F`zL2*ld~jCswAo$~`316Qf_BXLUggD`~&?p54 zOlxLH1VIOGO@QWCCNv6s)u?qCxDxWBLc^Ic=v!sB~rf8WYA?&x^{R?b!gB~OXSKNy5VScC*obi~37 zrZ5VdunD0643R((qqYn*sD_L6|9vUwhOY&LLKtykXb>aU5q($?E1(C3PzZBlh=yv&hHZm~m|zI3@LIkQB$3DrlbD8U*oFhKU#&NWF7+`D z;esqk5Ugm2Hy4MYFo!5e5P>*|h4@wqF$RO^dBk81t1byXbnv@O~s7dG;Pw4;- z?%cA%@84Y3eeFzGc3ad|$F1w3_hV;GYSLlEud zF7|bZ4xx`t*N;v~X9QuB9|M#G0bK2+e;Q$yjHh@wIet2ciX3SWA6b@Zww4O9k-t|E zb19B|iBH2+62&xuY7l~c1pwfnYm5eEhIg2UX-mCEWe$-6ZvcC`kdh$@d97HAnE91a z83_f45Kx(rp1BaWm2nUGl3jKN1MzyR0d?0UisNty@{I85}ZEpnh_CX zLv|1X+Ls9WQF`|p{8VhG@C=N$j=gqEiK$drMooz}f{pYLPq2x@vJ4w$o?Hokm3Mg! zxRV~TdMIiT!zeFf|n;#XsQsNnqmxLU&C+*esBj=b*hDMdeAVMae`aW zunId`n%^g@Gr@no;E3Wklcjp9sd@*i`hBiSbX~Ti%Se-MNRk3Us-}vn%^Iw&`k6jJ zmKi~<_Su?frK~b%X6PrJ#mb}nq@!Z+rN}A}ziF(~8j~{WV@IlN?b(%_IuP>dgW4*m z_xXITrk|^oYRu}Z&&sNwOrWhlyRt&dldC122@$2!Sq#=-r6{^z6G^o62mq>3 z49~!IBF3lR$q*VKEb2FtxG;Ma0kV`Ar`SmlNn5T~SqxN5r&Wt*cWazLJ1jz5vp09M zGaI#4x)6RFvw^#?h>K0MmQv($`d6A_9Y~Z0fn6Pr0y4y9dE&*^s7JkiEx=AUVsc5XD7Pd1Pw)+LNw`;G_iD$#` z|C9H65cb8p@>;KWX{-Pc2w^8K$=19MF>3u4fY1O8>o*bcwG5owSZ%typP9VUny%Qo z5EUR`ACbOyTDyB2ycg-ZGRV3H!M>oE5U8uVwVJbu+rO05u&u#Qw)AVY1P&1TPj)2{ zK@ggIIGJo}xyyEWGYX)08Iu&85aha|0%5rW!ElB!dPJHK9t^_#n-Kt4umeZ0a$69V zyOfQ0ok#1YHlS`d8941U}Yp$VG_L0kZ^iO}$=bcrU4%*Y~Po3`fz z1}O`b!Y5p#;SE>^}yyTTd3e?rU2|0i%L*j-t$Sp~<+4zU5^ ziGHl9!4HA8%~qoF`ooge#VEYR3SqxRixI|*R>%7gi2TT+S`f+H5Y3!%kgUemJV{5j zPhkKDYQW9ObikSfvEhtmiRaC=Bn5Qb5Fj8d*%Hgl;0w^Rss88zoWN6cs1WpQ&kgaL zf7}o%V4J!u5dUmu8l0mg+;J!D13|!=C^!P%+Nb->ct=XUDwm|@+GYKW{}4fTzZhQO3}eg(gI<>M@oj) z(5?~z)TzeL7BSEnhoVF+5lG$A*ZkD~yJWt>ciapI-pr`U)Xm5Q2Vp?da|>;Q=NDbI*ryA4Sdrwny(A_SmyPN(_-B*^3||~k6~$SJa7mdX`A;P-s5d%_nfT663`AI z$pyCppAB0u(q7$tx-kcnRQu7Vec5$J-}h+WW0oupjV1t)+z}z*ui(&eQsBBnhpVJX z#AVu<$5U(=rgS^D9p_e$sB%f|()kS$Ga!k;Y!S~5ThZ+U)4kt;jXMrb;FMOjBmGrk z2DLr_gT(lif#pmYkuk%ytW^z3#%U52l2Z!IE%9GrZWf%tFD8` z=i!l!#R-`Tm{91Nc(nNa>+N07j`hrLjpojv=4=k1zdYM+%F|IiEVa(+2l2DSz=PV3 z5!~+Ww@&IeXX@$_=!|aYh<>BtZiC;h=)q3t>i+0VUhj0|D+qT5- z0D$F%^yQQqy~;uiwCTWN2gN?%w_L}_5HBu_T!7DV{|(H(v=rZ68viV3Z4f{pgH2lz zB!+b4MJ^VvEErFonhn=0T*U&B^2D<8s7)``JP|pVh|Eq9nKhDP{LIxy@j~96Ex!;l zuPn+-y%6Efs}*Sx!SGy{(^MS81JUyyzw<7;dRLzj6aVuCarDeU^8(TFiRf=#zwFEV zb{}u=Y_CRuwod~O09 z0)TO0!-o+kR=k)}}$3|h2m(289f_bUJ?A-6t#p|Z)r zygvG}-0PR`%)dQHya>{>Fv1rvJ}gEG|Cwc9$vi2?tb7oIV3>^`+f1DJplFMpGqcPv zsaUc!cW<6n`FC?r*)k;uzFf1mMcjj3{{|jh_;BLIjUPvzT={b5&7D85TJ`z#o?-x` zO?!PV0BBvaYY!U+?%c80!vcU!UHy9Y?U80F2JMx5`t|MK$Dd#Se*XRa{|7L@rlJ!9 zD*@MA!aKFf+YT)8x*BV(v*LPdt_g}bFhdPDOlpW-m`Uag4o4)hL=#U$F+~+uWU)o) zqLZQu7lpHKE(sgNF~_sKN^eFVe_RfLQ7~bs7-pP-1V|;9WU@&opM)|>DW~*-s;j8V zfJ&w&(6LJ|w@X4xF~=mT3}((e|I?XB$YirkH{XOaPC4iF5k>>)^r^-#y?oHS?Q|dw zPeI$vf|g~LAx4ze24%ESM<0bWQb{L0sHy_*q_i_1_T*H@9Wn)V#vt?bWzrk3R-EWRXWE*`}(l zEIDPBS7y0omtRJfW0+^A|G8$HZ^k)i4rSK4XPrrTtzs>-r% zz4zw3Z@;bP`ES7oC%kaOL$;Kv!xv|~amOFeH*m-&r@V5@FBddw%s1z}bI(6_QF2%T z$V3%hR5|7JVNkk=bkk8+opp;`KRtC-R$qyB*l(8|A>D1qU8Ub+&%N~8i5K4Z-GfhQ zc;1VD9r)f4a-R9tm6sm+2_=N^3Wt)D&o^2gsEec#o0pL+PG-*kTM zm#4q_{O#|){r%7X|DXTvm%aYsZ-C-U9{>-yz61)efCMz310!g_2O{u&3alUlFDOCw zQE-9~>>vmm=)n+LP<|Nvpb6`T!V+rmgd7~92scQ=7^+Zz3yj|iVK_q>zL17Iv?2de z=t3R-(1$%NA`)+y!5o6_XJlIz2RMNyG$q3gE0H1<&tyd`PEm_oR7n=Uc*QV!k&0kU zi5SmF#x$C-ifp`y8sFGOErJM+7Lnr??}(5*+L4Z9{Now}Ima^A5sh(lqaFvT$U*|L zk%D|AA=_xkJszZwhs0wd{TN9`PSTN{gd`{>iAYK=a+9L$Bq=|s%22X$l$k6gCr|0h zRKjwVwrnLX|6j>RO=6Olu|(x8by>?_-m;jvJmxMf*~?1$;F8UJrYU{7OkmPdn8_q2 zHI1oFWO7rP&Sa)Ep&3qDP7|Ef9OgB>nay)_0LaNK|0cj zezc?_Jt;{mYSNXWw52M&C`)7N(wV|EqZl7XcE?ykX0{ZMe9t=nv%6%q^-P3 zt5xHQ|JAu_wXR&ft5@?1*1d{Vt$i&kV9)wj!pgOb0?aeXL&%J6Xh1 zR7tQJ6nXfmbSOWZEkh@ zID1L~X}bk(aD_YEx@nHM$3<>(l?%A7E!Vlvg>H0d21V&s*Sgom?!?-bUG8?*yWfSE zmGA~$@|M@U=LMB*(Ys#uw%5H(N}SZ*J74VU;g%2ImY#GfCW5Y0&69| z1x9d!6};f7B#yxkhH!)>TqP+$44)ImaE3KJ-ve*h!yg9ma!V`U5SQ4*CkAeZQM_Un z|F_t4Dt2*)c z?xWGYZg#u3-0g;UymR|*dDq+Cs}1$N^}X-oLYv?J2DohL9dLpdyv7bTc)}I#L|ZT1 z;SZ8Z;1)1^Ljd7+%@SI4^6 zGM;s=cimXp_IlXGE~$o(-Rx(tmz5`ucDA=&PE2=u+~qDHkI&uicV80R@xFJytH(Ot2^NUfG$u# zm*|S~Ymlwj4&(p<3j_e}qaGEkKN~Q@55$QL$bl6E00;nz5@^2*|J)BKR6!CP3a@Ct zouEP~l!`61!i!KsCUnC4YrPo!y_Tav^Xm#JIE@_C5v_2(8!V0ultYU+ff#6z3#@?; z;E5^(02`QrPx%Qt1b{u<4?;A=qxeH4v

W$HI4|^`qV#r`RzB~#I7R$&B8CBJ>aOne z{v8DA?%!Hz>oSD*`tN4h>z2YVJ&7;*nyDg$Z?&U))L@vPu*AX6Ow5shxl&pWQMAdSv&9ZFHaw^RFs-UbN@6s>@=lF0BnX>L7D&WAl&}dC zjQZ3=pF&YHvTkS{EJM653`@iknWD{_Kq#0(BxKC*N@MU2PA>0qF9X2uLJM|jYePWM z+xWn*49L;0*PgMWX?aI`uqpy7^9%Z3EyVG7)N#;F?LdImEw?C6NeMzR^D#K{#)OCV zlF2dp|LjEMnH!%o9QScH%W_3>k<8f-Hpemn^K&;V#45LPgWh9S0CT7EUb_k~Krb_F zHuE#P^Wkpk6-g(e#BA7ABX_kacX2cKmN_@fg|zDyDViebUlZ8nK$SnRv>%Rg zdMVl)E=4U~)qJLxv;IRhpo*S8L-n5FM4j{bp3`}oEY4PGH93g|6J$vGajoJh% zl(BaHrjewerJY27<#^FDCh;G)kqeKEtA{idC{3)SNrd-D{{_i~6z~|7=&h^CuVI;z z^=cBV*neuxDr}p!Y}~nH>zcj$HZNPWc-NNP3)k*nzJVDI?km_ZVZDtPCx(pptXs!^ zEk~|Qm$KZ+aRDgq9Qt$N%%eqvPHmdA>e7!-yN=Boa9jNOu^&pV{$}PiTh%W-fs4z#CUGhj~D^s zQ@?*UCE`=Wu)Rq2B{}HCEs5;WZ#}`#uqU5O5Tl@kTkJ8XEEOooLYktC@+UtpC0<5DgvtyE~o9#aHT$T3S= z(?>TuYww@;C~NUTru@~->*49{e z6}DGhgC#auUyD_CSZ0?+_E~9tzHO&F&x3Zs=+a7x6`Y9iRWq<8jdh*1V{YO4OJ*a z7_!BRa*>n@l-LjnBu742+#(7`5JfbqkzMloA@gV$yTp8vVMava7jZZ)drU(xgC$C%9A$#VDoV$M@H*o$k(o?o zE|Zzfbmn#_z=Uz=;~2nv<~6aIO>J(In<<=}HM<#3agLLm+9u1u%gToM60G$S0Rm@Ts0)2vt-fs}HvDdTX1a z+Xjun?Qn2hbn7GDUYNusRtOI`fsJnb;lO7cisc9k&8|;CQ7@L^K&CM`) zHY_;NdY0~fe@ZJ1S0guO$vMl+) zj%O$r~+KCwxMc1 zHLHMwDX}evpC~#=r@6Xet+pPNA_W-A#CB`=|33s0RYki&ela zoX^JVv`L+2Q?r@g1@G;)x9D%}fSZ|B41pE85^JtzQ5H4xBmWoW8n{bDP?pr>#IJOJM7#i0vG@W2t$2-_>L92)wA4jh}!185FMX6C^e zR)HX{ZSb;B^B~XY81SGlsGVYM|I(rVIEyV>&RnYR`apx}vI$iJuHiBu>bR=+Lc>Db z!XNsfAOFH3^caE!1jH?31TG9q1kVl&f@bv6svm-)LvW1l?9LtzP><}zANGMb#7qdm zqfd$u2`H^Xgo95~a0LSc2fsuIo6Rmv5DHb01$m=N*kMorfQg&~7X%{V*Stv4g7>a<{=xz0%V$@%KAkc{J|jb zV;*b)w-&+!5`hi9qagm_0Oi6HQE&^1t^tLynUKybTJaU{!ysgl7VpXt{{a$-sXCBM zssDz+&!7>TD1Z$pffClx9>T#BwlN$fp*VC8BC6!?t|A-!fgkW>Awp0Y#f~Rb0b+6? zkF=2+(d8S%aT~`mXzWBEzM&U%Arve^8fxL=LPHLJFbErE9 zAMGK~#UId-71WU(;e$Uw;vWD~APF)!4$`5_kt1~>9oMlP6(VcuK_8R>XefXTe!?2y z&~R3=BR>)*O$0?G!XYc77a-JvuPHkMJK^)@1`9MAls@fKK>qG09{-Yq34;Aw63Q&oH12a?MhPX(3AVKLZp$LE;^{^VRmVHuj-?ZHQNNH>=$Ym)I9akJM%v@^ubIbtEHQJ=_Mih``nO3aLFQ#+Iwf%H&8>{MkHSpRADi2y)SDRp1m zR6C|rE?SOM+mc8!&?J0yR)tk0T~$3?%UewqRi(9DO=4S3rb;SCjHTc~M;s%Qk-U>6B)T-%jZ;q+m{NlvwrU;PyTFaTf$HWQHp1Lh!H zw*yd>i33mpLu^78lZKm&4KxIW(hiF+Ws~-5S~g)^O;dF? zJef5+MO0;-_Gn#JAz)S>WY*ob172NgXirXF$|M5_;zskuB>%M59ieth&3)fYjc5NZHbhSxh zN1$>qS8_8~W2XUc3nCsmcj2Uwv9J*Xguv^1gdeb0IGL$t$%6@sq8kdKc#XFZ4go+x zHFCu8M}{E}e79MDjTT{uY3*WcnUW!URwm5gCy>&$MxcBB;dHWec#qe3lh-*)0dUXt zjLMg4&$oEfcX=6N1hS?SpN?yU7k=Y+e%JS2S=aQ$Ry=nWF@RTv_Lq43S9#Z0A)0p{ zo;P|4Ms8a!e7*Ohe5VoW!5_N84Q3>P;}>}un0+_RF8?Z6e90Gt7g&A45?Wi*dB7&BzV||HP#w5 zn1em|KtOhxIKphQpe8_7H4o%~6N6r?$~$`4D?p$V2!tQ5p%v0#W*>*L5+tQM_Y>>_U@`gp*sfg!SSA@r;l+xsjz*aSic>QKnk8 zVTMISLc+I7m~AbpC}GE!LK0-z`eNupgOz>6mH(AEnwLpp`&gNY`E!rCFH`x7vAA}{ zNj^>{UnLls#5g=5tA}w&1vbDAf??fCb{NvY&ZM{Ewn&f#!;YERl&6Hq^!Y3Jh6me< zDbV?qahAgXR;~tmUqnPxUSXLJnyUtyCj4O<>zIHm*?`ZsqR+V&(s`ZRIT;~1o`aE; z^Z1{0v?$h%B$WZtayFnPBB4_>a5eP;OKgl1_DyKT0wGr3BR`i_H^a!i`8_lC%{JC_mJ zywPZXw>vH_wjiv54kQ8{B1L~kO}-Zba|Kv6!aBVjTYAYmcmsS)y8Cdz`%emdf_vnN zJG-?(TVoKBkGdELA?0r3`#|h@M*lHfN<#cerp>|!8HPRGY)WONh-*`K4U5j>I*Y)7aeI3}f zqt}Q1*K@ttg*`inUD=JD*Z-S6*pHprlU+KNo!Xxr*`wXsryVq^o!hTn+Oyr;xBdLS zUE9OGBqTPb|3N`lSHiF6n=1X%A%(0LBCDGzQLWh|3I$jH0GVb?K|!Ns{lOYk4^C*L zIe~k?B^Th!+t3+ea~WQUJC(iH{L`&O;0eBSIvq|hzTl+;2k`+Cz~}^;V@xMj*)l%X z8YSe_2;vu1;!Wb;UtU}hYv4WJ+o+@-5}sg6pgDTUgDn2TpsCYv@VRZ!Z&Ci{vwX}U zy~IhPas?$`eA>wfP+1Mm4>@8cfs^Pcbn-|+vw?+4%R6JPKPpYh`Y z@EafTBVX|!-|s8m@h^y5Mnmg?qU%#4V4p&#ik{c9!`@TK0wlo-2G2$pL@&^QKoW-D z$3#%^9Je+=42q+QrmY{MArGo7_rJ#kM4=9jkMWwQAF9bDY+o60U-$iy>1RYxpBv&} z-n~EDE!kW~c)v${ANUzU`0r-p6TSC;KlsJqd&KZLJ`@A*BqY0k7tcTZ(|=$D^%+|M z;=~LCq(EN*0>B6kpZ@vdW<>=602JET>nBX%!ifa`@l)1h;s!BD1X9SM?O(rWd3;O+ zu(1h+i6%tV3IB2uz=}p9k4cP)Gv~>H1^3}H2>^o_m<+YF^Ed4#!zlKUD%Gizf>Eac z6rKoXPhYhnBRql>NfPM7lq)BqK=#iamZveL9!+|Z0#m9ZK874w@+4WDBRaUUw=Wlk zu0mC?g*uS}8hiOJv87PrmOXQZRR9UO&sGzK8|Gjx)pf*(%m4IQOSrJ%!-!t*hSe)Z zB~Hcv{yl8=0x@md{c!6>Q~P#qsJn&HlsRN%aQ6K8JxIdbRArz_7{Jo)kF*Jnf5 zE}i@K=+(uCCy)I*_4L-qgD;QXd;9b6*_+R)p&-3DhZkDLtQlvX{j(2fsZ9l!Raj}o z6=Ht<^Z(8nh)qc0PHp@%&o!!SxJx#+>C=n~UA2XWJ+=MgjbRn8NP!vc81xM~F{3MiVeB28`Z{Gno8F2X1yjW({R(^+Y)HQ@+M zjAM^I;>f5)^@n9lbn3_>o_g}> zmOa*BF_8)$G-6LZ)u=eAojq20s6;AUWnyX9U|J)83wa5qm}Hs>(PLBkgJ-BX5_%`7 zK@M5ukx4GudS@bC_(p>qbo2khO=!9;;DWVg|0QS?XmKvJPtbJ~o?6Sr# z)&HiZbrve?o@`-2hl%<)qe?Hs-qso`(w4@lGrJVyv|nQi75g~LBzF9Oq|6IQ>-z@QFZ(@jvGXN)q* ziLLx`%O{UK^TjTw2D8dCvrIG3HrKqd&q2rBGtocml;1!50ZJzw`06VSzy21h=?Kn3 zD^^YqM%ZqVay;g{%L5{KaVolhTs z?|;*u6BD2UdidxAp8_Q~c+DmlDI6gR*``7g!q9~*oZ$*}(!v|&B!@B-p$_?C!ycMYh%tPj4~a-a z90t*dLu6tRYj{K_CUJ*Lgklq|_(c5ZM^uP;;N&Do!NlZmeT=z;`*gB33IBBuOFCdf zD2jm%b?9SEtw=*za-xC^%t0FY@P%yF*p>{~K^EA^N;z~;fC2sZ5nBVwM_Jrz#byO`YmitEyE&an-6&!D?5pI@YkB#H?gZU0S^=RT|h zrA{Hf?QC#?N7LG#w3TZORAr*V+qx7N0LmSeUXNRq)WOz2qW`JxZ=3tw&dN47tDQ}2 zhg%ihq7J*d9j&I+Th_y1mxa@Pm}!Y~-U*c!v=!R#e0AdA+fF3F{k<<~4Xj@TBZj~S z=5K@ln-c~n*ufKqFis@AUx5MNltG2!rsDV$*x<0NTI>(wIRJF3~1 zH?x}!uZ?ev;~eXF$2{(_kADp0APafOL@u(Cg)HDBD|yLGZnBe~4CNfNxUo%6W0bFq zdcJ-_O5$joJxz@LyT&@${>RuD>*Sc;OuZ0b3 zSr>cR%x<=`gPgQ5HH;mb3G&w#aUyBkBh}9yGpmtZHhOitLf-~=ZNxopC70XW=+5!F z!QJk3>*U?Yj<>w;jqiM?J86?!M>c~DHdk|7-&z(8!PjW;gJ(J6O=h^ly@t+*OMK!K zueil8j`56ZeB&JNxW_*Z@{o&sw_%Y5cEuer@{j`N)B zeCO%jfEAL=^PmfT=tM8N(Xr%npesCY#%Ra3jQ`F9La5>uN;i4br(X4=Yklin$2Y$T z=kM+@-RbHL_~Zs&Y^m%I(~=G zyx|XD>93b9?CaIS+9zao!l7N)61BA5!;SawQs4uk7CU93a(P&1p34;rQ#-s7gop#V z@`!J}>t8?Uq`|`yrS?M|CLPhw>*fic{{5v#ET|V39A&+1P&+>=e z2AbpJJ36>3_=@uZDFf8-57YNs`WF{v(f@xT@qb!|J^nB$ga=3lNPZJYffWdCTLgh1 zWqpCt7n(JBKV|~Kz&H7z2>6F%5?E>ew|#3AfCJbB)YmvDh!ZL(W*!g-y|4;tU}Q3A zfj#JhKZrVN_FX)vH6F-WK}HbmFb~);I+yTwp0|OsMuRrif&^$>?1varC}uAp0Y^rJ zKnR9mD25XugawF&6cT?xMlOKS3XnpA8Q3mg_-0!8T~+vkW*CJOLWdbAhhyl6e@K31 zc!hbmB57!RU!o7nkO7c#f2;R-asejr00}s;0)RkB^#BgeUGM`6>5qZU$t*^$RpCr$B+OOg%# zzz^}@21=nNrST2DpbMceBhpX{U6>{q@+qNdF2WFTk;y8$K$(__nTMg2amWUgKnaro zCgI=-w#l24Ff{{#3PS}7pb!cOWe@FO4ISkceRCwLU<~8K4UCc#v&ov1$qJQunVC5e zJ82>UlO&*tP`QAS6XF4cz?-;R5}hZ(ofzT_-|3v{ zAfEUcY~D$n{rR7BY5$-Hs&RE06XZfX{s1*ik%({COOEN0w*`?p28k|YktOJemxBRi zpbhr0Ju?OX0mvAyU=np9K#>w6H=!l(kOnI$BFI%N=;Q&3pbqw6BR659`GBGxv7+_R zqBohI;uTzW2q<;&0-rS^9~u(6LzLkV2|l0#FYuuP7#M>=qzhQ0)NrCHN&!JSqyV5L zh}jjRwE|W!o-D+FN~tY2ilaJ8We60OQc9&(S_W7u7$Z`qP;#a#k*3$tr!$JCYKowT zil`E&pfp7qnGr^J=?{;2reox!A{q%KT7<#pW0BDh-Y`=Ex0uG4KNv6v4T%$r2pF*- z7NUVW|3EAYf&Wap@p%tYAlnHkZnLT*(5kO`60?aqNE%T)Sr7V<3&l866hH98s0#r$ z7NL^-^egO81)7r-q-v@aa-ldzQM6N%8@i@Lh#(n&2lb$%eE?UhqjuIYc`iK5iNU|Fe_j_`?4{+u0?CKv(~O7P^NvLdTnPnllm3}Yq0Tvu!TsN zK9&R2!2b`&fKH%#v7&ku*3by`;1Av420O$AFVYQ#z_x8$2#0_QfKdv{5v%yKfg|7| z{tzMpk+%P^wr`fDco^@4J#=DQ%b4E6Mv^qoED@ zV6$Euuz2_n#8Vo|Fa>a>8pw9NbgR9x%DtHr!@@ShmwRL7YnM(#y>L=_f&)>{D8Fs- zD>_^Y%Ao=qU!^D~Y%ERu#8GSyvP;E+;>1uqK4gr0 z70kwMd}SBx5PGXxf|HUR{30xzh%b!1SG!~VnmuS)8}tgc4oC+%dJtI99T-@U|j5&`-r%{;C`z$B|JqEBq(0ByV68$>G zjL{iAWV=f|{S2U`vCrAO&E5RXXvoJHItGWJ(krdfl+d}WfC-4O#-~_RgXk0#-~_;T z5NDtq6fmeOLe2_#!;$>cfzpE499}_95phehqY?$`Fv@r9w2DA|M_sS8QUBClLjjLc z554hQE&C7bAe};;#pn}*nnNwiVYQnz1WaFi;Vr)jy>6!9od$h*@sQpa?RPB z{n?Zl+Lb-pF(KKat=Xo%+Ms>fzF69>-P)x6+7pr5tnJyejoPtI+qw+^?MkL-ArDL& z6H}cV=7ZBZO_6Il&EI7hsRi8tfU0kpT`?g*woyAa?IxUZ$oPqE##oIMqSSsl)Nt~y z;0NNqRVP?0p;dJ<_H)eyCYVLjF+x!#9y-pQGPBfu1ra{sLpQUcG?Vq|U3 z3eCkdTQ2yGxB>ovT>{}GbKQse2NY7_**zf{e&HKFKo*YSAFknu!Qml}-5pNiB97uv zQQ{tc;wBE_E*|44{^Bw|;w-M>Fn;4Y(c&~d<2YX9KEC4tl;bwO;yhmDLVn~#9^^p& z<4E4*Q2yjhKIJ&k<4!*0R-WWozT{dSB|d7%-pF;(M@&?2P}b@3H=ku`gvHhuvZdm0pC zFyK&85f|}q8_^MyD^2yl4UTQk^=&LGFbPlUk*e2$k*=kMA^%#7{u+axN-gmcs7(|7 zU=x7E69f`1lTH*8rNwk-%hpHgrmm#`AnVFD6E$%Y$4=|YE+6-S=k@XIc^(()fsxs< z9@8%EZDH;1k?q_*AMmm5)gJEA{_W%r?b?3siP7!!vF@qS?&ALL?o#gRe(mvY?)P5r z;GXXMPVMZz@96&T@E-60-|qrH?gl^b^nUOPpYR0V@DRW7*skvp&+r2e-2Xu!0fG_( zQXnIuK77s~@LA_N*1Qn2Gf>DZ!-qA}f-rfaFnVU__Daih@-4BFFxIl54Sp`^vMv$0 zDy>2+BVeLpLJXs@3B=iSQLUdc(jhMrq4UP`=%Oyfa{n+m5A!-tBu0WHO2Q;g0wq<< z5Wlj&ydW&Z;_GI8vrS(n*#Py*fOlJOD_)-~yFx2me=A?FD`77(M`JWYqccSl_*GMKl`{}`?`PorqB7X-}x?wUH0DkdxPq{sqJ*;|~z+{{Id1=WifBMJfOQpn%5SK!_3HMGCUe z;zf)ZF^(7k;#0qW_8$I2xR8RDcK)U*v`8U}J(Vmq&TNTtph%Ge^VuqLvEj&w5GPhN z=`g57iW)~888>iWESXA~I)zFTz|yAwo<_}xl`7YbQ?*v5>2)mFi)Oc4RZDheTeKG8 zvK70wY+bifv)X-&H}1l@clYM)>X+`|zgG(>`GeELT(PGSHX=4d-Q&leJzCsVHU8D*Tke3xE~nAySA zzjtS>PN!Zy!y@F0UT9R2VUDzXzM4>|KL1^K%!?{8;2HSN4fzoYJ=kF-Of84<$vGa&)=_T|4laD_8@Y9dKw?-hr5o6RzryOD+EYB(mGO$AyZ0yme99=3I z&mt980ECuju&E~Xz`16XG{`GULJBR!P{XShtna@3_Uq3g z490L|$8_wukQYnRuyMobQVgWR3o~rOB$N^~2|)w7vC8tt#3Z2L-&^&4>xJz`fc4+>1XYM%fy+Y}#p)1diQ$GZB6y>NDWF(lJpmR{ zp@*rHcp-{23K@X7RFGg|jxMga%aXdGp>FJz~8fvSgu9|A2u?AXejjYaE=%>M+TI{RI{<`R`%}yKbuDQ<0>$Q=# z2=2DkuAA+*-NuOSyzTBA@Bh8^PFnD}315is!2M1f@WB-aU|id9>#F4PaxNHY=b)1= zX3jhJ{BzJl7kzZnOE>*=)Kgb|b=F&V{dL%5mwk5HYq$M&(;+YS_RR6c{deGl7k+r+ zi#Pswqmw$fx>$m@Y{PRzJTzo9K!;k;f!3V9Qz!L!YKRY199=_9G13!m>K>+0_M==Lc z9B94>PEdmtEFAGVF^umLKLPjX5+#MMA*>;_Hk|m*MSI57HB;c<`5$g z1mOhD(Gwlggoiy0%Kr~LI0zyZ5r{!#ViTSCL?}j4b>lk9I>yAh>wHjiQcNL7xTr5L z+AfS=Bx4!Pct$j)QH^V4V;kN0MmWY%j&r1A9qo8WJmyi4d*ovu{rE>f22zlNBxE5C zc}PSiQjv>fWFsBvyxJgvHu9Ke( zWhz~nLn;QviWQ?k7L3u3G)*UqM`7R@dB6#5bmI@Lyj}q(Q49-#!x^bS1QLKZ0+iSz zmoq8BEhxA|(ovud_TYywVtBk?26LFNq^32M$V6ziLz*!Rr!a|0PIxLWmJyky%(MWL zY0_kuLFr`~DgS^6k+j1bL12#lbTbZvK*S%&DaAWyLV*tquygRDPBpK2I&E^(oAmo< zKnGe-&_Pt9ezfR0gGf<_0n{D>C1}JV8c&!KZ(Q)O!~y+Khw<5SIOHoJ00F3nU~DmU z6!?M`ol=UQ;ckZ58Hj+sxjdIN#UJ0$MJU=NkJ~)eA}5eVHmnHJm!Z@uDp-0vNgiR8}jIAJjf@<&I5@ENj`wIu*A5w9rM5jNiN|~pEQ|ERj$sJZT#ld z;r}?nJKgcUaHv-H$Ya2pJaCemi(!Gi*D6^iZj_CSMmGnN(5_ogJ^ay)0RuWC7@&ha z_R)$)WM|5P?p&hb{OQukIcA)m4v$SYLnKCyb{k*=D=PG&7%|g=C5?!KFTw#-kd`Eh z(1?ma^d@ImVhU6C2@lc*j&H0Y50ge>5BqV92oJG`0=kHb{`w*eRiX%BJ&!d$QqBmvfxT{xu-eu4fzdJ%amtqPz!0){hK0nT zA8msb15YpnD~jQ!ez$^Qv<@M;Z={D%6vICHP$rPZ*>oQCL{>Njtss`Nm!wv8RbVVq5|P+ujm=sz&0@wP7`xyMBlIOx2!8R0w8F^ zAMZ$mq)o-y{BeR8w;Rj`{zXcP-7b-9mrO;$<)jc zF87Yq zACi-K*y4&;$w1mQ_-w7BI*B4*2;)TtZ6FgrL$Kkiq~m zIF6~(1x;XuZ14wuhzB>AGD=g2)cO-zs62nrx+S6jHZa2D2nSDKLQ2SkAV4#QNJC1X zK6_w8HihcS{8yR77pyMHGofw1{I=LnPFQ zYOKRG#KtAmMiWd)eR;bb;>L^w$1+rgGi1yZN&qm+vY>z$_!rtL1KvoEnh=O^hy)){ffvAlGSIDk$c5R;h=QETe5lH+pa86_ z2d#{%6{|{+8G~5_MZz@9tIRdUq?n=HJfkEJq|_4)SOlH0g-m)F9mu4M2r#MxnWChb z!vwAbd?=LYK8B%y6_|v%Y^c^;2)0xTuG|N&REYH(ON&6wG*HcgBDTr2O2o8GtsDXA z(g$m>f}%7Shseim>bZTyN8>V>w+zfsl*<*ngqbMF%>P`?qfAPf8LfU821h6cqEn_; zc)tS~hqcH+?byJ(%*(y(ORWG*vUJUm#26vjO_eNAfniCx=*`~*OQ{o1isH-tn+Ss` zH5){o5jX(_%8CB_hE^EJ)Zt2RD1t@dm%k`TlgWbHoUMZ5I!lX%k>Us(0FcTO7#iY- z`P-9i`iC662m7ID7af5Z{f8NS9NN%_WAGw_ z8G$zV2jgo}7nKVrmC-n`2oM#~^q?G<5QquHfU5x0j0mu}Di25$s}8{fkl0a+_|Z%R zO&cv449HPDh116J94~#+D4kLoN<3`(3G3vh0RLWAK4R5mdX9KvU(C5g5uo zuq%r~2)pXD&_UOHiOpSuiyv|28iAbFec14SVG}vgJRyDyj5dUMBHmr}-Iz=eSGyJnpX5|nyGnjY{&qu7* znNZhQZCY7H!QJT!jU_vdtyo>1h76E5dx(cUtBznLonrhK9Ts)&YS|xCwr^ zhE@m?q=DObOo@1q1G^F}UBijQeJT<*ue8vn)N0rMW!!Z2U#o}`SV#vIi--ilTo(2R z7cRH(Wr)!NArr0z75E7ihE$aW;u02#6SfEo#$XNRV2m)>csK*{QQqp%-Nd?zE#+aZ z2;P<$ts@4E9qwT)9^kjUT-wpq5Y zDBIYQoMDO^qE#s!)A`lFQ~yn@e{Er~@L(_oU`nP^a*2x+cHw4iDvj2>cq!sn&g-fW>Tet zd7PjemRKXsr`*e^B+BKr>S32nVrRzXTG*0PPUTg$hgN>&Jf()IyECgeV+Q--t9_U1 z3}zYLWghlrMoklPrlni%A!v>=X=VuPU0FxXUgOFDVprxZq6xz5#+9Q zVxGi@K2V6L(}zPVRb7V7-Nk2oiez(sVaKgxkDk?xYG;f|XOOW8F!aUPA|_F`Q0L^0 ze!?gUSU5+J1{E6$K>zb#5ST@2TuqEf;bUkqO{IxU&a5*2;&uKUVz%3+sINMZ0w2%; zF{rd;zUYv?X8>qsG5+a80hu;NQ-1VmpnjK_Mi-hM2%BCAgLRKqif5!VXWuGn;5}-e z=<0XjWSL;=K+Ec|#^^D|=tk9Ept}mJ(~4ne>ZgwCjOJYaHHopF>6)epw{4Dwe(2zU zJn&i%HI^Ve6)$BbWviBCx%OxP25GwPX}i|dYT!ngXvB(M>C_3ZDn{FU(aV^&<#)E=j2S@CR&s}VRIbAT+?UC(WB93bq1|1_aF^!-hE6q!gNz(<{Fej&6VZtc#9((}dS zUI@Gfqu^FRz!+SAFo*4C)c+G>X%1}Y&eqwaZaseQ?G_aCw(VVOm~o*&jKH-E2e}^F zY=@C=SaAV7D~;@uWpDOL^52f{i#}Q{_t<-l3n@?UUE7lX)+{!s z37YMLp8wA00aqvL3hXQwPvPzI1sij4C2|FZbAiEZ)JqEtwTH-|fH61(QZIEf^ZExKS zZO*YMJ)?xQK$oeTY-e{^o@YLeZU2Y!T>R_AR|TxA6E(rkgk4+hCE>S4gU~> zY@rSnP*4-o4c-7;F_-w5fjrn6z3+FX%w>;y;D%&}aB5Vu+DuWqiZ55NG=RuOVFUmS zVpOOYK_Q2>fBmB6@d02WfDK}rPyjGu1dxXMtXWYpkwP1L{e+2B$j~7~i4-kjq-a7^ zogfq^u4v@&n8=A7KK=WbuT@Bb6ER2$l5rvmWdGb@>3MT1#SuUM?Zf5LpiKZ2w6yyd zF4ERA+U#YOO7SW`Wldg+K{8QAN36D*h}NQF`f71qQ?BEKmK!OGlPXC-)b860= zunkk?fXbe~YB4D|aoQL7#0-n<;ZyX2Iqan1B1>SKoc}xMlhkoE&OXraqKYcJWMhv#Z6Sf=NO;&2)IZ+f_M~A)j8jiJ zyrcq(Cag#k6g+>F31UhA@N*3>v`73}%q>bsYyGUrXPEjZv))uPQ1j0^Z<%Q(1!jchPo%2Ql8v8-k*cLQ#Eh~DD4`g0 zT~F5iGmkS>Vi^{wMHGes9iKX*N-ur>3BaO_KKkmU*;EQ7254w0S+l%gnORTMI*O;Y zl*VD{L};qXrkileN$0pihWU>^dcumOpHemTR8*?ondhF+_88Vb{&Z@LnB4r!9;*ji zD{HiwZrZ6dpN2YBzk;Po9l6y?imtilUk1HNIBcK`7KCxV?Dz{=XFJ~m?-Lw%1pY)?&z0n=k@_WrcTFXr; zP>_W*1|RHCP|+n%on8$!KaVd+{6?y&sP4`j2c7(`ANJCQFKF{x9t=BP>6-NbC=^?a ztuQ7)#sBcp2X%o@eB|?o`Tl`EiY#glS$a?4RyLRLjsH(f6g;0IeziRAIS+aw;Xxyix9XJ-1Qtem)UzH52{Xh78WB9j!wT|%L%ebwFL{qNULy@NNk&R?k&$$yCMy}p zPG%C4o*W)1IVs9eQWBMzG-WDTxk^ix5|*cQN?X42le5HSEln4r88K&l+;boN z8s{Scjs!ATyTl8sHj&&l>5XREz!*ZY3u&lBAOGTT#wt={7!_n-4${a+@Pg<>%Q#{e z*hq&x(y=B>q(P5!SwSF-p$>NvLlUT>028vvJ$`tvDkAZK9pI#nddveFx{!n&r8$Bp zIKmj|NXIo&k%vpk?^M#9PX_ED3v3Xn9J;7POy&s)T08?A^+?Ayc_E5C@Mi^jxDqxF z3XW@Z)12TL0SO{{&vN9mp&A)zL9vNYg))>dJOG3`+o6w}>H?ms839oZdensqwIE|i z=Q`W@&Up3*qD_4&Qm612Cf4wK^t2~F^;s}~Sc9qAsK>1;;gbGz^rImiX-N+QQHfI2 zqQ61wM!{26qb3xq(Hntf;M5LbwE_>U)BjB4DDZ?yV6z`Ab%rV6ITaPG;G`*CX-i-F zp^fyEtw2rcQMRg1cDj=n@RaA16hZ+WIHDNY=*O)h(abwOCu_8&j!bE4Q=EDeIH0ZP zPxDDX5aku5`Vc8e%?VC%lC!T*4Cgq@sa)=6SEt_tuXn#mUhPuly4^+ZXU{8M47&Hb z*e!2i)Ei#;uJ^rjiLZI*%ii(&SHIHYZ+!{8-2mhFzWvQ_g6Vr;1{daZse{sACtA^q zvgep;SyXA3i0@PrnKRwBn_RgvCxG@?8IDq~y6MS1m`L502~m;@OSh$7VvvZ*Y@2 zj|m7Y_L#;-2=Zl$Xt^pV-tjkq#ADu7naY)I1I<7*vV>3|b3*=AMv8o6X*Kz?SC&?c zXWX%Y07)TRJ2Q2r4Cp66DXnQfjDA(TVkcP|(uT>jjRC-EOgox6pQg^JMICD3lp58i z-t?*qlWJC9n$)c(%&T2JYpT?m*0-KYu5rEVoAi1j5(cKy7!95uE7~w2?lH5Q?d)em zJKEBoHnkgF?Q0)e!`R-ovq@5CZi74A;vP4-%Wdv+qdVQ|Hq5Z!OzmZ_JKpl1H@)j^ z?`OvjnfER?NlZf7eFHq;0v|ZR3vU1Lc(+^K@IE-hJB9%^94g`x-$W%w=In=0JmVC% zc*8qx?tHtO+!)um#48^1`Gn@*1|K=e4`p&_v%KYndb!7IZu6VtT-~R`hb8Xd2cOsB zuGhBg&wXwOLFq2%N-xMD@`pk(nFD&w#z51bj>4!fJ?qj2dBCaebgN^%#r@vm%Bybm z=7s&)UOzk9rLOh2!#(bDpUUaFB_?(V@7g19j+pwG^tm&f>TE}Q0xv#@!86Ysg$F$2 zRi$l!3m)-7QGAcbuJVFMUhHgv-u2q42*)+7|eJ%NKWqyi#{ z%Ix(UKU`kNNgxGUAOSi_3ETqrfF9|MfS55LbDUrWUZ4u%VCWej^B^GaXaRA|U|~2Q z^FW~RIgUNFgEs_$Gu@nCj6(%U5!QIe?u}pxVj!3xp%OBoG*Q3@2%Yw^pbIXZ=yBa1 zVPO(3;gWe_4zi)#oeuvzSVGbHK^<^hw(ZQ4J;BfYoE`+jl0*}?S(YK7R5mE#4Q|`s zJx5S*Ox8VyqqvGLgaS>;!;GAq(pg?5${Qlq%ObYe3CKb=oXQqb-wF_dp-e(pydb#+ z;vg2HBPEAWR8ayx9+WsBCl-b$f?^YhVjJ>e!<~+wP?c90+kDKTBO!!sQNY4g8*<1( zxdcTh)?4o!Of?Ln!J)?#d`Cef#V!KgCElDg7DqKcNk-{n_bHhQtVAk?8#6lN{A}R} z41u8dLmoL%87|rKgk$=UBRBS9KMq{NiI_bqjWSM~2hf2&d`hH@f+ge~y?H<&^g=hm zUBA#nADEM&bz}d^VdA_!WJIQcv+bk!rQ;ZKn?WL^y(lC@UWy3>gTV+_Jr*N%bfiR5 zM@ahPO?DhW7M4J!3PDEO73oJlbd#F}+zUuR+{q6+Q~~jMW6l}nv)Ls0jU>4NCDsh3 zv?R|@JQg-s&?Ok6G2LORJf)LFB~F?p!Rh2i@uZpfq}t&@J@`X6SR_B1A2~9mMkZfb zdf!xv8)?9$TxKJ4^vpiALL*>JEc#$rf@G;c<5@bUz@;S@#w1(1f*)2JM`(jTyh9q$ zrL=7T8?b^sk(LJ{f_a1?;TeW-d|e??MQD9gJ#?TYOuL-7$=QjN(ZMvp;I)iM|2nhs&I?bmCTBlKr zU@k}*US@}A%@sb`r(y7=VWg*MvS(?&5oT&8axPiRfJz1!COyhxYgmIvsno3?!r}!e z)D7r@qELfA=941aFV)a(NoR!aW`-$dBiaKzKmzVmW)VIc-*rb|iW>>wPvDRd-z4ee z4JiKrT!q!i&`5|+;IK|Vgae&OQ#fU2m;lf`+(lKC4BP!?C0d68G0z{3;hIVjI(*nv zgi~bn9iP^N%7GA__Jf_`sTCyxq;^X^gbCSIst8@`oeqYX8fu#64GByFq(-VeDCJGi zi*LS$NGPg07L1{a38DgO=bfFCHpgE01D}S8pFY<_#3~B)!>rchoQf)@4h9Ci!C7#R zJxuDalmtJZsGazmS}5wlFzS27C}Ef?nudu#9M0B=sdvz&ko?0BUCm1dseW*eKKzhs z)GA5t9j{uQuR>|Rnp2`lmXTLvL8*+Xms;kujez`|LphRL$qcIp?u!2^ z*i$a>hX)V>C3x(reqNhqfF5{k$x?zQ^r0XA1*1g9Bfd#C_ya%0gBxI?vbYe!xQ;&@ zOXIM?$3{jtc!J4NLLMkZVtS30#EO3CLo?KgRiMH(=mIG0iajA=9o>Yov_qJ{46GD` zqihk&62~{_!pp|&%t}Scn2)RQf-@2$%3q{4p3MK3)0REkudGA^eiQ;syzU$&`N@+*zCbr z3~}u2;5tR&9xk6K?m4}!%gU|H((O^8LDqVWDag#Wg2t9)YvZV>MhFMgN-h7ds6*9e z)lGz}qgI`;;4Q|ggT^}1!IX&7)*A0{R{$Sf)Z2vp`H|Md7W2V-+} zdvBXp7q(N#!(mUwW1HA9t+S`b186&mTDMpS+(Hpa#EP9!6ki+l4(l-F0bS}f`07MI z)KL(PKtKIMHx$B!V|XDP!f^D3cpnC9*uy%6;87eNKrOa8i(O0xsy@_$myCcA+`%0b z!W?RWr((wlK!&Ll6Yt{ggolNNXZVG0I9ilKGqHwx^Z3g6c!dYKkaKvEPXvMoi-I%Z z2t2l##5Q%zwqlaZ;j#FMdX)>}Wxn>Ca z(-C_us1b2cu4@`<$HN)qSe7%#mP5Ni9}Kg@F^Xq0xhIW6 zNV}rj`x|{b3WYoPYyMExhuHR^@r^~CydeZS-H;=N1 zbB$sO{Jz6RgFAbtvwX98^|3R2Xrp}O7SKDSx~eyXJ_Q_q@;{GJ07MIF_TIUn003b^g$hw1`{xdeiiHyu=Fs-fToH-@C}?Tt zZ<<7f6x#gr*GyzW8Pxu*YqF7o8GHV!sYr>Cq)C(s8$OJ<5opkuGmpIhYLg?HnKf@x zP})zJNRsxTHf=*SD#8&=#{Jtjtd0~PI>gYK8Iqz!j2eYf!0^P8w08gg!HX1RM9`oe zZrQ`v2PsYtpZ?)f^a2AOx{Lo8+q1`*T@}4BJ%YT+@#@LT0Murjyt3uX$)ib^Hhmg( zYSpV*w|4y+c5KZg(m|mRCIAzidoG_8h##kZ-NY(kqTSBBC_Tz9Tqr^R7+eT6cAD7DKKy#zPeA~Aawx~xK%_6I4E-@j zKj5Cqilg3k0glJMjRgAVy()pY2p&ar1+FJ~{?L)NX8Z7N z#u^PH$eC^Kz69U1xd@eyqJA1@kC`mvCvIm}LQ<};{3YvXZFL@nm*Ti_|g;uW- zc=Z+7gl@A(pJS-i3&)>G8@5=Ay1fkC>c(9tVI8GC7yuf%!?t1q_nmBAdHPt$sS~xb z&>ku=GZ}z#&sF~@U2hdCvQN_d?e*7SRa>>yS7V)3Hwwl$gy^D)rUVsx_E`lJL|)+6 zM?kgG*f5L%U>H7xp;UM>TX9{s)?0HO&Fitd78~q|%1(QswaG^N?XTlDTQ9fImiunH zgR;ABxbtRPA-?zayKlS;*BdCo6DNFd!zrRvH=r(Fi0Z?JyO07Wf-#9yY=LU-x#$*E z;fSsZsxzi>{PCuVWlh-59%mJ`&z^diny3U>Y265f?eq}_@GwlbMxnk!WRsN1KMiBbe~;C{S8g4&!s1VDJ=p&qlJYFY=A<@AeRjdt57 z^r$dMev|)vYvX6sJ#rIy*htKPJ<%ydV1B#n3`!QGh(#-Qaf@H9;uEz96fTaD zie4lm7|kd~HKs9)YeZuiH*&@|(vc!|j3XS|D91apv5s!E4Q^IAo#cVUfe0z$^UO3v zgq$E4$jRENLf5Er(d2!Za|rhwh6cJ&r5T=EjR{0ihaj{qc|DuR2nOR7TsRN|VmRdn zXM+EZQUS$ZYUxKb@{mXXY(Na}ON|PS6{)c%xlPoe$%G$j zj?!{LfxN^rFae!qLROH3esQ^9sL|}=MWOW54UPq$Wy9&jxp#JXAyFfxhi=@UxrZT(godHS&k_4+p z5v-;V&}DCS6gZY+!jXziumYX@*sawlfQQG>#T)xmP3h`U4ql`J6q;a#H0Xk_%}nJV z{I~`$Xdw+1YUOAAh{h(^+g|qmVL2+m02#Ux-f!?i6|~qzHgfZ{VM6dD{jhF%AEOUt z6e@b!nx1P4L>GSy*cgjRjz~tZ632*x7^Bz(C_-TjZ_1Yr?V!q@(kT-8(wF}({J3v^ z_uF3}Vn8X5nTIn}!HfNt7$GOFuZr=jU;f6yzW|`Fb+M~m?$*U!InFIz^f6u*XG4V( z>|bfXD_-)NHx3t_i%B3XVd(K@F8)|CEufrb0Dw5a1C~aC4@|Ks$418~9&?LDp z+~zJey4S7lX)gqFum!L`1bk&SXNjyT7=aUHHBnzq@roDjQ|9n=m)}Mq!q4wlEnO|g+d6P1-UGyJu!+nMNqn@dH4beyKyFL z8XR&`)v4sN(o@0#B~J{zLFbFv2R3k=V+xpj%O>%M$`c~rIb`Fx38xF6OMY?(FP%G5 zQ3{E>dl!-efCrtx56b(|9`iiOCqa~F#y2kYv)6o2{J8l>IM9iSKk@7cX}O+W9&3X6%253v-^X2^RJWnb@oX8lz^xWj0T<8gYc-FKe%gp8o!UV(U(a{q9C;l94U=Y8;ZkIUW{pW(#+ z{qdKt{NXzs`4j;@^qWup=j*=u*uTE;jqm*KTmSpo4?p*l5B~Ct|NP{aKJ#7skKF-f z?~LjDyBrCeDAZJ3FX$}*0^TTpQGf>=p%}8EA22Ku^lu|nAOq$=8uGyxgsC-N#1VD@ z8`8lZ(!siLCA3sv1p>hs)ByvDK@voR0#ao2!fWJiPa-_P4z^1j>R}$BK^G)p3R(&- zj3FJ;;TlpQ51?h{sOKZB0TA}99x$vD{w>E=01#;58Lz1!HgqYj7fXUtpunC_q3a79NrEdgCz!5$$1YHpTB#;6v5CeHZ16$+&0&oBekQfed|Mp`Md?>We zCeX0q1>43`-~zL}^Naua+{$+N3Fjk}JQiBa1S}j&dA@ z(b$mUD)mAsr}9(e>CF5GDQ^#OJ_Os~GAd0&EYGYhfWv%n5hsF7-zXs56jLe=Vk^(m z#4bYx)bh;OayELhGWW70|8gYEQZNH^H1PjYlzws}g3>ef!Zbm%G=)+#0U$M7^EF2! zHi42gWwSPE^E88EHc|67c{4YGVmEcOH-+;zSF<=NA~=C_IF<7_Cn7nGQ#FxuIpOUk zpE532X@^L2BX&|!y3;$q6Fj44^1g#c!YWZ(Gd#beCexEWg`y<*tUcpXKIfA@>(f5( z6F>74QsPZMxl=#?6F|RnJ!4}9>fka+!0znf9BL*&|8qbaG*aNjFd>veE7U?S6hkvq zLmh=b29YW9&qT^CM6)^b}KSA|RRc20E=0a`tP+e~}cy&>IRaS?USc{cM zsbf#J#Zka&Pde3D_0ulfGg{}m0$bSU;h_}mS=m`XMYxGgH~vVmS~ICXpa_YlU8Y$mT8;TX`dEq zqgHCCmTIfkYOfY+vsP=jmTSA#Yrht3!&Yp^mTb$`Y|j>L(^hTQmTlYCZQmAd<5q6x zmTv3TZtoUv^Hy*7mT&vkZ~qo>16Obdmv9T$a1R%86IXE;mvI}{aUU0QBUf@KmvSrD zaxWKiGgosrmvcMUb3gwVbVFBkN0)R<*K|)8byHV$SC@5L*L7bPc4Jp|XP0(s*LH6g zcXL;Fcb9j2*LQyxc!O7XhnIMZ*LaT?d6QRpmzQ~)*Lj~8dZSl*rgIE>Nwi@A7>y?BPfxQ)#?jiq>qsW^_+_>I{(kKK5R;dqbb_>Sq=g88_P{kV_y z_>c*CkOBFR@i>tg`H>yDkP&&36`6>~SdcIIj7J!B^yK|~%Q-x`kP@Pk8RL^lIh0L# zlo6to7vq#!Ih9>`mH980>Ee}Xc~5S+m2G*JciH`Vxt4!dY$#zn+uwk9a@+V z`k^H{q7!D|IunRk|4?8IhJFyX)GzL4z6uYq@J2WM`5h+_PEt@PEd$KjV zvN`|zvOPPq7hAL=8?=uCvrW6S6Y#WEJGB9Ov^V>;J3F>Nd$vQHv}+r+ahtVu+qG}o zw_*FYWjnZOTYrn&xQ`pTlUuo$o4K3Yxt|-lqg%SCo4Tvpy006%vs=5jo4dQ)yT2Q} z!&|(^o4oV&1AO2E&O5!)TfN(xz2AGib&oWl1Y z!O^qBQ@nLSoWxt4#7!K%jLJmADqm~+|SP((7&7+(%}WqF~a#^4|ISs16O0z zR9I7^(HR3*DS`($0UNsE57GukMQl6=w$d*>)APJ-L3|JRz|_aw)Kgv6S6$Tw0T{4B zAJjqC(}5Yl9MKcpG*;ja2!Rp~!4LAFZ6h5{A00J>oh~MwWSq+$+QA#}!eMBYV0UfV zo84P+;=I-K+S!u=vR%~wRSe*O)x%xfQymzdK_7Au1Zy4D`2Y@fd@^`o66*iUBx+UnyYE$KB*pJrl$s2Uq?Xa$MEhy)~DN#G>qB_1&xV4i7dC zIP5a^zrEB={pzzG>$e`! zPkqp{f#pwO)%T#?l^iIT6&`R!2c)16jbR#~bYX3NB<`;+L@h*#o;L74C_Eq#dLbPR z3)%^m@BKavMq}z_A~J{__F9Dx6U)ydx7;{!BA zCNb}F1FQhg|5fg_g5Q)T77#E>^Ob{P9d=4hu14oJk z01zWe1VG`7J%7z!Ox#GZjXi$3R7j-Yv7a!J6Dja8BFbmE}ts#Ih|%9Sl&#++Gm z;tD{ru+;Rd+QsuZ|jx_&qzM60xOBMizLH72Q zf=J?%y?)%VXyo`a=+L4^IbwKXh*dFcSj9POr2^#&J(_4e%l7n}t~^0((giA%s8OUF zb7=c#u1ML>Wi@m5%(29+f2dP;9!o(>JAc;f*cC?%5$TP4`~D3)xNyXuiW#RXM~tTA ziMjm(ekz!-NEFEa;ZuYg&p(rxK}RI{G33aSCx0Uj-uQ83iKbJlzKwj4`uC~bVbQMF zJ^c6bH`c9;H#cToT89I2*Dx9GXkXp4mQjPV$lDKF*YOt1!av8#W?@4Qw3<; zfrJ@%*iA&jB;(klo;_3==_Hg=rbeYbR&JMtG+F-RPe1*H^G9|hbV8+;Qu^bKZxN-r zCTeTC38zFO91-O|`^4g3glDC-RtAIAW6wKoyl|YCV5T-^nP*OPLNH;z*UvP*wbo9h zV*0a>GDs{+(U)O9YUZB+h_+gXs%V9tf;R8hmnVEuwPZY(AHu`dO0RZ3iAonsVxor=T7uta`HIN$7#<ZF=s*7>GL?Aqrpya33VZ=3s$Xz>5R2q&!Y!VEX;@WT)*yu=y_!I2_1 zDuM${j64d$%#AnB(_+OiR_r5aCm@rSFRsq&k&@rN6<$Opppx1=*YKiBFHJRrSu;Zo`&%%Vmkjd#;e zN6k^I9%9as2ZT`KIh3QZmbjSZ=xj`__QW+&-?suTxGIDj9?=LKpmL2apj_RQJ?&UC z#p=&;uo^bEE zBuGq1LHON|U%vV17if3Vx-3ok-Vu2Xw%A&8y}tPNAfNdt5x=gX<- zfe?(K1Sd#A2bIV~GGc~}FqR`2A%qXW07u3!wy~0N3=Svz3@~h`u2`Ah=c-K zga}eJ<&S9`#{-E_hdr2w6goN)icL&=zOGt>g!Kl=DbSYk#r%$o=kHsy^X9P2?e)RxS+ z<(fhsGLegH-W`7!OHa;@noIo0I-1}-2W=7}+1#eJ9BD^9N>Yzkj7}EE2!Jv|ixteLKKIGbe){vD5t|stfFTVLVxvVUgBlzH8WnVf zNdMqRHxSW;W5(-c+4|@*;}wN={39Qwz`z}6wr_O`gqt!^c_1T`=Og@RzH#vU46gWd=j20@`idt@RtOdzyAbb>54 zC8cn*Vk^p|>}6+`fgJXu86)UscdrB9ZuU(+q8e{L6fu_;Fi{?gsOnhUYY+I!H%KZt zj3v_G(Fq00QRvXsSkC{8h4kidy<8)(Sev<5h)6c0Gb6BpPs-j(hK^O91r1&ir&pvz z5d>$=WkeKAA=iADZ-edT**4W+X-S7V>H}%?M#|wrV&=j8nAt=&ED?En$j1X7?1T{v z5&GJiB=7dgP>!;cr%dIsh(Qg?%^=8dv|NvYqo5+}U_!GzW;jgraGwdU z3MlXdPAHO!-ADs=E!&ZijvzhL0nMFHljk%K^B>(PgrEmqh(kzHEpzdJO}{pbJjhWZ zil$nlA3Z}SO9-kiK}|R0V8%QCInaa_w4nu+;1oZP!Lno_M)>vVP>cG|rRX$|D>Unf zY&f%cG7v~4LSp~zj6eu?xI-cQPzNm-s=;71HCdu48o)}X*Ab#KlV*nN*{RxRa=ma_ zl)NEszy}l{fg@1q4@5~u+R_66edBS4!cM&6*D`~z=tL!c)B-6W=meekagA2+ z7s(+XdC5<%FgBi2OuSr@#?iGUk0S-zW<&KzbWVYpAE@RZx9}zFn&HpfTIbhngP5@W z)qgNng=2t4I!yW=8}nM(WpS}uNctM4S60R~q&Cv?YIUL0bt37;P}(Wt^ZNiDn9~p~mA}v#1?{0XHhKI}I<$j_O&Dy4S}ZVQi#;6IAw( zVjYt^rq5_CO6_#Qp1Q*$iNZz53VhR}-t3txdT6A-bL5W%jb%@E@x@DrO&uuv+JE}< zSjdlR6edvA>a>F(PUC8mboqa(bv*AbmJY4G`R;nYAUCl3*4xAOf3kM0#ecNtdcy#F zp*464xPT1UfD|+VST<%}hIrV3T$%7OAoC2suw0QhW@-QdbORxq!~{_SSU@suGiF2d zg(&})77;6GEE#tZ$Dn?KlpQ{_7mgqeR0j`{a2_K;0XV3GsKJ9iI3l^Hdn)4(ap6B2 z=MOVT7PN(PvodxRp?p@?b23PSu@ryQhe=(SZ9`XM$W~{oHey)E5hFl?d@>AmAO$|4 z17bi2@OO3FryVD78qD(oo8?tsM`PQ?OxE{);Ad-Jh<5b%V~1!qP56XkL0c37ginxz zJJ^GAB!JriiIO;kl<0t**omI_i43Cx^)_a}fPv4TfgjU&8Y2iDD0vh$Au0d@w_p$S z&^&9_gkHyaGlOR^_8GjWXMYxkWkE_n74Ra~RE|cs8RRHB+-Mf=p$|dVbmGSm-ZVj#18YsArBdmnskj3qKwRljCNNf z6giBvw2ZKDjJ*hoBw3Osd6EwzifVvm5vT^i<#^AKiXs$~tVnPP6@g}ElPd`USru)Y zMGt)dBto(&<${7Y^eDz>{-6SMbWStuUH1+idPxXvx6-yY<28IF;6$Sv%={uoi zm}0mjYl#x+DV_9@GF>H2973Or1R5ovGPU?8l!*ayXqR+2j~me}oFz>AXqsc;0S6`t zZoz^UQkU{zoChg(+^CS22>|;^i}pzt=`o=m@}U$;k2w^b35bT&sh$6?Qk=kHijaq8VW69E5Tw2-q-qcb&(%#%pbK>n1cUShHscT4 zNjuJ>gjT>U5eEP+c2ccyrN@B*QV;~YvZeXZrLbfcVY*5bAR7GuLp`|xVo;3`VFIGi zq(0FI*KrI;L8o>q5i5YF{h+6_f(CT*T?0uUUnvpN(hqj2MQgelZ5l%g5~s(+o|K89 zWW{Wd>ZX$_r+DI6`=ClTfdOLhqTU3mqG}eUnjKWiC%BMPq`DREw*r%ZOOVBv1e$Bs zQ4id}U)xbqg{h!uD3;%l4gUaM7SgI#ilyzRmJ5j|TKB3BCIQYT>3XMmdZIIWulSm;6eIz!>6?bsaI}boQK(ZVX7WWXI|rL_A0I0cJkSYnLoeTgjO1&+>Mj? zAjS7V#l_pjUi`%gGXcqqu*qAB{w5AIIi%Gqz0r%XYs|1{HbX1H3eONF>EIZZ(2-#I z1d|{Q>d+5RVG8?YHj%gs*w7C801h%C2~!{{Y~~1tjL3?t3ydro;ROI5UdtiLh zD4o(7a=bU%(rOH_&w$1;c?t#V(qTZx(Hp%-3WD9F0um5KHh~guHA6J(s7HI4M?G6& zB~*^|>JwP}KiX1lJN0T2_rGwCo^(eNC*v)Y+1eq-M|z|Q3kRhblfcl@H$9}IUD`K&+NRyGn7!JpO%Wt$k7eE3 zvOU|hUE3B?qc^?TsEuW(a0;ww)24meIZehl&6Bo$++B=>hXRAh-Q3Ro+|WG{{EFJa zUES1e-Pe8EsvX_j4T?j+3s*B;kJWeGUEbz>-one$*}dNE-QK7Tg6JLJ3Rn@Zks2ta zQu3YO`n}&KnFQ|r->6O6*u4X*#NP&9WfV{b&-@LmkOzZ(;1C|+5`J5Iy#@e&;qAQy zAJEtn-r*E9)Qj!mB0l0Iu226qApuK};wpXxD!v3PzTz(G~VLyUE(;N<2t_M zJWi4l@Z%rw0TQqQK_287P~;zQaNe%%NK;%sx z-sR+-9eBS4N{^x)m z=z>1zgkI=|e&~pv=!(ARjNa&u{^*b%>5@L_lwRqUe(9K=>6*UjoZji4{^_6|>Y_gC zq+aT#e(I>6>Z-o#tlsLb{_3zE>#{!Uv|j7Be(SiN>$<+{yx!}+{_DW*i5$QQ%TVmb zo(veb5yNim$DZu&O6>p4z6{9zC(9n~(>}z|&g|E&?AhMz)ZXpIzU|K*?b7b-iVE)K z{_W&0?&rSl=5DR%&hGH;?&@ys?+y{wuI=}(rSZP+`R?u5PVWJ4@Aa zkMvPb^iyy2O%L@)PxV=E^(cSyOyBiA@9dlP9{tG?W3L`%zY%Aj8fiZfYmXXjUsh*D z_8lts@DcYB@%I1!fcFrg_k0fkeqZ-_Klf-a_DCs0tp%{h)`j}g##f*Oh~cfLx>VJI=uMLqehDtM^Y@=F(XNi z2UntOiE;lT%#bG=rgS;;W=xSUch>aD5U0$KLwo8J%F||2ph!11Evhu=Q(!FcO+u3qM6$>9 z2pmYTpuvL(6DnNDu%W|;5CKTokwn6Xh!`_!+{m$`$B!UGifl;1Lx>k6Q>t9avZc$H zFkixe(#6XbA2D;n#2`Y%2MQN;3LOe_#EBU(ZuCG&v?)TKK7l%&O0}v}B3!)K3_`)G z*M=Tk*sxJ!!i%6^)2bba;>OseYlVg#TlQ?+yFz!cAfjZ)2($@VD7nIA%M~Ji0Xv1E z)h1571y`i#$HWh{_!)v=}><#s?K5Dd-@P!oq3DA62tIERNk}|iY9HtbZywkQ?kg~rArd233n*T+E^kOWF-*hPk3fRWf#EHoA( zUK>HG`iKO|-(Fi0| zK!XTvhQ#Qk3;o)X2?8GKV~(~uB`iX?>dGsfz-}}V3q}}0M73ECw3oF;fVSxb5`3!Z3zcDIov8xtP^}S3FeEOw-;ndp zJF`5{&p{8J(9}Ugfaj4AQLuH_U!UBwL1d?Zv_cq!P_+mvK>vN5+cqPsv)n$ZU^35D z&&}-1M$b&N(MVtF)n0^Y*qEgN5MidNnwQG3y_S2iDN0<1Zn<-)!9X461I26Im;kVV zVOCjfK|~PYf^PZfpUZ3J<^@T|1(;hTQJd?Y7gXkkNeIj=2XN|`6srbh0D+bQy_us~ zk}Jf)c{O5vP_9X>n)99my_e&v=hg6HZOx{a9u0mMG@=U_7g>V^6NpNIWG*OC{*|A?tktiJ$z`Q$K=WAXV$L!3>I(5e8x`gE)gA03;|s`t?d6X*!^s zyq3VOc+i0%RKfQ^s2=edgnS>wS)?pu0{nrGEd0ZlRsVJ{Jkwbx2=MCM!7v59d|}9l zSUOATjCeWY0U%wXgAo9BK!hvI?n)4oh!QQgg)E`vLSXWe83i&$Bc2HWsta9KLM0F- zMiDSd1Ox5}vVl2BLma+n!c$~`p7m&tdIiy9L9}Nd?qLLaaZ1)eDA1N1uFp6E!9Wfy zX}$wdAV3qePYmj^AdaDjMW;Lx2woK@6TQG9BbZ?;J(kH$wz7Sm1f~2$Ny>p_z=y3g zR0nN&OPu9WWa;6hSr!x)PL@V|nsg>EDIm)gNm3w`JS8fNBQ{f7GL_C#hz^1Pk#Rai zc2qoA!y0o1t}roqS4^XZz~BSaw2_LILY^0>@c&9IW(ad*3|$$s7EdbD^LSlK=b18P zIu3PZd0OEo!+duI(2U>+25|uSXjZAMzQyAH_P&cKEPM6wBwmpQZ zacZgxo{G!{l4YudET+bG+Mr;%Plj*`UrVKlo*uM`r(WGAP&U!6_ z4upu8yMhw zaFNDt=&^(OR7EMPp)Dvu2{maVI@vi`n*W#yclxa8xr(??4^aRQ8vVyVyrIZ~oP`2i zfg~aq^2;10P$P#!Sq1bbD1o@N1`)78X3FO^k^zouwHh4aUIPLZs;4#PX)9BRb)M{6 zU4pGY^=oKz@l`l%=2ovep^JjOzshZz! zy;b+Q_qp$T>zsSeNnhaO>WdbQ++iDx4XYBls-bXb5ah5>@}a9N40}|B6c7vMJ^R!s;Nk#zrCv)okJ2RWESX zS(uBvPIqUaU11t8{C+cK?xwEVwI{yM>|rG(bK_&3xsDB5Kwm!6k*p&gp*6+`UH`8% zN(#mAU3|yzG2#yEIk;JtsY4ZlnUbxKU)v>bgkR^=jYFOz`sDxa!nB`K1mGm`)X@D)N%^5U zxk4AGhq1|YQ`i#)Pgi*Kw|$8JGVw0IG%0=7m$Wx6uSabbcM%F z!LBR=AR;lOx!u8Sc;|-LZ1{$R^jjFW>YWINA z+-u;Hv68mAzpoU|PH^)vgXx6gc=ibM7Vy{$2SKtl_g4}@I7txWU}XSxY9g7#2DO)& z@gFq?7Z>*kH6du6-INP!N`d0Jhswnij4ktF78WlsXPtACoRPvR0axUSkp!g|jwxFM zV?4Don*hFHC5&V(;K)d=|?1 z?Hdig$m5J7bh>70w+QB07+uaihK)R>^d3ePXZH{8=va6#Izh4n!)Fvt=WtHn_)fAl zo!&Vi+p(@TU@2EuC3_9Q4P_60L@9#7F^*INP8LA?p|mvhn#v;?`JXeYTr_XRlZVNS z%vOS4YXarDWLfaSPm|H(EPly_@TZnq($kb#;q*Wkbl$PVDhKUS7pL06=B5%^71?}38){nK+d&>c2f0HtY)r#M7$MA^ei1R$+4Fh@Qu`N+P##K&Z@qARq z`GiIWM)!u3Eik7Wn$Ar_XDE!n#q2s2=&Fv+vuOlbY$#ZJ^48sI*orfFg+~AEww+^p z>K^YdzOL~-T;_Yn(?{PeJx@5A-3r3`WM~QuyuA$7??LmuV3jU->0NftzR8QVFvbyI ztNTz55fcWQ2u|&syFH7t>6C&-otoo9h%!5ij919xA|-9Jk_Z(;nsg*-4liX&JrASu z9`Jk}!Ih+VaagQ*;+M6_nd60K)hfz?q$_f!^Eu&7Rv7H=pow?Wr3ld8zISo;c62<%IR1c zHmFFSPG#X`J;d$HXI@0#td+d^o-1TsOs5@3SeYPsQw)mF9-KrHkvcFTfpQc+H7CWd zwJ?Ov75v{ydJExD9om#PFuGyzvResTf`agyoVn~mDxQenG`Vaz7|x}NGV46L#k@^s zJ8UM&SSD`rXtg3ijS6i(fpNLD82mcB9P$-7plUc{s&=TA`GQSoo2Q_H!?;z|nD|GH zaS9o3r(uOi^tMQy-3*LSn|ehgl%+D4Bhrkq6TO7?CJDyqYgdbHrHF(^mWDmV&w9uw zt1Wg?)PY<43y0}PA|)<1QYGe^H8zF+%bI9>q!&E5qUA>{2{{`mMJ(qOMtTME5>=?$ zcvLa~V{@ucj!)ZIY53NokO$0`ZG_AIL+~mdj2@7LC0a_EH1`%a5Asqh6-9X-){#29 z`R<4OBX=sQH7nXPCwbxAEbhoK>Q|=lSDiSQtQsoo6lGO(pB7vC4&+=+XxvlCenN2n z#3$~ay3>5B0gJ>QJdQOes!z4%%P%I)4g;^}DC&(gMap{pl5M1Rf(v=FrNjZboy`JS zIbtli@e8y6)s2@0cFu{O52o^FPh%}2F^!1Sg`Ts{!>rNa0Zj2PW!uJTDztGP?> ztm_HIQ_4?r_7%cR6;$fuiFOsyX3TE!$^}0QTMw$J)Q}u(Z83h{9G0J~=fyN8s;Rd6 zaW1-IocMZ>8X-ld3{Dloa&6$pR+`uw;W*viiE8SH1iZ;SFis?aN>Av6taixP_?bFV zmRnvOI7q0P?**_;DK!UgwqNKW5FSW7r=fn>fQxRh@z6bG zNdxi?@aqonISn+-7Rq!Ew3hI-(WMN|4zRR6VWEXH`mhf3y|h(-DLVafI*R!tS%M{) z+%(7G6_$=;Icnf@RzJgJKi_VD190Fu?O>?L;ILT}H4s#q*+t9oKsm)uO6B$4FOu&* zv3xXgZR^PM#^esS;V=EkU%X{@*EJ|8VrW4ko;569;-Nyx!!IWc^qiyW#uB-xWF_FO zM>2-t5@=6mySoLFc_;p6lnf8VT&9d6oh<%VS)1ufEajFY59ebY(h` zq+(6byRJ4BD%9g6ZHLnitLIf*lQ~5Z_!&)%|CY7Zh&Ck?7wvW)=M#74lGCzm(YI~^ z+1gE>RGM<)r1`|VAQ!#8OW97`mE9)B{>;1@?H}J#qlXj1>5&9H(ZtzKIKr$laq`oN z*yw4zxEg|y5!V}P+z43;(c0^k@*S1JRiFreAWq~D+D13y1oY-625eYl=KUcjQ=Cq;|RUW6OGN}OvPnAMk`6qlYO)Sb6ZWE@Wevsly1m6W`aXIdm`R(zR|tLxaC+n9g` zh9HQtDnJGe;@5ib?q0&i24K$awvGHSn4U(SFAO|y@T9%&MpWSUC#sYhPzRZ9IgSUo zQVwLkV30e~s-BNy$55?T#>K_?VO^A(!;b#ISgDbD1XDDf<|VH_k)&xb+!9NNP0{%! zslvFBZJp;h)eZvQyuGN)-Vp;i$IXuxvEmQ#O6jl2`Yi)amnR_a*t1rAI#y(Mrpy(r z$&r=T>Pn#|%Q`XdZ19|9x*nhLV-V-d8h-4m(;5_qYu?`7e>=*}ys39PMwr^l*QV&7 z=hZ=381j~JdcYzz=ZiHs6Yx-Yd^=8J5cMGse6xSua%mmZxv{>!LBb~aLj{uUw~m^R zUU3e^cb1OLC5g+mJbB5f3|iC6d(oTCCn2n}DFLjNT;`Akjo3ci;7Azw3hEzHmL&zg zjEQ7Kin4M-@(Wx@qXsj>Xo^I@?dkS#X0hb>_wNrRM!!gd_ujv=cO1jon}x-{Y#OCu z?a^a$-We#{8T#}7L+cJ8u!<5%t{rFp`?LJ9!pGBj{x*U4zfIpO5{^szZHMJ-`&4c- zj!fe&h85v9ft7ukhsvhb<$tgyLzI1|RBcpuc#qaIE0P*^*JR%|z#LOxAWalK3yLik z#+U(TsrR=Hre~4LLQRJe#ztaX4WE1tC;b{v0x0Ht{+UJE_XWF**mEz52m8bspEERp zFUaFPvOcNyF(=RSO9{fi81FaZ9`LXpXvRi*?tQX?e)eR+$O;1G|9)OnA@~0!2RHBX z`g`d07AV3EqBE+bV+JYxG536M6#bks_%Arr|0vwR?_9Y)<^yTNh#iC7?s{?teen6{ zKWIABpyLUtXhY$AdWIB={iKv}thnri&iRCST)9G#!s*1Fc4?+IgUU)h=YcLqN2G2_ z-)W@V-183=T^aE94?3OYXR7d~cw?EQO31S>XFe=bh1&El6ggTA0=7yIr_0aZtsp1; zQ+h7WS3i(b#j$oY9&COfpO4iwVCA7k(g-5SH~fHHdxtTM7p>wKA1`1Bf49H4UTDx< zelNdduB?7iMW^!ol6vG4@D%p<0oyx4av<=(p?d!;bit$mj(?~g9{`L0e@FG$bV{|_ zt0q$aBdSNQEz11g-v1xMh5x@k_>WZy{dZX9fAXsS#{)tT0QP@5^uR2QAle-@Z*sW*Pb%}_OsRHf-F%63$m#Z*&iZ%H5D5eaJhcsr zm0B#cMNq)pbd8i0RJ)Q5y3)*DWH4@H!fW1aH)gZ))vRT+({s7WezvFeeQ)6BnX>n5 z$HoKlz;Ndjs(k`xDSzKDvy{oiuTb!RavA9$6RxhpP2Nv^|GbC_#e6=@^>-e<3;#mN zK){TT=c&0c*{_*z1lJ0+a!NzDFyw$7 za@9}5&v{TDWxkZ&$CYT#rsJvy7*6%4s*zbJ6=ehqQED1&m4s`? zUdqo2K>PtUSiosaH_+%_`EfmSJXAJXFHw^aIb*os>@;iNDca zGS~A=%SyKBtiww0RT-5Ew&&7~+3TF+p8-G1WKEls<@$F+->3I}AQn-S8WKAt@GzJK z&9=Xod6)Nlc0p6<&&>*#cLDX&52dXZ_JMzTEDYe}cYS(6EYDWeiUXQRG&a^tH@Cb$ zPB=u#)Se(dchHOWR&g8}d~9VpjEy3?nL(aDx! zd;{+S9IVRKzf5H#K3*cdm+YTZK*s)HVC3!!qsut?^FR!xFDnr~7en1}W|;X)L((Yf z=ltAR63pUgsCJ*CL~xsqc$ahJcBp)5s21Uj?5SQYMB%(Iux(LCLQJ&dMF_E^R%XciH6}f%B+s{*L+4xNJTVz*1oA&9QTNA zMQE+X1s+oBe3YhDYaOi4te(oCmcmg@LR8NUqX#UfP6J0dQrRo17&*;I+DvhicE^9! zq?qxJlLta@*4eD+DMD%r(eTkObh3vdN>b*K|{mdGsy;iI@ zs2e#=k5(2BuG!|htjrk3p7qfF5f=Tm4{S5R2G~i@<5C-KdcZk$jM4h z^F=r{vyg}nJ0#zu z2bI`F7KU(G13_PgUlFq`mwoPf2KUnF5o zr*iB}*dTM7rWwi!NZa}7novXZgcriReMV$V&H9kHo0$tOLZIU`3Tp{$5)yZlkH1={ z$%3_#Y>yN9n~uU`Z8SvPn=(bnt6*`=aFGW=0kKpnH9sCT4et4^MA!s@CR_6kON}C|SEi<1-z`w843I z3>G2&D{WpHqjR@#aYo})QxX4KGRBrckQZ~x&0g0)C$+-fqspyHBOYvN2Bx{G!#6=C z{XiH38lGF|k>6wVDxgLyGvNI*aVJuLGg)dcI3-(UJoyL&NCw@-c}G4Pb#(;*%X!@M$I)&G6eUcbImbj>t`$MaRs`sLn z2FO~*QHIA+7qUrC{&U!{ws+D&uY{`CQz&RM6@c7KKD%mKSW&u)+h$pDK$$O0-Swy> z_C?{YomST@Oqgnl_Vz~_mar2|)Q4(dHa{1qR0l(97wpmpU95H?8yhQdY+H&&QPvyyUIRL7~}1JPQH zHz=~NOeTfAc4&6TzU1mWe1mSHw@3#bG@2b-g-%1mSEgcqPXxJ0I1|MI1wh1BZZXgX za6l%*&&PqAB2VrMMBW7A(glgW7~`t#Gvz|2G;zazn_G~Kq%iF{(`zDWIhB8^!`PjC z7p@#KcPtwrRhtX+emw>#YlG?qRcI(w`e6HO)WR9U?5^@*DLFZ0SIn+%f;0^0aWn+g`eaIpPR7ab@d@DO zBUYJ}AGgm}a0<3?$NB1*t8W(=x64%>{^X$>aYPLmv*7r<5cVzs_Pt)jStkGO6*<>% zp2jiEV-pVAa6WTGmUGi=2fLS9!7kTfAwSaMWeQQ&g@KEV>!gCqXqXCK7^^hwbGU1? z8ZXzh5}p@i|KTGeRUD&ra#zhnKW8Z#;_?r*U?}y)Hh}rd#J%<0OhM zt4j%k+t%S63qIign3K`ci>%!;N-hMf&rhrHoy| zV>~5z@KS_S*~F%8G_EoLd*wkeQU>-aS6(g2zAaHS3SS+`rl_$ts2Q zoc~g6ydxpTXfU=PCgo*|p3xH?rL04!c1%1Qn2+fbDEYp;TEtm z2x@6I0K-f)h%oO34SH77YQPRGV0uX8gcZ>mpm4&b=9BF6eVgZ8@N~=du;Zop+XUmC zX>x8)Wv;?*xaJhnm4=xXp_nyfsUOp0f$%C_*+14qLXE?g*R(mhqSEsyH1;@pj}SJZr( zmXJUfMlmdp@7z6-(v2_uZ0KjzNk{kClKg^)*(xuLF}dJnQvsJc>!7#j?8AJ{BUiTdsAJV|$ zSHndx7A*W7r{IDHBIaBi8r2?~y`V>GB9-5{ct)%7zdoe^*5Eq7-b{uWxk5o}`HLb| zIT7%0XFZ@GbC${0jhW%tA4n0M;TX9Qa=Q@~UQ2ajobR6Ky2cm9U2X^2>A)I^8`T#F zWOR4NJaW=vWM30LfuFJZ*2Sf^DJ6*so(tO^zF-alk&ufpUe$4kcy{6ySE~$i@Of;} z`3iZd_>yq+&gx&RTbW`CO~bl8}#K97{kk}6-#&`ZM`flBrO9aT~ZhXQwjZJf_4bvTtze{-v2sXgB5 zl4^aA#J8%inxK`NrDf{0rC4Dp7T1F;N>0ipM?ojjz5>*Uir;A_CV!nbwyGbnthd>Y zjHlzg;)l^KutbHxlHfvoZ{~@N=aU}Kn-L98iN8%5PzL;VXf(o0ZP6yyXC=T}qyzeC z@xiJly53a~nXu;Nq_u9sU$gBXRR4*Ci5S|aiR`UO`H1uJ+m{3w;Jnz~lnoD_ z*D10dQ>oJ)jtK$f7}(?HWxeb)4f`3Lp%uNM5P8sS)9?OP5wZL5jQYw)psy0H#j{q+ z#OM^h`4-kFctT8?!HskPN5h3z%aL%C}sL?6}+R@se7jFzEW0rm4 zqn!3dI)iwVG&~7_*c=Gkyv*Lbx#l@1&80KjY8QY!W!P+*-^%}zz9+irr3RX2m*~+~ z?n~bi^4bvg8KP|7_)=s@HPW&(hz|0CE&1i<8|~1-V_tZ}PO!SXqfmSmfnKc0it<0L zoi2>g$@FuW@Q?bP)A(>kZvNH{GB-ob1g6Q5G0kr_wgnuv!v@t@^xpI|2BoWiId)@r&m^|lhJhOI0g+sdi=PA8@6(G=k47)qUw)64P4Wy%uXu*k)je~#8 z1kcSllD-mHx*KDtOo^|4GUjxGPfaf$2)*)KR<|`Q5u_<&ET*bFc@mEUS?nPs`n<_2 zG5Vdrzl^{5?Fo4Agn;pURPkIk{&`#!jO~?9!apJQ`#nN-Sd%tf8-T++*H?3`l*_(R z{3T@7fIVHpK=(?Be>k2X$emrqxj0fgzMZoKY;>+N0E7Gftel_!685HXGiHq`CFVC}4 zVz^)L)v^=v<-Vgs_wM6#-dBIk-&f4Dg(0pSW^DFqL9BvMk{ZfZ4TUrAMKcw(#8@e>TRdEcKasV$bFtGN>h?gkJ zpN4j&e!NCL(Zy(Er7Ek=gg$l`6WXwV)-Jy0KNW5^5R}j3rx@(4S7QhWlkuU*np&@s z$gk>Cw)CFv2l(FPUb8wcF%y2hBR@CNN~Dqa3knxi76EXHxWK~t;*%4(9LauN5i(z* zSji(~fiSj9B0|d83ctR-ub-!LuzZe;Hhs>HtMY8WVb{tvzEko(Q+2*Hohl3fUO=0i zc%Ry*E)Lc6U(|@*k$&Q%gCEwF?r+PzGWZ|DvcnUZr4>vkjnL&rU11`&xO(dis%@P_ zY4udJp1eM2@uvN6*pf=K%NaB?ac&{nPC8Lq89nr!Eat4b^*FQmg$ zU(=z(_hMXBSD<&lLs#fEL`*N!mlVV$!m$NJcC4tRa$UTY$0$f2MSCXJF!H9qMO|;3 z8Y)cTp1j0@J+Unc8{lfd&yU<`%99LF(DvnTIPdrjmh$X zZQ2(%EO16kaN8*}Q+1{dZBCt#^C`cR+z8+T9BBvk_2M8Oc+S;DGy5=3$%p(`CnW!X zpZWgpG@p3|F6m4B3H8%78u}+DNmdBkoI-_%y$_gDc)auYXz20BCr=nWcaynZg@1mk zsTgrk;PHxGwDx;X#7TYqtH`q#Q;JbvS}9_qE??s*MSq>*dj0!qP)aH0cHQH3%>Bn? ztkVKp#_QN$ms3hl{@foa`S+_|Is$N{5g^J1Bp!sZXbGVNkz9x)K{-JDx~=fC@%JjUKO+oWzRyQ5XzYd?)>0UL#IWa zSF#es1(qnS#+rQ^A=#(`&+Aj;lw8CZlH(IsN~`l)ET&kt?ESVts|$uLraD3%Vr!^1 zMDrKZJk;23Ckr(sdlu6JP!EqXxN#W97c(A@KV-b^6Q;OWeEJ09%8X;(B=KM=Gg-}* zm7*w{(LX5bDaw_d17Ev>BPhFI+?7)zLbC*AKtQwM%B?P=V`9>tS*zy8Yu-OY8lRc> z0_Dc13Qo(Bo=QnQ}Jr~O5!l@NIwyb_Vcgf`<#gb}{XEOTU z7R$vOSlUC0?tXp$FjGC78+Yjm8G~R8o{|GK582gzgOt*ovP+bQ{Fuy??96ic{rHjO zuR_DvyX9vSp+`zMvPSVPCPg^vo+=cwZ*&A#DoGL>nB*eJj5JoNs3tsVz{SQHVJp=# zp>85TSX_=DJNWjer;a(nWP0oygI~c={9~1@f+{mx34xP|Vle<@pPUU~gfRTkBT)K$ zr9o4lStuET#2sGI1qdU7zB8~QUZg!duE{J$wzb8;oRv;%M|0S_w%vvuEKW^;g8-QW zU{1|;=}ndlOQpE-PC(W+I+NQ0%i8oTE*t$6oBpo2@;>u6;zebW@Fis3OdbpB0E2bz z#o6ou?Y6;A-TbA9FkvE8?V0-==0rXeEgjC9LoG?o9BD<}$P;zmz4!V6a zkDw%g5^|8IWbcvsi?OOk*vuU}M4|l@ucL(HwTuN~vQ{gm^%wsi6Ia0loD228J`Oy@ zh208`t=fN2*OqLWUkiYSJ)>AbE7vHulZn8jZnqIAfU=zh8dzw8beuDn8hE$!LPyUI zKAgDGq5!-c4P~yc^xT+~nmklXIE*83q&APy{`!{4H=Bj3uN9Jy4vD}n?Pi%#16r1g zvISBf@>*_M9jkr;3bR50!hp6R7pOr5X*?{mtKQVzfZ)|N04lCl=h!%n{_^XvnN^Yp zSq8}E`es=!>;xo%b{H3Pc$S=6_5r5F%smEHrk~GF?0DS-eAy=FOgNMXZQn48)}^AA zqv$O7I}?qweQb_m0H*kG7rDM$VdSXGx%tY*)B{{bQ|9conpN%>$tk#k{X$|o|;EV z^Fl7){Bw5j4@u4+@0$#tyj3$~+R$$LN}1W7>;mYGx9m&vyVJsFq!{`h#v%)NG|?Uz zG(JhcMZj_r{^NYr^+hw=KBu^E@N?h#QTn$q?GX@l^Xh;k;^awes7m&$n+e6?6N4wn zaNfYntZEq+l>Vavs$nx+VZ=5;Q6#v~|F+(vjK$=^ca`A_3#FW$^QpZ4{KKkmxb#S_ zZ8sTvySMZu@2TV>*26zhyr_)<(d@a1Tx{ToEs22{`+ea4ufZgAe{4SBJ$v)Jlb9s3 zRbyXfVF!^M`D+VhRk5i!eKXGox+#^uy;0O2dSP*S`_2)&er4S54FuWS0ice8qlpA( zTDySqLiwAad-FOYpXj>GXe+&lSQmFaxFfb)CyGnJY`+=y)$HZBWR9VGftZ_#-j9DQ zP8p5r<+5@lv0}A9!hoEK(F@d6SYLBAK|?FG&kJOZ*qQ9Gbaj8yZqFX+Qwq->?M5X$ zqJd|0U0a=vxM?ELb9}Jfa578?jOfUMj=-Z^0tR0t1`67kqTd=RQKa#bk*7Y$g;LQg zCIUKyiX=WnL4h@hJ6Zw~GbKPEG8ByLi+as&rjq~f_G~kh$49t`_n)I`2bmFaidx{f(3v+>hGr>3rBPWHut92PH8Aw086~*pBM&tEGB! zKaNoTO9uNms?<6Ps2G01x}efbFnstqv&N|iQUkRvh1!&k?C}bOJmgbv84emmND;ln znVu-=oEVE5Nn;*k@sdP*#&_o*-Qvx~|2@c6qeikIwN&-GrZ3~!Ck&tukh$BgNSC5f zsxV$HX-PCdZijXm&lxf5bZ46~$eoJt z*FGN;sToIUX(YoCTJ%JXCV~x3tp;+FSX(pAc|t@&+B8$CDz<|+<_3+RCoNk}E$y3h zjgshxzIpjcq%l%H&bE8Y=1p){Q^0R1F@QIat*?;0t(h(A6NVAz* zSM)`Yd_+jgh7G}A!jalt(RfHY5Jg%guX7fx^6487u`3BVWYS^3#c^di{$~0DIsKe` zh^`c!MBm-QBejw(P$@duOE7iS`KE^z@UjNoJgz|#BwF!OH6(g$Z%YS7pGLy3S?xMA zy*-|}q&>Ag)eV_^GCrmIQn7qkr-NQXMo;T3d-kRHkZZ{-CRZyJrMtk~of4JNgA&o= znZNSZW*wZ*_SH?gY9D0l@{Q2HK^n~=yb5Q|_kiOnu@8)Rmud?}DM>v%49jmsg@1m%gF~Kd6gRZsTSO0hDzjWZm zTk$?>iE$1#(J{!#&v7V&)|)FCExg#c+OzA~+w=GQDQxyp125tkEa9xzU(Hg%=T_7Go5hOW8n zXl>?BOYZ3bpum=m{|Qdlah+VVT*mhNkdD+0^iIKyYHV-$c_Rb^gaYyo$Orm$oM84fB!HRVW`%Btu$h;9x;B|9g|wM0<0d zLOPzzx-6a61{VPKHQ~F9%%&~x7IQL^qngV*ZL$yWb0%6UW30*XZn6QIg7hgJCvS-{8d^WIAf4W{fs+Oed5XZMO`)A`)DOYdogI#19~3{;*$afU6}{TH0*4y0sPZu@dQ6%+Zyh4kG~1XndAP zoi;C{_)|9W^9%lWdo*hWAp(q#>mbH0(j^w+w5V>oG{XZ8>#o!j@IEku&}lwn z<;M__b(TSI%pE6cZ8ub$%8`x?6~d6rJ@Y7l(9j1ujTiY#b@ippRdB#e6Z&kDde%DktT@_zq=~6*yEby=mP4Sw_QE z_=^$0c8~Off_WSO1if=9^Pwzn)y&KFNN-=?;-&E4mIZT)J<&J#0GCfd=Px$%M!0SU z?gzE5=KRKnba!M|#1i@%jMFISw2q4r=D_04t=6}}k@C=a(jlq~lfruFYH>(E`Q9z; z&_kmsGyc%4&OPIL-}_(sZfw2!gPOyIL*B240hC9XeTSmKAR8xK+YPrXnoLff#8PgE zegn#c4dxksWQjS7uqX-Z@eq_b3UxU81^_55ptPuuqG^xg8S^ZkGcr@7t+b%b{YO{< zAkbQvt}6>l8i;bEP7+5vbwT2?I5PouA?{j9@>Wnr{A?4xlN?5hJW8*SMbC`8<1}Wk z9E}rH!*S}{<@Af=rw_bxHcmP>p`wMQmeQVHMJLaC%4(mVa0i_liFmssyp64e3U@o? zq){pWWW50eOJ9NtH$AFeRc6PFfQ4mnICYXB^>~8$LU}}e}DdEVJV?7 z2zB5-c1d{-x`Yp-L5_07cJ@uM)zH$x>oC`ZDl@2V_zM66f z?0^SlA+NHjJt3BX>C6dn;fYIs8v#OeAT0D>FWI*xjopzk;RUmNq-Q0$HaGT~iMZG@ z#R0y!u%P~$3jnB2qZ~GHh~hG--%tXeUwL||_@;sdo@TDSdWRKyN-E?LButey2#UESI}L7OrqnH4}?9zQ}OZ}W`8+^5iuf4>v} z0LCv$P43ZNuPFk083GOSZW`!45sjuVN}YwjVW_=3gOYTPoqdABe7){FBJRVJF%eHm zW3XgCcz5598SgzwjVmBJ8z`<97aI3*rXK!G3n7VDNbF@sKnapawF1W$k;_#9DS|^s z%8Mf5C7jkc#c?#+$9tJi{C7=4Jlc-+s9-}$N#@i9Dw#NLPFd*QER4q&(*AIVWavRs zlGYVQ?&z*8@dx{6h8zH5-G(dK3r(83@9#)3?F*plg{Mu?q#u2cqfUD2_&Bfc+tgtu z)p~ZDWzvhMq#Z|8423_NUPC+I@iaVx_S8p;S$ee?;`hHpH@*t;Rr*Ps1*-LktglD2 z^gU&Fq!{zS9e>6!;Sp`t__dHL0^sQ)gomr!kPrYwl3#ihT={xXC+aUiP)tzUiaYPh zsNUZEcO-unsREauVOBJMrDeu6DxN;6#~=eDuu?|o<G2>LkvhU}O^W{o{A}f@{=$PYf$&KAJ+GKYQ3Nq_&Xea9$V5^ulNN4~ zR8>1lso?YDuMek^NZ45EP0}{oIGHV7Tw4_@en`JRX^%Ct3YEc2uuc3qER<@_>nXgHu;k7)sK!&YTY8Hr-SP zgjOggh_}_YR>>RU8b~)sU>C@RooMkidORlHw~+Hwc>>6l*C=l7Cai!Vx6m}dr?Y)< z(#Wo&vaPJe-Ng^46RKJXsz)>o6&$DTp1_YQW&t?_tmp?_KtQtRD1k7Sc9Sj#-dCr8 zS7`XZ@o=J19EG`rKw;dPIsWBT!6ciM#=6o-0-^?}Td}rK9mv$Pmf($Rdab~Z(InnM ztzrVJSEIZQEnQnWna_&Fqj+EBw|JjxPqU)V>dIzjd>T{~c^letZJO4)+~;l1DmOk< zwst;?jLPhola=%8dUe_^@FF`p%B$|+V#`N)|1?r?OtI9-34@Cfck6k78YLVUDLc-k zKJ`j2vkdy^Tx1&e2R`>8c7fa`KsjY{n_G`#@KmdQ2lrEEVAwon&4lLVp~&I`j2sMb zrXlOqjwj=LQGY@y?%lY3nteK=$gbZeh)czOqpcnl-4&e-;Nx#!bt?_%wSIr|OgZ5F zap~rWPH|Xd&4;1OK71wmTZB`pLQkzN_WftGtSEUlpjY)V6ID~c_8{j=+&q1WXE^B=Bf*?u9q8ylL69bT?3 zokf{d+;hkV)jnoNF*Gf>yytxcg+m1vV5ZtueNS2}zkb^ibp5_I%5PoHefF2@%Mcgj z<*L;s1n_p^crGLcYjdW+{}KO>_kEk1vrPDePV~{4RtEJl+sqA%zdW0Dw#v6+PLUKE z*JWZKcQIuF0atK$yNSB=5oy^16s6KcEbfw}6q`sG!MB%P1Ocg5X#2;jH<@N@9I{kd zv-x0XxAkq}8h(?j1#$kKc91`0%C&O$J|QcK1zaVyVf=JjW-74WV$zP(uE`0e%eP3a zdep)(%a5~+jf(aZ&T<&qj(z%3!?B2mR(w*@SHpLuP&ID+6O2^QwD1O6Tp;C3Iv7*E zajc~(PN|AMoUk4KniiG;s#P2qfy`iT9lf!tD??3 z0sBxAmw+~rT1k5{t(8}$jM`P@#Xq!|7U~2V=-M-%@N!4lA zsV#dNnG>jac9DAt3D8Q&EX^u_zLgpN@ihDMX*LYx*I;X>ok8C@+{1t4;$Unyvyxr= zO&;rCE+0|xqA(Qy5dn{GCD{KMJz?QVP|3|h9GvJyJ5p0*?beok7=OcMT-c46RK z3t6$UoEiuV--0R&K@iljjv}LcDUnAKo>D$=n;b1qieU{}$H*sBS64tdwJ0PVk3(D{ zKPB>LwBs*=*hnG3W70Bb0wYvBg?n5HB0O;8fK;;mOI$NVshGeJ7G^rlf%apFx81p3 z>#aq-)l}BCQ*OjC(*oS~4aBC5$t?}Xtnw@?MRra*J`L18n&iDZOX2F|)?f<@Z_tP#6EqqMPIUY5@25U&~p~dxwy}DQT(Bp@tES%vi8s z;A8!5Q_#`c@V~Rvdrw}G8UA;pKXxuPrT8;v-NQORHbaqo`LGfi)Ku|7i^@PJ(Lzdo z*NU%pSXzTkEv&!6nJtM2bkJ_@fYb3LlizGI)7p{rGfd&_Il%VU)<*u!NVmO63d#A- zlea8=V*(dJ&9tA^GpsB)lJ*N3Ob*4UT#)p8!0-T82Q9a$N$Rz>Cp0dU2?RC#>*j(X zBxl<>oV2v5oG)~5_tVT!tTFm|)TOqiE(oglIrlaG;>#`%otp@nCwI#5zLl4%<>OR> zE2;78+fMHCR$2@F73$@H8N1kw$!B(xZ~;*1+Xpo)f6c@a-s_!9eY98&zQeY(6w2GQ zkd99)W_YUP^80g1wY*jA?+n5~oPU@+9=&blwaC3c_k2ykkqA&TEMp;1H6l_y)Ci`m zwGAMcGTXc9z8ray278nowmst2TZW+v_!X>HX!-W~<|bE@C#86$PJ_IP?B|J_E7vPh zcWEtB?nGX#L(L#r7GLS_G^iGHfv@x$41WpyM?`NDGOhd;14flCIjRH<@!z$^7{w7g z$FZ6a(JdI`LtIbaqUEyWSotMbB)gtq0Bg|W}&M81${t*zj#N8uEk@CG+@9hF9=vk{|sae zeF~&I#0c01*f5t^ghhn@O#I@(-SR-B)>xY7eB&=#k-5{XqK7w#;u%jwq_JK+a}MW8 zK2MU+k3KA5|4DTaZ5I99eIm_Yh)iR^}SNyG!fQuoDQaxvg(sf8>+L-u*b4xS!N{GX8NU@;JbClmqzAfB(>|G}S3f>~Tc zB`iu!QAXB?#Nb&3n>f@!NI_T_lk5?M5T@7jp-$Q*-uacp_$gHmuH7Wz9Rc#45BObT zn4S_cRO;CU=v1LYT%kqSgbDRTCou{%lo$8e-R%j*9ok-7)M4(S;Y_GucCg_WzM()E zpc#Dx{3)RP(O({|;H69vX)v1!QXUGf3LWBJA|jBEJVpkfPh0c>1`veegbrFsP%&%< zL1aKYd0Gml8+4^d<^Yj3FrH}eAhRiiattE#ID;Z223mdJLFgiX=|V3aN#Om*2VTW9 zrbQuEMFgeWUJV2Za1yhXpALW`5JCk8G(-xS7+Y9_9Ti06;lj6=|I=ArBc**rs~|)z zQiKS><3*&F6i$TSWli~oNO#;?KS~4>5C`J{l+pN3K_bLI#v?_n<4pk5LM}u@GDC4( zib#;nLAFRjl7z=WgC=5^fsBbET98G$qf1Ie7QSG$5y=O3KAQq}{C~MVVbzG6noa!ay1XPCjHvR);xdV;t7VJqF%6y=7cNn3|LkHz^`OhD=-f z%2^_cZLwq}(j9X=$1*^J_Av!lR+UEyTK^aXfQj0aNtClW|4?jJ6F7o_ zoTgs&WL<^g2Nc8;XxlOnLmi<>;TePnZ4=P^XKYLaM%;!h_NNP&#d{sZP)LEL+-KP2 z=YDcTRllyoo`iWMn0P<}f#6gf+NO@4|5RV4mu@m3+|j6ta>R++9g7lY zk|x)Z@|uSVsg+2_KXRT9G@Ub;;*C0AnM#C6v>bZD(UtyAmU^QFNuZPZC{QXx*&R!V z1}XE{mxz++kvgS@HpNOlUS3{eU0kSq?gwRoWs+qCik@jg7^%gWMbCUG?uqHaScOu6 zCq<;mGYOdoVyB@}L}GBNL}-YZ)S;KbU@}#NrW&62U8;C|%aEXiFWExb>7%2Tk|vnK zDTJJREtFa$nopcszI6_ghGs=r2mr+5le|$DQX5{45UWNhzYxcV${SBwL!M^qbxCSL zxab^Ksdm^DnY5`-4yjJagm8pMXW?n2lIuc%|EsHVs2b{NO@6DqRvV?pC<1<=K&X;e zplUjScAWEL^slAqSlhKda8Oc!zyTz zGbzN(*6hvZERF(z7yy7DKrPXxr52*jqJpLh$p_8aEY7NewN5M1dE8B`g3vAIUm5I1 z;8gyhow9lmYz!XRs?lGKB+?vH%LZWEs@T<9UDQsk7*s7m&4t-6gx#u@e+=H?zS9JG zT}uqZ9r(c=06^s4K_Q?JTv~0^Me0iS|6lZ#-s>f$niQkp@ualg6x9i3P>NsBs@}Qg zU`re=APK83t?tY)hy&Fhcyw*kf~`L7Y|mN**i8s){VmmADcTxt){>=nnik|rF6Bz@ z<=#Z*Zl8qUs9Nyu*G8$@8g84C?%>@e^Xe`~bgxQf$BPHQXt0Z(MHZ89U4FBLL}#%06^3ElgJzdh~lMMh;dKlZ(3v_(MaE` zQU!mAUomhiB+6xyGSY|0B(b((XaImJ6a&N-4ZccDxG?Io4y3@Y1Rr~h017S9#ikaS zg)U^KiB%BSW&-OfoDxgea4; zmQHe|?JDpoYj^U|laL^*?sB}wq(lT+E20$EW9p6JTo$@ z-ke0#a=zZOq+z3|;#&xzvXyQrXj&0JGb;Gm^TZAGK|Dd8@LNF~G{C9_KL0WeAIW=` z0@69bdmbY^FEYTIEAu`?N|Uo>1_1z&mTBNX+}5W_KXN{M%s!j4_bRJ7qhU7}azP}u zKoc84k18k&vn&%cNRYJj(KHg*yuNO+ZHpak%r`Jc%*8a%z*z{Pan1x zZ9>_?U#&JL6Dk1oUOmC29yf7$9mb2cuuOL{n|3Th=l~mNTiIrf>HSso?Ja99n-9s% zwCV?QJNFg4ZFzyP)>wC>QlVb>wnzy@b5l`wi*B;|26iX+OS}!&kzPv%H*iN_clbh3 z+>?Q__v;ekV?*VTwl!?m>1$ea(cDB;Q~<4n1?F_ zioYa@Q`Nzk{{VPNtp8AWdTe;n3@2R^8%4YUh8~oX$8ZDhX+E}J3cUbE5CbPDBM#GP zZ_81W-}gcTsT98VM$0yuyRDucMu?NIKy0^q7w-I2IbdCRzGgW=8Y5U=qK~w$7cX~u zCzPGzUZH=~p_}f38#8vp`G?DSAEIlG=lDVFczsT{MbNpSvv5LWF>ok!jA>;-{}u`@ z9YO?wW72YeHq5EV-KpQSNEFR+LAgL=p_PEyp4)LjsQQMG1g2#gV*P1FRONs z;nZ55|0=Q)r83U}5x)}vpu4DnSiRG`GMnDJQz^R#q_tmrw)YLb`-HyFdpNc-Ltuxq zBJ2g0a=@SJwogO}I9nD4ti|f0ukR+k&(!yZa?~PJ-Zn1n;<0 z#IYwkQw%ST(jDDiJ6SZ~GrN#pM|EC9ybr99%}YeCx_hVYdpJrHmr-SMf%~ur;J>qT zTv~eUAzSw`@qa7C3VD@J(7X>&JI@n#MtZn&W9#H|Jvfd_;Qkb(W=Qv{``|mw7nnvtnNhHlQ)-Qwv|G|ALNEih&5X% z|AP)l;m`ioy{9`Wojd3=JK>v_;mdWmqtiR*PO70={~lrgM(pE4sK)%k1{I4CKn%0u zgJ3~}K1^_8)-BILhY!mzV`eMJLxNc(;&PaQ4m4(^9u_GGfK@SQ&i)w0II*I^4IMv1 zlt|Gc!4tBW39LyFz$9kSUI93mWf{wmBTJr4S#TvymoPyREK`h`#)BL^mipi${|i-R zJ00H4IS{N5u{z88*f8fONGDXSs+`%f;LEHJhwNgej4arQM#~PNYAvmgynFjPoLbf4 z%D7x-78!#R8rs5X1v4XTaV1|22g!JR_2P2rhdN+BHeI*^)nQX=_WT)iXi=v&aa!g) zSmePuMMq!sy1I2v*sCojT-Y#eQ@RB0#w184qr=e5g8Kj>k$ZRVQI#`nE>T$%hdL&i zjEZ~g<>IevcHO)Bcg{dXhowPoP&QDZDs6kNrW5o^=0C6qWG0z2_FK&aoRV=PEUJ>x z54jZ-^5rI!2y_M#wvxeuC!er8@S*?>Ix8`Y7E=u}#_-aMubNtHtUBQ||Le`5`m#Gn zN0LlJsX+%Jl<=SvD70`j>O3Tk2TCZZ!#ogh`pTihW-RN+q=Y<*CoY$|QpObxs&S|r zx#G?Ph#i)9vCiIvA1`+{JOAi`I+N|j?7kvPrBM%8G!U>5gv`(T}C@3pa zMtzvoyI*h3b;!$BvUG%D!8#1Svfd+;skzPs%!jFzE^^}|-PR!??(jqE7$1E|EN1{9nmV9} zo;N;P^Eg)Qu=LtEHjgLmD&%v^MG#$E*=_UXbkj;uZL$5WH!Yr9^_i=pe6dsA+1!Qq zS9$vexU2+G+BVZ`(yh5Lz4^VnG$#r?!{UjM7_!@M+a|dzNB0H`_2bU2DbfWG0)Q=M zd}$D0Yu?bT2^|t)rj_)pY*nF;(|nueealjr>DvrS`XS|?{{@U(m_fTW?1M%&zV+lF zWRSnvYu_)TCJ8b)(Pxy|>LBL}%br#2@zuV0*Z770qNS(D+lO`A-EZrzs|Pll=p}Lo znH+N(*Q3?AZ%BnI6hWw@pmfQuEe7#du%cxk%p{3L67iHpTm_9&k;)+<9HCz%vL1sJ z;V&Qb0SW#!3=!=pg&S0mlG1X$?HNP^3dzaU&i1-Z?Sy5#f(Tq}1f%;IB?vt5;nBFI zkP|NNIWGfZzj{YI!%astI<$xvSJOf_Ep31bx?=RWRvNJw@p)+CNqthc6X-ccF|?Xt zii(7w0-aHKKN(fDm^U_Q(eWUE2~o*R_r{{^O<+{<|HaH`w#U3lvM{o71|kvx4cJA? zMTWeiL5es;i>y(UuFGRtLP;Yea*~QZXyOxD=MX1KBpUj2h!8FEIa_S&I*{3sy%SrIHA$YS9@(?xH#k~AjIjZ(2=K`Kxny!FyF2Gb@V zeegM4n23V9OA#g0Qp97XGd`%=lGRWdCu&Xfo=4I;qPr ziDC|kVDPT_($&~ckO%caNRQ~zC6@G=Til;Y|5u|+UV6lr(wFeUmCBfCc3L3_TixnITg0`c`*GAVhEazf0l*U0EDHyu_zJTD004|drhj zVk7mz6J)UrXxPH2rD^9d!zDi?6~h{_42h$D*eU>MLN9e|Y)4%v%R}fde!(?MF=%Vs zhrHDpmT;>~Xm>`F2_X2yQBnW0_vsuxW5W}I>5g|!RUyq_zwXE$Rciv|Z5CcWTI&u^=R)Iq{ z{kBn5fvjXlLRrfK^Iy9xNP+G2+HJXQTnbjMk8nFo&hplJ1<6cK25gW5U#*8!F)~4r zY&#z|_qh?Cuv+q4-c$uMC!gH$ewgdr682cFV1C1YtLWs*ba^5N3iDy9f@K05ctj@Y zk$f}==k_i+kpS=m4t+pfiQJVSPe8=B{M)jc{`Pfa)qx$z+X@mAE6~YUN~HWal0^?C zk;g?aSWrCL<(62tC+_Bp;v%a+|4Vtdz1?qoxoc=1>#2K32K7%w-5{D??|fOh@N6GRvsQU2p8kvRA@pK1cZdyhBdpC zgQDk~q#HAIVTzU;eKCQYV>i`qn*4c(t>gdP`J%C>SryST4pHe($+N0&mouFN5xO$Gh9(82Pum z_FpZN=}JBy`m8-%hwf$om7oK(5!d= z>!GXt><;*l&otr=^;!@7j0pEyX7L)&wz{GyoT2XiuF4wZ0DmF@0kBo(AkZeTA{RnU*|5i@}fp3Twpygw+L!2ChZSWh4OCj^6GE){KCtgkEISrB9<=&$%>_* z2mchOM>NmyIB%YINC}zH@Oo<^ps+9^0vZ&7*{Wo(l1~hO&*;|h@Y*iGVy^H6vGa~@ z44KXgHL5P^BMr+fHn`9Gz|R6X&;x%j1dZxJmb{~(|6V8+r!h>T57$)8>B9#3&VAjHF%QE_gAAAjW?%Me%sGOieK z8rd-+f8`ngWgPS7AWefP$U_S9@mC_UCSXVvZSNom60E>N8oPrUOY$1MEhe3h)imuP zwgdH4s3usFI~c~@hS593j>jw#AfFLg=CL06WGH_{962&A$`K&Zaa)vN2hpDI*Lb1;VpV4p@THG`12e4FUtgk~@H+wKP&I>9HjNWhy12Dv50nn@4bvLoS1m zE*P>q|A3(tVo5gO#Tg`F$mRvr8nO}$(;{YU(;Owy3PLZ*$Sj&tj&3a*3*sf=GA4)0 zA)c~3pdl(tCjT6g)6B6Y6?0gY?ICVT2OdH*c`{7;vL$zhG|vb&tB^Cb@gP1^Ek(0I zd=e7RGA&W>6`)ZF-V!F;aWHMJCIp301jRKYs2c#Tt_)56G5`QtZW)#V7*YWd$OHp~ zKo3CA$IS22N{}5}4MF2n#POm$40Tb$=FT|5PX~8_dvpLoA zG}r+2P|iE+O>Be!5CU}W1~fM8;6cx`F90Xjvay|lZ#Bjf^z=eMN8u2*MPs1xAt*ox z|L%YgC;=0!W_IMIK*h)+bl?dLq34RCJ5!-BwPr9%^dVjV00<#Jfq@rB0X(AxrD}p| zcy1Sp0vL3`xi;WDXw*g%G)Ku~C00QXSOFM9Q z|1@LgWqdZxAev1=E%jMvv_{z!N8Qw5k@iNtb7_h8RG$@Pg~|tD7FlOjSQXx|GL#j!J=;e zwtQk$9r2c01viIou5VYPBItrt==K8&cey%Z8E$cKEy7OP#Ht=Qa$U<*N+A+I18*_+ za7Ajh6n7*%x7;=tIT*JfNM&?=0F0;ws~n;wIM;6-Vs&%q1t51dG9W!_<%mqza6u?_ z_ak!)<8~XQZ03ammnOMcd0joDED-qH*g8!dU34g3_vYl+JtuegV0VD^cPHXj zfx*{{4j3#Rcr_4sC%m_Ekt2GmS9{MlEh1QK_zffzID|!bgh{xB|4n#aW{Db4IDuaW z;M5T_xu+CYnD1h2hLOc>)Dwme^WSW^hkf{me<_8t5{SQzh>iG&kvNH!cr}E0iJf?f zoA`-u&VdL;immvHu{evTMhuJ!Ii+e+wRm3SqZeZpaN=V)j@XRDc&$38qiT3q#(0h8 zc#ele2)H<8NyKEYSdHnpkNx?kPW#s1R0ON_>hO?gAav~AvuyIIW1VxeM`H#wGxxRO;4lTYs? zvAC3Fd6#*)m(4CzYB`f}*_V5nmM>YD!DS3UZkd_+5D+1f{{>~M1SJEAxtguniF@x? zzzdeEIf4*zaQT?z?2?GXd5@>DpLU|08^oL4@S5E@RVuhpAkBt2bTv30rZ^KN)@_S&{%(U7^WuLh%5Svvw#-3lNi`_ zrQWzfcrl_$y0IAAp>H;fGrFW%x}_Jop*cFGUAm7?I;Pk5DLi7Jr3#v#v)^X=r-3>x z;%b6pnVWmqsDpY)f`BxLT5OOysiC?chB~ULx~dg=s;xSVvD!;6qhA5}ptCxx#d@sC zx~$FmtkF8H)q1Vjx~kn4gxfl<<$A8^x~}c|uJJmr|MhyWBO0vv`mX^yumyXt3A?Zj z`>+xFtf?5T6T7h;`>`QAvL$=6DZ8>QyEJ-QP#*cRIlHr!vA{e#v_*TgNxQU7d+hxB zv{f6D8Cnw}s+ygE@fK^fX?wK+pb%WUwsqTwZ#%cM_&zlOop&2qw92Om;wGDFC314O znY)MoBBUePX`1`y0KgD9T0i-dZI{csxzkCr+uwerpi8>CuiLx(Q?BcIl1ac3Aj^g2 zn7q4tKhJx*%NN3SXuaFJCL+i?#Jeo!M3VQLpjkS#d*xPyLb;pzo~1xz5o*Dg*ulS= z!Pi)qH@PYDm%h(t`~Fca+p}$LPQxM68(GFX|Exj6cL>6d7{o=g!<7-mW!b{rd3HdS zg72UZ4xtd{n}}5$AXyx>O}vj?Tz`Wcjw7R`WW2veygN!9IqJj03^~cUW64RHz+C`? z+$P?}ng=pr7p&pJl~K#N+{dXs6J|yhUU++FY?il-f?DQOo`P|MUZUcv{(05t7uNhy2iC;Ji2wSohv^!Q0-OpR_ z%eTDDiJ8$Y7u1o((TNGt+hffG{h}SpgAyEw0o~&SeZU8$P+T*xmLV$K#)6~zX6kV4 zcpZsSLSbj;6YLcN|-m96wH{_2510^=NrSaX@3I2$`{))%`kc%C4%gX5= zVg&|HD@ZpCouA<%Q*Jsen^yM@eSkkcjEU8qv~5;_<@-Cj~_8+o)N2hdP-^H zjRyFC3-znn`j_9?e%AR#*!ZQMU3S^~`&$+;1DvQ7m7h$MFA1E z=sq1f{u#AI@#_#Iu3-U!*)m|dD)~5w&<77gtcoogs0$M!00|KSU^tMMC5jb0daRgX zQZZ&EOPXY6tw~1!dQ|O#|0s#$$}yp6Efd&Fm#8`_4~86h@?^@7M2i|d+VN*lp*;_> zthvk~uvCbeR@?yKkS12ZMnW{$abd%U1SQH$$kHXuvp$qAEjbkF$EP->QnhMP4^+EV zKVsF&)hj`;D-UJrC6$+`z6Z0A)K$|VuT*(H4jGw@tOm6eNv6_n6xOTN^k zW+2EClrj~bz-2OLJV~Zp7Y3CfS~H?Y+Jb!HbtY2?tymGA3AGp)j8uLIWQRi*=^l>o zO?XtCCdH*@pB~O+q@YL|3KyZ9_4wm?sR*eWYMHWz(3V}wMWCcaI?@$EmIOl5D=(DE z>QQ*U_?3)`4tY?hN0~?ChkJ==AfPwq7+<0sF50SGQb-}ju~KA^kOz|tW5q$v4l_lu zLZJEiW_0M7Ofj#NA@n2@4pCV8|{+-Fzj%+@`3!V$O-Yl z@JSN8hV4??9#jUpCwYdKv%NuA8p;9#aIkM>lHqME6x8XhstDb-6wMXIyj(?2j3in@ zNVD4taT`$FZJH;HLwTbwd88J7tFzi)MYC)Opg zt#QB^9vtw%0011Jy9g<#453%6-LTmMRqTR_l{z-yNcva%^p{Yo?RVjM1vcT~g~lypFHFXz_lM@9J6avgNL4 z@4oE)de8Bbx6hnAqKQv{<})5#pm(^DB(5kcXxUOyM7?4eFevI{UrDr=5X4E!f+WG< z$GG>feeDl!7|UM-JEAOQb>suOK?n;a$ceq_!W5u@O6vf?31WSV^3bicpLTZ95u6Aqs&lVjSX&V=Tmoeg_mC z|5CycUdhb+)|LuSP=XR$;Z#Dr2u6~Kk&ItF;|ZUH$1s+$jAz858hhA~9tClTMLc2> zm)JyvY~YC`iDC_@Xh(#IO^s|6h#L_SNIed+k3HeR98ZZ#RAM53hTI_{e;CA8^3aEh ztOz4XVnie=k%<-IK@8V6!y4LTWk`$SC_+KRhQ;D6Lm;BUZY7e;yZ|&eL!eRSa1)_b z1e=*eqd;b;#ANdAXe3$L(|Qw$)WOUv;6&!y+y#I)TrrDV?1`tQ_mMfsO<%WJMmxpn zIP5UZpQZ@sJLMzKO473=T=XVE4cfSlY;uR4T*fDNR5v)vk&dkBCXo`_n}w33|Dl1r zNixP*#zLGWl+4&;Aqd%&lFALGUsS1OMpB^ifE0^bWGG%_`N~>?@{5AxqZvi@$D_#f zq%W;0TPC{3HnIh%v}6Vl??_Zp8g)e6)Mi4uYRhCYBbSflCGyfpQK`PMVR4*e9U)ku zsX61A9sSl>GRoGCzI8X1JjEz#2o%pCp-i_Nt0Q+ANgnmauYg@!D+YUz73hJC*urON zo)E*RC6!f6rRf)+iq@!la;iCWWmT^_)I7Fylt)EKSI2h{uqLFiWJT=KLR-;`63K4R zXe&oON+FiI6sBj@V_HYM*52y1qi_{2YyF5Fxze>H!#K-px0y|ZNS3lB|FLXQY&91M z5yKQUF^Ty$`y|>P7LvC$u4pYfB=CL&s<|cYAx~>RyeikN%#|o^)rww(C`$;(3b3^7 z<_t@u0RyRf0j55pPTu%zwwQ6lQO@}`-K^%k82;P2pz$0s6`@umdER%4=!{A<#g{i( z8JPCj5gkZj#0a6Wa!hRCR@u13);^qz#d7Zdvp@bo!PQ|V)(PBnO|A!^{;Sl?P12Md) z6l4OgO2-TulYVnD79H0^x7pOFzO=zIjWbCjn$;OZagz_i;-;lHOeXY!jA?ujS5FSs zQ6{m#uuv2!E4zQWTO|DUi7Jgej;U4#Z%I#!)r(4TjuJJ*f-EOa;X3&L} zH>%0a5(HP;x|8e&Qreg3~HSyL)W?be9Urf^ZejR2euE|j`p|zF@*&-u!JKa zatdKUp9Hrhm#jfNqL3LEYzbd1QioUdh3}a=TCz3lzOT*@{UD4Klcg1?9?N51?3E|t zlC!M2Hj`@Jp4HPAga;7+4-a7=v(Df~t3QDd=w5SApASaK*PCXc$&%7;K=IR2!%u zb~sxyID|!bVkj|sWO9TFk%X8rb#O>Na+rI1NN*rG7r&GyBZeYjM}{9kd_MSsK?sOf zCy9N?fOm*{MNxx+h!oBTgQPfvo0USzVqgSV|0ZQnPLUR7l-Cgv*K6ElK3h`^800B( zS9v(aiJW(Md8Px2&>J$Li`YRR#h8iFMvS&oi$aErwuBkGcoV&Nj4`H+&Y&63SSFY_ zU)6XNw749$h*NncLTN`72$&FbKoayvBwfdRkMBL~50UdU+blaPk^ga|2-z6gw5 zqHPfQRVUev2=|TN<`Kd)< zjX|B)nE6vT&&QptJQ7(r`>d^mJ7*VmN}xil1b zK1K;~>v)vzs58RIbg3AImDEs7$&)*|diUpz%Q%%t*@a5!ZdzH8>gbh30hWOYj$lTX ze725zMES1Zf(mGaw}vltgGCOD1t!NsfGpnb*jJmFbwI zA#F1Wm2D=L6Ni*E`IWbM5XpF%0mgSyU|_+RlX3w7VgLYskepOWiU`CTn|XY(FUo|*claau5kcqd0H|2qT6s?j5*eZ~y*_$8!je6qgfNw|SjMzN?@ zTAz=KrEIqgl1i(UYNijGs(E^$0m`X|V|qqmqoK-`A37<}da7tTt-B+uG8uXl`m78Z zt*9EI=ZPP1x}NAtFs>>|usWKcIwy@oj)-%bIpq<&(Wiu_uXLlV$!eu-%8uUJ9Hx4y z@es!$BiuoRxzm2~)yJ(8#4 zV?OA33^wF|=&1!G|0E`hvO%G;Tidi=`?MwIlWRG$^VNM4MS(h#3?q9^pX7Y2NhI>< z40({JM2jzFcU^HPHByR&EAv04_^*CTf#|5R`KoKq0eAo)PSAk^oq?o`cDR4r5qNt+ zWk;_ELAhJnXdba)l&U_PYqbI6tCpLz0ob$5@Ux_0xRd(1k5~)=Tbfz>fwWt@3k0Mk z>A5v)5lq{)CvmuUcc2Ahx|LSBNNc>H7`s@Rv!sg_zU#DDyP02=O*-gNP+PXm8zDQJ zx~p3(Vk)pZ$Fn5ivp`#1!TV-anO1&k~DYO*g1Vj{mPg1;cn-KYHv?R8? zEz5hj3bwy%{}RDFnfX+`+J}v-r<(zsfZc1pHfx#ye6{-r!3cD-1p~gTAu)T~h&NRX z^Bak%DYdrXxk1A!{QD6qoQv#>v%Gh!vx~cG>%hIswb5(AIA`SDE|~I?%=F z00&Ac~YydxtjqzK2)U#uCK%ei_d7lUj~20f-Sydw!Eh}c2S zmut?u{1NG##rez;?fk|Q3eSf*&)8KywiqV&Y{zH8%wC+H=4uoejiAVkXal{{=#0gz z0?-F7&z)S)SR2s~tIg2t5j*WM%B&zny~HNH&fyq}866?)49FA|V>12BRhQHTHk<|) z|8cDtJ>@DSLc&dgTw;>ax+9n(W4+BbE3dihjtSA4SYZg`@|b2PyJb?W2vOJAN`&pn z8}QoI0Pxid%!M=@yp9*pTqf68X-cW2*gWJud##>*{nsqV*IgaikSQTGi5Xgrw!M7I z^$3>;0ft8TW>!Xpb-gZJ8NQBa(Db@y6jNOht3Ix3*3i5vCBP-PU^A}}Gnuj5f4~u9 zYS4AP+J&pTXWiMLTi97v5qB*~7XjC5R@^}C(c*l~bN#T^ecc8!+f+TITus?3my{92 z+{}%r)UDPC?E{hK*nRES;QgybaoS-wot7QgZMoKP=MirG+?=`AbC9+y;@npR|62Y1 z-*I`_6IbA0mEbx_-oh(4M@%8(O(B*S&kb$ZlKt4@>wp|UTwpVUc+9kJ8WqXU=+ z8j#B>F1_R+NpcqPb#|%7HSL;50q6SDE?_R^Ps=4`{*ZHu=G&;CkXRafeh^hw!x`1* zyYc6{^W~7v*<@a!TFwkye#q|VszQK%VP~Shp#$>Bdu)6Zz?_&l8ryD}|9?erG0-K7XZBy)s{zt$D0CoOl(nB0= zK7Twd7m7a3*?O8Htn5fm$!@+|lP)!pp6!d7=S|tu0E?XE`3io{*#}Vp&eWc$ey`mc z=oPB&6yogAPLl4J0lms>blw?qchi95tLkCtyZY`1f7rQ-hv#nXTP^3t%I@wSyzy@C z#?6|sDd_n==T5=zZ;t0I*V9C??*k8`caF_AEv5f{>xjF4Bi-;L%ja?M~^zyKMsRXVKpCWO&s9<^i1G;dogLcF7rYOy3pC|C@*_hn;7UF?0=G zj_{_3m0C|GDxivQ$4G#w^G9*^ScyYX52I5L*0TAPkxA8Ae|uifm|(wq>pV1VFZG=v zF@FE{fDhOe+VoF9l{3zgejQO2VcT*eGEeYDXj%xAUkKo52{!f5H{lC-aT=2x?9n}H zxUi8!z$LI=(9x|}`o~CiS(ia(>((GoDFK|VAQF~E>_ormhs%nlvyMo~1JPyw0x%>GR>th(I`xz%mof=*=AW1Pj32-a}AOLK8A{MUzx)&j&J= zb5S_^SQ;}xwJMd%$B2lu%B4zC1yL(3S{tq<|5snMEtyy)Voy)CzAC6ZMHOXKG5j7S zlchkz1Qwx7Hyw;MggR|8qFIYGmQ>m@tZYt4!Nb-}Z3W_1y;>QZqsXH)MO4~Pb~z?5t0-+T^-D<~n488>+aY{}*?>GDe)CvgIfMWP&D=+~|Pjnx9%Lys%sa zI+r=Y%I%R#fxr~#30anb#+LmwACz;H3VK4aPk-iOwSe|@lm>}*Nuw^^?`q0VS}i33Sr8G%6n zNc??u>W44hT)chLT41-GjvMOA>ipJ+$ql1&KDbB7?Cl34SWm)VHgo4dXH$% zOBM8H!#(>UB#YFmpt!h59WO=%j0dC)2Z3lk!ALP7RGi3xUZKPEO@w^FJD)=GC=i#e zA`Np?iUWx_87sDAjBL|m`TDp+F@`N|{=3iIuqMAIim;0$1fL|8r^Q1eBsNkIgA_t3 zE?I;P8oV$CC_wmsD|1FFFmVV<7=jhLgW&!IvVqIVPMEfF;0A`*5L1L=Wuv=a z8)Zj~Rw~m&7~2RlmcdF_!cvw9DOoKkfy+=Z1XD45W;A2@p;K0=oUT0QEGK45T8xx)#jcx$N-Or` zRj`V6oV|l;C}<%&|6;jtqO=kwP;aZTD{@M9x`?V}pHf%NYC;vNEa71}v(}Sf)izqq z=r;ZOScnEzvYn%>IW0RV%(hgsFlE`+cB|N|GFG&JByDU@do*Vb)Vpw%t~J$b-QRiB zv}4UkTjg6&xb9B1<6Wj}v&K@Fl2@iq8E$a{JjdzURh#rZrZY}4ii~pBxnt?&XVV*6 z{6>VX5;kmosoPk0CN#T|jb3~AJ6jXqH=y_%C2S5$0xjjUA(0cv7{saLo<8tADV2yF zevDSd-EnQl6lPSKNf{jLm^29SF_0lQSSb@Z%K4n~hPkrkVRpI7Kn}8HhWrvE4-*q*^%Zg3BZ6Aip$TztbA(Q? zD`a+tBq$o~xaRgG)n>-EeZcT=38dS<{3|I|g$6KI;lo<9A*jT|y4KhB1Vg}5DOOK~!C9hrzWW{UJbww~6({*0 zcAXYq2S(V(PA!X=JtAmVJKNpfCAbHE@K?|Jw=548%)5r`UbklF#cr|LZ@%`PCwu6* zB6X^BzVJtZv}A)I1VFGtDZofY86OXwYvjRZa!DC23IcXfxczpfMxE*AW>8W0Zo52~ z|NGVLj`xU|bS80H!F!b9)JmtE(594$Z z!@rC$qRQL5%lkWv=?sN)3FNcBRZ4})ONaxcyYtJ4+~YjnGl|#hI|!t_mSDig6TJQ- zyw;P558MD^K%Kw6+Y>6-}_d-OIk-^AfJmyVd(Ym=VAgWIV@{y*p5@73{o< z*??LT!o_30A{?|;&;wQg2I9fM=~Iic<1J>G!YaH%M94ynu>%X_1q?)-G0-Jo5DMn| z8xw?y?4c+hOhO<`h!+IA7|e(fj6EB~LG07Mt>M82{J|Cs!o(XwC_K0xoIw)IK$uZM z|NkpQ)^S1tj6ywRL_VAhAvnSTR6+n?!i=cC>+3unj4m=13Nti34+MlV#J~$t0U-#! z90bHSd<)+jL^_njiKswSbVdG)L{+py%>%(aoWw(X!UB{*LJY=3sxc|3f?8yW8bOGI zd4U)BfSr2@0C1dTa1fIS0x?Vv!N7pun+U&{iflZIag4@t6bN$sMjwceYP`mDY#eCl z#&AT4dQ6CQER$&zh;aPJc$5fh#71^Zh#=U4bjY!gNQiU| zg=~p)6o`#P$b7WM%IL?1yvB@NNRL!UfP_eagh-OKNPYasjM&GPj7Ws2$O56q+5dpK zj`RVUdP|Ur=LVelXyq6PfXWRk zwnYO>!YoWL6U<*TOvPMG#%xT-d`!rUOv#)~%ETY~+px>5OwHU(&g@Lj{7le<8fOem zmJk9lh#Mi$k~Set){I8eL`~Hcz1Eyf+N@36yiMH9P2D`D@bj|X+)dyNPT?F*;@r&G za2Qj{O(!58yYd7i^tIxQPRdx$&I1MKB(K4gPVL-I?(9zQ{7&%f9i|kyL;omG^E^-V zJOo6bjHL=s_XN)|NKg4R&-Hvy`wY#|%+03&G896;`{d63)Up2jPXaAa13gd#P0;lC zO@!EhM|n44dcO_QKLxGO+H_ErgHQ>zP!5I3Q7R?fV zjZ(_%BPp%Y*`!h{&C-^5y5!VSFa1(54O1~4Q!*`6Gd)u@b{Xz9dvcO;kl)R7P!7NB@0PNVP~kjZ{jl zR7<^7OwCkH-BdB1RLKN@LePXt;8apARZ~4xR83V?J=HaZ4TvG4OBtpOJqZ$Ugk>0p zK{5#jT~%J~RbTy8U=3Dbh0zb~QvBJDVX_8|V<6d?)hm&XVU1R4omOhCR%^Xh&GgiW z0RTNPPmy8m%0U}Xcgci@C00_3jlVpsVKG>6Vc3c7SdaZ! zkPTUp1x;-|B!bmgb*rk8U0IfGS(klTn02~Vjn|QQvRIAKlmGY)UvNOibH14US)dJC zp&eR@EmW!2RTl^tjhB%+O4shwJ?t=eBLR+=r0CtCqMD1<^dgg^+9ob4?P zV<@_@TC`1DwOw1b?Ng)0AEZ^Jnq^vMn4w={1J#LzWo=u&?OVV7Tfps5cy+FfJ&f!b zTfxGBjmWsbZCuBFT*!@F>1@?!B;4%LSVqawiI6k~c@KwB0m%(r(H&jVEnUX+Q~<~X zRp13xI0e=v*N#wK)^%M~fZbf1Te$^S#FZ=zg@#fvUEmE~;T>M$4HiSSv4q&8VG1#X zP+nnT-nc4Q!v&eSpwPeIo#M@2?cHAPEnU>bk>{PU&i~CP=#3<;HC)}rkTT;gj_qFe zeP8&E->CIeltqa8jb6;f-OaULgqQ>Vm0ti3U;!TBiKX0E9axKaiOv|dR*Q%Xu+iTo zU4dQw$4S|N$ zA}?q#FeTn%F79G47Tk*U%fJNRfJ@An132jHoTi-^miuBieq%U}W8q9;{Ncq5bi`CS z#XCNVO+>=w(}g9JV?iEdLM~*_JWk|AIJS(+lmAFdLw;mPj$}zDOgf%qOTJ`G&SdVy zU`_61PyS?3w#`ZoWl}C>Q$FQ`%wbesWmaxwR}Ll071&pvWm>LfTi#M5y=7hAWnOmW z<2BY^4rXB<=1Cr9Vm@YMPUagnW@T<>XMSb@HefnUf+X-VpM_>@&Sq`SO-|m;ATWY! zR*!EE=gI(QE`8qG=g;o03#rR zkuGT?sAlR2Xq1tHB&{Qs1^|~%4wI^ux9Cs41%yu6tV_u^~h)Lee9y}^UK!VxWfNDyPU&oElLhx+w>Fl`XYSRYTFqVRgoy@ZiYnJYU z3ju2>I7(!R>Df+?whpC{7HwYUY+C=RTzU zG$aN&9}Lh9iB1pXR_=x7%_lnm!ietl#<|{Vh0-<&iq1J_ zfbURr?|`%K)821KWo93BZLr2klqu+)R*wN+Ynnc1*dAz}FvIpFoqK z*Y2(i40lL&NkjEbtf@bFj$Jon_&YdWR^3C9x zU~$$#ScOd#9RTwxC!(eH@b|!#G6^3qr%X0~iS1Y%+IERIxD~BVk1YQeRe5Pl6pNXa^_E!Y$4Tf5Q^*&d5kd?A~)Mf|J(F^^5=} zZk9=zKPHfq-(bE@=6_XLWb?h#>1NF!xCZ%(x}aEZ=gBl?evc zbx&al67V?@M|dY!%mt<>w77G9$qO@{O!J<=rlxL{itvul9AL+oNAY-XKlwQIZ^8_C z`R0OnXK5g>cK|PFF#kXtmk;ZhN1t-Xd4#a{nC1d6sQD*%ZP>naguH?w=l97_p+nGw zDjY>!7}bu@1I6n+Nl3R^k1ne3KdWaRZY9x`Vm?H8G5{C@61ApY&;?PbgJ$B^s@H|9 zH}Euf18A=!`wMe~=wrnLJG}RS40up7Tg{#pTMx7YRk$s``}07c1!y3KO1O`~|LZy; zV`HnMtp~ff4@E~HQYU^zYASjDX_x? zJZ6x~?|KIYd>~s*2xAU|E&H=yhO}RM*|d%(qKwE7Zh_du6kP^XG=A5}pi)4Qk`a-* ze{SEk{x~E@>i_3{aJ2#fH+vODd$nhKluv&z1?eDX`A~!cb!TZKPn97@nJu8kDF^HM zPnGz0ZGiA05CFk~1`P^K2%yWsgI)k64AO8S7>EfkV$3Lzhb(3uKZ3-SaZE>$9+TzK zC~!zIX3CnVOi0q?$&?rYl>7+I8*h!=2T;DUlVi`WeLMH=-oJwn zFMd4v^5)N@Pp^JG`}XeN!;de2KK=UVrAQHiUrH7G;}22+K!8*bL7)^W_#hEMMkv7$ zK~e;v5fNMv#LyK7b-|E83If2Og8)FN5kU_TbRhsH5_Hjo2AN1AK~yA^1Q!$@v;~a- zfT7StSIk&pirbwKmq)yqGRh{KgaS+?7-i)VF{4o8NK4B!rV>l31*cMz%sd(8lvP&7 zNtc)y!U|QsoTU(K(4^uCC6s8w3SwAM)uoioQ0drqdbM_2UT=KlOe&$|)Jm2<_{NMg zr~ixsN+?v#RhLYK6k-^chfs=XA&3OGf*zS#s!U8}@KgvOLOLUgA#oMwgQuW+Bx+u4 zWSS` zhlH9`lgsF03X(!v)g89BWwgRx*D#me#C|4ZF*6tobV5rM6LxXN)dj3CoN}J)pUW@9 z9J9H!qXWC*ly#*(TdFzFo-ehFM)1B3EF)w%903S=O4EC`9f+5fLKh)YQR zgl57MqA-OjTpb0T1p(VldKjUfL`@1x zIY^d>W)T3~sBA)H2t#zx5DZktN7}pPE(ByQpuFgmueBVJUB z2LP!jw;(2PYGmUcF)~Y~gk^SHOo$!3b0tp7(vzbS9Y#p77tn1-bN{dN08{#;Mo1j- zldfw5H^KMK2-(pJmhG*gE1oF_f&Y0rD&GoSk0rwoB+!xlPmA`iJ*)tE-3 zF5CnmNGzyEc3_Y$RAiY=)6?0EP@^ksVGAcZC;*yZ(Sj^4iv$TGkTR&0F2+(II!J_~ zw0F1v8IF-D^}$M8`WRA8PI_x|jY?38KbAUUnYgi>Q?S@mt?)FH9Vr}61GyRRt;BU- zY#k$GB0Fb%5QFG57N73eJNPAMGgi@)Pcc%{pb7*BWHcZ!(-#xgB+EO#n<{pi3c>)YP~S7!cH zq6=|o1R@-lxJ3kMWfRggFdT{qK(vBU6=K9RHpD?QooI9wD&2<6C~7H!T7^>jT?eU+ zj6y{ePwwD{I{?7FcPNAfBhg%~{!Sx_8isu1`(FA6%)UPn0|5L005LRGlmT0XQ1{zk z`2rXk0#3CZFf&UR4z{ILoDa#uMT?Mkr!08yi z5cV;NU7}=#R2VA1TJU;R;@Z}fSvWM-5;d9hIlWMVXj&r{74JeU zD$=8cZKN>4q0!9dX%$^sWALjoR#5Yi5gXH9kHx92acwqL6yq5}q3lIS0Go2@!V=o! zHEMjV9m`=(#fQAF`R1pqmJk~wi^RnaBa0iB6vNLdksUCydL>pWwX}BAint&1nPf~y zSGM-Zh;f?j*zE+f7=de=zYW$U0sz>;rpdkAIR9&<;M5oQZEN6|eHdqJwcU9MGgOT2 z5o9YnIUJ}>zaIl`iRbvu7cu0AwL@3{5JD6_`6gtgN}ZIie8aS>qwB1Pa+Uu%U^5qp z#qq?9WOp3WLLWNOi*EFzBfXwLtKkh5ifQIb=tjOW$p6UuAZqLALj@VAbGvrxC$hRB zO)v;_JB?l-aa<|qvvn*6JI-;>8!=BT9cS>z1~cj8(6Nhq+q=`pq?|910{ILsMC*gB z>`vMxPe=t^?(r121sYg2)|JwCX0Jw7-C}0d&M~rf)bu^@8tLm+lX7NQil_0Zp><{G z&KKT=Fs^M0nBa-**3TcN?-x-$m6x~|&;Q~^2Y26!q5FMi*>h*jD#mtmCyVvAMcD_^ zrTQq)T{5gEeJ~fJ`Xn5F-h-a>^P@lg>R&(m+n=1_y3mQqu{r=k&9#EW zGS=2*s8ST9^)3X6KycUBLBw|v#FsgjK}f;Wjg;&F$=l*(LZqiIxP+A4JHx8B2^Q zgAOzo8YzV}SsMNp!+G}A~IqlI^rYF6VT{K)4kebq>YFG5d&6Ys%@fDIMgNT-$9&(K~RX8 z>7Nli1dXUoN92`$L7tlZn=5J_8E%fAWI?&Opg;uA^XN|DEDz?9$KS;Sko}JiY#6RJ zNg$mD#PJ^Moly;H#8qV#54=eoz=RJzqZB4Z1}su5D$gqpW1K}^>Ae+WB%(?*<7?C+ zM%>~SJ|HV1AfjLxU-=c5fg@C;qf!W3R^i4jLJw4(QAKLVev# zsU)hgMG)BE{vDtI-pHYl9Yh3F|2@OBVUDH*AV9|XD9Gbn@M?Vd>`h2})V zQEFvUEhVc+jqNodpO__hV9Q@FMfh=~JVxcUz)w-y*6an-UD{<&m1R60-sUu6Oibm5 zC1mZ)Ph}PRDEwdEMP(mI-r{*Xbrc;iJjDxsp<_^iToke*wm zkvWP-tYRCjcGN&fDmzjtV*aU86oKZX21F2qCI|_-W!|wqs`f}Lh;}OMP+wtchNb?* zhS85k=%IRw$Ti`rqH@n1hGpf<*%JZU6wuOSenbZ58Nl*{z^aMsm{Pz74-dfWO~~t( zQf$Rq?El4LEDP<&q*0=$irJfbLD@`{CeA5AHAKjAV%T+^b`odU5yTRN(+(V`(TJy? z3T7H@$uba$qo~3z%%^2toGySu&z6xw>DxW&y{V_P9`)c0|QOdJZQ#h@Ah(U_j+%5bm>Xz zpC!I2*x}#TlcEn!4jfP2pj7lUaS8!zhg2evHL|~Z2 zy*V6LXb%8?gaA|6<$SFVV8CZo#Vf#0q6h^vn8m?QFke8N0(%4l)6^RXiBLG}a6qeg^jrrI@L?3({o;ll1O_y?YQDA$0l%=jZm>`Q zWMcs&x*P@x8ky~m#|i7OhXqook{fh8`>SWShoF058hwx#uHgoeuymBn#2`$?)S!Lm4P=F5H0U=NVCK%Xdby(uCf&VJAvN*T% zHM=T6ON}CV#LXs7F7I+d`*J*@3?i8a0C+Pgo3gbeW5^w0b_78vlgly?L&hbAsqTU^ zv_k*U^fK5qPHV(W0|NBiwD#zLNDDJb|1upr^;1K2R9k8IO7+gXzzcjp?ka^=e_vPQ z24_Y{INeup1P)i9V0aK!s0v5m1Oe7fmUl>xsBVu&e1J1w#0OL}7^*~GCr1++m03IC zk)20e4^4tZ#0g?GV1pPmECU;Z=~^d*Ts!F@EfZPmHEuLX<%)-5n}=eXHDKqiWqWm1 zlXhvF_Gu@bB;GM&3L@DmGWCYC21442dUcYsG1I6w{~y)_Wy4KcW~E-r0s`E z3iollk^_?m7V{>ugx_Nx~&s zZi3=wK^PbtaIt2ikQ1!3(0YOv;v@F#|8ibB1p=L8~KqVd6E;IiYs|kb9R%5 zhyFY{lv8IRLmitg{CI)Osc8CIC2sSJ=~=<5P4V*GD}!hv(3g%Mb}T!ZP&4ZjuhLrr#aD zP=YBM;$9-?jY|2x=Z`LB}yL-g=NZQVclcJwMdw|_$^Qw>>jdrS21vw@ec=3K zsQfUnyn6J!S~R=Q41LkZ1nH4$&n&&*-bI;dyks3x%*$b}+tbm1bke^E06f9L6ha+@ z)PKxZA>=c5I6Y$hz#U);*<(GGUcGv>{o9v)Z*9Pq_yQ)c5VF@3#Tx|I;fb`1`jAcFMiy&2RHTw0HC1cLw-GJKILm0r^8UgV+XJ2`10Bfs~|DV zh`yOG50^lReK&4{K0f8QewV;L#qhe}-pR(dNB`$*$Ett59_ao(+7s~O#`)Cn&KUnd z2!G-3zL?oqYJ8cuWA+TvQ|>!I@rTD|5WM=}H%177>jlaxocpod2}tNd_U;`y_6-O3 zMJx21<8UPPqX!STzoNi@O2e|ao4(Wo1TG%~2^K6{tkYlTE& zMvWUecJx?KN=PYUq)OL`4T{vE&x>4RN2zYOPf1+_KZkIp|2twhaem_ zrDIWMM}6*8%CzauEX$ZVYpL-m(x_XTW<|+?Q`J8&xR(8xC80o+PbMv7W{svs0J0J~ zgEYddPqfU`=Jop*aA3hGHH2km%gRFlP5&NxyZpJX0bevR$2Nq=B1i50!=71<6c`qI_~Z3dUWWhPh=tdy>V{cy9JZGRXFSn zPvXw&M*5g~b!#60c2Qp%S1#SE%O^(WU6#A|Vc>hRbqhBmapM}xcxMLfDo=`4D-8wi zS0s`C`;R4^Q}e1|=GzJ8S&gh{Lx&{4YcgDdd4J zXd*c;qY_7?uq+o*asnY`G=x#5gAQY{BLD(H=ssRjp+u4!3F3hh)x^RvA{}>Q63Qs0 z^ymb-l5r!c4que0N@lL~NGW9)8~;$olW1hdM&QC6b4sqxOcOQLUeT^ig5EUJhc!8} z5JNnD(DAs8oaC`ipY+^ngEG1^uOlRtG_9Zw=17PbNk{^~k&0f~2>?!tfo8oTbO4|& zFR|qpUlY7y8d_wP_tM~$W>TnCG@5r84{An8xvi0$J*{&<&b8b zJp=$1e0n3HWYQ>9r2vH}30tlTY>C@}h~NUJD=-)e!j`CbC|v;7RV-a2^2-Q-c;_wX z+n1;SK!+`vv)508^p&^Ul!V(bvk4C#P14%}&NJbHYg%}tNtt=&VTpb4%_LG0awbEM zR%i>F?jUWGV~tf_*)OH6VgD1Wh(Sc?WiX9`*)@tavKV7EZ)UkBo--Z@C}>obSm+ji z)+;TKQU==KrJH(!6}JM7Xyp4$zO^78PLV`23^K4d87yo%6RM#+C^Z(;?mPNVUatkM zyJYgnDC?4!Ea>U2>RW2=fkL#KCy%QNStF{g-g)nW=-g>-gm8meqL8)?sS3(f$~Oso zUFzbBxL{)MB?bp-7l51;y!Z5%RM)AX9Y&9db%IcT-KC7=?PB$ra@V}E?v#-bMB#}i z(fA2Zs5@}1k_WzcG>`8wEYlvRJXen-rD~YFJpy1-Xew#znm;~hGkfUOUmv0np^AR^ zEN8PBO`L;!{-aJi^Z&Vi_K~YTHmFif+a#tx>nW{&J{eR;(&s+}im53)C>1k^;Rvc+ zO)=pcA2Z5#z6xQaAVnyFmAIyo@Rb50`|(AbRwJ;mG=xq(lL+|kv%wA;1R=AEivD2u z5EittME(3u=T8&`3rJ^U|f0u`fU&abQQjH50|)1Po(R0=a%CE|_?6cBumZ z7>-ATFp80k0^;+5Z!s*^eNqTxBZ(podWwqm!vX1TKBB0RS9A6RZFRLOQ97N<@SQhF}FS zn~4cT+^CnEY$hlRqDx53__BD$Blie6*_>VT@0GXgzS+ZeI&>q1F24E7_ycR(G8*Y^F)gN z@}j7umoz)c3&eqDKDoJ%LwxX2i((Xln&F*wyJM7-&pO%0@})R|;do%7W8No}O8DCtQR<4bj#6K7O?PfT6r&SNHXnX;TG zI)S=PKmR=?rmk$OP_gM%LV`7vqm<=QZJJ4K(lsgKq?aR>=}AijA|Y@S*jzuUp+9Ou zo6Y2^Vv4#~TE?}kFt7oyk$0k1p3d|mAVCUK zdNFpWvKdBPJ6636omL|hv4}#8!O(~J3pQOXCp-5kuq5`dAVq=33L%22!s_C%iS<)ULzMpYg=p^T3(@-6$G*8lr_?AYkttd3S74=rMk+i( z2}&fhg%f8zrEXvP=8}pb#5X?j^R1FMafJ4^hrRPJ%n4%f9uv|8k}syWJcaPW6rItTqx0r(kTOmooFV-{k0D}#J=`bTW(blB>IJ-xyiDlwB#j^}FZl*S zN0y=UJ}>lau5UCg60~CmYY+r!jX9{LE>a;B775lmEA=8`QjlyfX#ddjeo*vMq6eSw z1{FgHtAYrP5N4*(1{uQUbdQ2~&-a#r|55__?nC;HuWItm%sMa&6#~}!ulA@S8D1~; z3V}VckNLJQ0N0H5=+F*J4-cDdx9H^xo#71=A`V^2`@pXOL+g0-@c;NQ_a0&ZS8gC6 zkQpR!{X(Jr?r-<>@Ar%ZuIK~*#Bl#4tPv^U0LPEZgf9)L4tCz36dzq z6a;A@6pI#7aTGbNDo$Y(<_}0L0)$)%7K={}V^AV;QL_pWYe>)ZP%pR^!Wl+^{+@$9 za`0nz5ER2u6v?nbG?5c2aPyq73xRO+$k7%_5v#W1b`m zCwvEtT7pf)sEc4?DM>6MP{+g|g6|eXV4$+@Vq#x{l6D#@q8`cxiAKB1>o;_OY(x^J zHY+3i!)5TqKZ>9&orgcz1WAsFAR6-HbU+G5ge_^tPw)~V^wM8W;%~kXBKqj4Q0Eg|AeJT35iz&79}M4X`#=#mrcav%_M;pE2}O8?MtCW0{~Vlf*h6CINy676u( z00TVZ1@2{m2Er_>B{~S=Z&pE?fPoo=ssx40{eTP_Ov8g94J=`7EcuD+X!9&LZ&o@A z66mCFYSZU3O;g_Q%%Gtm2I9)VLkhrS$9V1_prI_AvnHm~JO?5&jV(L3lPoJFJ@dq5 zGSdP%GbI4?Ap}z(-BH%GQ{sF8F^9oy)-s$DlRyFVDF&1%$Sot*6FWb}JtN{YpXe>) z@*_&51e@+LgJ&-5lP{VL;VNT3HPb#HVnUe+KPzM}RkB0HY%@Q!MLHzgI*LAv2>__c=#C(5fWne82dFhLZBG9aDZ*Zqqu^OBCgLY!vC`(<Ed5GqN7$sW0#W=6^$7yf;yzs;^@K`v?3pM zQd?JpTMHITztji9l^6v=2eMP%!1aYB?hFG3JEF`353`8Ep<^{6LxPk;%)bnTutzAv(`%(7CA3QL-gljLWnXS z%3u#yZxfdwm>@Dnq+wvvPHMzB8UJ@?`Zjb6cV-^~19XJ{IHqPlMcl4-U5zwwjTB?= zb{2zlAY?XV^Y$)dH!Z1AP1p8FOcrG+5)xEaBF2V$Xcxfvwa>%@8g!u$a8xp0@^J}b zDD=oPh%2;khBLaMB!!nYiPv~f?sN$XJT?~bDr6UZfqTQkdu5eK(>M4;);UNv8)5e8 z>UXjPmw7|i-83gaNP<^;mBYG#3W~FIh_x%TvJ0-Li}CGb9)CVjEZXxC^0!wa3S2X=q&U7FVsscQk4ptEeT_J*o&7vT$N4J!h zBmM>#v}5O3l1l~>B4F@nT>m&QW7sPA_J%dW12)qxcvw)5VP<3ZL0@Hjz zz?h+EWl|xC0TWEm5f-Hk$HwEFODz5rB|v!2wynHK*n~F^7d${%WniQtylZSS5h#1?>-z#chyF2$BcFk(D?jVTy~mZ8buZD}<9}W0mg`V_&$JVVN&lBZomb zn3q_FC8CyZnL`~?YUx;ZmsFVvId1Ux#-8YyO`@5jb97_UWlT|NU(zmg5>WK`lOHgd zH+h*oIhXI(jwN}Tk^h#Rt$AEy@{xg;l9q6Yfb5OsLI_dtBY>DrOyB~&@tM)m;0n~} z?nW;$C7szA%lw%*os&=usCc)-Yw4$>(O8ZB#3jW{6HMV06mlWJ zY@lN~Bgl?(cQT3wcqIy$Up{1uD32*K3|q74THnqJfU%svO0U#4pffrQa6lR-|jgbe1HN{ewIvolrCC9OV^gO1tzv+S=??`kY<>=(JNixKuS( zUFivYkFc2o!9!fZ2m%DcqRD|qThI=t^JR9H0AN2SbeMy8Jh&<|c<(6B?Yf9LR>y)R zSi)e(DLc5TIRXZAJE8_C&kF)`@i&}`!5UP!*HsvmeY?kQd{s(?DzGbH!g(V|z}T4q z*_%RtpjoqiT`mAX6^h{*sDn*+E@SEAHh^WJ|7O_7I4!B2fsBKSTjJWY-8tA)-LYNU zdH;mlP1#5Z3h9V@tC#b}tOYX7H)I`>qVoV)I3wK9a@Io_8tXCFe`D1w7qkLp;c@f7 z7FR9M!ot(zX>mZvJ!m#BcWnv&$k0;Zm-BDBJaVg-#vvl&g}ukf-3LHE-X4?L6XMwS z_nJ+wb>AyL!cwL;Y#x{;S+!Pm! zhn?8LwA+7N6J`~IFc0_N!)@Y5CG1JtA~q{v(OLdjcQ;aAMXijgNx} z6Dq7wjM=a$3Kaqf)R39Qix+VzJRyr&G5`@3l3@l-W5kirUcERt5~E9uu|Otd=n!Ja zhys~8gCr9t#g`aO9-LSaC^Ml4IeP4Qb4bHvSZ+qlViD=VEDdc1IS8U-LI0c`ll{Tq z!%4A}D-r6Lq!ZEwoC94Zb90i}vRD)p&RNDxCBiaUw^qSTY4%vl0`$4A;o7{9B5xR7uzmJC8&deI* z>^XNbPN?)__Lk`_23H7vnD~(R<+mStzgc!%a)~h~QhpchC)_~t0a#T?d-au8fa&db zT1aI~n9zaBDJWHZJbBmOf7yYjjCHa_)S-I06-Clw<1wgTPxZ~`BL8fI^~REo%s8kW zbtv96+(pG@w_!wPq4gnKJH3UIZO&Bko^U@Em{1wdIEj!DY%#Oqk1*=RmtPVA;71{T z002M~9**YaL2I=oC2xgOk}rUSw~pLqYk+r}?5gRsTcF1q(ohR-Mq0HAQmh zqezs-@>iT8y{nzM21`}n8B$?I5_7>aJQ%_x!)h{p4@Vr4!ybYBr*2n58sESQX#gS4 za{vi(LM}_St;Q{N7A|TS>t>rn_kxS3P%rM*?NKY0&=D>ku=NUHA94MMOpYNf?moPWu2UyVhy!L)qE!ORr8$MSUfima*sFp9KTnkiLS8S+?F1vW>FBD~~JY zrypAf^Uv#^t&rT`4Su-g3B4sqUq-*S?dc1DL4G94FnA&2yxFK<-+S#yu;8)hRi-izoFO*1JpXi}Xi za%UJFBj4jhMLz%}2O7?Z;B~6#oi2Vcj3J~0NlFtbD%vC@0ANc>7?TCQ>_#bA%wBs6 zc#sOANn~RJl^Y|4I5^7j6?C+tVB$y;M8-yu^#AEd9<5`7fpjAek!l;Hgm6HOHBbj? zv=JFYwM3RQ(N9lATexgdgfVmk8k2m@*xVK;!UXa|bf8oz4nl%~_<|#0Ji#t@(le;6 zvSz0vWbt~$HjCU+Et5FQE@L!HNC6UsQZeIM(1<5xxNI%bjEK7aG)OD9hhPs2nKQo` zO=wJ$k#l^c+GcXaEUu1%k#vYXwfM|PfyWvl8(gH|>BzxAG9uapV+5}$szOFjOjaOE zLMCOQs|Ylc6GO-ba>Me>xI$kf@3?zD7iBBrpG%4J%#%mMUo_)SG8D>?L3gUPcO}$mNqS|dc+Xbo)%_%#--KRi`5mNGr zhGn%y?o)D0rq|t;x^9i_Y<-Y5l3W6FjO7G##c0vzhIhONGcRp{YaI`Ww=c@XsoECS z$cT8beLi?XAqsJa{q}di_OnDGNdH^fN};xq-^!@*AOo0^kYxz3Ox{DKw%WHgm#ZD& zCMcm{jW>Ky2Tg57G;f(oT8=bSt34)covIK{G2GH>T){My3yUc--QF{P?^Vc3X~L_OR`C7aHG{T$Pc_of_{H$*JoycHt=A zC}TyvH^H%auN$f>XE`K01_03zOyewf^5 zfQ@k`y7-7=djXo9Vvef9smXBGZY2n!?#{xYjgamhWmgqquMc_#3|T;<5>4=oPOiqx zsi*L7W zFES?4DR1*gfj;w6ak%KAxSFs}OE-kgI^hn&`XO(AU3gYy-);Z-*XMPDdqWZZnl9(~ zm3~C0w|e9+8Ij*FD)!BxJp;ppMfzzUA>8-e*%rnZu?t<~Mq_g5QR9=PRZ6P4fH2Kc!X+^gp9>K5_U3Z z<}k%z4XXB0L&Y>Ls0rL5KT7BjP52?OmJnljg;^+4zMwnwC5LJ#6oZ(C&ebr*(1vd) z5pm%RmM|ZA6MZIz6c3~r(*gi66$#3gORWYlGdK)2ICeKke5UwBPaq4+pcqA>Nf z7>|*bgLQ?FH5n2l36A1;LYlUK(C~@EVR-O?jjGkP%sma0Cc-=qa68 zl;rh9LeP?t;3vJr7mz54BVv^nftB@BFOi57d}u$mkcpcJXhW$c&*lSNSuY55hD#Wb z(Z+FMfq(xL*ecy71%X+Z^+s^Q(L#pRSqjsbzCuKM7%ckc10T36^0q9Exo%XjZ)y2`i83am65(rT_|{pbB-^k%p##Yr_n>U<#qI3rHtY%s>pMIh(babQZx2xk;M? z1OSd<7rqG!p`e73AcmY4S_6xEP3a}m_3 zn*+oiI*|&P;GBlQ3TFe4v$X*g@*CVDoE5PQ#d(~xIeY?tnhxt!@q3G3OOH@RJiRuRx}pU5dzz_=SuP=~~jo{mrvwg5ul6d|pG6{d-rs_8Ag z=5zn1h%c{XY^|wc=U5TWAe+0u7p$mc4EZ1MSucO^k`pGNGt-Nc1{n(@qcmy}H=3fV zw;ID~pSf9~VAT-I*__WAow>CEOEaUhiKHJIMdw6F;`y7qsf3tj9o(}Fx{#Xwd4@*1 zr86>jhRG@<#8)$0m>HN@u)-_} zu>&a>njUy88u&x~#*B#>nvhwvAUJTLX|$lSG(e$RmNS$-5SA_xWmOv{S8Hln3ltaT zN=9Ln?(-W%7ZN0(wHOgnb?UW%=^dblwPuU9{vt+T`x|%kmyu!zcY~xbWo-WkQMWH4 zwhD2!UQ1s0rJo4_xInQRiu)3G`+V6!xJIG2LaIr1h!L3hCjh{JTq`P2`w~*ClS+Cr zZLy-2s1O+Nxk&m75kr!DHX4w7cWw14z-Seziz$s1F|`{Jje8M-+feCIw$q5aDN#24 z(TEWtxmyclXK{_$k#%V+Mz{MC#ml>U>$u~iywR%=hwD~&`zO6?5x<)d&^xyZ5xy@* zUg(=qYx@$yixB8LzTKN0#CW-&t1$~KYBfIBz4ji_f%W-jWz^jssMqv#;C=r~?o0MV{Uje@lj1m94A*?`=3}2}b z^(&u#a*r5643f}#o2yW=l)y$ZvW1HfGAy1oY#rLWyn|uCT-q@l3=|%mHb4Okt)Ob_ zOMpXxlsAh_q{3IEg05BkfH`}#v9g(m$+W=&#+*rSjVXfurh&Qxny~@^0f!LeX2yu= zF0gV1MnDCkX@;x<0GdDx4Y3TspbMhVCcG%RMq&qkJfW$82=;UwCkhR~unLH9FOCdT zkvz!}aU8SB7r?*^qrk`?@&tyUntTDssSw20QOP86$wDZ~q-+8%nbjX{K@-t%d9+kr}Zbh z+{>^G%)%TI+e{0-9LT{u%q{aNF8~09aH4?h$)J3Ur>hV_AOwKG3Y?q@P8$++0%Lpn zDSYh5-8{&XXvpylcxNdyva2aPu%7KK3TYr6PtqPnwgLn3Odm(JK}ygg$IzYZ&=9?n zZKu!BT%*$L&a>>ou1wOVjLNgd(Y)Z%QellUXU(}RPJ$LG^qk1Nu*mbV(DvNX4}BBi zJkH*%J2Z{S%#_X21$mL8$s|3^?<~}Spwm44%+U49Hg!pq{k)!&?0I$#Hx@YK5SSRsMdY8}voTtCau3#lLqb#T)AoCoib*o$r3j?F+` z4a=4-5WgMV!~G}eyxZ;U*@;EZ_FUIaYo10yMtJSjoU99LDb%h_)Lfl^@3X~L+->ar z-tZmYjVV#_Ejyv&C-faR2$A2}fjR$?7BWjc2f^R}eGmbzDp_}hk%HgWadq_;5ijrp zA8_DCl1s}FpbL)9`wf!PMBxYF;2&N=K7fG`F5&+k&c!Eg;{9CT5wYSLKH_`&0{qS5 z-jNnT;Nl^4;x|#?KA___UgJ8BIs=|D5{}~#Zs7V2<0&-a4^G`1{+}HV4j=9G<*NeV87>uMA?EV@;+JFM6G-7~{#Q7z z;-aDh4W8wGj!?qk9bxVrMlL%OK0a^mSA{M@LBQqNXW%~W=X>fHXi4KEL@K62n3lfj zoDMJmumu6#{!Uv|j7Be(Qe%>$ooKx^9>l z4KTgF>%uG2T9`EwLDyr`6%s%hG@0HE=IWqB{Z^E}`45wG(;4=_MKv+OQHLm%`= zpY%$vaSFTWvL1n&zVyfb0c;#BZ!GojJ_b+B={!IP|EkxrOo2nyPgq~}W`FkaJ@#lH zDQaKEzs@RczxHxJ_x-*BfPe@{0Pg?yPWOD@_wrsqaE?Ma@ArgX_=cYygHQO0KlO*- z_>TYhmyYs~Klzkj`IdkAn4kHYzxkZs`JO+RTI~6vKl-F!`lf&SsGs_(zxu2n5tx1| zsnYtgKl`*_`?i1kxS#vFzx&{}_q_l6z#sg=Km5dB{KkL$5x-a7hWyOm{LcUU&>#KM zKmF9tZSGzD*q{B{zx~|b{oeomD-ZSHKmO!j{^o!F=%4=T-&4N7{_g+&@E`y3KmYVU z`!yT&^`HOxzyJK-{{Z1b;6Q=}4IV_8P~k#`4IMs&7*XOxiWMzh#F$azMvfglegqj( ziWDIurBn%-QsqjPEnU8Z8B_n}Oqw-q-o%+x=T4qIef|U*ROnEmMU5UsnpEjhrWZ|` zY#LSSRH{|2Ud5VK>sGE^y?zB7R_s`^WzF6+DQ4tZwr$!AbcNxe*OCg^UJ4C75ngm4?zA1 z)GxpR{VQ-k0sAxPKL`I8jIcokAuLG33lTK%K@KOxkRS~YWH3PzD>Tu=5<8?YLkmxI zF~t@|d{IRdSqyQY5p9eS#}7AL5yT&1q*2Bk1=?{&9cxq)NhXii0$R&-evdJi)40B2{yR`C4EVtCMM=sg)(n&DU6q8Ci&72d?H04~=%{H%m)6Y1^ z%u~=k3GEZmKh+FW(LwFhv(7Uk#WPVx7ey~q)`laF15TI$HPmF>K*-ZjMJ4qhQ%_}P z)KWiX)m2qtJylkNR;@MFTWLl0RaRTowIE!3o%L5+h2<60V+k5o*;9{*j@jPtyZ9Gm$f!pa>X?_Tpz|&m)&lieHYtrwau2>dc8#!+<4DjH{E^P z_4i(Q1(r8odgrwlVSJ(0mtcO~br|4=6&{%4f*C&8;Dj|^xZH_3t{CKtJYB9 zqnsM)o29;*>a4BqS?jL3{@LrW!LFI?oU#78YPGjsJ8ib#c02B|+nyWly5%mL?!4{Z z8}GBl?)z@R11J1%vv)q6-^A5j96!Ace;o116`!2(${pt$^2{aQobt}q9Xj;RI1R3e zh)jnF^@smdcZl_dTxSUOg=ANV_JnLl2={|@H;DIwd?yI_frJ-`cpr>M{&eM2Uw-xG zTYvub=wqLL_UdcD{`Ty1-+uS*d;k9T@Pi+J`0|TC|M>KiXTE*s-G?52>E)-Me(OcY z-!=~= z(vzlKWiEBOOI{k~jLfm6Clx_UBn7jRpzCEamAOo2)})CiSq>?T*~x8)(h9SfCIFx` z77{pO8N*0JGsQVhaz3$@zi~r0J86X?yi%RGyviIxr3~&6Gbd88XF2t`Pkw$Sh7v>$ zDUG=easU8z0N4fq3Tn(EOvD3#P{l5m0jg#2A{B_(W={YxiDH0COho~J9zsEjW%&O> zBRt5&F09eeo1j#sEbV7ZWjfQQoU=EaoF*w%hXi%z6Cxdu#jip|44(#t459J`Op)@0 zNwCTqMV-hHIy#Ig;6$p-pejwZx>c_J#HPH#X)t4XhTo}^A|Ut_G(pZi+sR@b@+`RZ=YkWC|avIzfqs09oF z@Q1K^5|k1_>ob-G0C_Sa78DrBQQd10`No1ELbwAVM!Jk*_*E2jfE|AIOVa-G;=iur z?XT#W5CQMEzz0qU4-CPIVk~1Bz|aLI5FuEHTs0XSo=65bD%4Y119l;e=!JC=Sp1A( zzsy)hFLVK6fruEcBsQ$;0N`Sh!dS*MzHX3(tju2`Czu&9T`-$~*$F@b059-?J3Wa3 zr8XqN$v7`T|UH~tlu zE8>Bs%GDzqpo5ojjMWg0A*jY$ZibplVv`O8(1ONsi#*HdAy?YcxqSa4a<<#aS+pkz zwXN+Af{+AT*0Pgv06>))a%JBRq`hNy^+exW5aiBxAZ6(2GIU{zO@yMb%z%cTZ=LI1 z_xjhtCddl(P>4btA`qZjFhXz*<6`?7RLCaCaRu!RQ-C5AteB{+Dbn1JT4W5&el|0R zaR_F!yAX(A-L%y`3Q9O4*B=(qqM?xrPq0-JtPr z5SCsw&IxI*Gb*7v9&lo!K^MpgHk)XkOF^k!@95VNqV`s?UFdF?c;5AH$n4riotJ26 z8aye=OL)@B)4azlk*V^{_5lL~S$e_9;N_afm-11cHrfS3g0#+c*v!zp4>%8O&l}{b z(LPA#3o?3`4}{M;_vz%_8}*2Y7T7{1sz#ctSpU3y>;*YNVUs~tfkYnl4cWb_A>YyC z@7?*&7i1y<`Wtm_;}Y2P!!EAXkM}5tO0t&0#8b(ZMX;R>#2eAq)Wg8H~0faATKN9 zglAy0-wU(K6NsY)|* zx+3==g+E9GDSN?eazTPpC_40k1QV(Ogq>RRK_8Gk`tyOKD+nf#!YRx|g22QhJP4^O z25aEIO&o|$L^~t&1^;8e1B{3_V>yF}Jbb%|{Y(D{BOEkPLBD|@fkm4rT_`Puphbn) zMT%kuUsS|CM8;%1q7!@#Iq;<1D#U0sCLCmjYp{euxC1~Cz?S<#7>h&$6hiw`#YzN~ z`xA(1Ge=Dn$4_LwC+w(ntUq%ELM8-4R)mNI>jrKFfX(`Y$FqnEs6vF0uUmAle9{3n zps-MpL%{+_tdhWlILL&&Ha%3vinPciDl*@Y#*H+lVInz$0>9pxIc7*IgSfNyOTQ~r zM?Wg2r;Zhnj{D;bTj}kghR_f z{jE zBpgD6Xvu;gN@j%&fG`%WNTLWDQvY$tanHUzml2azU^ZIZld9gIGDG3&<&)C|_`$dUQXW#7o(X z$8qexgCNW?q)l}+#}FJqE1W%wU@u@3wt*-Ck-UiDGpub=&flyE6%Z|GU_#}rK#73P zmjgr0)Xwe1pnn1mD*&zXN<`Akraw@=gSaxg8_x49hzbxs0-Qv+w8^*pyqBC$f^bF9 zaZ8tUJq#qt*Yq-bgb2MHJ|%=Us8s(5SL{I78!=x%P=vV6f>2P3h*0jd&yJcJI6&f*-1 zD7C*Stx_K7(g=AuFvZX{Wm5vu(AFq5H@JlEBhQ8U$Pz_P`6Eu+qr!>eHc$YCh*HUO zd`H{dO>{KWbv%fH+%;b-P(oEy-t@oY3O5oHP>67XUsJ$Cu(xuACxfs>V(2zU{XPJY zw~89oh&qkyqQ+OGOW}K#leE;>yw94{JRUX5%`-bzsZ4FXRy$zIYcs!Buu8-HwNpvb z2VIC^{FTZSxN$v|Yw*ci8?>rS2zT|?dc8>;!`ER2*nkz9KLid>Iv!=UrC}eA-WJl~4Q3&M2{ii}kCEf{>22Q3}w3{GtR**x317 zKZ*zfApit$!@5-1QjR!CL4^ijScMHUh=cUCi2?>*7zK>txhF7$6oWBfNCmB1h@CY* zp9NZh&;g}o+6#MHb`}5FuJzjHc+=HDsQXBRf~u+_*tCsgDV~a@gH^;2_yB}>fw_HB zp{QGp0DwSShM4P!Ah_F%h&q342qQ%uGW%P(9SFfiA0IH>uXWtV{f>;(NGpIc-5Q7@ zK&TfC5mG3qoTQAhiz|MG+|ec7N6NI{@IQr;9YVAvQc|_A?1RG<3?UE$*Aaq6GqKXu z-QDFoDOe9F$Rk%`h!k{#Ghipra00B;1yJw=O%T-3<=yC&UOo!jLK?_lA*#5Y-tFaH zHj+MF;(-C1D6NIs?ls@@9i!?cB&CvF^mX6&HK*8u-}$BA`uz>{wcq^J-~D|I$>rbw z1>gXtjI9*l0yh8P11=2wMc@Ty;08t{igVxymf#8YClsRK3&!9KUJ3=);12fS59Wws z1>q4U;SyE}f;Hh3R^b(v2o7f97lvU8_TLz$;Tj&{)3xCo)?o~$IN;sk9|mFvcHtl< z;vz2H?=|8iR$}@c;w5(CC%#Y=hTT^`D#qe0-oqK@4M;H*NhuUZITSAzV=%T9 zK53Lod6Y@{;z~(lG{zJ}d1E<_V=`9bF=pd3cH=tE<2yFvIksasUgI$K<2@eaGp6H1 z2IM{#WIQJ1M*d?ye&j@UWJC^RN{(bjmSjt2WK1^XN?znizT{EvWKtI8Qby%dR%K3J z zYR2Ym*5+=m8D<7&YW`+%US@LMnQ+eLaZcxQUgvW*=VMl9 zZjNVfW@mY>XL@$$c82GBp67n%=YQUrT0UV54vxn0kIOme&%qqO;T(qc9EY|Xg-&RQ zUg*Km9EzT3i*9I*erS%4XpB}Gi4JLyW*U($X_9stlTK-rzEPG&8;`E&!r5q68BHl@98b9_p9Y>74!>fwm$DCJy#_pQnx=sGc9Gt{#7dys~+sFF6^&9?6FSlvtI1AZfvDCAcBVM$);>&yyD8n?9A>e5!UR^ z_UucO;?EZC(RQuNChgNk?Ns98)MoA09w!`j?bw#>Wa{DIo$cGkZPk|S+~)1xh9o5R z?cf&f-hGPUHtyr@qtHg~^( zrWWt=Ht!dD?(wxV7NALufj@nl626yn~=-~iXoTsDz|cj5b!J4@+}YHzu-zP2lFt`2nHAPGB@*ypz4 zHFxtjABZ%E^EnrB5U2Ayzj8Up^F5#NEa&q-C-N5u^g(BGJty=-_iyh+^hFPHLud3y ze{V;J^hr1G`KI(sKXE(9^i9WbN#}Gir&0j-^H1-kkkW)mn4`nl%ZqRVTCo4W^F-4| zJXhR4a4G7hZ{o(vV8l~F%Kyp*)|3q>gLH+*p{T37nt8BGEW0m@)B9ATNWQPAe1=&3U%ZXI>Z`wEu?XXYZ4+G| zF#EKuNq7}HyV87vcr#}Z1z26$zKVp&w+P9{eARD!onXtn=OUyj2EKpxLZt{(Nd|#e zdifoiCjhupIe4;|`?!$Fu6X{enEs;~G15}R6==hd7hQ?%FTSnAxT=Jd7tYNJmPuRt zmx%tcfK55t`^A$8zjuhe&wbV=_}AcpMN>C#y4JP;h#|X}DYO6L;~>I>3KueL=DRRS{Wz2@<$p~N$Gia42F}wEX z1400Fph1I#kOwDb%xqyC$wL;iWM>pUW#;UU%ZwE-X6>4l3^Zo3ypk^zTcrq>1i!P$nq258vIR~9|Y#owGFXjQmizw+GM4Lqe z;?|UC1DX~QefDjb-$Fv%K}dqdK*QmK4IP*uGYd9YBT3?sry+U}9)y}|thol8Y|AzG z9BdV)v{hajWg%lLI%wt$B-{;E)Mc3|HOv&3AyndRCo*ILUU4Pl0Z&+}(4!tPp68)M z-U;agnYNkv5D)d42OTDeaOWb7KhEgjojyp&WOUrk=SXX?SrUj+ue{(GrkQHGsYH;$ zg+ZtF-N&Lrlr1GDLQ`6Wl|ztO6B4Jf$~r5pwc7uBt6zsHc32g-`g-dal30O}sde#H zhEmoPu^fEzRYjnqRE;JgGsgO`!BEtaHVmvAfpr@+1Ca(zD@Z6b#SSGZ=8zH_cUqau;LV0r5Tqxz?Y013zbY7Xbqtm)l>;-kkrp30Z_BPby`$!LsKVva!@H#%d)8= zzML_>N_CCU3Y2Qw->-S=y*FEKMWg0J1RMW6F3ShaRjopv2@MekT%*l5$U!UUQAi<&*u5u1?2*QcDWhz%NhrWXlF%5a+fB?NsP|q7 zog5)MQz3we>x?Ldw8yI9ArdnRC62fjrQu%|e|+-G&uh_leLqhLCVndAOSk{coX`nr zrcIhNsf03ZE8xW>7Q+e6kGBuG7qa{BzXxA2PjZq9Pf&sqs{xQg4#$k|NiRp#J4#ZN z0u5c50u%|_ng00aJJiw1biP|1Ti(Dv_rVW5hg${(X(z6g%_o4+7+?Vph(D3E<$K{{ zMTp3!5Z5UMcCw?L?Gorb=rwQ|5A6TQN4RAZ&E(|;RubBDghCWZI8AQsQiumr2STf< zFCnV(g}pk(2^)%!89HQ$of?uw)hUH_3W15uoKeA0*oQv}IbjMzvBJXnK!FUL9(OY1 zJX4IvHV@kzA`{scPXW$GB2-HI-bb4Fc~L1lQ=Bs*ajCvV<#CGqBq&2E%27rHF=99j zDN~seRau5F9&n4lK;;pOBm)a*`E7-we3UjC=EMi+G z*1ecdJv$FEeuaknVBGUq8`^tL0Yxjh4>a$hs!1jl44PWD0eTg&B+WWTUoX=ceJ33?nVTeuK^JwT2=AIdY|Z~ zrOC(?;DkhRZ<$>#4oD`@7zWO|a$ea&b|&2XWn>!?v}!p@Vcz_%Lc&{H{)%>o?-gC8 z60)rS#3I0nJ#pn2$=RfPX|$H^)&>5O*>6&ouA%UX`mb~Qq;B@5UZ4q0Kq}iBW3_6T@%Wxc$!n2g$=ugCNQ;h@jtngS-@l) z>SqoeX@$%$tBj_u4{-b3OfSUGob~|@K!?(A&H2a9zExX_@i$qvt{7}yh@zSIT@NJn zo*AL-5uYvYag#fdxI2WD)y)Hcl2lP71_1a00AiT5LZCDq*YP>8n0+JD za;q?R!nyzAAs+}zSR!9&tiqE8U%0{%-f(zdCfghN$g8ot+_Baexz|d0{V<$B76V$M zH7AS5Iqq?gqYvK@xjDjf&TxlAyd!D;!*qgx6sAKsK3wHVIGA^kZ1rMW34VyZ2~5yJ z+~E+Lz+p~oLFkHYyyG7y`;ohc>j5)FRo7*wST0PLg-~6qGr{@7cmD9pm`WKwW~a;5 zt$D|&*deMq`M{8VO{J_NfC$#e*ZjKq)w90!vBKObU605h_vLVAx(U4RRFPr?y&)#0 ztX6W(lKlV!z-2G7n@==6i7DlnP(J>pN;LGJkIh~RRYftLA)-e6U~7vTeYL1PJnp~K z><9n3AIp0i+(KuC1Ob$ZE=YAp=9A`_6`_O)Bt-C7Fro*`b&TUU+nyw(!>B_`!joF~BImCZ7q7OVGPu&%;{aT-OqQTHy z7D5C75F)i?6Gzb#9x|UEc0`GEg9fHvGL+#4W}Zj22_#%VLikh=e8kP9AmJ5N7wOv* zR*Wu+2rIf4lTF`#C{qAt9xRd*0QMIB@rPcl;(J)4iGd+Jb{Yyc1UoW>D>hG#NRTbu z;z58wQj}ag2Bbg^qz_<@Vi4r7u$y0OKy4h;7nw_g31YR?*n{=czO962UB*@*pY)|$ z<1vIcJ>Nz~A5aRYEY?DJ(;Hf0N={$BWgO#Gh7H!EpAl%92{GDUMDB%Du z1nq^TLR^?r+GOG(q+?D*v`yO%E+%7I3!|xx(R>?T;N@d>re`KrkM+uD%F0ykh2nro z&^QC>)CtV>q&;0)L}a1_lEm}zKmpZ=^no83Nkm5afNfF&ZkD8&t>Dt+#49*T7RUto z!GtkwAW>Exgjk7;91{j0<_|7}YfoME<6bTW+v zD&0k~;&!5yk)fW$9cQU1Cw0B1d8&$1l7tY5m+|onS$b3mrX$FdU|ae?LdgPA_`>7x zB!Ok)kT4@>>XvaLCw{UYMYJYXeumNYn&J^ElxD{qT!nJdRRA4IVxp2g#P8+kb6u8{ zP6U?)s9O2sGbjU4*ppB`#Pca=LTD+MK8bbBrsx#Id}5P;fk|v-N!6{2qxfS_T+tV) zM>t+0$yF&ys6c303u(0(LrfSSC0UJCX(key>s-a0CdD<_DP{jE7WplNpE3Rf|rkoGJvJKI)5^1XSRt7#%8+{wu(a z(ia9SV<;q_L{KycDoKoAY;~POVBox#l+{fH1}bbvgsHs>r$;41Se(&ATxo#iW=a^< zy!K0iUeu&|i8CMq$0FrHOc*W%5_}YhMTU;TFhs?sg)IO20Gn8Z$9^n00l+(A8(vs! z&PXUiz^u$Bi*C4QzDmT-Rt$h9t6;9j#5zRJnh~E$2`%aB@Svv;AXlZTsnBxR$$C`E z@TzZK<6N@sK{x^Eq+!2ui8)TH50I=xK&*sinHk}!!-S|*GF{gGtb_&7KDO8lL6A}; zY($J5GcYbhI4dMcBf!h+PDMEnne9DH1I325;mb#;yqOU>NED1z|~CjWci{E{d0Z zDhdFA)S3a`L~PCNPKOQvKtjNhS!l6L`3^%JaL4v? zhlWK)F%GI*Bry64Nxwv|Ldly%Axw-}L;yeSdnN=AH1KjTO`$?2d}O8<0qYK_AezSI zmrO8MrEf=gA2jG)P~0d^vBgmM1i#2C-Egdn0nCL)$n+jW3!ckp*lnl8Fmlo`L%{DV z$ZrV8YZK4!K@@@F*snrqg18`GVVdL4M6eE{FL#{ND!d(tPM~dEs$JD_L{Oj|52f-Z zuZT7!Y3y+^1#%waF<2V%$0G7%67nH4aw7jb@*?+fU`p~GLo&cfn;TFYL-;R@0v#w@DH7P>KYAinX&>T6(6JqR`mZYkY1bmcb+5h!f)83$M~AG6DD1SX{dC4{pid0S8@&@#Y|1xYh6T@3;n z^FRl4i!Pr)>(M=zA2U#pLr2$5s53CP^BvtT)izo_?=vOzbDlla@uUJci!&<#-Y}yG zKsPi&mjv7@1u=kg11$wNNhg+QNnQWxn{1_@FGn;mTVYi=1E`_SYJ^C*{oF{G^f>?V z`5~Y#$n!(@v_vB_LcoMfXiHX=Bg&HCN=LIo^NZRz!zqkHC^!dy1Q;%})-BtHTE7Qd zM?_k~^;<(kT+{VjKSW*Q^<6teUi0-{uQh43^+NPDV2_Vn2linbHtf!$jifUqU3A`= z5dYHIzD93DaKL7=vSx3#W5C|!aW+|$FSUSPLIASxN ziobY{19^`N`Hwq9jT5xl2eRwZNr_ zJ3}jUMOO)OP}l)2$Otc>N4NZ5_rfR}m^Cy6hoFcA$O?D+78@JrkxXpJC?Eo?+;%_a z`AP@@Af)+>pogOfL7e~4!azXBC>+A$^h+ezOP{C+hor$Z&xadaIzhnzOmoh{}eDf~GUx zWTUW0rTVI$hpVduqmRdv~B9fuYwCFr)$xKg1I~DhVQSH$lb} z&Ihn3I-r+MAe@4EARugDLLT&TMnt=#f4VMId%GTbu-CbbwELRBdqPlv4!j8XbR2HE zlhda0EXKN@8+<~jfDqjJv^R*Tck(WXxkSWR$B%hLfV{_#{2PS)$De%3Lj=i-Jj<)R z%ac3Ir##A=e9iyA{LRaJ&cFK1+x*Jw{Ll0J&NDHB<1Ddf`leH%h}LgZ54yi{In+nJ zOBiHxOMO{1shDEQJHEgRd;q>bgg0q2M|?oiC&UL}y*KR@*Oxs)oPAdy6hvGGsx&Ex zWS~TBQV?K0QyqlZb3{|Xy-0lh*NY|J-@V~mmELE>-yg)_i-g=GM7ynh<2%IUYeW!K ze%HSMbuEkL?|nn471xu+``7mR)w-2y zLI9jXN&+yJY*&qA{QzJi3tBP&X5q$_JC|-~G=2iXlg3oVAQ%XORXgbR3!U=C;ieGF2_A&D%~hv~$S!pJ3=Y|_amp^Q?>DXFZ|$}6$VQp=g9<7>+=!36s_0YDK?yC?&_fYTRMABlWsd)^ zB>Bpq(Mc(-)Y3~a%@imd-qO_5PjUMb)KN+O2|!IMX`<9sS#8xmupqN0N0^+Tg&A9B zy{Ndz+;Y{|UmfZTL?wZ(kpxGAV~3s;TJL*`A84DjSxl$yFHJKqH0wLF#Rm}Zt+^B|$ zStlMc*+o_%0nlS0M?k_?Kb&>$I3{nSHM!@XgI3w;N%Kdujj?t*@8QS}oY1b5aU&?c zVd9|K>?=^Rc{oK9z9+oZzVXK8f(rzfXV5vB}l}35AL%q1j}n>XCJ2Y!-6%?)AGr|D!=tK9lWF zeFZdA?!@<#tHFc>^FyBjAtLs<0I>R#69*(E=L8sKn@c_7oM0MIkh_VN!Cq!yf)nC)M+q z9t5{9Bv3DOA&NT4up~mbLxAuI4PFS6kYxj-7|Hk(06Y>HV{BLk8^XcX zT~ZH1R0`k5%NKA)T^|-zX)L3;{qN2eb@8GBSd_Tut~e zkdaA-v68h+$i`xKOps{NV_;#NFQFMF_6g5yBdg<9pyfJ^bn*Wn3;Wjtec(;bU~L22 zaZSx$LI&|51{zRdXDi-^lCvcy8P`0BIu+B-#6Sa{q)VO0=D9hVDeE9zgGMP9(lwDJ zWCL#?*XF#45PZg>Y6-#TD_T^}b#exsFF{&l_{qO&?sK3Coy2H-A(M@Pv=#ho=tDWj z8iztArU}Vr#{j@hdGgAh2oYyw1b7evsxqMB83sWQ;;n~#lor3ABb$2tc^g8KVC%L>9N`gO3tp6qL~1)b<5g z$x_y`nDwXK8d)|@aF!AsVZ~v55i_<$H6bUMUjhN}2E;fc6`?p=OA|7O-cXc_!$k>V zqe6_uF4nOwS*bJr8VYwpBNmrjZD(Bz+nDWYbE)tIB`9G+W}Fd#1k!~m1Up@>=61Ke z{cUP7!rrLx*0+cO?m^mWO}K_+t_ay|XFp3;gzz;geg&*x3EPlV?&rF#1*3M4Ip0%^ z0u-S*3_s&p(5FJfyWkZsdCeMPxr9$wx}C3k?Q1>l9R#eA0Fo_%2Tl>sI77wbi-?9t z0}Rw@MMrgw!p1L+0LIL87~hn|^wic6H7_%>w$-mfbTF@&tk?&Kj*61y3{iyG8P74%B%L{(=N}(b$ZbyYg@l?# zEq9a45?rN6{+Om-J*awb&E2Je&V_|0nffxC-7ej4RkR)@>B6%@G z8aOgk51ZOGo?zr`_q^WdO)InQo=z7syNfv58cmh8>E8C(&DBqn#I06T#|4d557@1V zhB73o*6ymk`?SM=FGDb3FX|@5f}IX-REYmuwxg*4n;h|_^X+tSl!w(fsU_d0aYDmeMN<=I$a z-LDT~L1v<|gX5N4*V%5jpTp$dE2`U{yEYke=T8TT2uY_Z<7^+?eMNac^S>DaEnEPJ za(z?i7GGTqv?HWiYkxSxBmQvi{8=H`d!1JmLWih~86%3{(W@VQID_>LA(R7g?Fg86qQ@4nhOsD$n?))SyKa zEvn`YLb >LxDk2(ep0F9)}9EIu(5Sc0Tdxa6bp^!qGd;V(Kna@824}vqp#Q;!WhfQ z89(aMc#b5(PaW-XFdn5ZB;^igD$9LOhuBYVzkvat#iY9z-g!dNoQwooXEGAh6@9q*Ca%J3b_(kY__CXuE$ zHpdpG5CohF?%J|1;Uet725GJ#68^C(He<}z%oj2&+`zIBTLvc${m{#*JjEl^$Zd_4+Z(M zCBiWYhw(IPFg0DVDM!fv5Y8t3gmEW}kr_)cE=!L&SF&#olgdCsF>SFa58}m8Z>>l| z8rCJfN|VZGW6%uJd~nP#U^69Z^PUYSxlN^pk;-InS2(9cPqZ*+1pr=a zVrY-h&XY%R%|#hP2V(RkT$F#D6v)2HAq$kh6k^RZks%zd_<*i;qK-tfG)R3DDK(=D z8DdC_)bjRI`rdN?A)<64rgX|~4HtA^44?uUytEZsng6D zQ|<%~5{3`68b&Yk&z=&qdt$=@ug`QM^#+qv3zO+VRMi7n^&wOs-mo(xlrl@1k*|i8 zR?B|cSUG4mk$jyq!OIV+WE6H+EjfM{V1 z<&rW!6)q$YL1K1JA+9u90wx+Cv(4N}>n4gJkX9#Abj1t`61LV%F=AOg>Q`56Y;mD% zDTg~xLT~w2=@tTLp-^loBx*B43Qgi_U6aAImRadm3rq6S;?N=Nmib_%XE9<>c{XWx zmKiknA~+XxJvW>(w{%B0bRQyhQTKE=*L6h~c2`$+9fEaVw{vfobaPjAYnOH#!gga9 zcY{~|cZZjEf7f@}7I_x}cxP95o0oW>*LaiHd9xPN7y@v;qHm{h*(4+gMprT&BHD_q zX36&_0<~nS>H8jn8lCec7}pszP*(`GQ^q1!BCp&^f)tl+UpNIan{jF{RaEgOnCiwp z1BO%fWoZLAQ*{>n2=ztk*AHo?M6IkLW}v`+mH#x&X=jSCSa1F8Gq`H<+4T2T_!j{D zcX08xM=56*D7aX|=@4!&-jvZKC{0QG=GY9ag9(#X=FE$(4kSEyA>KEBTh(sr;D9rx zwu)g8ACe&o?iC;y0CL1^T{ufCc>5}6iAN|;aWjTz*avIaEO#{V5+hkB5h$NijW2@# zebJbV+c%BNcyP~njpvvJ>$r{cczyR6KG~R$|JaZ3*cJy_`2x8i+}Mue7>?gKk@fhH z%@~sDSd#IWk_mZ{9ht$b$_=by62iH@Qtz>rZEkxqo#9!!)UuspBBu@mx*+=hE@kMJ zsjKV&;G>qouPQ8>-|T)H*axs#n~j;9DeIfVd7QIwr5@T=Xr`$M+Mp=`q1Bgi+6E(Z zni*=jA$t0!86v28Ix&QLA&B}SlKLT*+98@6sExX)X<4dwTB)Pjs;3&OtD32=nya(g ztGC*zyIQQjnykYbs>>R!&w8lGTCIJWs^40xA!ArcVwzxfAuymm_vsa6NVg%wHW5Pqvn}JCNuoDL zd8bAL98sfFp|p%|IJaSnw`T&nx2lmM+aMnTwmGA;yA`lYqP2JHwPPDG!0{(xHdokp zvmdCj)$*|!g1yBIBjB4(A7Z}Udm-#wz8zb?+nc}R`y%oi!1r6g`<{27r2ocsZ z@Y*%|1XxblK)QOPl1AakJ9Elaw#s!v2mpb|6{(6EMhNZ*DpY9y6b|8m3@Sitmy#AK z4XoKKG=azoL@7i8WQDTHpL{(f7f#C<%>_iwm4eL=_z65hmh${SN;>gYxGZ@<&ex;P zVPevWY0fR(&>9aTgc8e>f)}>jW|F)m#GK5H{LIrl&vgOO$#NC+;DnN*ivCdt6kUiK z9YG%A>UiA8J$=V}JR^8L*nhp(g&o*^{UU}v*>@e;kKNddo!FJ#*`uA=9b(y^UD}%+ z+NYh{w_V$>o!Yw{+#3Sht3BJTz1+q9+ta;C*FD_59p1^^-Q#_~>N?Idz0y^fCCxUU zzkE_>Jm5*<+HTu>R+}Mmxh`_~1zz9-5}wv@J0?D0(+}eR1B4b-F+MvR{vj-$Dq7^> z5r*}I)`cW*vKOL7g5ct3Vn;gJEJ7Y3{!w*4J}*#SPsT(k-qA+{_3qh&AL7#us-9#{_7!r z>%HFW$3DKOkCVCH2T~p;E*?L}SK#U1O}eD+6Q<0@81J=WpZR_c-}EkG{O|d`#tENP z&TbwLUn}-{@x_oG^&*uWzwQb9;3q#WhJq+60x&Y4D14$SHliaw;>&EK^LgU)?ZWhJ z0`-GJ^;aTFF8A`sPocyjH35C1(LycQLcnBS#&5R&_v4~9Tw_ave~r3>_+Q5|cr7!8 zA2pEQESi5bo!Lp2r;6>i4-eZyofQQ z#*G|1di>}R0HpY}>kh3sEP>)$6adjumW(YH?bUFNWqc=Bk4g z8<&O5OgU_|X1(f0N6l8mfrKeq3+{?SIR<062eAZ@rFJ!lJfq!ZI_w>AMfZ%rj2dYFm=C5xq)q&qfO;^3esI zKvKm#;j7G}K0z(D(!A0ewQ8ywrZCQLNgIpRVs|vL(-%ux5eaJ+J$Bo%#VM2yZu@Y; znrGmYK}-5I>GeSAsj|y5z~~}L(EyAQ3N0Qk(~B;m)ZumBxawz3-xF;Bz#*Ej0t`ul z>rx3p$cA0e=mZII2O)_s(;zR8TkZqusw>|5>*(nL_~(Tqgt_LNdp>yR82ugi;JXv# z1WD24VUP(DI`fA?#Ouzs51D7uOEH4Gj)>Q$mtGR-K1jYdim*>_yy1u|&J5#@LtasY z6DlP8>b1jv(U=(%lm$uu&`XcI;t{TBz1v*p+84ajrOtlPa2@{66+r4KBv`Pch!E_c zitzwr6?G6oAZP&%VpM`)05AkAwl}bqIHGfdvqT^|^(pH`Fnx2Yq2D-GF$^@MgeL?B zFH&d_{UMBh{)--ZIQA_YD$zCDDVNyBq$!(hfDRWbU!$Zp1}4QP8O$q4e#++yf=scB zR&LYMuChmjlRO77GFY0k}VR9XZ&PIoRknd#s`lgB8kwF z#|&pc!XS_Yr6dRc;>YqiOO-ESB`r+}lUo*~YZ{SOFX?5HIU+%fZjdiSg+H5Chk|wq%a7>mu8ABloF->CF zs7Ckf9ywh?j8Ra+U8Q=`7om{~Pf&sqo10ca?1|5k=<}ZY^k+p*fJ|oa!W5&}#3n)k z4E78pWsNKdN&neCU6|sd$b4o&BDD;ava}>FeJMZ%N>Er41fwR|Xh%O9(pE|or=HWP z4~pP{QQBge0*RC_EEg$e1azQ94HFMIMpKo7Vs*|UsUrn)(nsDipZlz6-=aFzoEmhL z2vsOUS>e$CimX5n6rE-i4FXe{hN7k86jRX58Oum~3avzKpHT(E(~|hqqaYP&Q$43w zzT)+!R5fhTD5#M#G^!AQK&CT9VF-HtPaptzgGro`icn~gj-mvpDH8XRojrD_l`W_{ zt@g8(m{w1zU2WfLCJL%f#YgA&0I5AJJP?->XIinI@ zyQ+trRX1!JL5&Nh&oC zWWD&^h=02qV4(p(7FrVN5EvXvfgBjX-z7*=^<;@OV4x~5DA^jr2VeQ7<-GN(-h>6> zT>;boZB5xEuf8UNS%m2KzU`HW!G_qsSVH1cC-y;#g+|~SA7sY-B}k8dj1XdGgj4kN zfo9>A-v+G;mNoSOI2!~FbVfOr|FtpzSkWNa9y!6uHCow(To5Xc^UBc3GD`QCV_C+~ zAZqKFOv;Ss%piwOM$-UR^+a5G(aT?6YzZ{FJYaeyBauFUB5G9oNXD!(xL;KC{vh_uXTJ*@UMC8Mj#slBc0S?3vIxkbnq0eSB} zZ428C+V&CaRkeXjJTZ-V%r^h5C)PeqW|dx=x2mRtM8uLu1kUCJIo^@=;#(o_#UzH{ zh$LbA;E>f6qDjLy;E#`2^o!+9K6DFG$gQzU#j>q~)2G z?Lb~;5TS#d=Ok~M$vJXeh1i@RG`EPVJP-)4A4KarH~N5O26X<#{N{08F~r*rb%Q7w zBTU}04|?@5_!Oz+29e)r@l{}ZXVUINAIQ^>JMmC)(+xUA2-7H*O~V_}$*x5IyV@P1 z&cjB2^o4<^(KfIncaf)p&&FEZ$25#C-C;&&D#RjlJw!%9V-=A9yX#Y~=;K~ebUNF# zEAcAz*#c6t0*1ZpX-}iu2RqU01e&j&FMHd+Yx)FP!6^3=#71Gi_R>d~<{dKp4?pwR zzn8K2&jfw#OEI~!nVB{e425P_Vcfj-cHvsZu+NP6K?a6^V( z%OZjef@%H-fC^F~YjS5V=!1drB2HvAPCzf!mJ?x@f1G3z@zQCd0t|lFE7T%CMY4bhAzw{3cUl-~ocM#)a%R!Cir|AGp_V7B1`(lGh_A?t zYGHa}vs;g56LqLE?j?t@@FY6G24I#<_Ju7h>Xj9?XjTG6BY6y{h=qs!z5f&n668Lf($T3BN zEPx~s;e-nxaFHgNGCSjt)HIO2s39>3kB)efuB4Kj*FH4GkPra^NECN6DU`lAE<$KE zz2Y%P*nSDPhy*cbUSu{YI6{kTnRAmIv{am=l$!6_Zl9k*V~RP5BT86HdND zFnwtdWqFkh5tf?>mjdyWo@jRMhA^Wz6{RRQ%;yUcxRR>>XpzLHX{2eAaXFN+GI^E9 zl%;8!42UF)sVfiBE$6tJxw#g)T5_Hh zK{W>Hh?xix`cnjY1P#o&W!GtW65^i;7@gqAfbQuKNVsG=HWfUuQ1DZV|7iuTDG)kf zXr-_LDwru%qn`h%o90536}d3ub)mK>DLOC)ML3`cp^G2tqGhpMMyWKgWQ7QUI({&i z3&B&3w`}575h`$MCb^^9DG^mjq*>r0PX#_HsDiWq=#{0JkWacls`hnjhh9V)5mX8h zMA$KBKyOQhZa%sY^ww8mm!y`1f?N4*cV~HuR;3LAq=uH75&?~`=cG*vn~x}EI`m-g zQk!AwaM*dJl6j|Nr=<~rFVUEpA4wGi@srP>3@XV}&L9Vc$1&HC2M9B%yOO3=ai|e- zqrDfZ4l$|?QK}4astb{-^wTAx+Nz}bs-_yNs5+~vTC1#zqp+H*vbw9Z+N-wutGMc_ z!I}`OTCBHvti)=g$m&HSk$61{Jz?^b%z&xCCUt$|s(5ur5bIi{2pg}4Ij!2-$RMWhxMwOCFpH0B5PAhghQv;b)CP zk1Yvv_xXi~sbZfy453SOqicqU$dB~t61^+BYZYd2)HF}RyT0pqz)Nbelws0C5VPwm zwVRIyp}IX;p{=WtpnJRul(1%;d>D3^h{8(z7)}SbM~lvsb-EBP9a5IRJICtICI%6Va0pD^##0>XcBY! zmWZjr9^Aq1d$}A;njl=lBK#60ynh=UvnkxdEIe*59K$dyi!|(qCcMHpJi;%m!!lgM zJ#52#>BBdCvaP8Q3tXWM%zb$O>MU-EF=t6MjVr}+QE@W5FV?6Fpa2S*fG0F%zF_qd z@|G$lK@6s_Rl7i!01#Enuu)qDI0YtLsXz&bKnbc9OhPPCX1rH)IT8AcrTya}&S1u% z065`ee0iFX-)OC+sm3nB#(Mn5HRi`Mvc_#33XzOb5nEqM#Z=Z)$&if3HD_2i$$99adpJ);)wa1L6$Wkhr4grSP`U*VK$fRt^ zn7mW4OQgSyr{#Lg?UT%?j1Y$uQ-qAhlIRdmrU(cuA<7^(R<>KQ^ja-Nzkair&TtB& zaLLHjm7eU#)BLPpY!YJsoDheUEO;2k`drWaoX-n^&jRbu0u9jnT+sb|(EqFu0d3FK zY|sn+(DxkC4vo+dUC|PK(G;!F7`@RNP0$wI(O5XqBAwCAdb2Ar2QDGVd&QGyRiF|k zA?ce=Ycs_)JsPCZGi<3JTM~CbkW6f-5rMcj$0-oC^b*(LmY2KJY$Mb=J)aOUjlH*+ zdoy3UyGvI6OXY*q9(S)CVbmtE)dFF&4p#xs<9F7i)$zA7AT2Z@R2^ibDCt>+olgo?Y4c-M(tw{-o~Q2*KHV0ustt$m+cck~>~(uvZ@9L!*EQkD>!l zU-~GKb4E_-q9yl7# z3;dlU889RROw!5_CLpfgBhKOHLI{PtJtQtY4i4d>psW?j2J@ZZyTalkPUDUT-~ui< z1b(z3VFwTY-ax7F;yT&C?#sW@b6*1Y;~OqDW*M^;&cp{%d~K3;FI3-HYl@7)Rw0qBCx=YGBrgZ}4* z9_WeQ=Zk*micaW_9_faz5QzThlFsOqo)DH^>5!i3j^62*e(9R--kv_{puXv*9_pmd z<*IJoCylji?n7=4=lm+=k(dx^e&plL>lOj7QV`BOBWVp$u^o{X6uJ;|hBF*C5yyTJ zFYp2%Q0#0mXV5O~J2CBk2@%=es5Rm24x#LpDem6>6~V0Sj~VUUZtM@i>=y9>wQdk9 z`IJ@v5d`L*-Q(Wu4599-x$P0L@AwYx{x0zSPVnqL@CIM-0iW>1e((y9@DK0s5r0I* z&`rp$5cF>E3z6}+xwyUV@gLDdA20GFPx2+dx3I1h;cfCOKN024@-GkbG5>b z^EZzWF^cn9F$R|6^FI&tmVyYK(epuX^g_QJMSt``KlC)A^h^KqO}`mW5A{gD^T-7s zTLLCqZ?G1z^*E{~+Sc_~FW!4b_DNBLayXuwk@j*3puW-eYWVgU5%)?s_mNTeoOJhR zk6QuOKFa|3ifR)k0{DK-t$hFTRV)RNf%rcG092&*iMSb$Pew{8`Ncu`kxvtuFNc@^ zpBbF5hMwQ}b2BlNGwdM2C|g92qtEg#&-zQz6|52a-ZA?>QTwTJ`>#JY|E~MJ@9W^| z`@t{#!%zIh4>PWB{K>EU%g_AHPaPre{LwG{(@*`?|28;n{n@Ym+t2;oUwSX{@z&w} z<4^wCA^zo0^1m*wHC-P`@*MzB2$~=~!U6yCkN)>B8t`HI7t#Kw4-h^C0+__i;6Y}| z00?B*(BVUf5hYHfSkdA|j2Sg<UN01>!jwD&qVl7ypW!_qVrSoA1WtXZ{g<=WNjSFmCK#f~Lw z5df!UN{3AWc{3+Qg2j|tvr^XXUA%eq?&XWMr5Kid1rP3;#h|SsgfH^=3Dnxclu{s% zG|UWfMaFX{=bdwqY17J=6^|xe+VpAEsa2N+kO-PHNq78+0kFAa=)@Om`?*tx&Cg_5 zRu3m$-1zaXEnOmK4s*yZX3qZXw0azXoj8xhqf=}8`t2l zx=qVBr(fUxef;_LH}rs3A%&8v%LsrGe8AwWglyYyK?WOiFgfL zGV_odLawz=`jD9rNeV3pjYcd{FES4DWjvlvq;N(WYqask2@y=j4U;(k^oWC>=)!SG zB8!}nB_UeU!ASgg-~<|jq@zwQTTpCnL}i)~5jyb#l&eFFu+(xd9qc&85(R4#kV!V% zbn{I(VLGdr9Zf2#%8oeTBq8DA^z%h2+c z1t!>Fn?nLwWR^=l6#$qCmYL?DD@;1AnQKm%TdJ$J`o>T(Fv}2D;N=BPh|Xmw-9;lp zP{^y(R-3_At61Bs%!UDA13F$?m!>rZazZW)9~z@Eppwbsq0B0-6kWrafu@z4|E^eY ziHHq}ZYUEc@mC)#1gdhW94e|}XjYjjBFQ6iTq(#aEXZ(QJ}Mlc$0Vu@bjI6~3v|&J zUcKVhRZqP4w}Tgcc+=*LM%0Nmc;j}2pn2t?ynWa^I!Gx0(lbHgv)A4<5vEiDV3gC`2F(LX1ODFoT8YSVPVdKT2SNYq#?S=4Kb2QG|jO2ay};P&Sdz zfzW}QA&5bskqS>xf)e;*Ml3$E0zGKZg6xq90S$;i8$RzNCv-*%SNI+puI`38+~Ep+ z=s_vsP=`FcV)(rHMKFqQPtTGF0N!vAXQUz&fUDiu4#$x7wT~jE3S%AZ2qER%F(%%E zh5$2HH|gLCZ^8N)-yq?>hCC-am4K9!($b+5$c2pmvU{UIbkLcEAkPPibOs|C;>c)0 zkUKZ)NNz?HyNe*w2V}dO4~jwzllfo+Y>*2yVv(lOIfRp*6bLBIa{}ND&LKLG0*pQq z!H1CQk_MUNMqU}qS=REF3;||BgxL^c!l;_5!)7t{_)Tzzvm(yYBt)jCK!GR^C(x*# zO$fP4L#k3D)8dmj?Rigs`pKrm302kP>LD(fgOsR>e% zN{>4+!IDKo$&idHuAR4OsexcRnd$9JB1!H4XH%W}%}||ZbzmX`l;GpWXdM(M3O&!B z(DqcYhP5HB5-UeC&>3z5fDY;svMfj~&|ni35X<}8_r=xipEm#c`_1cJ#fPSmbG z&;eeZfy+^f2t*atNEs@nN@G=MPXirkVj&{fhZHui)Re1pO@)7jBdh!VaLId6LZx$9jkxhHh!JLXP{c+mkCRU@SOFHQnH5Q;5$ zju~DBgO&SX5VI+&viqztW!ceK0I0vzeb4B01DrmESjO8U>x@-LCrzp~I(ax&Og_iV zoI=lX2)VF8_Da#dN|*_i-0zT?QRGBgHd3D*CjeAYjAv8~PgQ|%Lt5ltf#lLAPBsXX zNwB)K=mKme`A{)HOGz*<8O#|WuMfU#o--RH&1;58o@K@iHG9~`gC;b7$O*&4^jIKY zZtjmEs*&Ley3mt;m9~;Q=)E~lo&tr$1q#|cTb%04PM)Tbsa284BF-{a8ugG%9pwR= zqymdQ4{#G_9?_O}&Lm|>t1b8cYQnOC70lBxNqYL=RXZft7|Hd1JEQ|)6T8C^f%dQu zQfx|RTieVTaYXJMk>wdhkOZ`0s51fruhw_l>rNGtTH>}sW75gih_9Wf6mAlw$=MT; z=W0J&?~h4XQ~Q=kzZoLtkSLPAp{-?yYg3y!JaT|tX2={U9%dtHtt=OPbv?r#V?_ds~R?cgqg>cEBxz?-pJJNZ6}+(K}UR zLeq`RBx3AKaN{^=Tvgc)zDPBt?o%*nDqTtq6O~UgYo?bGQLnH~7SgiwlV_lAPZsc- z!$$Rtj0XV2wp&T(t8Q2SUi{=4N&DKN%n-TLy=)E%JWd90bi^nAE8Q-6mNE-qcN#?p zV}MT7@$QgFD_-+(`b%IY{R9f$A@me1ghnkCt`nz;6EbzhY0_9CY$yH!1oit z_#?piGr;;o!246c{A0lVbHM(C!2esl0;IqLw7>+!zy;L62IRm8^uP!N!3hjN5p)Ru zGeHSN!4mwJ6lB5w6=aALbU_kCpF^uXMfn5UKsRQ%hHGk+*HawrE2DJVJR;1N2%$7c z8ISZktd3j0Jz<7un7)mWksxrA_h2j!pqLV>I*?*8^3s$nl#4CQ2onH~a`LBO5<;Qs zF|nbrSV;@vV~8@$q=*PZW*9?{ShkJGjAoiRliN6EE2=kShBzcJ@1uxivqC78!cTg_ zD8vj$WW-*IL`XD{+5 zgvE$B#aCQKRUE}zEX7%rMTnrqShU4j#KmCL#a`q^!ZAjN@WovO#$Oc1W+cXAWX5H5 z#%YAcWJJdQhEPVyqC<(u!b+hbOrp2sON=Uf!iA#Wumyz~5fZ6`&Cr2Z;w(37x{}Mfmh#7807z^aNX*lySJ05!FtLr$0OGTZ z$q>jfi4`cCI^^Rr{t^P<3ZRa=9_Nur02mL2WJrb($j#t^L)a~bYA%pK5a)r6iNq>| zoHLBb8JWRpt%EG+4i8!a`(kf;MAYmaDe54Y9qa$|w%o=ew zmRJ&+3zu<$gj12XJEV-F`oWuAsLxr3{Q-qgaF=F^BsaUgyh}D!+c^M;&DpHYUFgG# z(=%NF1=|cVk7zPtFa_Ei&YV)D=tu=jI0Q=InwF{zXgE&fOrSP`2uv!V?hGl|ydBwe zPT`!S^+^Rvu+BANrcT6&Q8S=WfX?J&!$X`x8{$rPv7ztOsp)J_N_fxUf+N0RPxow$ z*7_@i03L}@O$Ut#2(5?-rHBfhhzott2Bpvqwa^a5&=1v67vazm_0SRp(GwL>gNRW7 z@C;E2b@YRU`R*4W+R-MycHP&B6)?iiEVP)20b=6~q)?}5|Wu?~tX0_I5 zmDOm~)@kL|YW3D@1=nn~)omr$ZZ+3$Mb~gu*KyU=ICWNtAl7(=S6HQ2TD4bP#n)Zc z*LTHM$-=RUzyN$}o>%Y~(IT6nf{4spQ-)>JJgPbK;Xy^Sq&L_FXJ`dV0W*$#G;Csn z`b-O8;Du2@1iLyifZNTTv=nIj*vk}IlD$7bpap~Bm`#-kXzNdC0ESgS1lte-Js241 z;FnQ2gv|gLO=y_Q473fHznGm_p3N+eeV3LUSMy(-OM80&{f^jh27Sb-PdK^*~Q)4b=_g2-Q49}Z1UaRwcXy;-P7Vm%-|ZY z5rs5ptH=YBjU~^B(7{4%SnH(^OEbC`_<$FfSdQZf?}dntdx7n3!j~}Lhd?Cs{RmaU zIfy_JVVPL1Szn4!-;C(EF|ps6px>96-;2m!fmmMv_5uE7hypfXjX2-{PGAPU-v-7A z1t#DJPNfO9hzP#m3a$tYc3=*M;0&(c59YTJF1Qhnhz<7O4yNG$6t>_J7GJXW|A>T4=@g6Fh+zM`C>6XB0-L1DLxBNT?R{z3WJP^ zxVYqls9s6_lqE@yO3XLVj@b`G)5yR>$WXL+7ya1uy*4(D^e=T!lKLeK<%VGTXd z1V7T}zTjtnCg^>BLdU3OjBo;4undOw9n|4tga+b#hUTY;3Kz2uLm(J_!Gt>C+M%ek zhsYU(U}Tf1D&|s+jJB?CGHK6n5R#rJsBr0uwh>BJi%h07^@EFB?qrd^=^^fB(!6GQ zL+R5nBSp5Mg#8G~7?s0_=w1SxPNry$h`1g?X1h4*P_}9&d7Y|m5ZuTJuEvO`7Haxv zjn{~c*>Fgrf@rt^0N(Ho7(;6~?qzLz)N5<%uy7CmzA}z~W%O-8^i}w(H~QkNN=s z6$6&TtdIgZkTLRV)dtOrp0QJH9=8sQQF{^Nu*k!(D*!lIijWkj=FMTDwU|)pSdeV8 z`0d2*J?2&xjcV%Szb;Tog=9TxE<{^4xvjOI)5PdTLmPm%IYZvlt3 zVVLockZ%NMlIyl^l4}b0aE1?;h(l2v(FX4u?}#5)an(R?kPy`u_ERS(;_hjQC#JNz zNe1H!9^cz(kWjdwwg~XMhzL(b^T9nvLmg96wo1Z<%TV07uZ5v>l=zXP1$Ym!o-_rI8v^ zvX@Ngg<@FVRcDBLxtDy|mw5SSNmmO0ls4Li;TKQ;>54IR590K%c{zK*7k!bLXBUZC z_x1d`Z6`QGlNib&(;`{Np=Ct@i_u^NS~Spqw>D`7SkjbB@daAF&c+} zX-ba>Qnws!H*~?snyu-YPEV7A<(n=)-u8BQh6QAJ9yIewQ}NMmpm0#I-tJmg=!$7R zOD)ikP$#88CuLtd%Mta9pqwGoKFw*K7JDDV37w4|aL1d7m2a&T7$U=Y_n}A1-|0W2 z*XeAWbfCD3k|)G!xd~8*4aTy0%h5iU-yBh&dd1X;q<@GFu!H=mn3Zzw*IqOz+4X^# zbk6yCSPOcy5qT@k2yg5g!~O~XxuF~lCv#9&oZJaN&4>uCPn^ou_`ImCis*WRT_FEj zAHIG_e^-3#MRX858y!!H8d4lBs-hlh2p|%ogf|;D&>#&W1O>H?7;2z>0eQGMqbN}$ zHd2@gs-Uz;h78&u)cc6rA0iAweGa-9P5>bhvbH-kAxL#17CIGP^FzUQeVHLHzi!Tf zK?t|s&FG(>*N-9_`g|^$RK+nM+9ViAI{W?UpZ^IUh8X=qID{cW>B?}1Qy2x>yq(2C zf`<703)&!nI8>+*q9#5L0(ir0p|ez>WG!UY%HY9+Cw4{TWop!>O`(8Aswi?K$&qD> zEhILIk|UJMk^vBz5G6DJXB#!C!pQ8^i;8oWF^ffE0=1W zVH8UOn6waDDo>OsX<}tmD%p`kFb<1aR_3H)Fkk4<>I_oMwPmwJtT`%_*oDLD`dFc7 z;GsYZm6aVc&@eM%0t-DZXm>K@%9bx<&a8Pe=gyu#gAOfvG-agBlDS!%Fk-`p5hseo z%M6!DCn%jo>RHw_Owzu80}n2IIPv1fk0Vd6T)33xQp8AEE`2(6%>ZN-Bj%|PfIQMq zKuqR+qz)-Xq*7&OmF)SWZ&3CYdA4Syt2{UXrM=lUGia|WZtxy7&_E(md+7~>g=hz& zWDQ_;>E=TgxBVgi(n*a0P}c@*pae}U4_1_(GFOo_pELDUbYFh4-G-tM8vx*xe-WDK zq7N$0CK7n3jVDrh?H%NSRyA3Y1_rvlkl{!tz?WKO2$EnfBGzAPZz~~~0S|m*f4VD1V0gW?%e2^$g#3=b9hy8sRXjOw|c~E*>R@bVlufiIu ztiK`UmXl<8`QJeS7BgUh6w{@K8R$cvb0VZ}#1p@(CQ9SlRcs`~+O?xKQ&?6m zSxhWs3&0x;yyc+&0-UbFB-MK_p6-4i!vFmE2mB#-1aXqnjky#k`j ziVvRuNKJ1~#-v+@9s~kJo0eEr(4!%^`(;ey5@E0?S7h2;X*0~sh*GUik`AsrQx^{I z?tM_tJXr*hHLO4XdPSleu3F!ei4ORarIQ6bNvGqU{4lg(_Sjv}zR%H(-D)vfE%q=+{0UTnHB)u+r!#!vVxykcC1VA`y#dM9E2k za?Pp25u2Eq2m#;;E%6KdgwwS)!R~4#vrE^YLMgx8!4G!;Kp5{(2(4I(c$V?gzeuwG z0d+wwU(m3I8bdQ5W#A7hVgP_203ZhS1#V{}d1J3iakcf7L^i`HBN@eb#{Qg4LWijw zoKUA5MKZFCk4$67I<$%``oM;Oik-VaS;8%TF-G5inLJdY2bOtsB0JmZPMH9)dLx;o8*NFfMGfh5mbuPpmi5km0yLlkZI!i} zGth&MWKI1kMFN3Cf*}D#7vHiARf=Ja%Dm?^pfkzWI@(IjFilt`$x}B&RE%Q(#Ij82 z+YJCzQH*DFRHO&V;YhV9pOIyaX&%jsD??gGjXe!mJMES^eelqQM0BDg0pCVb)}Shu zrJ@&2A4+5@l9~lCc~g~&Piu;??>%)iIvwgqacEO|*(pLriK8*cQoNSFG^RgQ;PS|F zRAwY~nv*$QGMcJ2w{;RVtE@>L=2{t=^6aeJwCD*>Rh2izT5fl1rcc`Vx?r@`9OpW1{tv=u`6q)N$`6>{aQgJP8 zxAMu((61YQwODxm5JTI{$VYO4#0Au5*0i{2- zP%j7&w>=c&ILA7+TtQL0;~@qiJ@+LJj10jo@1dt*Yas#RzFHFZlBA)N>1tx7t15jN zwljBJr-Y=Lp$rveUWg^il(d-B-&RC;ANFlfbOU8RNu|g~j-LG}RvFP*gMo)+;Fzyk z=2t3jom6J?yzs2&%)D77YG#Jas?4$Z8W62*4xE?29Ofi1a58fLW{rADQcaR*V1$6G z*>3X;(m#{g%G;#wTDJV-SHn6qE`(ua8GTu8J~yN4=`9D+8|z;KJJ{9fv9MPNvQC2r zNHOZw4FR1}vSiRN>+};kK_i>GYPgW6{0)hfYcVvYZk|A~>NrJ`0ag~(q>fD_ZA&&X z)?M$C?Y4}vWpqOmAg&t^lp#dSM;TX3<+*j-0J=%iZCM^>P;t@r_xjD{%ut&$)Mgzq z1*qzHcMNqP%EGwkx+==lo7v&Kj1Uk9nDd>RdjpNV%jY}egFe3XhDi|7qN&ImB zL}TCIbC=ljcUxR91KF`7-RUWvJ?b!6YjXWsIqlAV-Du4IOB z*+b22U0qc20NtJJ)WliF>r_=WKaI5{eXVn1n*x#A*>+XbY72)(-1%eG!z|Km}eOlSnYxLX=cV2nkAf(P#*X!92(W+RQU~8)xuU*_j4`)R6|>la<{bP$=L*m_T@R zpbv~7Gn8OioFJotNYBJVMf_k8iqo1&(%>~;E>#8xBAlu*jd`t4p^(K8;zA16jH!*? zXPIDI0O9j79;*3Qgt*<|wbc>oTz^2q3^Jie?BHqi;9Afi#KA|!MAx(VK+e$K|9F@U z7DW!;2oKuD3sQy!V&GaR;^-YjBOYRtEn-U^B8##_ z;4va&tPxu>HVYs5l@KrnZarCEAQ)!wz#%-)(1c*|m|VD73;+O3ocK(u(Zn#+!Gq{P zsAxv697;xk#7`j?F@i>@RTW^QBSom-U(8eO0o}YnBT9fo0(q8yi~vG_qq?cbIr@k? z`hb_X$yEWxf2Bu4q9a3IV>YTwNnMOHD3wga&&upmH`Yal0Lg`biiX%0sZ0$$65*i$ zz*VrLH@RaZft5wIpk_>@IacI@_{q9V$`dph8y-Zc`Neq=7(&#HKC;N`NyR|ELP35b zkX)pvWh7@1B{bN=q@<*!a8agQL*M=Xjv4;rqvXciX`DgQ5=+jQOQOnHRbzCanT1f} z0nO!FxFuai#9Pi(U6#dN+9hAsT3o`Vs)1Nu;-z5jrD6W1V)i9qa+PCFBVz_8VRDpY zKBi^v-wVYMu5?H@5e`7oWE0NI6JQ6U9U5Q8i zLe`KP5Q4_Jqyi-z0wuVH8nPKP=z=MP!Y*)DA^OE3IOlUZr=bAAh2WJ&fI=wb1OSvk zYrkb##~jdvO(f@5_twb|5fo&XMmpPfNEz$-o|eR=Wx216ic=TOmQHpYXdXYp{SiGHVy70pR~C};d9Qf(-K$^@x7!(JR& zq;(lk*hGCU=*+RFdr|^?Qs#gm6@RwqU$G}}3MaQ*C}t>>1RcT@nWkMV++3t)4bCT# z_6L4?m2o2HlUC-6qUUm6RbH^xTI}YVmPMS}1)ajFo7O3w-f2Y0DOuR*oa!l{@@bw9 z>Yh@dq4Fl8_Gx|ws-HG0qZX>59;&1!Dy1%JN&KlB2Yhrqou8xGW zR%p=*CmUfg-Rs^zs z&A1lq*Fcv@SfI8JX0?84Y=moUWo2z;O6!GXx?ZaX@+!ZQ#J(~FSk~%R-0DcI>_XYa z%R)oTiiFJ4Y(>~?%-X8XvMkTS>{{e3(C)0z_AJr?(hjZC7A?~rEmm zHZ9dYtyDybDWkGWPsXAhYmb}A>;&5 zKuSk07`Qq^D};vJ((Nt)#WDaxDo6?!*#Rx|3NNGrB23E50&7XQR&v@cFN^}?eg+RT zh2ZjnD5ODujYK>?-)6X$fvk^nTu< zVa4Gl?&400Y^;JFu!5ruE+S-^H?aZfnl9?DPv#CU=X$P1G>G?#E*nVhqYy9az=-7D z?Ht(4xFUomATIUl{{r)Zi|Vqj_QJ00wjb|?FZ_Db+)l6H5^m^DMi2-AAh3enrUFBj zh1sy(`i@`i+AZ$tu4;gu?Ysu@e8V5jQas4>4H?u@fgT5kD~#SFsc?u@z@=-d-^mN3j=ou?W#&BFgXP zcJ45lAyRIZ9?p-AX)xT%@f=6XGSV?}P{a#J8CmSHX5{ff^zrpdg$Z9qAQ$pM1OXxI zv2jGmW)89s5YJr2kRkIiAX~;BZzd&|L<0S>4`?!F=5G9+hN zD39`INLkwK|4E$`-e(L;B!jXgk3=WGGG+w9D8GOx6Y^wCGG_pDMX2&BSH>}4GAb{# zGv{(MOY<{Nb2L-)HCuBwPX;q%^ETHO-DxH@C-P>PaIf{#9iwwPPm6OXkvhBcJ3m&r zk=9zlb3NPh<)~P!nC(6Db3YG<9s6@Y3v}a%s%Qn!KqGWQ$I4{YVcr1I*(!8I!}9~a zrbIX37(i!6YxE%?!f;^pMuT)mTQp|ep|pH7NNco6iw4Hw%}Jv)bgDFPxb#aqXH1Xu zO>2wsNWzzWnR0wtAF+WfKm*d@bWz_kKpXYa5t4NshHxx3Q&(G3tMQUD%TrrdRC5L` z6R@^e|MhNRHE47-S$(xslXa^gMT{*&S}%j_701JV<8@0LAxGDAixon4X1l7>IY+B% z_Ax@`tAzGvv(RYUvMw8uX;+78lXhzpM=!f}Y$r5Q%XV$s_HE;KZtM1L^LB8UmO=G) za0~Zv6L)bN_i-yUZ6o(`Gk0@4_j5ycbc>d9OLui!_jO};c5C-`F9&BgN7Qk5c#HRV zlXnEBV|kJc6oMvHf_%Gn1my)~??>k}T5S!%ntQQH~$SnMb&qm&n<5#)|7@x|HMd@xY$< zxtOyi`8k~G0iw@YqDd6qp*jGab6iYh}w|hIP&s?N4ouE{QwP&c}W;-a|M_z-hWn{V( zcl)=e)0W)2v@;z;ixu)^#?hUZzZ?9tdv|Pi4M(DTu&oOIs+KaD*sd=ilu*aQ>$_B} z02(2BATGL=A^X8I;Idm=U?bo|i+swj1+~vL%>czPVB%`V-c173VV@YlF(A83hs&!+ z%(L;jXvJ67ue}TWZ{WPjOC7+A#=YA`vVbAdL%m3>cWVn-!)FEni~*YC|5Yy}a2;Gl zhv&st>vy|ig&x!`q-Z;6JopC>ufAu1i|^FPULOj-ZpK^8>PC1mV6bFVU{mP*>tX^0 zABEWiF4|`i05}3L6vN#z##X!c*+&ZOV7=Ba1J{d!*E`H1WI`|K{rvJhfB?WDgoWZG z@O{_Luqw~{(f!?m%iI5JNBLs$oUz@%Na&BgOF)Y0cgEuve&o-e9n>x0BLZq|jH3-c zns@%<>Y2swM%G98)^|O`lSS39QLnT@m(oQb`a%q2un2>FL}0b+lU3|TJ+SFbGCR!d zzYVb$3s`a^pwFX2a6Byckm8$cPI=KAgBB z5Ke)yCi2YaQzC##LVZyb5)|jKmI-;l#EFZl)q&OklsuWT;mWB9J<1e1%t}OyUBu$x=NYWwd`T3Gf0*r z!W{;ntU=CnPvV5Os%F-VTuGLVJ2UNQ*|TZawtX9SZr!_i_xAl8c<|l;I2DWL#`4;X zZY7qUo6reH&2>3&VwS9{aP8Z>+&7#;2?? zl_yG+G_hjl7oipOD5Q`>pi*WsHV8aqK|)}{uDO(m8HEx@Fj@ve+yHoEAZJnu#i~{$ zQ;5Jombs2W1_{jWwk!m4=9E!D2_+?HT0tTyP9~%T6GK=@$(Pv3nnNaMOqr3D6aR}) zpdBq_216EoP)b5mLZOAA!y+n;87D8y5TW~^$#1{@mXwGG>jZ*Emm{xS<~f8;;7cH0 zOc`a9O+o=Cl7s|Y@Ibm;EGWhqYrJu%5}Q;}MMOrNlEW824DnGp>14)EJoV&LH!i(| z5==-_GIA76RH0K#gyuL=Fo{+@Ga=%3?D0o|f+Xp&|CuUHYAO8m+U3W6107@;x*TE$0Tm;pu?NrVjV1cs1` zNCzp3z!jrmc8%!cq;A@mSZ6AEG|dy^tgT+=d{_$_Xf{izX$cgBbSEh;+m*tgi>X`xfaiggDQkmtd<%%fd>+<5*$ORgpPF*#ehcE5;CXjoZ&Y2d70BdQXh#B%NjWH z4HE=ni`C6+4g#^p@gjnO3_=4yYm-Z}3KI)_QE7m?GGHbt#)Q#=@EXm4t%K|FEE1GRTY#j*c(ZxFD|tvcSSPP)-aXB5+Vh zKuaC4hPK*O>pE9L&HW`X;UkF$H6pBtO`p;VlRIgOoZSdhj&AxVXDc?iFDH$EV8C$COOQI^^z{P^hy;c z5>3%1L}$$;8%k=4%+T$#Sijlk|2ct~&MXE~jWfL5P|ip>kaUhTRaB!POGlD*a*>t9 zaOELjDUeWQAcGsjUFdd)o6>MEM}7R`GM#8dW~vgVF_r0XigO$cn2w*<^npL6=^PLE zt(GOJ&UJ)2Q==XgsYwlw`%tOWn5N4hEDA!J=qbMdYGf5frN|LEGrwC&WKhGngCFhy zfU(}85KFOL5N88ADOSXWt$O1wpkWQR?k12)`3N&gK@k9m0RVpZ>oXGqI;l`{DEJEN zSV8rd2uEvt0l={<#*L#=LAjwQ)!V)okC2l7Q3 z@I;7P6}f@9o^gezwM}Nr|9aNPuC-_yysHn$c)7qd)M=NU=#&5}STPV5wYrkzRZpW^ z+CcRWQl;wH@QD!^@-ZBwbyii?Vnf%mw1@G8t!$mzUiUT$B^i_3p)9vUf?l0S7nZQ@V~Sx7*_=$ldwb15&P$utp%k%( z868Su+q3Nrw~4qQn%q=TjAw)d#I5tia0`>F)@|6eAEt3&G`wNHM#RAafv|)rLdy)( zMR8k{4@xZTl@m`S#cGr%NpwsR&H+@(zPa&ha!lUMB{m_ilIu9#$YK}6c(^_w9S407 zWGFEs!q(*sf{$FU{|p^VoFjEc9?~ouzFC&L%dD_VTWVlKA6jA3izJmRnC2^Mk03gn z%}^PPXiHxj(+o41dUTNK_4%_KM^;QS{&3Uq=ztBiT2p8plUjCuEqj38@bGMdxdkHR zCb%eRLMrf5->h~^tjZ>N!f^9FH3|@q`df>*@XvzMwm_*6`ftQle!GyRAGnk2vTQ`Ex|1 zZuG#dx9A`Sbexj>_S%&F=sjK{plOI}?yi)`?Z z`J4s)1O+mBskp?>iXQ5O)an(QB@4ta&8Dm_;1Bu4OE!Q{Bfux6L_-JYVA?QlyzH*| zybp@@&*FT|_hvBnI*$Jg5FxA&8qUi(bYKi3C;#Yz(r}Lki_i!K%zfD72>qq;E^8u6 zKx2*!A%tM8h>$i^0EPVR;+U^qqQU+OD2KjKHw-AMBm(7_;Tm+R0-^{wKyONti3Fu5 z`Md|&c&%=9N};|es3w98iAUM0DGYN13vK8*JfJv4%$+)EHnz~oSWq!ELyZhb0%tsu^iFy97`e{*>N4+u^ma`9py0|b7Xqt3x`6d zSNi9P7o?;KooHn8Y5LTyb+!!^NARvP5+OvW6bXX@XY#5zPE)bJ2A@eX7^D!y2BqWnDnZYqH(=tf{Gb^((BeOH#Vl*ok zGjlUNvoz79H3c&@4N_m05+|#ODmCa)PU>T(N=+z|F2?6Np7J4wlQ@woJx(b-GN3rc zCG$v5NB|&m?#1;6LKbO*2M$4}P^cDTCHO8dOaS15Dkq@gDJ6!X4yx^U|8T>4a_}bj zWVp=DH+iFX(sSn`r|?d)TCS!JcZl|;1OPOFPDC+RX5u_KCOr`%W>ACf%n#fM6g{Qn zI5C5}2RW6R?7 zP3v@;;M7j}bWim(q5!q%-1JWSR8Sf1P|tKuo5N27l~5aXP!|;y|BZw7=z>KtiA!I! zDC6V_WMO;W4wA5PIY*ULpQraq6+WDRPBe<6Qo zLSYvq5-w)q5Qddlht-$>fJ>IaPG;m(7R@#qZ&a*=PDoB~AV~WN!bq-UAY74H+yxVE z)knMxQ{$p@kToTg)lHf;&t?xsqUBT)Z%UGNSwlfxBZ5|K6;~hRnyR8k=Cxj7w8A#= z%LJlVf0gZ?r68;XOQ>Z4Z;Tk+#9(8k?h@8l*W^_*G+=|nT!qYEe>KlI&l#dcD|8qlCo8()cM2Y;)`ZPmS z2Zc>gR%Mr_BzRR?B{EGPc1CX2RS2Rf>LOOJmM*f^GPo9Nt+s2y_G<@1YfHjww^nS? zmTbdzZO7JD*!FGRR#xSGLmq|ic!No}gH8B@Q85*7 zad?I;FCQ%fc5inhc10$yrjLYBU-0*GlURw}<5ZVeJc2+703k^xCKYV-oRHTUTERDT z;0X-jR8C|WfFTuT$8ha{76gbFQUMW!JiiocDI($ML&(NtVnn?Zw}c$uAjnIurn zn336?uXq=-m@_b`p|#jDAljiP8jB@5;4b>1HQJ&hdZRtMqboY2L3*S`nl30BqfeTn zOFE@X0;NCNq+NQYRhpz}I;IbLre(UO|8@GNQ_5K%Gmo9Qk4>U0CDxH4`gth%caA_%zJ2VTId|4xABmou6sh{QTfK4KXl zfvv}yE=;Yg&2A#F8X@XhHpuz|^g8h5?yTe5u6>~OJO`|uj<5Z?H}3L}!sirwL#-1c zuCrPq@LI0BW2$N61^QaB=lZg5L$Qgbunl`QLR+&>yR=bzuT^_AOgpt-d$nP^wON}p zTsyXJd$!qW^+ua6@FlEqgR5;XIHQ`li~Bs9*tnCMU-HsBsOo<;j=B9y%#@qDtDBK* za9oDay0@FVaU%(#y1T<$JlwE1|Aascc0~w!#VTEnyx%Ji#T&lkyI&NIHqRqej2U1WTbCEdj>B4?MsV{5RMS zdM+Hpi&ew7!^1y(SVO$OPkeo#N)i~)iowI`1d9!1L6eXe#cO;xk=w>m&$a>>C!{Yp zc6{2x?Ieu!-Gst-pqb^`n_qx@+I)OB9=my%+{c{!H>7;Ss65BBoM4DUNtVIOm%$ZX zJb=D@E9hCv&wL?Gsj1NX<#4BtUZcsj!_9-d$%Ys>cz_VzM4n^$^rCyd_iEbW+&4-H zdgeUPyTj0ZOwie!(V6fN{~O}1IPR(%!X+Es(w7s*FP#RJbX+(+uealZ6kF3t!qXx9 z(?_Y)<0I8aoz=@l)LR|axjWNioz`pJ)^8oxb6wYWeK+=73ObF~gI(B%o!E=r*pJ<~ zpBmYho!Oh+*`FQSqy4>RUD~VN+OHkkvt8S_-8;UU+rJ&$!(H6Ro!ptdA7!oi zr=IFV-q(S>>a$+!x1Q@6-r~C+?89E{$G+m3itNuG?bBZEo!#5l-tFHW?&Dt5&t2~8 z-tO-n@84YtkW&ieJ@5Y>@B?4)i4*Dv-|!C~@e>~j_gnEB-|-(G@{QB%BcJjs-|{d2 ze7+v@H=px6-}B+4@IPPlN1yadzm!bg^iyB;SD*EL9p+gd_G4f6BR}+K-}Y}G_t75c zbD#Hn-}iNY@_%3WhoAVHzV?eB`IBGyt={F+1M8O``lDa^IbQRp-}l*{ny{v;XOIoAO7QC{%w8x=imPCAOH2f`SYLu`yU{D z2pmYTpuvL(6DnNDu%W|;5F<*QNU@^Dix@L%+{m$`$B!UGii{|wq{)#eQ>t9avZc$H zFk{M`NwcQSn>cgo+{v@2&!0ep3LQ$csL`WHlPaxfQi>QUOQTAiO0}xht5~yY-O9DA z*RNp1iXBU~tl5lBOP*cJwyoQ@aO29IOSi7wyLj{J-D|X|5U6|u3m#0ku;Igq6DwZK zxUu8LCI6PJ^0>0)%a}83-pskP=g*)+w@eFIwCU5RQ>$Lhy0z=quwR>eS~|Av+qiS< z-p#wW|L@$dfBy&b;~Pw9cbTpH98H_3PNP+l_2ehxYH_!;2qJzP$PK z1)Dxa&%V9;_weJ(pTBoF{rmXy>)+46zyFSY)%M?j1QuxEfe0p89BGsV2S6sO@S@5o z5)NYpLJvkbVTBhSl%a$as$wCA5Q4}di5iADqK74JC}M{OedywfF}A28j0MSfBZ?-z z$m5JS5~SmaET%|gjYj%dWREoFXycJiCfTHrPb!HdlR*wNq?J)(`QwgD0{JDBR7#np zKwD0!WtD28$>y0pqzR{-V{%!hmtlf=rI>k=xu%(Qwi)M}fzC;2pLYI<=bn0Y$tRZ%In zx>iZFNs$3@9B{(Su*4?g213CSTkNq1C95nm#v&{1v(Yv?t+Le`?JV40{Z5$Qn;va>X29Y;ndbZ#-|uCx0Aq%p=3x zvdK52{Ibd^v)r@G2;2NK&O_@QG|xNxoHWo23%xYaPaEAd(nl*@HPawFeKpiu|4W@U z)l*v?HrBe9EtY)_k{qMjIkGLJ+gienrrdGT4S?NvYdyE#b@P3<-+6=Gx8Q#h9=O