For some reasons RISCV is the only arch where the vector table entry is
called __irq_wrapper instead of _isr_wrapper. This is not only a
cosmetic change but Zephyr expects the common ISR handler to be called
_isr_wrapper (for example when generating the IRQ vector table).
Change it.
find ./ -type f -exec sed -i 's/__irq_wrapper/_isr_wrapper/g' {} \;
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In preparation for the support of RV32E optimize a bit the t* registers
usage limiting that to t{0-2}.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This patch is doing several things:
- Core ISA and extension Kconfig symbols have now a formalized name
(CONFIG_RISCV_ISA_* and CONFIG_RISCV_ISA_EXT_*)
- a new Kconfig.isa file was introduced with the full set of extensions
currently supported by the v2.2 spec
- a new Kconfig.core file was introduced to host all the RISCV cores
(currently only E31)
- ISA and extensions settings are moved to SoC configuration files
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
When returning from z_riscv_switch, depending on whether the thread that
has just been swapped in was earlier swapped out synchronously (i.e. via
regular function call) or asynchronously (i.e. via exception/irq) we
will return to arch_switch() or __irq_wrapper respectively. Comment this
fact for clarity.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
After the introduction of arch_switch() in #43085, ECALL is no longer
used for context switching by default, so remove the comment stating so.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
ARCH_HAS_USERSPACE and ARCH_HAS_STACK_PROTECTION are direct functions
of RISCV_PMP regardless of the SoC.
PMP_STACK_GUARD is a function of HW_STACK_PROTECTION (from
ARCH_HAS_STACK_PROTECTION) and not the other way around.
This allows for tests/kernel/fatal/exception to test protection against
various stack overflows based on the PMP stack guard functionality.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
_current_cpu->irq_stack is not yet initialized when this is executed on
CPU 0. Also the guard area is outside of CONFIG_ISR_STACK_SIZE now
e.g. it is within the K_KERNEL_STACK_RESERVED area at the start of
the buffer. So simply use z_interrupt_stacks[] directly instead.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A separate privileged stack is used when CONFIG_GEN_PRIV_STACKS=y. The
main stack guard area is no longer needed and can be made available to
the application upon transitioning to user mode. And that's actually
required if we want a naturally aligned power-of-two buffer to let the
PMP map a NAPOT entry on it which is the whole point of having this
CONFIG_PMP_POWER_OF_TWO_ALIGNMENT option in the first place.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The StackGuard area is used to save the esf and run the exception code
resulting from a StackGuard trap. Size it appropriately.
Remove redundancy, clarify documentation, etc.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The NAPOT mode isn't computed properly in qemu when the full address
range is covered. Let's hardcode the value that the qemu code checks
explicitly until the appropriate fix is applied to qemu itself.
For reference, here's the qemu patch:
https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00961.html
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Overall diffstat with the new PMP code in place:
18 files changed, 866 insertions(+), 1372 deletions(-)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add the appropriate hooks effectively replacing the old implementation
with the new one.
Also the stackguard wasn't properly enforced especially with the
usermode combination. This is now fixed.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The idea here is to compute the PMP register set on demand i.e. upon
scheduling in the affected threads, and only if changes occurred.
A simple sequence number is used to stay in sync with the latest update.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stackguard uses the PMP to prevents many types of stack overflow by
making any access to the bottom stack area raise a CPU exception. Each
thread has its set of precomputed PMP entries and those are written to
PMP registers at context switch time.
This is the code to set it up. It will be connected later.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is the core code to manage PMP entries with only the global entries
initialisation for now. It is not yet linked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Set TP in exception context so that it gets loaded into the CPU when
first running the thread. Set TP for secondary cores to related idle TLS
area.
Signed-off-by: Keith Packard <keithp@keithp.com>
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.
RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.
Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.
For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
impl
New KConfig options for 'A' and 'M' RISC-V extensions have been
added. These are used to configure the '-march' string used by GCC
to produce a compatible binary for the requested RISC-V variant.
In order to maintain compatibility with all currently defined SoC,
default the options for HW mul / Atomics support to 'y', but allow
them to be overridden for any SoC which does not support these.
I tested this change locally via twister agaisnt a few RISC-V platforms
including some 32bit and 64bit. To verify the 4 possibilities of Atomics
& HW Mul: (No, No), (No, Yes), (Yes, No), (Yes, Yes -- current behavior),
I used an out-of-tree GCC (xPack RISC-V GCC) which has multilib support
for rv32i, rv32ia, rv32ima to test against our out-of-tree Intel Nios V/m
processor in HW. The Zephyr SDK RISCV GCC currently does not contain
multilib support for all variants exposed by these new KConfig options.
Signed-off-by: Nathan Krueger <nathan.krueger@intel.com>
This is painful. There is no way for u-mode code to know if we're
currently executing in u-mode without generating a fault, besides
stealing a general purpose register away from the standard ABI
that is. And a global variable doesn't work on SMP as this must be
per-CPU and we could be migrated to another CPU just at the right
moment to peek at the wrong CPU variable (and u-mode can't disable
preemption either).
So, given that we'll have to pay the price of an exception entry
anyway, let's at least make it free to privileged threads by using
the mscratch register as the non-user context indicator (it must
be zero in m-mode for exception entry to work properly). In the
case of u-mode we'll simulate a proper return value in the
exception trap code. Let's settle on the return value in t0
and omit the volatile to give the compiler a chance to cache
the result.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
To do so efficiently on systems without the mul instruction, we use
shifts and adds which is faster and sometimes smaller than a plain loop.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stop using &_kernel as this is not SMP friendly. Let's use s0 (after
preserving its content) to hold ¤t_cpu instead so it won't have
to be reloaded each time it is needed. This will be even more relevant
when SMP support is added.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Rely on mstatus rather than thread->base.user_options since it is always
up to date (updated by z_riscv_switch) to simplify the code and be SMP
proof. Also carry over SF_INIT to the mstatus being restored in case
it was changed in the mean time.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The move to arch_switch() is a prerequisite for SMP support.
Make it optimal without the need for an ECALL roundtrip on every
context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615)
After:
unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is a per-thread register that gets updated only when context
switching. No need to load and save it on every exception entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The minimum stack alignment is 16. Therefore, the stack space to store
a struct __esf object must be rounded up to the next 16-byte boundary.
It is not sufficient to do the rounding on the __z_arch_esf_t_SIZEOF
definition. When the stack is constructed in arch_new_thread() it is
also necessary to do the rounding there too.
Let's make the structure itself carry the alignment attribute instead to
make it work in all cases.
While at it, remove the unused _K_THREAD_NO_FLOAT_SIZEOF definition.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and IRQ locking.
Use the regular IRQ exit path to let tests validate preemption properly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Complete revamp of the exception entry code, including syscall handling.
Proper syscall frame exception trigger. Many correctness fixes, hacks
removal, etc. etc.
I tried to make this into several commits, but this stuff is all
inter-related and a pain to split.
The diffstat summary:
14 files changed, 250 insertions(+), 802 deletions(-)
Binary size (before):
text data bss dec hex filename
1104 0 0 1104 450 isr.S.obj
64 0 0 64 40 userspace.S.obj
Binary size (after):
text data bss dec hex filename
600 0 0 600 258 isr.S.obj
36 0 0 36 24 userspace.S.obj
Run of samples/userspace/syscall_perf (before):
*** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010140): 77312 cycles 77437 instructions
Run of samples/userspace/syscall_perf (after):
*** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010138): 7040 cycles 7165 instructions
Yes, that's more than a 10x speed-up!
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Same rationale as preceding commit. Let's create pseudo-instructions in
assembly scope to make the code more uniform and readable.
Furthermore the definition of COPY_ESF_FP() was wrong as the width of
floating point registers vary not according to CONFIG_64BIT but
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION. It is therefore wrong to use
lr/sr (previously RV_OP_LOADREG/RV_OP_STOREREG) and a regular temporary
register to transfer such content.
Note: There are far more efficient ways to copy FP context around but
such optimisations will come separately.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG
shouting at you all over the place is rather unpleasant and bad taste.
Let's create pseudo-instructions of our own with assembler macros
rather than preprocessor defines and only in assembly scope.
This makes the asm code way more uniform and readable.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The thread->base.user_options field is an uint8_t. Access it using lb.
A "copy" of it is made into __esf.fp_state. Make that field an uint8_t
too and access it with lb/sb.
_callee_saved.fcsr is an uint32_t. Access it with lw/sw.
Ditto for is_user_mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit 8686ab5472.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit be28de692c.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit b0458201cc.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:
sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Use CLINT to send interrupts to another CPU. SMP support is kinda
incomplete without it.
This patch only enables it for riscv-privilege platforms - specifically,
"virt" one.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Secondary CPUs are now initialised and made available to the system. If
the system has more CPUs than configured via CONFIG_MP_NUM_CPUS, those
are still left looping as before.
Some implementations of `soc_interrupt_init` also changed to use
`arch_irq_lock` instead of `irq_lock`.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Enable `arch_switch()` as preparation for SMP support. This patch
doesn't try to keep support for old style context swap - only switch
based swap is supported, to keep things simple.
A fair amount of refactoring was done in this patch, specially regarding
the code that decides what to do about the ISR. In RISC-V, ECALL
instructions are used to signalize several events, such as user space
system calls, forced syscall, IRQ offload, return from syscall and
context switch. All those handled by the ISR - which also handles
interrupts. After refactor, this "dispatching" step is done at the
beginning of ISR (just after saving generic registers).
As with other platforms, the thread object itself is used as the thread
"switch handle" for the context swap.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
isr.S code currently gets CPU information from global `_kernel` assuming
there's only one CPU. In order to prepare for upcoming SMP support,
change code to actually get current CPU information.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
When XIP is not enabled, z_data_copy() already falls back to an empty
function. No need to ifdef it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Non-standard `jalr rd, rs` pseudo-instructions are used.
This commit changes them to `ret` for standard return pseudo-instruction
or `jalr rd, rs, 0` for no offset jump register and link.
Fixes#41100.
Signed-off-by: Henry Hsieh <r901042004@yahoo.com.tw>
Currently, is_user_mode is 8-byte in riscv64 and it breaks a 4-byte PMP
region protecting it. Because is_user_mode is a single flag, we could
just fix it's size to 4-byte in both riscv32 and riscv64.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
In RV64, all general-purpose registers and pmpcfg CSR are 64-bit
instead of 32-bit. Fix these registers and related C variables/literals
to be 32/64-bit compatible.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This commit enable PMP-based memory protection of code and rodata
instead of relying on non-writable real HW (e.g. flash). Use static
PMP region with PMP Lock bit to protect them in both user/supervisor
mode.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Implement new mechanism of arch_buffer_validate() to support checking
static PMP regions. This is preparation patch for code/rodate protection
via RISC-V PMP.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Thread init related to PMP & userspace contains 5 parts:
1. User/supervisor thread clear PMP context
2. User thread clear it's context
3. User/supervisor thread assign to different entry
4. Supervisor thread assign mstatus.MPRV for M-mode PMP protection
5. User/supervisor thread setup PMP regions of stack guard if enabled
Signed-off-by: Jim Shu <cwshu@andestech.com>
Reorder the memory domain async functions to:
arch_mem_domain_partition_add()
arch_mem_domain_partition_remove()
arch_mem_domain_thread_add()
arch_mem_domain_thread_remove()
Signed-off-by: Jim Shu <cwshu@andestech.com>
Simplify multiple ifdef case in computing region number. Also move these
macros to core_pmp.c because they are only used in one file.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Using struct riscv_pmp_region to modulize PMP CSR handling, including
PMP NAPOT/TOR mode handling. This patch can make us more easily to
add/remove RISC-V PMP regions without considering register handling.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Cleanup logging API in core_pmp.c. Remove old printf-based debugging API
and change the log module of PMP to individual MPU log module.
Signed-off-by: Jim Shu <cwshu@andestech.com>
This commit add 2 minor fixes of IRQ handling:
1. Save caller registers before calling z_riscv_configure_stack_guard()
in RISC-V assembly.
2. reschedule and no_reschdule code paths use different interrupt
return path after supporting of CONFIG_PMP_STACK_GUARD. back-to-back
interrupt checking is in the reschedule code path so that it should
jump to interrupt return path of reschedule.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
If no thread use this memory domain, there isn't any user PMP region
translated from memory partitions in domain. In this case, memory
partition removal doesn't need to remove user PMP region and
arch_mem_domain_partition_remove() could just successfully return.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Although CONFIG_USERSPACE is enabled, there are supervisor threads who
don't have privileged stack using exception handler. Only let user
threads to switch to privileged stack in exception handler.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Enable ARCH_EXCEPT macro for non-usermode scenario for RISC-V
Macro will now raise an illegal instruction exception so that mepc will
hold expected value in exception handler, and generated coredump can
reconstruct the failing stack
Coredump tests running on renode (for RISC-V) can now utilize fatal error
path through k_panic
Signed-off-by: Mark Holden <mholden@fb.com>
GD32V processor core is used non-standard bitmask
for mcause register. Add option to configure the bitmask
to support GD32V.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.
Note that:
() For assertions where it can bail out early without side
effects, these are converted to CHECKIF(). (Usually means
that updating of page tables or translation tables has not
been started yet.)
() Other assertions are retained to signal fatal errors during
development.
() The additional CHECKIF() are structured so that it will bail
early if possible. If errors are encountered inside a loop,
it will still continue with the loop so it works as before
this changes with assertions disabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In some cases the 'reschedule' code path is executed when the current
thread is the same as the next thread in the ready Q. If this happens,
the swap_return_value of the thread is ifalsely being reset to -EAGAIN.
This commit prevents the rescheduling code to run if the current thread
is the same as the thread in the ready Q.
Signed-off-by: Wolfgang Reißnegger <gnagflow@fb.com>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Enable RISC-V GP relative addressing by linker relaxation to reduce
the code size. It optimizes addressing of globals in small data section
(.sdata).
The gp initialization at program start needs each SoC support. Also,
if RISC-V SoC has custom linker script, SoC should provide
__global_pointer$ symbol in it's linker script.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Increases the default CONFIG_TEST_EXTRA_STACKSIZE for the 32-bit RISC-V
architecture. This fixes the portability.posix.fs test on the
qemu_riscv32 platform.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This reverts commit 7b09d031fa. Because
context save of GP register is removed, we don't need to initialize GP
at thread init. GP will be a constant value so that it could only be
initialized at program start.
Signed-off-by: Jim Shu <cwshu@andestech.com>
RISC-V global pointer (GP) register is neither caller nor callee
register, and it's a constant value in the single ELF file. Thus, we
don't need to save/restore GP at ISR enter/exit. Remove it to optimize
context switch performance.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Plus added implementation for esp32c3 SoC.
Signed-off-by: Felipe Neves <ryukokki.felipe@gmail.com>
Signed-off-by: Felipe Neves <felipe.neves@espressif.com>
The stack frame size, used for context switch, is rounded up to 16-bytes
alignment. Therefore, we need round down the pointer of top of the
pre-populated stack frame so that the preserved stack frame size is also
rounded up to 16-bytes alignment.
Fixes#29535
Signed-off-by: Shih-Wei Teng <swteng@andestech.com>
In unshared FP mode, only 1 thread can use FPU but kernel doesn't know
which one, so riscv arch would enable FPU of each thread.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Both operands of an operator in which the usual arithmetic
conversions are performed shall have the same essential
type category.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This patch introduce new API to enable FPU of thread. This is pair of
existed k_float_disable() API. And also add empty arch_float_enable()
into each architectures that have arch_float_disable(). The arc and
riscv already implemented arch_float_enable() so I do not touch
these implementations.
Motivation: Current Zephyr implementation does not allow to use FPU
on main and other system threads like as work queue. Users need to
create an other thread with K_FP_REGS for floating point programs.
Users can use FPU more easily if they can enable FPU on running
threads.
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
Add exception descriptions of mcause id 6~15. Also print mtval CSR for
memory access fault & illegal instruction exceptions.
Signed-off-by: Jim Shu <cwshu@andestech.com>
The only user of arch_mem_domain_destroy was the deprecated
k_mem_domain_destroy function which has now been removed. So remove
arch_mem_domain_destroy as well.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This patch adds weak sys_arch_reboot() function to avoid build error
with CONFIG_REBOOT=y. Some SoC has already had own reboot function
but others (Ex. qemu boards) faced buld error.
- openisa_rv32m1: Not change
- riscv-ite: Do nothing, remove and use arch/riscv function
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
Increased stacks required for RISC-V 64-bit CI to pass. Most of these
were catched by the kernel stack sentinel.
The CMSIS stacks are for programs in samples/portability.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
The CONFIG_FLOAT_HARD config previously enabled the C (compressed)
ISA extensions (CONFIG_COMPRESSED_ISA). This commit removes that
dependency.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Most of kernel files where declaring os module without providing
log level. Because of that default log level was used instead of
CONFIG_KERNEL_LOG_LEVEL.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Adds a new CONFIG_MPU which is set if an MPU is enabled. This
is a menuconfig will some MPU-specific options moved
under it.
MEMORY_PROTECTION and SRAM_REGION_PERMISSIONS have been merged.
This configuration depends on an MMU or MPU. The protection
test is updated accordingly.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Since the tracing of thread being switched in/out has the same
instrumentation points, we can roll the tracing function calls
into the one for thread stats gathering functions.
This avoids duplicating code to call another function.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The IRQ handler has had a major changes to manage syscall, reschedule
and interrupt from user thread and stack guard.
Add userspace support:
- Use a global variable to know if the current execution is user or
machine. The location of this variable is read only for all user
thread and read/write for kernel thread.
- Memory shared is supported.
- Use dynamic allocation to optimize PMP slot usage. If the area size
is a power of 2, only one PMP slot is used, else 2 are used.
Add stack guard support:
- Use MPRV bit to force PMP rules to machine mode execution.
- IRQ stack have a locked stack guard to avoid re-write PMP
configuration registers for each interruption and then win some
cycle.
- The IRQ stack is used as "temporary" stack at the beginning of IRQ
handler to save current ESF. That avoid to trigger write fault on
thread stack during store ESF which that call IRQ handler to
infinity.
- A stack guard is also setup for privileged stack of a user thread.
Thread:
- A PMP setup is specific to each thread. PMP setup are saved in each
thread structure to improve reschedule performance.
Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
- Set some helper function to write/clear/print PMP config registers.
- Add support for different PMP slot size function to core/board.
Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
Introducing core E31 family to link Zephyr features (userspace and
stack protection) to architecture capabilities (PMP).
Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.
For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.
Furthermore, much of the assembly code used had issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This set of functions seem to be there just because of historical
reasons, stemming from Kbuild. They are non-obvious and prone to errors,
so remove them in favor of the `_ifdef()` ones with an explicit
`CONFIG_` condition.
Script used:
git grep -l _if_kconfig | xargs sed -E -i
"s/_if_kconfig\(\s*(\w*)/_ifdef(CONFIG_\U\1\E \1/g"
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The core kernel computes the initial stack pointer
for a thread, properly aligning it and subtracting out
any random offsets or thread-local storage areas.
arch_new_thread() no longer needs to make any calculations,
an initial stack frame may be placed at the bounds of
the new 'stack_ptr' parameter passed in. This parameter
replaces 'stack_size'.
thread->stack_info is now set before arch_new_thread()
is invoked, z_new_thread_init() has been removed.
The values populated may need to be adjusted on arches
which carve-out MPU guard space from the actual stack
buffer.
thread->stack_info now has a new member 'delta' which
indicates any offset applied for TLS or random offset.
It's used so the calculations don't need to be repeated
if the thread later drops to user mode.
CONFIG_INIT_STACKS logic is now performed inside
z_setup_new_thread(), before arch_new_thread() is called.
thread->stack_info is now defined as the canonical
user-accessible area within the stack object, including
random offsets and TLS. It will never include any
carved-out memory for MPU guards and must be updated at
runtime if guards are removed.
Available stack space is now optimized. Some arches may
need to significantly round up the buffer size to account
for page-level granularity or MPU power-of-two requirements.
This space is now accounted for and used by virtue of
the Z_THREAD_STACK_SIZE_ADJUST() call in z_setup_new_thread.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
MISRA-C wants the parameter names in a function implementaion
to match the names used by the header prototype.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
arch_new_thread() passes along the thread priority and option
flags, but these are already initialized in thread->base and
can be accessed there if needed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The optional SOC_CONTEXT carries processor state registers that need to
be initialized properly to avoid uninitialized memory read as processor
state.
In particular on the RV32M1 the extra soc context stores a state for
special loop instructions, and loading non zero values will have the
core assume it is in a loop.
Signed-off-by: Karsten Koenig <karsten.koenig.030@gmail.com>
This commit renames the Kconfig `FP_SHARING` symbol to `FPU_SHARING`,
since this symbol specifically refers to the hardware FPU sharing
support by means of FPU context preservation, and the "FP" prefix is
not fully descriptive of that; leaving room for ambiguity.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit renames the Kconfig `FLOAT` symbol to `FPU`, since this
symbol only indicates that the hardware Floating Point Unit (FPU) is
used and does not imply and/or indicate the general availability of
toolchain-level floating point support (i.e. this symbol is not
selected when building for an FPU-less platform that supports floating
point operations through the toolchain-provided software floating point
library).
Moreover, given that the symbol that indicates the availability of FPU
is named `CPU_HAS_FPU`, it only makes sense to use "FPU" in the name of
the symbol that enables the FPU.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Adds handling of the FLOAT_64BIT option when determining the ISA
flags as well as introduces a new Kconfig option to enable/disable
the hard-float calling convention.
Signed-off-by: Corey Wharton <coreyw7@fb.com>
This change adds full shared floating point support for the RISCV
architecture with minimal impact on threads with floating point
support not enabled.
Signed-off-by: Corey Wharton <coreyw7@fb.com>
This operation is formally defined as rounding down a potential
stack pointer value to meet CPU and ABI requirments.
This was previously defined ad-hoc as STACK_ROUND_DOWN().
A new architecture constant ARCH_STACK_PTR_ALIGN is added.
Z_STACK_PTR_ALIGN() is defined in terms of it. This used to
be inconsistently specified as STACK_ALIGN or STACK_PTR_ALIGN;
in the latter case, STACK_ALIGN meant something else, typically
a required alignment for the base of a stack buffer.
STACK_ROUND_UP() only used in practice by Risc-V, delete
elsewhere.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The core kernel z_setup_new_thread() calls into arch_new_thread(),
which calls back into the core kernel via z_new_thread_init().
Move everything that doesn't have to be in z_new_thread_init() to
z_setup_new_thread() and convert to an inline function.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In arch_irq_connect_dynamic the 'level' variable is only used on
platforms that define CONFIG_RISCV_HAS_PLIC. For the other platforms
we'll get a warning about an unused variable. Remove the need for
'level' and just call irq_get_level() where its needed to address the
issue.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
z_isr_install is not suited to handle multi-level interrupt formats.
This update allows z_isr_install to accept irq numbers in zephyr format
and place them in the isr table appropriately.
Fixes issue #22145
Signed-off-by: Jaron Kelleher <jkelleher@fb.com>
The mabi and march options of the compiler and linker commands
were previously hardcoded and depended only on the 64BIT config
option. This update allows these flags to be set by the config
options currently available, plus an additional option to
specify the compressed ISA.
Signed-off-by: Jaron Kelleher <jkelleher@fb.com>
The set of interrupt stacks is now expressed as an array. We
also define the idle threads and their associated stacks this
way. This allows for iteration in cases where we have multiple
CPUs.
There is now a centralized declaration in kernel_internal.h.
On uniprocessor systems, z_interrupt_stacks has one element
and can be used in the same way as _interrupt_stack.
The IRQ stack for CPU 0 is now set in init.c instead of in
arch code.
The extern definition of the main thread stack is now removed,
this doesn't need to be in a header.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add TRACING_ISR Kconfig to help high latency backend working well.
Currently the ISR tracing hook function is put at the begining and
ending of ISR wrapper, when there is ISR needed in the tracing path
(especially tracing backend), it will cause tracing buffer easily
be exhausted if async tracing method enabled. Also it will increase
system latency if all the ISRs are traced. So add TRACING_ISR to
enable/disable ISR tracing here. Later a filter out mechanism based
on irq number will be added.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
This is no longer needed, since all in-tree platforms are only using
the standard mstatus formats. Remove it to avoid the complexity.
Signed-off-by: Olof Johansson <olof@lixom.net>
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.
This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When compiling the components under the arch directory, the compiler
include paths for arch and kernel private headers need to be specified.
This was previously done by adding 'zephyr_library_include_directories'
to CMakeLists.txt file for every component under the arch directory,
and this resulted in a significant amount of duplicate code.
This commit uses the CMake 'include_directories' command in the root
CMakeLists.txt to simplify specification of the private header include
paths for all the arch components.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Use this short header style in all Kconfig files:
# <description>
# <copyright>
# <license>
...
Also change all <description>s from
# Kconfig[.extension] - Foo-related options
to just
# Foo-related options
It's clear enough that it's about Kconfig.
The <description> cleanup was done with this command, along with some
manual cleanup (big letter at the start, etc.)
git ls-files '*Kconfig*' | \
xargs sed -i -E '1 s/#\s*Kconfig[\w.-]*\s*-\s*/# /'
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a common definition for NUM_IRQS in arch/arm/core/Kconfig and
arch/riscv/Kconfig. That way, the type doesn't have to be given for
NUM_IRQS in all the Kconfig.defconfig files.
Trying to get rid of unnecessary "full" symbol definitions in
Kconfig.defconfig files, to make the organization clearer. It can also
help with finding unused symbols.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is an optional feature and no logic for it should
be present unless CONFIG_IRQ_OFFLOAD is enabled.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
include/sys/arch_inlines.h will contain all architecture APIs
that are used by public inline functions and macros,
with implementations deriving from include/arch/cpu.h.
kernel/include/arch_interface.h will contain everything
else, with implementations deriving from
arch/*/include/kernel_arch_func.h.
Instances of duplicate documentation for these APIs have been
removed; implementation details have been left in place.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This makes it clearer that this is an API that is expected
to be implemented at the architecture level.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface and
has been renamed z_arch_kernel_init().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
z_set_thread_return_value is part of the core kernel -> arch
interface and has been renamed to z_arch_thread_return_value_set.
z_set_thread_return_value_with_data renamed to
z_thread_return_value_set_with_data for consistency.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
k_cpu_idle() and k_cpu_atomic_idle() were being directly
implemented by arch code.
Rename these implementations to z_arch_cpu_idle() and
z_arch_cpu_atomic_idle(), and call them from new inline
function definitions in kernel.h.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface
and is appropriately renamed z_arch_is_in_isr().
References from test cases changed to k_is_in_isr().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface
and should have a leading prefix z_arch_.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Various C and Assembly modules
make function calls to z_sys_trace_*. These merely call
corresponding functions sys_trace_*. This commit
is to simplify these by making direct function calls
to the sys_trace_* functions from these modules.
Subsequently, the z_sys_trace_* functions are removed.
Signed-off-by: Mrinal Sen <msen@oticon.com>
Consistently place C++ use of extern "C" after all include directives,
within the negative branch of _ASMLANGUAGE if used.
Remove extern "C" support from files that don't declare objects or
functions.
Background from issue #17997:
Declarations that use C linkage should be placed within extern "C"
so the language linkage is correct when the header is included by
a C++ compiler.
Similarly #include directives should be outside the extern "C" to
ensure the language-specific default linkage is applied to any
declarations provided by the included header.
See: https://en.cppreference.com/w/cpp/language/language_linkage
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
SR and LR were used as global names for load and store RISC-V assembler
operations, colliding with other uses such as SR for STATUS REGISTER in
some peripherals. Renamed them to a longer more specific name to avoid
the collision.
Signed-off-by: Karsten Koenig <karsten.koenig.030@gmail.com>
For now we enforce the medany code model for 64-bit builds as we get
reloc issues otherwise. The instruction set and ABI are also set to
soft-float usage.
The ilp32 ABI is explicitly specified on 32-bit build to make sure
it is not using a wrong default if the same toolchain is used for both
32- and 64-bit builds. The archittecture options are the same as the
SDK's riscv32 toolchain default in that case.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
