This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit f0b458a619.
This is a pointless change that simply increases footprint.
Existing code already supports compilation without multithreading.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
This make MCUboot build as Zephyr application.
Providing optinal 2nd stage bootloader to the
IDF bootloader, which is used by default.
This provides more flexibility when building
and loading multiple images and aims to
brings better DX to users by using the sysbuild.
MCUboot and applications has now separate
linker scripts.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for a pending FPU IPI
from the arch_spin_relax() hook and honor it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Enable single-threading support for the riscv architecture.
Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.
The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
use helper macros from csr.h instead of inline assembly which results
in cleaner and more maintainable code
Signed-off-by: Manojkumar Subramaniam <manoj@electrolance.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Looks like some implementors decided not to implement the full set of
PMP range matching modes. Let's rearrange the code so that any of those
modes can be disabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's honor CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT even for kernel
stacks. This saves one global PMP slot when creating the guard area for
the IRQ stack, and some hw implementations might require that anyway.
With this changes, arch_mem_domain_max_partitions_get() becomes much
more reliable and tests/kernel/mem_protect is more likely to pass even
with the stack guard enabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
When CONFIG_SOC_ISR_SW_UNSTACKING is defined, it's up to the custom soc
code to remove the ESF, because the software-managed part of the ESF is
depending on the hardware. Fix this in the ISR code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some implementations may not capture the faulting instruction in mtval
and set it to zero when an illegal instruction fault is raised This is
notably the case with QEMU version 7.0.0 when a CSR instruction is
involved.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The FRCSR, FSCSR, FRRM, FSRM, FSRMI, FRFLAGS, FSFLAGS and FSFLAGSI
are in fact CSR instructions targeting the fcsr, frm and fflags
registers. They should be caught as FPU instructions as well.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
- IRQ state for the interrupted context corresponds to the PIE bit not
the IE bit.
- Restoring the saved FPU state should clear the entire field before
or'ing wanted bits in.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For RISC-V, the reg property of a cpu node in the devicetree describes
the low level unique ID of each hart. Using devicetree macro's, a list
of all cpus with status "okay" can be generated.
Using devicetree overlays, a hart or multiple harts can be marked as
"disabled", thus excluding them from the list. This allows platforms
that have non-zero indexed SMP capable harts to be functionally mapped
to Zephyr's sequential CPU numbering scheme.
On kernel init, if the application has MP_MAX_NUM_CPUS greater than 1,
generate the list of cpu nodes from the device tree with status "okay"
and map the unique hartid's to zephyr cpu's
While we are at it, as the hartid is the value that gets passed to
z_riscv_secondary_cpu_init, use that as the variable name instead of
cpu_num
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
RISC-V multi-hart systems that employ a heterogeneous core complex are
not guaranteed to have the smp capable harts starting with a unique id
of zero, matching Zephyr's sequential zero indexed cpu numbering scheme.
Add option, RV_BOOT_HART to choose the hart to boot from.
On reset, check the current hart equals RV_BOOT_HART: if so, boot first
core. else, loop in the boot secondary core and wait to be brought up.
For multi-hart systems that are not running a Zephyr mp or smp
application, park the non zephyr related harts in a wfi loop.
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
Commit 4f9b547ebd ("riscv: smp: prepare for more than one IPI type")
didn't clear pending IPI flags.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We can leverage the FPU dirty state as an indicator for preemptively
reloading the FPU content when a thread that did use the FPU before
being scheduled out is scheduled back in. This avoids the FPU access
trap overhead when switching between multiple threads with heavy FPU
usage.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
FPU context switching is always performed on demand through the FPU
access exception handler. Actual task switching only grants or denies
FPU access depending on the current FPU owner.
Because RISC-V doesn't have a dedicated FPU access exception, we must
catch the Illegal Instruction exception and look for actual FP opcodes.
There is no longer a need to allocate FPU storage on the stack for every
exception making esf smaller and stack overflows less likely.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Instead of saving/restoring FPU content on every exception and task
switch, this replaces FPU sharing support with a "lazy" (on-demand)
context switching algorithm similar to the one used on ARM64.
Every thread starts with FPU access disabled. On the first access the
FPU trap is invoked to:
- flush the FPU content to the previous thread's memory storage;
- restore the current thread's FPU content from memory.
When a thread loads its data in the FPU, it becomes the FPU owner.
FPU content is preserved across task switching, however FPU access is
either allowed if the new thread is the FPU owner, or denied otherwise.
A thread may claim FPU ownership only through the FPU trap. This way,
threads that don't use the FPU won't force an FPU context switch.
If only one running thread uses the FPU, there will be no FPU context
switching to do at all.
It is possible to do FP accesses in ISRs and syscalls. This is not the
norm though, so the same principle is applied here, although exception
contexts may not own the FPU. When they access the FPU, the FPU content
is flushed and the exception context is granted FPU access for the
duration of the exception. Nested IRQs are disallowed in that case to
dispense with the need to save and restore exception's FPU context data.
This is the core implementation only to ease reviewing. It is not yet
hooked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Right now this is hardcoded to z_sched_ipi(). Make it so that other IPI
services can be added in the future.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Given the Zephyr CPU number is no longer tied to the hartid, we must
consider the actual hartid when sending an IPI to a given CPU. Since
those hartids can be anything, let's just save them in the cpu structure
as each CPU is brought online.
While at it, throw in some `get_hart_msip()` cleanups.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Currently it is assumed that Zephyr CPU numbers match their hartid
value one for one. This assumption was relied upon to efficiently
retrieve the current CPU's `struct _cpu` pointer.
People are starting to have systems with a mix of different usage for
each CPU and such assumption may no longer be true.
Let's completely decouple the hartid from the Zephyr CPU number by
stuffing each CPU's `struct _cpu` pointer in their respective scratch
register instead. `arch_curr_cpu()` becomes more efficient as well.
Since the scratch register was previously used to store userspace's
exception stack pointer, that is now moved into `struct _cpu_arch`
which implied minor user space entry code cleanup and rationalization.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
RISC-V has a modular design. Some hardware with a custom interrupt
controller needs a bit more work to lock / unlock IRQs.
Account for this hardware by introducing a set of new
z_soc_irq_* functions that can override the default behaviour.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some RISC-V SoCs implement a mechanism for hardware supported stacking /
unstacking of registers during ISR / exceptions. What happens is that on
ISR / exception entry part of the context is automatically saved by the
hardware on the stack without software intervention, and the same part
of the context is restored by the hardware usually on mret.
This is currently not yet supported by Zephyr, where the full context
must be saved by software in the full fledged ESF. This patcheset is
trying to address exactly this case.
At least three things are needed to support in a general fashion this
problem: (1) a way to store in software only the part of the ESF not
already stacked by hardware, (2) a way to restore in software only the
part of the context that is not going to be restored by hardware and (3)
a way to define a custom ESF.
Point (3) is important because the full ESF frame is now composed by a
custom part depending on the hardware (that can choose which register to
stack / unstack and the order they are saved onto the stack) and a part
defined in software for the remaining part of the context.
In this patch a new CONFIG_RISCV_SOC_HAS_ISR_STACKING is introduced that
enables the code path supporting the three points by the mean of three
macros that must be implemented by the user in a soc_stacking.h file:
SOC_ISR_SW_STACKING, SOC_ISR_SW_UNSTACKING and SOC_ISR_STACKING_ESF
(refer to the symbol help for more details).
This is an example of soc_isr_stacking.h for an hardware that doesn't do
any hardware stacking / unstacking but everything is managed in
software:
#ifndef __SOC_ISR_STACKING
#define __SOC_ISR_STACKING
#if !defined(_ASMLANGUAGE)
#define SOC_ISR_STACKING_ESF_DECLARE \
struct __esf { \
unsigned long ra; \
unsigned long t0; \
unsigned long t1; \
unsigned long t2; \
unsigned long t3; \
unsigned long t4; \
unsigned long t5; \
unsigned long t6; \
unsigned long a0; \
unsigned long a1; \
unsigned long a2; \
unsigned long a3; \
unsigned long a4; \
unsigned long a5; \
unsigned long a6; \
unsigned long a7; \
unsigned long mepc; \
unsigned long mstatus; \
unsigned long s0; \
} __aligned(16)
#else
#define SOC_ISR_SW_STACKING \
addi sp, sp, -__z_arch_esf_t_SIZEOF; \
DO_CALLER_SAVED(sr);
#define SOC_ISR_SW_UNSTACKING \
DO_CALLER_SAVED(lr);
#endif /* _ASMLANGUAGE */
#endif /* __SOC_ISR_STACKING */
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This reverts commit a7b5d606c7.
The assumption behind that commit was wrong. The software-based stack
sentinel writes to the very bottom of the _writable_ stack area i.e.
right next to the actual PMP based guard area. So they are compatible.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Change for loops of the form:
for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
...
to
unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
...
We do the call outside of the for loop so that it only happens once,
rather than on every iteration.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Change automated searching for files using "IRQ_CONNECT()" API not
including <zephyr/irq.h>.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The interrupt stack is used as the system stack during kernel
initialization while IRQs are not yet enabled. The sp register is
set to z_interrupt_stacks + CONFIG_ISR_STACK_SIZE.
CONFIG_ISR_STACK_SIZE only represents the desired usable stack size.
This does not take into account the added guard area. Result is a stack
whose pointer is much closer to the trigger zone than expected when
CONFIG_PMP_STACK_GUARD=y, and the SMP configuration in particular pushes
it over the edge during many CI test cases.
Worse: during early init we're not quite ready to handle exceptions
yet and complete havoc ensues with no meaningful debugging output.
Make sure the early assembly code locates the actual top of the stack
by generating a constant with its true size.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The software-based stack sentinel writes to the very bottom of the
stack area triggering the PMP stack protection. Obviously they can't
be used together.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The IRQ stack in particular is different on each CPU, and so is its
stack guard PMP entry value. This creates 2 issues:
- The assertion ensuring the last global PMP address is the same
for each CPU does fail;
- That last global PMP address can't be relied upon to create a
single-slot per-thread TOR mapping.
Fix both issues by not remembering the actual address for the last
global entry but a dummy address instead that is guaranteed not to
match any opportunistic single-slot TOR mapping.
While at it, lock that IRQ stack guard PMP entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Z_THREAD_STACK_BUFFER() must not be used here. This is meant for stacks
defined with K_THREAD_STACK_ARRAY_DEFINE() whereas in this case we are
given a stack created with K_KERNEL_STACK_ARRAY_DEFINE().
If CONFIG_USERSPACE=y then K_THREAD_STACK_RESERVED gets defined with
a bigger value than K_KERNEL_STACK_RESERVED. Then Z_THREAD_STACK_BUFFER()
returns a pointer that is more advanced than expected, resulting in a
stack pointer outside its actual stack area and therefore memory
corruption ensues.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This commit is fixing placing the vectors section through
zephyr_linker_sources(ROM_START ...) (as done in the ARM
architecture port) so its order can be adjusted by SORT_KEY.
Fixes#49903
Signed-off-by: Mateusz Sierszulski <msierszulski@antmicro.com>
QEMU requires that the semihosting trap instruction sequence, which
consists of three uncompressed instructions, lie in the same page, and
refuses to interpret the trap sequence if these instructions are placed
across two different pages.
This commit adds 16-byte alignment requirement to the `semihost_exec`
function, which occupies 12 bytes, to ensure that the three trap
sequence instructions in this function are never placed across two
different pages.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
All SOC_ERET definitions expand to the mret instruction (used to return
from a trap: exception or interruption). The 'eret' instruction existed
in previous RISC-V privileged specs, but it doesn't seem to be used in
Zephyr (ref. RISC-V Privileged Architectures 3.2.2).
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
For vectored interrupts use the generated IRQ vector table instead of
relying on a custom-generated table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some early RISC-V SoCs have a problem when an `mret` instruction is used
outside a trap handler.
After the latest Zephyr RISC-V huge rework, the arch_switch code is
indeed calling `mret` when not in handler mode, breaking some early
RISC-V platforms.
Optionally restore the old behavior by adding a new
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL symbol.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This is really useful only for one case i.e. when testing against zero.
Do that test inline where it is needed and make the rest of the code
independent from the actual numerical value being tested to make code
maintenance easier if/when new cases are added.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Retrieve the pmpaddr value matching the last global PMP slot and add it
to the per-thread m-mode and u-mode entry array. Even if that value is
not written out again on thread context switch, that value can still be
used by set_pmp_entry() to attempt a single-slot TOR mapping with it.
Nicely abstract this with the new z_riscv_pmp_thread_init() where the
PMP_M_MODE(thread) and PMP_U_MODE(thread) argument generators can be
used.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A QEMU bug may create bad transient PMP representations causing
false access faults to be reported. Work around it by setting
pmp registers to zero from the update start point to the end
before updating them with new values.
The QEMU fix is here with more details about this bug:
https://lists.gnu.org/archive/html/qemu-devel/2022-06/msg02800.html
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts the bulk of commit c8bfc2afda ("riscv: make
arch_is_user_context() SMP compatible") and replaces it with a flag
stored in the thread local storage (TLS) area, therefore making TLS
mandatory for userspace support on RISC-V.
This has many advantages:
- The tp (x4) register is already dedicated by the standard for this
purpose, making TLS support almost free.
- This is very efficient, requiring only a single instruction to clear
and 2 instructions to set.
- This makes the SMP case much more efficient. No need for funky
exception code any longer.
- SMP and non-SMP now use the same implementation making maintenance
easier.
- The is_user_mode variable no longer requires a dedicated PMP mapping
and therefore freeing one PMP slot for other purposes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
5f65dbcc9dab3d39473b05397e05.
The tp (x4) register is neither caller nor callee saved according to
the RISC-V standard calling convention. It only has to be set on thread
context switching and is otherwise read-only.
To protect the kernel against a possible rogue user thread, the tp is
also re-set on exception entry from u-mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For some reasons RISCV is the only arch where the vector table entry is
called __irq_wrapper instead of _isr_wrapper. This is not only a
cosmetic change but Zephyr expects the common ISR handler to be called
_isr_wrapper (for example when generating the IRQ vector table).
Change it.
find ./ -type f -exec sed -i 's/__irq_wrapper/_isr_wrapper/g' {} \;
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In preparation for the support of RV32E optimize a bit the t* registers
usage limiting that to t{0-2}.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This patch is doing several things:
- Core ISA and extension Kconfig symbols have now a formalized name
(CONFIG_RISCV_ISA_* and CONFIG_RISCV_ISA_EXT_*)
- a new Kconfig.isa file was introduced with the full set of extensions
currently supported by the v2.2 spec
- a new Kconfig.core file was introduced to host all the RISCV cores
(currently only E31)
- ISA and extensions settings are moved to SoC configuration files
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
When returning from z_riscv_switch, depending on whether the thread that
has just been swapped in was earlier swapped out synchronously (i.e. via
regular function call) or asynchronously (i.e. via exception/irq) we
will return to arch_switch() or __irq_wrapper respectively. Comment this
fact for clarity.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
After the introduction of arch_switch() in #43085, ECALL is no longer
used for context switching by default, so remove the comment stating so.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
ARCH_HAS_USERSPACE and ARCH_HAS_STACK_PROTECTION are direct functions
of RISCV_PMP regardless of the SoC.
PMP_STACK_GUARD is a function of HW_STACK_PROTECTION (from
ARCH_HAS_STACK_PROTECTION) and not the other way around.
This allows for tests/kernel/fatal/exception to test protection against
various stack overflows based on the PMP stack guard functionality.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
_current_cpu->irq_stack is not yet initialized when this is executed on
CPU 0. Also the guard area is outside of CONFIG_ISR_STACK_SIZE now
e.g. it is within the K_KERNEL_STACK_RESERVED area at the start of
the buffer. So simply use z_interrupt_stacks[] directly instead.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A separate privileged stack is used when CONFIG_GEN_PRIV_STACKS=y. The
main stack guard area is no longer needed and can be made available to
the application upon transitioning to user mode. And that's actually
required if we want a naturally aligned power-of-two buffer to let the
PMP map a NAPOT entry on it which is the whole point of having this
CONFIG_PMP_POWER_OF_TWO_ALIGNMENT option in the first place.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The StackGuard area is used to save the esf and run the exception code
resulting from a StackGuard trap. Size it appropriately.
Remove redundancy, clarify documentation, etc.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The NAPOT mode isn't computed properly in qemu when the full address
range is covered. Let's hardcode the value that the qemu code checks
explicitly until the appropriate fix is applied to qemu itself.
For reference, here's the qemu patch:
https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00961.html
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Overall diffstat with the new PMP code in place:
18 files changed, 866 insertions(+), 1372 deletions(-)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add the appropriate hooks effectively replacing the old implementation
with the new one.
Also the stackguard wasn't properly enforced especially with the
usermode combination. This is now fixed.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The idea here is to compute the PMP register set on demand i.e. upon
scheduling in the affected threads, and only if changes occurred.
A simple sequence number is used to stay in sync with the latest update.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stackguard uses the PMP to prevents many types of stack overflow by
making any access to the bottom stack area raise a CPU exception. Each
thread has its set of precomputed PMP entries and those are written to
PMP registers at context switch time.
This is the code to set it up. It will be connected later.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is the core code to manage PMP entries with only the global entries
initialisation for now. It is not yet linked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Set TP in exception context so that it gets loaded into the CPU when
first running the thread. Set TP for secondary cores to related idle TLS
area.
Signed-off-by: Keith Packard <keithp@keithp.com>
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.
RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.
Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.
For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
impl
New KConfig options for 'A' and 'M' RISC-V extensions have been
added. These are used to configure the '-march' string used by GCC
to produce a compatible binary for the requested RISC-V variant.
In order to maintain compatibility with all currently defined SoC,
default the options for HW mul / Atomics support to 'y', but allow
them to be overridden for any SoC which does not support these.
I tested this change locally via twister agaisnt a few RISC-V platforms
including some 32bit and 64bit. To verify the 4 possibilities of Atomics
& HW Mul: (No, No), (No, Yes), (Yes, No), (Yes, Yes -- current behavior),
I used an out-of-tree GCC (xPack RISC-V GCC) which has multilib support
for rv32i, rv32ia, rv32ima to test against our out-of-tree Intel Nios V/m
processor in HW. The Zephyr SDK RISCV GCC currently does not contain
multilib support for all variants exposed by these new KConfig options.
Signed-off-by: Nathan Krueger <nathan.krueger@intel.com>
This is painful. There is no way for u-mode code to know if we're
currently executing in u-mode without generating a fault, besides
stealing a general purpose register away from the standard ABI
that is. And a global variable doesn't work on SMP as this must be
per-CPU and we could be migrated to another CPU just at the right
moment to peek at the wrong CPU variable (and u-mode can't disable
preemption either).
So, given that we'll have to pay the price of an exception entry
anyway, let's at least make it free to privileged threads by using
the mscratch register as the non-user context indicator (it must
be zero in m-mode for exception entry to work properly). In the
case of u-mode we'll simulate a proper return value in the
exception trap code. Let's settle on the return value in t0
and omit the volatile to give the compiler a chance to cache
the result.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
To do so efficiently on systems without the mul instruction, we use
shifts and adds which is faster and sometimes smaller than a plain loop.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stop using &_kernel as this is not SMP friendly. Let's use s0 (after
preserving its content) to hold ¤t_cpu instead so it won't have
to be reloaded each time it is needed. This will be even more relevant
when SMP support is added.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Rely on mstatus rather than thread->base.user_options since it is always
up to date (updated by z_riscv_switch) to simplify the code and be SMP
proof. Also carry over SF_INIT to the mstatus being restored in case
it was changed in the mean time.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The move to arch_switch() is a prerequisite for SMP support.
Make it optimal without the need for an ECALL roundtrip on every
context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615)
After:
unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is a per-thread register that gets updated only when context
switching. No need to load and save it on every exception entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The minimum stack alignment is 16. Therefore, the stack space to store
a struct __esf object must be rounded up to the next 16-byte boundary.
It is not sufficient to do the rounding on the __z_arch_esf_t_SIZEOF
definition. When the stack is constructed in arch_new_thread() it is
also necessary to do the rounding there too.
Let's make the structure itself carry the alignment attribute instead to
make it work in all cases.
While at it, remove the unused _K_THREAD_NO_FLOAT_SIZEOF definition.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and IRQ locking.
Use the regular IRQ exit path to let tests validate preemption properly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Complete revamp of the exception entry code, including syscall handling.
Proper syscall frame exception trigger. Many correctness fixes, hacks
removal, etc. etc.
I tried to make this into several commits, but this stuff is all
inter-related and a pain to split.
The diffstat summary:
14 files changed, 250 insertions(+), 802 deletions(-)
Binary size (before):
text data bss dec hex filename
1104 0 0 1104 450 isr.S.obj
64 0 0 64 40 userspace.S.obj
Binary size (after):
text data bss dec hex filename
600 0 0 600 258 isr.S.obj
36 0 0 36 24 userspace.S.obj
Run of samples/userspace/syscall_perf (before):
*** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010140): 77312 cycles 77437 instructions
Run of samples/userspace/syscall_perf (after):
*** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010138): 7040 cycles 7165 instructions
Yes, that's more than a 10x speed-up!
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Same rationale as preceding commit. Let's create pseudo-instructions in
assembly scope to make the code more uniform and readable.
Furthermore the definition of COPY_ESF_FP() was wrong as the width of
floating point registers vary not according to CONFIG_64BIT but
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION. It is therefore wrong to use
lr/sr (previously RV_OP_LOADREG/RV_OP_STOREREG) and a regular temporary
register to transfer such content.
Note: There are far more efficient ways to copy FP context around but
such optimisations will come separately.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG
shouting at you all over the place is rather unpleasant and bad taste.
Let's create pseudo-instructions of our own with assembler macros
rather than preprocessor defines and only in assembly scope.
This makes the asm code way more uniform and readable.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The thread->base.user_options field is an uint8_t. Access it using lb.
A "copy" of it is made into __esf.fp_state. Make that field an uint8_t
too and access it with lb/sb.
_callee_saved.fcsr is an uint32_t. Access it with lw/sw.
Ditto for is_user_mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit 8686ab5472.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit be28de692c.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit b0458201cc.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
