Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Setup the stack as early as possible to catch any possible errors in the
reset routine and handle also EL3 fatal errors.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The content of the SCR_EL3 register is overwritten by a later
instruction. Also no need to route SError, IRQs and FIQs to EL3.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Instead of having some special stack frame when first scheduling new
thread and a new thread entry wrapper to pull out the needed data, we
can reuse the context restore code by adapting the initial stack frame.
This reduces the lines of code and simplify the code at the expense of a
slightly bigger initial stack frame.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.
Changes are related to converting the integer constants to the
unsigned integer constants
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Zephyr is only supposed to be running at EL1 (+ EL0). Now that we drop
in EL1 from ELn at start we can remove all the EL2/EL3 unused code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Remove the useless CONFIG_SWITCH_TO_EL1 since there should be no reason
to run Zephyr in EL3. So just drop to EL1 by default when booting from
EL3. Remove also non-reachable code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
- Display full 64-bits register width in crash dumps
- Some values were prefixed 0x, some not. Made consistent.
Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
- x0/x1 register printing is reversed
- The error stack frame struct (z_arch_esf_t) had the SPSR and ELR in
the wrong position, inconsistent with the order these regs are pushed
to the stack in z_arm64_svc. This caused all register printing to be
skewed by two.
- Verified by writing known values (abcd0000 -> abcd000f) to x0 - x15
and then forcing a data abort.
Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.
Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.
Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.
Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.
Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.
An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.
Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.
Fixes: #26486
Related to: #23063#23062
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Enable interrupts before switching to main()
in cortex-m builds with single-thread mode
(CONFIG_MULTITHREADING=n).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the location counter ('.') is within the area that the veneers
should go, the current solution will give a linker error ("Cannot move
location counter backwards"). This patch places the veneers in the next
SPU region in this case.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
CPU Cortex-M implies Mainline Cortex-M, therfore, the dependency
on ARMV6_M_ARMV8_M_BASELINE is redundant and can be removed. The
change in this commit is a no-op.
We also add the ARMV6_M_ARMV8_M_BASELINE dependency on option
CPU_CORTEX_M0_HAS_VECTOR_TABLE_REMAP to make sure it cannot be
selected for non Cortex-M Baseline SoCs (at least, not without
a warning).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Using SCB_CleanInvalidateDcache instead of SCB_DisableDcache
& SCB_EnableDcache when config the non-cache area, in case
of the cache will effect the configuration of the non-cache
area
Signed-off-by: Crist Xu <crist.xu@nxp.com>
Work around an issue where the emulator ignores host OS
signals when inside a `wfi` instruction.
This should be reverted once this has been addressed in the
AARCH64 build of QEMU in the SDK.
See https://github.com/zephyrproject-rtos/sdk-ng/issues/255
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When _arch_switch() API is used, the tracing of the thread swapped out
is done in the C kernel code (in do_swap() for cooperative scheduling
and in set_current() during preemption). In the assembly code we only
have to trace the thread when swapped in.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Cortex-M SoCs implement (optionally) the Data Watchpoint and
Tracing Unit (DWT), which can be used for timing functions.
Select the corresponding ARCH capability if the SoC implements
the DWT.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.
For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.
Furthermore, much of the assembly code used had issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Switch to the _arch_switch() API that is required for an SMP-aware
scheduler instead of using the old arch_swap mechanism.
SMP is not supported yet but this is a necessary step in that direction.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Provide a TZ_SAFE_ENTRY_FUNC() macro for wrapping non-secure entry
functions in calls to k_sched_lock()/k_sched_unlock()
Provide a __TZ_WRAP_FUNC() macro which helps in creating a function
that "wraps" another in a preface and postface function call.
int foo(char *arg); // Implemented somewhere else.
int __attribute__((naked)) foo_wrapped(char *arg)
{
WRAP_FUNC(bar, foo, baz);
}
is equivalent to
int foo(char *arg); // Implemented somewhere else.
int foo_wrapped(char *arg)
{
bar();
int res = foo(arg);
baz();
return res;
}
This commit also adds tests for __TZ_WRAP_FUNC().
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
The same code was being copypasted in k_thread_abort()
implementations, just move into z_thread_single_abort().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This isn't needed; match the vanilla implementation
in kernel/thread_abort.c and do this unlocked. This
should improve system latency.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A check was being done that was a more obscure way of
calling arch_is_in_isr(). Add a comment explaining
why we need to trigger PendSV.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We implement an ARM-only API for ARM Secure Firmware,
to set all NVIC IRQ lines to target the Non-Secure state.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
we modify the ARM Cortex-M only API for managing the
security target state of the NVIC IRQs. We remove the
internal ASSERT checking allowing to call the API for
non-implemented NVIC IRQ lines. However we still give the
option to the user to check the success of the IRQ target
state setting operation by allowing the API function to
return the resulting target state.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Now that device_api attribute is unmodified at runtime, as well as all
the other attributes, it is possible to switch all device driver
instance to be constant.
A coccinelle rule is used for this:
@r_const_dev_1
disable optional_qualifier
@
@@
-struct device *
+const struct device *
@r_const_dev_2
disable optional_qualifier
@
@@
-struct device * const
+const struct device *
Fixes#27399
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
To debug hard-to-reproduce faults/panics, it's helpful to get the full
register state at the time a fault occurred. This enables recovering
full backtraces and the state of local variables at the time of a
crash.
This PR introduces a new Kconfig option, CONFIG_EXTRA_EXCEPTION_INFO,
to facilitate this use case. The option enables the capturing of the
callee-saved register state (r4-r11 & exc_return) during a fault. The
info is forwarded to `k_sys_fatal_error_handler` in the z_arch_esf_t
parameter. From there, the data can be saved for post-mortem analysis.
To test the functionality a new unit test was added to
tests/arch/arm_interrupt which verifies the register contents passed
in the argument match the state leading up to a crash.
Signed-off-by: Chris Coleman <chris@memfault.com>
Saves us a few bytes of program text on arches that don't need
these implemented, currently all uniprocessor MPU-based systems.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
All of these should be no-ops for the following reasons:
1. User threads cannot configure memory domains, only supervisor
threads.
2. The scope of memory domains is user thread memory access,
supervisor threads can access the entire memory map.
Hence it's never required to reprogram the MPU when a memory domain
API is called.
Fixes a problem where an assertion would fail if a supervisor thread
added a partition and then immediately removes it, and possibly
other problems.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for ARM Cortex-M.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Make explicit what registers we are going to be touched / modified when
using z_arm64_enter_exc and z_arm64_exit_exc.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The default implementation is the same as this custom
one now, as the assertion that the context switch occurs
at the end of the ISR is true for all arches.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
unify how XIP is configured across architectures. Use imply instead of
setting defaults per architecture and imply XIP on riscv arch and remove
XIP configuration from individual defconfig files to match other
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit adds the support for HW Stack Protection when
building Zephyr without support for multi-threading. The
single MPU guard (if the feature is enabled) is set to
guard the Main stack area. The stack fail check is also
updated.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
For the case of building Zephy with no-multithreading
support (CONFIG_MULTITHREADING=n) we introduce a
custom (ARCH-specific) function to switch to main()
from cstart(). This is required, since the Cortex-M
initialization code is temporarily using the interrupt
stack and main() should be using the z_main_stack,
instead. The function performs the PSP switching,
the PSPLIM setting (for ARMv8-M), FPU initialization
and static memory region initialization, to mimic
what the normal (CONFIG_MULTITHREADING=y) case does.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We extract the common code for both multithreading and
non-multithreading cases into a common static function
which will get called in Cortex-M archictecture initialization.
This commit does not introduce behavioral changes.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This patch is simply adding the guard area (if applicable) to
the calculations for the size of the interrupt stack in reset.S
for ARM Cortex-M architecture. If exists, the GUARD area is
always reserved aside from CONFIG_ISR_STACK_SIZE, since the
interrupt stack is defined using the K_KERNEL_STACK_DEFINE.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.
Two new arch defines are introduced:
- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN
New public declaration macros:
- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF
If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.
Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This now takes a stack pointer as an argument with TLS
and random offsets accounted for properly.
Based on #24467 authored by Flavio Ceolin.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The core kernel computes the initial stack pointer
for a thread, properly aligning it and subtracting out
any random offsets or thread-local storage areas.
arch_new_thread() no longer needs to make any calculations,
an initial stack frame may be placed at the bounds of
the new 'stack_ptr' parameter passed in. This parameter
replaces 'stack_size'.
thread->stack_info is now set before arch_new_thread()
is invoked, z_new_thread_init() has been removed.
The values populated may need to be adjusted on arches
which carve-out MPU guard space from the actual stack
buffer.
thread->stack_info now has a new member 'delta' which
indicates any offset applied for TLS or random offset.
It's used so the calculations don't need to be repeated
if the thread later drops to user mode.
CONFIG_INIT_STACKS logic is now performed inside
z_setup_new_thread(), before arch_new_thread() is called.
thread->stack_info is now defined as the canonical
user-accessible area within the stack object, including
random offsets and TLS. It will never include any
carved-out memory for MPU guards and must be updated at
runtime if guards are removed.
Available stack space is now optimized. Some arches may
need to significantly round up the buffer size to account
for page-level granularity or MPU power-of-two requirements.
This space is now accounted for and used by virtue of
the Z_THREAD_STACK_SIZE_ADJUST() call in z_setup_new_thread.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
MISRA-C wants the parameter names in a function implementaion
to match the names used by the header prototype.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This interface is documented already in
kernel/include/kernel_arch_interface.h
Other architectural notes were left in place except where
they were incorrect (like the thread struct
being in the low stack addresses)
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
arch_new_thread() passes along the thread priority and option
flags, but these are already initialized in thread->base and
can be accessed there if needed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In CPUs with VTOR we are free to place the relay vector table
section anywhere inside ROM_START section (as long as we respect
alignment requirements). This PR moves the relay table towards
the end of ROM_START. This leaves sufficient area for placing
some SoC-specific sections inside ROM_START that need to start
at a fixed address.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
_vector_table and __vector_relay_table symbols were exported with GTEXT
(i.e. as functions). That resulted in bit[0] being incorrectly set in
the addresses they represent (for functions this bit set to 1 specifies
execution in Thumb state).
This commit corrects this by switching to exporting these objects as
objects, i.e. with GDATA.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
MISRA-C directive 4.10 requires that files being included must
prevent itself from being included more than once. So add
include guards to the offset files, even though they are C
source files.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Race conditions exist when remapping the NXP MPU. When writing the
start, end, or attribute registers of a MPU descriptor, the hardware
will automatically clear the region's valid bit. If that region gets
accessed before the code is able to set the valid bit, the core will
fault.
Issue #20595 revealled this problem with the code in region_init()
when the compiler options are set to no optimizations. The code
generated by the compiler put local variables on the stack and then
read those stack based variables when writing the MPU descriptor
registers. If that region mapped the stack a memory fault would occur.
Higher compiler optimizations would store these local variables in
CPU registers which avoided the memory access when programming the
MPU descriptor.
Because the NXP MPU uses a logic OR operation of the MPU descriptors,
the fix uses the last descriptor in the MPU hardware to remap all of
dynamic memory for access instead of the first of the dynamic memory
descriptors as was occuring before. This allows reprogramming of the
primary discriptor blocks without having a memory fault. After all
the dynamic memory blocks are mapped, the unused blocks will have
their valid bits cleared including this temporary one, if it wasn't
alread changed during the mapping of the current set.
Fixes#20595
Signed-off-by: David Leach <david.leach@nxp.com>
Zephyr applications will always use the VTOR register when it is
available on the CPU and the register will always be configured
to point to applications vector table during startup.
SW_VECTOR_RELAY_CLIENT is meant to be used only on baseline ARM cores.
SW_VECTOR_RELAY is intended to be used only by the bootloader.
The bootloader may configure the VTOR to point to the relay table
right before chain-loading the application.
Signed-off-by: Rafał Kuźnia <rafal.kuznia@nordicsemi.no>
Select either SW_VECTOR_RELAY or SW_VECTOR_RELAY_CLIENT
at the time.
Removed #ifdef-ry in irq_relay.S as SW_VECTOR_RELAY was
refined so it became reserved for the bootloader and it
conditionally includes irq_relay.S for compilation.
See SHA #fde3116f1981cf152aadc2266c66f8687ea9f764
Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
Signed-off-by: Rafał Kuźnia <rafal.kuznia@nordicsemi.no>
This patch allows the `SW_VECTOR_RELAY` and
`SW_VECTOR_RELAY_CLIENT` pair to be
enabled on the ARMv7-M and ARMv8-M architectures
and covers all additional interrupt vectors.
Signed-off-by: Rafał Kuźnia <rafal.kuznia@nordicsemi.no>
Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
Define vector relay tables for bootloader only.
If an image is not a bootloader image (such as an MCUboot image)
but it is a standard Zephyr firmware, chain-loadable by a
bootloader, then this image will not need to relay IRQs itself.
In this case SW_VECTOR_RELAY_CLIENT should be used to setting the
vector table pointer in RAM so the parent image can forward the
interrupts to it.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Co-authored-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
Create macro for TCR_PS_BITS instead of programmatically looking up
a static value based on a CONFIG option. Moving to macro
removes logically dead code reported by Coverity static analysis tool.
Signed-off-by: Scott Branden <scott.branden@broadcom.com>
The workaround for ARMv7-M architecture (which proactively
decreases the available thread stack by the size of the MPU
guard) needs to be placed before we calculate the pointer of
the user-space local thread data, otherwise this pointer will
fall beyond the boundary of the thread stack area.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We fix (by inverting) the logic of the IS_MPU_GUARD_VIOLATION
macro, with respect to the value of the supplied 'fault_addr'.
We shall only be inspecting the fault_addr value if it is not
set to -EINVAL.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
It is possible that MMFAR address is not written by the
Cortex-M core; this occurs when the stacking error is
not accompanied by a data access violation error (i.e.
when stack overflows due to the exception entry frame
stacking): z_check_thread_stack_fail() shall be able to
handle the case of 'mmfar' holding the -EINVAL value.
Add this node in mem_manage_fault() function to clarify
that it is valid for z_check_thread_stack_fail() to be
called with invalid mmfar address value.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Some wires were crossed when an older PR was merged that
had build conflicts with newer code. Update this header
to reflect were the 'nested' member is in the kernel CPU
struct.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit renames the Kconfig `FP_SHARING` symbol to `FPU_SHARING`,
since this symbol specifically refers to the hardware FPU sharing
support by means of FPU context preservation, and the "FP" prefix is
not fully descriptive of that; leaving room for ambiguity.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit cleans up the section name definitions in the linker
sections header file (`include/linker/sections.h`) to have the uniform
format of `_(SECTION)_SECTION_NAME`.
In addition, the scope of the short section reference aliases (e.g.
`TEXT`, `DATA`, `BSS`) are now limited to the ASM code, as they are
currently used (and intended to be used) only by the ASM code to
specify the target section for functions and variables, and these short
names can cause name conflicts with the symbols used in the C code.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit renames the Kconfig `FLOAT` symbol to `FPU`, since this
symbol only indicates that the hardware Floating Point Unit (FPU) is
used and does not imply and/or indicate the general availability of
toolchain-level floating point support (i.e. this symbol is not
selected when building for an FPU-less platform that supports floating
point operations through the toolchain-provided software floating point
library).
Moreover, given that the symbol that indicates the availability of FPU
is named `CPU_HAS_FPU`, it only makes sense to use "FPU" in the name of
the symbol that enables the FPU.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This operation is formally defined as rounding down a potential
stack pointer value to meet CPU and ABI requirments.
This was previously defined ad-hoc as STACK_ROUND_DOWN().
A new architecture constant ARCH_STACK_PTR_ALIGN is added.
Z_STACK_PTR_ALIGN() is defined in terms of it. This used to
be inconsistently specified as STACK_ALIGN or STACK_PTR_ALIGN;
in the latter case, STACK_ALIGN meant something else, typically
a required alignment for the base of a stack buffer.
STACK_ROUND_UP() only used in practice by Risc-V, delete
elsewhere.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The core kernel z_setup_new_thread() calls into arch_new_thread(),
which calls back into the core kernel via z_new_thread_init().
Move everything that doesn't have to be in z_new_thread_init() to
z_setup_new_thread() and convert to an inline function.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit reworks the ARM AArch32 non-Cortex-M (i.e. Cortex-A and
Cortex-R) exception handling to establish the base exception handling
framework and support detailed exception information reporting.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
In the ARM Cortex-M architecture implementation, the concepts of
"exceptions" and "interrupts" are interchangeable; whereas, in the
Cortex-A/-R architecture implementation, they are considered separate
and therefore handled differently (i.e. `z_arm_exc_exit` cannot be used
to exit an "interrupt").
This commit fixes all `z_arm_exc_exit` usages in the interrupt handlers
to use `z_arm_int_exit`.
NOTE: In terms of the ARM AArch32 Cortex-A and Cortex-R architecture
implementations, the "exceptions" refer to the "Undefined
Instruction (UNDEF)" and "Prefetch/Data Abort (PABT/DABT)"
exceptions, while "interrupts" refer to the "Interrupt (IRQ)",
"Fast Interrupt (FIQ)" and "Software Interrupt/Supervisor Call
(SWI/SVC)".
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The exception/fault handling mechanisms for the ARM Cortex-M and the
rest (i.e. Cortex-A and Cortex-R) are significantly different and there
is no benefit in having the two implementations in the same file.
This commit relocates the Cortex-M fault handler to
`cortex_m/fault_s.S` and the Cortex-A/-R generic exception handler to
`cortex_a_r/exc.S` (note that the Cortex-A and Cortex-R architectures
do not provide direct fault vectors; instead, they provide the
exception vectors that can be used to handle faults).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The amount of shared code in exc_exit.S between the ARM Cortex-M and
the rest (i.e. Cortex-A and Cortex-R) is minimal and there is little
benefit in having the two implementations in the same file.
This commit splits the interrupt/exception exit code for the
Cortex-A/-R and Cortex-M into separate files to improve readability as
well as maintainability.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Use calee saved register to preserve value accoss sequence.
Procedure calls are mandated to follow ABI spec and preserve
x19 to x29.
Signed-off-by: Sandeep Tripathy <sandeep.tripathy@broadcom.com>
Zephyr being an OS is typically expected to run at EL1. Arm core
can reset to EL3 which typically requires a firmware to run at EL3
and drop control to lower EL. In that case EL3 init is done by the
firmware allowing the lower EL software to have necessary control.
If Zephyr is entered at EL3 and it is desired to run at EL1, which
is indicated by 'CONFIG_SWITCH_TO_EL1', then Zephyr is responsible
for doing required EL3 initializations to allow lower EL necessary
control.
The entry sequence is modified to have control flow under single
'switch_el'.
Provisions added by giving weak funcions to do platform specific
init from EL3.
Signed-off-by: Sandeep Tripathy <sandeep.tripathy@broadcom.com>
Single mov instruction can not be used to move non-zero
64b immediate value to the 64b register.
Implement macro to generate mov/ movk and movz sequences
depending on immediate value width.
Signed-off-by: Sandeep Tripathy <sandeep.tripathy@broadcom.com>
To remove the need to have DT_NUM_IRQ_PRIO_BITS defined in every
dts_fixup.h we can just handle the few variant cases in irq.h. This
allows us to remove DT_NUM_MPU_REGIONS from all the dts_fixup.h files.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
To remove the need to have DT_NUM_MPU_REGIONS defined in every
dts_fixup.h we can just handle the few variant cases in arm_mpu.c
directly. This allows us to remove DT_NUM_MPU_REGIONS from all the
dts_fixup.h files.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The current implementation to preserve r0 and r3 registers around the
call to `read_timer_end_of_isr` function has the following problems:
1. STM and LDM mnemonics are used without proper suffixes, in attempt
to implement PUSH and POP (i.e. STMFD and LDMFD). The suffix-less
STM mnemonic is equivalent to STMEA (increment after), which clearly
is not a PUSH operation, and this corrupts the interrupt stack,
leading to crashes on the Cortex-R.
2. The current implementation unnecessarily preserves additional r1, r2
and lr registers. There is no need to preserve r1 and r2 because the
values contained in these registers are not used after the function
call; as for the lr register, it is already pushed to the stack when
the interrupt service routine enters.
This commit removes all the unnecessary register preservations and
fixes the incorrect STM and LDM usages.
Note that the PUSH and POP aliases are used in place of the STMFD and
LDMFD mnemonics because they are used throughout the rest of the code.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The selection of the Cortex M systick driver to be used as a system
clock driver is controlled by CONFIG_CORTEX_M_SYSTICK.
To replace it by another driver CONFIG_CORTEX_M_SYSTICK must be set
to 'n'. Unfortunately this also controls the interrupt vector for
the systick interrupt. It is now routed to z_arm_exc_spurious.
Remove the dependecy on CONFIG_CORTEX_M_SYSTICK and route to
z_clock_isr as it was before #24012.
Fixes#24347
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
The ARM architecture root directory contains `aarch32.cmake` and
`aarch64.cmake` files whose contents are better suited to go into other
more purpose-specific files.
This commit removes the aforementioned files and moves their contents
to other files following the convention used by other architectures.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit adds the GCC `-march` flag for the ARM Cortex-R5 targets.
Note that `armv7-r+idiv` must be specified instead of `armv7-r`,
because the GCC internally resolves `-mcpu=cortex-r5` to it.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This is a minor clean-up for the ARM architecture configurations.
Note that the `CPU_CORTEX_A` symbol is moved from the AArch64 to the
ARM root Kconfig because it can be selected from both AArch32 and
AArch64.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Those are used only in tests, so remove them from kernel Kconfig and set
them in the tests that use them directly.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fix documentation in kernel_arch_data.h and kernel_arch_func.h
headers for ARM, to indicate that these are common headers for
all ARM architecture variants.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
In the Cortex-M exception table we rename z_arm_reserved()
function to z_arm_exc_spurious(), as it is invoked when
existing (that is, non-reserved) but un-installed exceptions
are triggered, accidentaly, by software, or hardware. This
currently applies to SysTick and SecureFault exceptions.
Since fault.S is shared between Cortex-M and other AARCH32
architectures, we keep z_arm_reserved as a defined symbol
there. This commit does some additional, minor, "no-op"
cleanup in #ifdef's for Cortex-M and Cortex-R.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the Cortex-M core does not implement the Security Extension,
we should not be adding z_arm_reserved in the corresponding
vector table entry. That is because the entry is reserved by
the ARM architecture.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the Cortex-M core does not implement the SysTick peripheral,
we should not be adding z_arm_reserved in the corresponding
vector table entry. If we do have SysTick implemented but we
are not using it as the system timer, we shall install the
reserved interrupt at the vector table entry.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Write 0x0 instead of z_arm_reserved to vector exception
entries that are always reserved for future use by the
ARM architecture. These vector table entries cannot be
fetched to be executed by the Cortex-M exception entry,
so having z_arm_reserved gives a false impression, since
it is a function that may be invoked in the code. This
modification is safe since these vector entries are also
not supposed to be read / written by the code.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The current context preservation implementation saves the spsr and
lr_irq registers, which contain the cpsr and pc register values of the
interrupted context, in the thread callee-saved block and this prevents
nesting of interrupts because these values are required to be part of
the exception stack frame to preserve the nested interrupt context.
This commit reworks the AArch32 non-Cortex-M context preservation
implementation to save the spsr and lr_irq registers in the exception
stack frame to allow preservation of the nested interrupt context as
well as the interrupted thread context.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
In the current implementation both SPSR and ELR registers are saved with
the callee-saved registers and restored by the context-switch routine.
To support nested IRQs we have to save those on the stack when entering
and exiting from an ISR.
Since the values are now carried on the stack we can now add those to
the ESF and the initial stack and take care to restore them for new
threads using the new thread wrapper routine.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Out-of-tree code can still be using the old file locations. Introduce
header shims to include the headers from the new correct location and
print a warning message.
These shims should be removed after two releases.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit renames the `cortex_r` directory under the AArch32 to
`cortex_a_r`, in preparation for the AArch32 Cortex-A support.
The rationale for this renaming is that the Cortex-A and Cortex-R share
the same base design and the difference between them, other than the
MPU vs. MMU, is minimal.
Since most of the architecture port code and configurations will be
shared between the Cortex-A and Cortex-R architectures, it is
advantageous to have them together in the same directory.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit re-organises AArch32 configurations for consistency.
1. Move Cortex-M-specific includes to `cortex_m/Kconfig`.
2. Relocate the "TrustZone" configurations to `cortex_m/tz/Kconfig`
since these are really the TrustZone-M configurations and do not
apply to the TrustZone-A.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Rewrite the comments for the swap routine removing the references to the
old aarch32 code and rename z_arm64_pendsv() ->
z_arm64_context_switch().
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Rework the assembly code for the ISR wrapper and SVC to share the
entry/exit code that is currently scattered amoung several files /
places. No functional changes.
Rename also macro.h -> macro.inc to fool the CI.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
z_CpuIdleInit has been renamed to z_arm_cpu_idle_init, so
we need to correct that function's name in the documentation
of arch_cpu_atomic_idle.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit adds the required memory barriers to the `arch_cpu_idle`
function in order to ensure proper idle operation in all cases.
1. Add ISB after setting BASEPRI to ensure that the new wake-up
interrupt priority is visible to the WFI instruction.
2. Add DSB before WFI to ensure that all memory transactions are
completed before going to sleep.
3. Add ISB after CPSIE to ensure that the pending wake-up interrupt
is serviced immediately.
Co-authored-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The current AArch32 `arch_cpu_idle` implementation enables interrupt
before executing the WFI instruction, and this has the side effect of
allowing interruption and thereby calling wake-up notification
functions before the CPU enters sleep.
This commit fixes the problem described above by ensuring that
interrupt is disabled when the WFI instruction is executed and
re-enabled only after the processor wakes up.
For ARMv6-M, ARMv8-M Baseline and ARM-R, the PRIMASK (ARM-M)/
CPSR.I (ARM-R) is used to lock interrupts and therefore it is not
necessary to do anything before executing the WFI instruction.
For ARMv7-M and ARMv8-M Mainline, the BASEPRI is used to lock
interrupts and the PRIMASK is always cleared in non-interrupt context;
therefore, it is necessary to set the PRIMASK to mask interrupts,
before clearing the BASEPRI to configure wake-up interrupt priority to
the lowest.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This never needed to be put in a separate gperf table.
Privilege mode stacks can be generated by the main
gen_kobject_list.py logic, which we do here.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The nRF53 has different region size than nRF91.
This patch is aware of Erratum 19 (wrong SPU region size).
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
The set of interrupt stacks is now expressed as an array. We
also define the idle threads and their associated stacks this
way. This allows for iteration in cases where we have multiple
CPUs.
There is now a centralized declaration in kernel_internal.h.
On uniprocessor systems, z_interrupt_stacks has one element
and can be used in the same way as _interrupt_stack.
The IRQ stack for CPU 0 is now set in init.c instead of in
arch code.
The extern definition of the main thread stack is now removed,
this doesn't need to be in a header.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
z_arm_exc_exit (z_arm_int_exit) requires the current execution mode to
be specified as a parameter (through r0). This is not necessary because
this value can be directly read from CPSR.
This commit modifies the exception return function to retrieve the
current execution mode from CPSR and removes all provisions for passing
the execution mode parameter.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The current AArch64 interrupt system relies on the multi-level
interrupt mechanism and the `irq_nextlevel` public interface to invoke
the Generic Interrupt Controller (GIC) driver functions.
Since the GIC driver has been refactored to provide a direct interface,
in order to resolve various implementation issues described in the GIC
driver refactoring commit, the architecture interrupt control functions
are updated to directly invoke the GIC driver functions.
This commit also adds support for the ARMv8 cores (e.g. Cortex-A53)
that allow interfacing to a custom external interrupt controller
(i.e. non-GIC) by mapping the architecture interrupt control functions
to the SoC layer interrupt control functions when
`ARM_CUSTOM_INTERRUPT_CONTROLLER` configuration is enabled.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The current AArch32 (Cortex-R and to-be-added Cortex-A) interrupt
system relies on the multi-level interrupt mechanism and the
`irq_nextlevel` public interface to invoke the Generic Interrupt
Controller (GIC) driver functions.
Since the GIC driver has been refactored to provide a direct interface,
in order to resolve various implementation issues described in the GIC
driver refactoring commit, the architecture interrupt control functions
are updated to directly invoke the GIC driver functions.
This commit also adds support for the Cortex-R cores (Cortex-R4 and R5)
that allow interfacing to a custom external interrupt controller
(i.e. non-GIC) by introducing the `ARM_CUSTOM_INTERRUPT_CONTROLLER`
configuration that maps the architecture interrupt control functions to
the SoC layer interrupt control functions.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
We rename the z_arm_int_lib_init() function to
z_arm_interrupt_init(), aligning to how other
ARCHes name their IRQ initialization function.
There is nothing about 'library' in this
functionality, so we remove the 'lib' in-fix.
The commit does not introduce any behavior changes.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We align the implementation of z_irq_spurious() handler
with the other Zephyr ARCHEs, i.e. we will be calling
directly the ARM-specific fatal error function with
K_ERR_SPURIOUS_IRQ as the error type. This is already
the case for aarch64.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Correct documentation note in z_irq_spurious() definition,
stressing that the function is installed in _sw_isr_table
entries at boot time (which may be or not be used for
dynamic interrupts).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The ARMv7-R architecture supports both Thumb-2 (T32) and ARM (A32)
instruction sets.
This commit selects the `ISA_THUMB2` symbol to indicate that the
ARMv7-R architecture supports the Thumb-2 instruction set, which can
be enabled by selecting the `COMPILER_ISA_THUMB2` symbol.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit introduces the `COMPILER_ISA_THUMB2` symbol to allow
choosing either the ARM or Thumb instruction set for C code
compilation.
In addition, this commit introduces the `ASSEMBLER_ISA_THUMB2` helper
symbol to specify the default target instruction set for the assembler.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This function is widely used by functions that validate memory
buffers. Macros used to check permissions, like Z_SYSCALL_MEMORY_READ
and Z_SYSCALL_MEMORY_WRITE, use these functions to check that a
pointers passed by user threads in a syscall.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
We need an unsigned comparison when evaluating whether
the supplied syscall ID is lower than the syscall ID limit.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The ARMv7-M MPU requires power-of-two alignment, not the ARMv8-M MPU, as
noted a few lines later.
Signed-off-by: Anders Montonen <Anders.Montonen@iki.fi>
Upon reset, the CONTROL.FPCA bit is, normally, cleared. However,
it might be left un-cleared by firmware running before Zephyr boot,
for example when Zephyr image is loaded by another image.
We must clear this bit to prevent errors in exception unstacking.
This caused stack offset when booting from a build-in EFM32GG bootloader
Fixes#22977
Signed-off-by: Luuk Bosma <l.bosma@interay.com>
Upon reset, the Co-Processor Access Control Register is, normally,
0x00000000. However, it might be left un-cleared by firmware running
before Zephyr boot.
This restores the register back to reset value, even if CONFIG_FLOAT
is not set.
Clearing before setting supports switching between Full access
and Privileged access only.
Refactor enable_floating_point to support initialize
floating point registers for every CPU that has a FPU.
Signed-off-by: Luuk Bosma <l.bosma@interay.com>
Add zephyr execution regions(text, rodata, data, noinit, bss, etc.)
with proper attributes to translation tables.
Linker script has been modified a little to align these sections to
minimum translation granule(4 kB).
With this in place, code cannot be overwritten accidently as it is
marked read only. Similarly, execution is prohibited from data/RW
section as it is marked execute-never.
Signed-off-by: Abhishek Shah <abhishek.shah@broadcom.com>
Add MMU support for ARMv8A. We support 4kB translation granule.
Regions to be mapped with specific attributes are required to be
at least 4kB aligned and can be provided through platform file(soc.c).
Signed-off-by: Abhishek Shah <abhishek.shah@broadcom.com>
We lock IRQs around writing to RNR and immediate reading of RBAR
RASR in ARMv7-M MPU driver. We do this for the functions invoked
directly or undirectly by arch_buffer_validate(). This locking
guarantees that
- arch_buffer_validate() calls by ISRs may safely preempt each
other
- arch_buffer_validate() calls by threads may safely preempt
each other (i.e via context switch -out and -in again).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When entering user mode, and before the privileged are dropped,
the thread switches back to using its default (user) stack. For
stack limit checking not to lead to a stack overflow, the PSPLIM
and PSP register updates need to be done with PendSV IRQ locked.
This is because context-switch (done in PendSV IRQ) reprograms
the stack pointer limit register based on the current PSP
of the thread. This commit enforces PendSV locking and
unlocking while reprogramming PSP and PSPLIM when switching to
user stack at z_arm_userspace_enter().
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Modifying the PSP via an MSR instruction is not subject to
stack limit checking so we can remove the relevant code
block in the begining of z_arm_userspace_enter(), which clears
PSPLIM. We add a comment when setting the PSP to the privilege
stack to stress that clearing the PSPLIM is not required and it
is always a safe operation.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When returning from a system call, the thread switches back
to using its default (user) stack. For stack limit checking
not to lead to a stack overflow, the updates of PSPLIM and
PSP registers need to be done with PendSV IRQ locked. This
is because context-switch (done in PendSV IRQ) reprograms
the stack pointer limit register based on the current PSP
of the thread. This commit enforces PendSV locking and
unlocking while reprogramming PSP and PSPLIM when returning
from a system call.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
In this commit we remove the PSPLIM clearing when entering
z_arm_do_syscall(), since we want PSPLIM to keep guarding
the user thread stack, until the thread has switched to its
privileged stack, for executing the system call.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Thread will be in privileged mode after returning from SCVall. It
will use the default (user) stack before switching to the privileged
stack to execute the system call. We need to protect the user stack
against stack overflows until this stack transition. We update the
note in z_arm_do_syscall(), stating clearly that it executing with
stack protection when building with stack limit checking support
(ARMv8-M only).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When configuring the built-in stack guard, via setting the
PSPLIM register, during thread context-switch, we shall only
set PSPLIM to "guard" the thread's privileged stack area when
the thread is actually using it (PSP is on this stack).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We do not need to have the PSPLIM clearing directly inside
the PendSV handler and outside the function that configures
it, configure_builtin_stack_guard(), since the latter is also
invoked inside the PendSV handler. This commit moves the
PSPLIM clearing inside configure_builtin_stack_guard(). The
patch is not introducing any behavioral change on the
stack limit checking mechanism for Cortex-M.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We add the mechanism to generate offset #defines for
thread stack info start, to be used directly in ASM.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We introduce a macro to define the IRQ priority level for
PendsV, and use it in arch/arm/include/aarch32/exc.h
to set the PendSV IRQ level. The commit does not change
the behavior of PendSV interrupt.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit adds some documentation for the exception
priority scheme for 32-bit ARM architecture variants.
In addition we document that SVCall priority level for
ARMv6-M is implicitly set to highest (by leaving it as
default).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Same deal as in commit eddd98f811 ("kconfig: Replace some single-symbol
'if's with 'depends on'"), for the remaining cases outside defconfig
files. See that commit for an explanation.
Will do the defconfigs separately in case there are any complaints
there.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This commit fixes incorrect Cortex-R interrupt lock, unlock and state
check function implementations.
The issues can be summarised as follows:
1. The current implementation of 'z_arch_irq_lock' returns the value
of CPSR as the IRQ key and, since CPSR contains many other state
bits, this caused 'z_arch_irq_unlocked' to return false even when
IRQ is unlocked. This problem is fixed by isolating only the I-bit
of CPSR and returning this value as the IRQ key, such that it
returns a non-zero value when interrupt is disabled.
2. The current implementation of 'z_arch_irq_unlock' directly updates
the value of CPSR control field with the IRQ key and this can cause
other state bits in CPSR to be corrupted. This problem is fixed by
conditionally enabling interrupt using CPSIE instruction when the
value of IRQ key is a zero.
3. The current implementation of 'z_arch_is_in_isr' checks the value
of CPSR MODE field and returns true if its value is IRQ or FIQ.
While this does not normally cause an issue, the function can return
false when IRQ offloading is used because the offload function
executes in SVC mode. This problem is fixed by adding check for SVC
mode.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
CUSTOM_SECTION_ALIGN is already defined within an 'if ARM_MPU', so it
does not need a 'depends on ARM_MPU'.
Flagged by https://github.com/zephyrproject-rtos/ci-tools/pull/128.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add TRACING_ISR Kconfig to help high latency backend working well.
Currently the ISR tracing hook function is put at the begining and
ending of ISR wrapper, when there is ISR needed in the tracing path
(especially tracing backend), it will cause tracing buffer easily
be exhausted if async tracing method enabled. Also it will increase
system latency if all the ISRs are traced. So add TRACING_ISR to
enable/disable ISR tracing here. Later a filter out mechanism based
on irq number will be added.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
Enable the shared IRQ for the UART line and enable the remaining tasks
that depends on a separated declaration of the TX/RX/Err/... IRQs.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The cmsis_rtos tests are failing because the stack size used by CMSIS is
too small. Customize the stack size for the aarch64 architecture and
re-enable the tests.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
ARMv8-A SoCs enter EL3 after reset. Add a new config option
(CONFIG_SWITCH_TO_EL1) to switch from EL3 to EL1 at boot and default it
to 'y'.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
While QEMU's Cortex-A53 emulation by default only emulates a CPU in EL1,
other QEMU forks (for example the QEMU released by Xilinx) and real
hardware starts in EL3.
To support all the ELn we introduce a macro to identify at run-time the
Exception Level and take the correct actions.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To be able to successfully compile the kernel for the ARM64 architecture
we have to tweak the compiler-related files to be able to use the
AArch64 GCC compiler.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Introduce the basic ARM64 architecture support.
A new CONFIG_ARM64 symbol is introduced for the new architecture and new
cmake / Kconfig files are added to switch between ARM and ARM64.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Dynamic MPU regions are used in build configurations with User
mode or MPU-based stack-overflow guards. If these features are
disabled, we skip calling the ARM function for re-programming
the MPU peripheral during context-switch. We also skip doing
this when jumping to main thread (although this brings limited
performace gain as it is called once in the boot cycle)
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
In zephyr_linker_sources().
This is done since the point of the location is to place things at given
offsets. This can only be done consistenly if the linker code is placed
into the _first_ section.
All uses of TEXT_START are replaced with ROM_START.
ROM_START is only supported in some arches, as some arches have several
custom sections before text. These don't currently have ROM_START or
TEXT_START available, but that could be added with a bit of refactoring
in their linker script.
No SORT_KEYs are changed.
This also fixes an error introduced when TEXT_START was added, where
TEXT_SECTION_OFFSET was applied to riscv's common linker.ld instead of
to openisa_rv32m1's specific linker.ld.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
openocd linker sections are not supposed to be part of the
vector table sections. Place the sections after we define
the _vector_end linker symbol.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit adds a Kconfig symbol for specifying whether the SoC
implements the CPU DWT feature.
The Data Watchpoint and Trace (DWT) is an optional debug unit for the
Cortex-M family cores (except ARMv6-M; i.e. M0 and M0+) that provides
watchpoints, data tracing and system profiling capabilities.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit addresses the following issues:
1. Add a new Kconfig configuration for specifying Dual-redundant Core
Lock-step (DCLS) processor topology.
2. Register initialisation is only required when Dual-redundant Core
Lock-step (DCLS) is implemented in hardware. This initialisation is
required on DCLS only because the architectural registers are in an
indeterminate state after reset and therefore the initial register
state of the two parallel executing cores are not guaranteed to be
identical, which can lead to DCCM detecting it as a hardware fault.
A conditional compilation check for this hardware configuration
using the newly added CONFIG_CPU_HAS_DCLS flag has been added.
3. The existing CPU register initialisation code did not take into
account the banked registers for every execution mode. The new
implementation ensures that all architectural registers of every
mode are initialised.
4. Add VFP register initialisation for when floating-point support is
enabled and the core is configured in DCLS topology. This
initialisation sequence is required for the same reason given in
the first issue.
5. Add provision for platform-specific initialisation on Cortex-R
using PLATFORM_SPECIFIC_INIT config and z_platform_init function.
6. Remove seemingly pointless and inadequately defined STACK_MARGIN.
Not only does it violate the 8-byte stack alignment rule, it does
not provide any form of real stack protection.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commits implements the support for dynamic direct
interrupts for the ARM Cortex-M architecture, and exposes
the support to the user as an ARM-only API.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
With this commit we add support for Dynamic Direct interrupts
for the ARM Cortex-M architecture. For that we introduce a new,
user-enabled, Kconfig symbol, DYNAMIC_DIRECT_INTERRUPTS.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The system power management handling code in the '_isr_wrapper' enables
interrupts by executing the 'cpsie i' instruction, which causes a
system crash on the Cortex-R devices because the Cortex-R arch port
does not support nested interrupts at this time.
This commit restricts the interrupt state manipulations in the system
power management code to the Cortex-M arch, in order to prevent
interrupt nesting on other AArch32 family archs (only Cortex-R for
now).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit enables the CMSIS-Core(R) processor interface driver for
the Cortex-R platforms by default.
The CMSIS-Core component provides a set of standard interface functions
to control the Cortex-R series processor cores and will be required by
the arch port as well as other CMSIS library components (e.g. CMSIS-DSP
and CMSIS-NN).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
A single menu within an if like
if FOO
menu "blah"
...
endmenu
endif
can be replaced with
menu "blah"
depends on FOO
...
endmenu
Fix up all existing instances.
Also remove redundant extra menus underneath 'menuconfig' symbols.
'menuconfig' already creates a menu.
Also remove the menu in arch/arm/core/aarch32/Kconfig around the
"Floating point ABI" choice. The choice depends on FLOAT, which depends
on CPU_HAS_CPU, so remove the 'depends on CPU_HAS_FPU' too.
Piggyback removing a redundant 'default n' for BME280.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Out-of-tree code can still be using the old file locations. Introduce
header shims to include the headers from the new correct location and
print a warning message.
Add also a new Kconfig symbol to suppress such warning.
The shim will go away after two releases, so make sure to adapt your
application for the new locations.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Before introducing the code for ARM64 (AArch64) we need to relocate the
current ARM code to a new AArch32 sub-directory. For now we can assume
that no code is shared between ARM and ARM64.
There are no functional changes. The code is moved to the new location
and the file paths are fixed to reflect this change.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Fix misspellings in docs (and Kconfig and headers processed into docs)
missed during regular reviews.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Move PLATFORM_SPECIFIC_INIT declaration from Cortex-M Kconfig to the
ARM arch Kconfig in order to make it available for all ARM variants.
The rationale is that there is really no good reason why
platform-specific initialisation should be a Cortex-M-specific feature
and that Cortex-R port is expected to utilise this in a near future.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit inlines arch_isr_direct_header function that was previously
placed in irq_manage.c for no good reason (possibly in relation to the
FIXME for #3056).
In addition, since the PR #20119 resolved the header circular
dependency issue described in the issue #3056, this commit removes the
references to it in the code.
The reason for not inlining _arch_is_direct_pm as the #3056 FIXME
suggests is that there is little to gain from doing so and there still
exists circular dependency for the headers required by this function
(#20119 only addresses kernel_structs.h, which is required for _current
and _kernel, which, in turn, is required for handling interrupt nesting
in many architectures; in fact, Cortex-A and Cortex-R port will require
it as well).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.
This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We allow the run-time, full paritioning of the SRAM space by the
ARMv8-M MPU driver to be an optional feature.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit moves the function mpu_configure_regions(.) from
arm_mpu_v7_internal.h to arm_mpu.c. The function is to be used
by the both ARMv7-M MPU driver, as well as the ARMv8-M MPU
driver (when it behaves like the ARMv7-M driver).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We introduce MPU_GAP_FILLING Kconfig option that instructs
the MPU driver to enforce a full SRAM partitioning, when it
programs the dynamic MPU regions (user thread stack, PRIV stack
guard and application memory domains) at context-switch. We
allow this to be configurable, in order to increase the number
of MPU regions available for application memory domain programming.
This option is introduced in arch/Kconfig, as it is expected
to serve as a cross-ARCH symbol. The option can be set by the
user during build configuration.
By not enforcing full partition, we may leave part of kernel
SRAM area covered only by the default ARM memory map. This
is fine for User Mode, since the background ARM map does not
allow nPRIV access at all. The difference is that kernel code
will be able to attempt fetching instructions from kernel SRAM
area without this leading directly to a MemManage exception.
Since this does not compromize User Mode, we make the skipping
of full partitioning the default behavior for the ARMv8-M MPU
driver. The application developer may be able to overwrite this.
In the wake of this change we update the macro definitions in
arm_core_mpu_dev.h that derive the maximum number of MPU regions
for application memory domains.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When compiling the components under the arch directory, the compiler
include paths for arch and kernel private headers need to be specified.
This was previously done by adding 'zephyr_library_include_directories'
to CMakeLists.txt file for every component under the arch directory,
and this resulted in a significant amount of duplicate code.
This commit uses the CMake 'include_directories' command in the root
CMakeLists.txt to simplify specification of the private header include
paths for all the arch components.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Use this short header style in all Kconfig files:
# <description>
# <copyright>
# <license>
...
Also change all <description>s from
# Kconfig[.extension] - Foo-related options
to just
# Foo-related options
It's clear enough that it's about Kconfig.
The <description> cleanup was done with this command, along with some
manual cleanup (big letter at the start, etc.)
git ls-files '*Kconfig*' | \
xargs sed -i -E '1 s/#\s*Kconfig[\w.-]*\s*-\s*/# /'
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Clean up space errors and use a consistent style throughout the Kconfig
files. This makes reading the Kconfig files more distraction-free, helps
with grepping, and encourages the same style getting copied around
everywhere (meaning another pass hopefully won't be needed).
Go for the most common style:
- Indent properties with a single tab, including for choices.
Properties on choices work exactly the same syntactically as
properties on symbols, so not sure how the no-indentation thing
happened.
- Indent help texts with a tab followed by two spaces
- Put a space between 'config' and the symbol name, not a tab. This
also helps when grepping for definitions.
- Do '# A comment' instead of '#A comment'
I tweaked Kconfiglib a bit to find most of the stuff.
Some help texts were reflowed to 79 columns with 'gq' in Vim as well,
though not all, because I was afraid I'd accidentally mess up
formatting.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In the cortex-r port we are currently using GIC as a fake cascade
controller hooked to a fake parent IRQ #0. And in gic_init() we use
IRQ_CONNECT() to connect this dummy IRQ.
Unfortunately this value is shifted and offset when calling
irq_set_priority_next_level() that tries to set the IRQ priority on a
value of 0xffffffff.
This value is offset again in gic_irq_set_priority() that actually sets
the priority on the PPI #31.
Fix this avoiding to set any priority for IRQ #0.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
