HAS_DTS has become a redundant option. All Zephyr architectures now
select this option, meaning devicetree has become a de-facto
requirement. In fact, if any board does not provide a devicetree
source, the build system uses an empty stub, meaning the devicetree
machinery always runs.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
_k_neg_eagain is there for used in assembly where including
errno.h is not possible. However, the usage in ARM was simply
to assign value to swap_return_value in a C file, which is
no need to use _k_neg_eagain as errno.h can be included.
So change that to use -EAGAIN directly. Saves 4 bytes in
built binaries in rodata.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The exclusive load/store instructions don't work well when MMU and cache
are disabled on some cores e.g. Cortex-A72. Change it to voting lock[1]
to select the primary core when multi-cores boot simultaneously.
The voting lock has reasonable but minimal requirements on the memory
system.
[1] https://www.kernel.org/doc/html/next/arch/arm/vlocks.html
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This provides custom memory range check functions as
it gets a bit complicated with cached/uncached regions.
These functions are marked as __weak so SoC or board
can override these if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There were a few small nits that were pointed out in the initial PR.
Fixes the LOG_ macro in the arm elf implementation, replaces a few stray
mentions of modules in llext.h with extensions.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prepare data for upcoming
call to z_arm_on_enter_cpu_idle(). The main difference is that
z_arm_on_enter_cpu_idle_prepare() hook is called before interrupts
are disabled.
Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
Adds the linkable loadable extensions (llext) subsystem which provides
functionality for reading, parsing, and linking ELF encoded executable
code into a managed extension to the running elf base image.
A loader interface, and default buffer loader implementation,
make available to the llext subsystem the elf data. A simple management
API provide the ability to load and unload extensions as needed. A shell
interface for extension loading and unloading makes it easy to try.
Adds initial support for armv7 thumb built elfs with very specific
compiler flags.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Co-authored-by: Chen Peng1 <peng1.chen@intel.com>
Co-authored-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Armv8-m baseline support various instruction carrying exclusive-monitor and
acquire-release semantic. By adding this guard we let armv8-m.baseline
fall-back to arch defined or compiler built-in support for atomic
operations.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Add an ARCH_SUPPORTS_ROM_START kconfig symbol to mark architectures
that support ROM_START as an argument to zephyr_linker_sources.
This was added so that features relying on this feature could
depend on this kconfig symbol.
Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
This adds support for using coredump with Xtensa DC233C core,
which are being used by qemu_xtensa and qemu_xtensa_mmu.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The old CONFIG_UART_NS16550_ACCESS_IOPORT has been used to
indicate whether to access the NS16550 UART via IO port
before device tree is used to describe hardware. Now we have
device tree, and we can specify whether a particular UART
needs to be accessed via IO port using property io-mapped.
Therefore, CONFIG_UART_NS16550_ACCESS_IOPORT is no longer
needed (and thus also CONFIG_UART_NS16550_SIMULT_ACCESS).
Remove these two kconfigs and modify code to use device tree
to figure out how to access the UART hardware.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_switch() is basically an alias to xtensa_switch() so
we can mark arch_switch() as ALWAYS_INLINE to avoid another
function call, especially when no optimization is used when
debugging.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Let's make this official: we use the suffix `_MASK` for the define
carrying the GENMASK for the attributes, and the suffix `_GET(x)` for
the actual macro extracting the attributes.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To make the stack guard works well, clean and refine the MPU code. To
save the MPU regions (the number of MPU regions are limited), we choose
to remove the guard region. Comparing to add an individual region to
guard the stack, removing the guard region can at least save 2 regions
per core.
Similarly with userspace, the stack guard will leverage the dynamic
regions switching mechanism which means we need a region switch during
the context switch. Otherwise, the other option is using stack guard
region, but this is very limited since the number of MPU regions is
limited.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Refactor the stack relevant macros to prepare to introduce the stack
guard. Also add comments about the changes related to stack layout.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add the stack check function z_arm64_stack_corruption_check at
z_arm64_fatal_error to handle the stack overflow triggered by the
hardware region.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce the ARM64_STACK_PROTECTION config. This option leverages the
MMU or MPU to cause a system fatal error if the bounds of the current
process stack are overflowed. This is done by preceding all stack areas
with a fixed guard region. The config depends on MPU for now since MMU
stack protection is not ready.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Clean the thread->arch during the arch_new_thread to avoid unexpected
behavior. If the thread struct is allocated from heap or in stack, the
data in thread->arch might be dirty.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Accessing mem before mmu or mpu init will cause a cache coherence issue.
To avoid such a problem, move the safe exception stack init function
after the mmu or mpu is initiated.
Also change the data section attribute from INNER_SHAREABLE to
OUTER_SHAREABLE. Otherwise there will be a cache coherence issue during
the memory regions switch. Because we are using background region to do
the regions switch, and the default background region is
OUTER_SHAREABLE, if we use INNER_SHAREABLE as the foreground region,
then we have to flush all cache regions to make sure the cached values
are right. However, flushing all regions is too heavy, so we set
OUTER_SHAREABLE to fix this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Dom0less is Xen mode without privileged domain. All guests are created
according to hypervisor device tree configuration on boot. Thus, there
is no Dom0 with console daemon, that usually manages console output
from domains.
Zephyr OS contains 2 serial drivers related to Xen hypervisor: regular
with console shared page and consoleio-based. The first one is for
setups with console daemon and usually was used for Zephyr DomU guests.
The second one previously was used only for Zephyr Dom0 and had
corresponding Kconfig options. But consoleio is also used as interface
for DomU output on Dom0less setups and should be configurable without
XEN_DOM0 option.
Add corresponding XEN_DOM0LESS config to Xen Kconfig files and proper
dependencies in serial drivers.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Co-authored-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This is the final step in making the `zephyr,memory-attr` property
actually useful.
The problem with the current implementation is that `zephyr,memory-attr`
is an enum type, this is making very difficult to use that to actually
describe the memory capabilities. The solution proposed in this PR is to
use the `zephyr,memory-attr` property as an OR-ed bitmask of memory
attributes.
With the change proposed in this PR it is possible in the DeviceTree to
mark the memory regions with a bitmask of attributes by using the
`zephyr,memory-attr` property. This property and the related memory
region can then be retrieved at run-time by leveraging a provided helper
library or the usual DT helpers.
The set of general attributes that can be specified in the property are
defined and explained in
`include/zephyr/dt-bindings/memory-attr/memory-attr.h` (the list can be
extended when needed).
For example, to mark a memory region in the DeviceTree as volatile,
non-cacheable, out-of-order:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_VOLATILE |
DT_MEM_NON_CACHEABLE |
DT_MEM_OOO )>;
};
The `zephyr,memory-attr` property can also be used to set
architecture-specific custom attributes that can be interpreted at run
time. This is leveraged, among other things, to create MPU regions out
of DeviceTree defined memory regions on ARM, for example:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-region = "NOCACHE_REGION";
zephyr,memory-attr = <( DT_ARM_MPU(ATTR_MPU_RAM_NOCACHE) )>;
};
See `include/zephyr/dt-bindings/memory-attr/memory-attr-mpu.h` to see
how an architecture can define its own special memory attributes (in
this case ARM MPU).
The property can also be used to set custom software-specific
attributes. For example we can think of marking a memory region as
available to be used for memory allocation (not yet implemented):
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_NON_CACHEABLE |
DT_MEM_SW_ALLOCATABLE )>;
};
Or maybe we can leverage the property to specify some alignment
requirements for the region:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_CACHEABLE |
DT_MEM_SW_ALIGN(32) )>;
};
The conventional and recommended way to deal and manage with memory
regions marked with attributes is by using the provided `mem-attr`
helper library by enabling `CONFIG_MEM_ATTR` (or by using the usual DT
helpers).
When this option is enabled the list of memory regions and their
attributes are compiled in a user-accessible array and a set of
functions is made available that can be used to query, probe and act on
regions and attributes, see `include/zephyr/mem_mgmt/mem_attr.h`
Note that the `zephyr,memory-attr` property is only a descriptive
property of the capabilities of the associated memory region, but it
does not result in any actual setting for the memory to be set. The
user, code or subsystem willing to use this information to do some work
(for example creating an MPU region out of the property) must use either
the provided `mem-attr` library or the usual DeviceTree helpers to
perform the required work / setting.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Add Xen domctl API implementation for Zephyr as control domain.
Previously Zephyr OS was used as unprivileged Xen domain (Domain-U),
but it also can be used as lightweight Xen control domain (Domain-0).
To implement such fuctionality additional Xen interfaces are needed.
One of them is Xen domain controls (domctls) - it allows to create,
configure and manage Xen domains.
Also, used it as a possibility to update files copyright and licenses
identifiers in touched files.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Xen-related Kconfig options were highly dependand on BOARD/SOC xenvm.
It is not correct because Xen support may be used on any board and SoC.
So, Kconfig structure was refactored, now CONFIG_XEN is located in
arch/ directory (same as in Linux kernel) and can be selected for
any Cortex-A arm64 setup (no other platforms are currently supported).
Also remove confusion in Domain 0 naming: Domain-0, initial domain,
Dom0, privileged domain etc. Now all options related to Xen Domain 0
will be controlled by CONFIG_XEN_DOM0.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
After 79d0bf39b8 was merged, the inclusion
of <zephyr/acpi/acpi.h> with CONFIG_ACPI=n caused a build failure because
<acpica/source/include/acpi.h> could no longer be included due to the
inlcude path not being injected anymore.
Fix this by guarding the header inclusion when CONFIG_ACPI
is not set.
Fixes#62679.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This commit provides the users a way to disconnect dynamic
interrupts. This is needed because we don't want to keep
piling up ISR/arg pairs until the number of registrable
clients is reached.
This feature is only relevant for shared and dynamic interrupts.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
This works by overwriting z_isr_install()'s definition
(possible since the symbol is now weak) with our own definiton.
Whenever trying to register a new ISR/arg pair, z_isr_install()
will check to see if the interrupt line is already in use. If it's
not then it will not share the interrupt and will work exactly
as the old z_isr_install(), meaning it will just write the new
ISR/arg pair to _sw_isr_table.
If the interrupt line is already being used by an ISR/arg pair
then that line will become shared, meaning we'll overwrite
_sw_isr_table with our own (z_shared_isr, z_shared_sw_isr_table[irq])
pair.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
Since the shared IRQ code will also use the same logic to compute
the _sw_isr_table index, move the computing logic to a separate
function: z_get_sw_isr_table_idx(), which can be used by other
modules.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
This commit introduces all the necessary changes for
enabling the usage of shared interrupts.
This works by using a second interrupt table: _shared_sw_isr_table
which keeps track of all of the ISR/arg pairs sharing the same
interrupt line. Whenever a second ISR/arg pair is registered
on the same interrupt line using IRQ_CONNECT(), the entry in
_sw_isr_table will be overwriten by a
(shared_isr, _shared_sw_isr_table[irq]) pair. In turn, shared_isr()
will invoke all of the ISR/arg pairs registered on the same
interrupt line.
This feature only works statically, meaning you can only make use
of shared interrupts using IRQ_CONNECT(). Attempting to dynamically
register a ISR/arg pair will overwrite the hijacked _sw_isr_table
entry.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
LLVM LLD fills empty spaces (created using ALIGN() or moving the
location counter) in executable segments with TrapInstr pattern,
e.g. for ARM the TrapInstr pattern is 0xd4d4d4d4. GNU LD fills
empty spaces with 0x00 pattern.
We may want to have some section (e.g. rom_start) filled with 0x00,
e.g. because MCU can interpret the pattern as a configuration data.
Signed-off-by: Patryk Duda <pdk@semihalf.com>
This commit separates kernel_arch_func.h into two header file,
'cortex_a_r/kernel_arch_func.h' and 'cortex_m/kernel_arch_func.h', it
also removes some functions which is empty.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit separates cpu_idle.S into two asm files,
'cortex_a_r/cpu_idle.S' and 'cortex_m/cpu_idle.S'.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit separates thread.c into two source files,
'cortex_a_r/thread.c' and 'cortex_m/thread.c, it also introduces some
changes.
1. Migrate 'thread.c' and 'cortex_m/thread.c'.
2. Migrate 'thread.c' and 'cortex_a_r/thread.c'
3. Remove the 'z_arm_mpu_stack_guard_and_fpu_adjust' function as this is
obviously written for Cortex-M architecture.
4. Remove the 'z_arm_prepare_switch_to_main' function as this is only
used by Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit Separate 'prep_c.c' into two file based on the architecture,
one is 'cortex_m/prep_c.c', the other is 'cortex_a_r/prep_c.c'
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
There are too many differences between Cortex-A/R and Cortex-M on irq
code, e.g. Cortex-A/R use GIC and Cortex-M uses NVIC. For reducing
the complexity and easier to maintain, this commit separates irq_manage.c
and isr_wrapper.S into two different parts based on the architecture.
This commit also Removes the part related to the option
'CONFIG_ARM_SECURE_FIRMWARE' in 'cortex_a_r/irq_manage.c' because
this code is written for the Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
There are too many differences between Cortex-A/R and Cortex-M on swap
code. For reducing the complexity and easier to maintain, this commit
introduces the following major changes
1. Separate swap.c and swap_helper.S into two different parts based on
the architecture.
2. Rename 'z_arm_pendsv' to 'z_arm_do_swap' for Cortex-A/R.
3. Removes the part related to the option 'CONFIG_BUILTIN_STACK_GUARD'
in 'cortex_a_r/swap_helper.S' because this code is written for
the Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
'irq_relay.S' is only used for Arm Cortex-M architecture, so it's better
to place it to the 'arch/arm/core/cortex_m' directory.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit follows the parent commit work.
This commit introduces the following major changes.
1. Move all directories and files in 'include/zephyr/arch/arm/aarch32'
to the 'include/zephyr/arch/arm' directory.
2. Change the path string which is influenced by the changement 1.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
It doesn't make sense to keep the aarch32 directory in the
'arch/arm/core' directory as the aarch64 has been moved out.
This commit introduces the following major changes.
1. Move all directories and files in 'arch/arm/core/aarch32' to
'arch/arm/core' and remove the 'arch/arm/core/aarch32' directory.
2. Move all directories and files in 'arch/include/aarch32' to
'arch/include' and remove the 'arch/include/aarch32' directory.
3. Remove the nested including in the 'arch/include/kernel_arch_func.h'
and 'arch/include/offsets_short_arch.h' header files.
4. Change the path string which is influenced by the changement 1
and 2.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
The prep_c.c source doesn't depend on the ACPI library. As ACPI support
now requires the ACPICA module, the extra header breaks projects using
x86 without ACPI support.
Signed-off-by: Keith Short <keithshort@google.com>
The old acpi implimentation is replaced with acpica interface
and updated x86 arch porting with the new interface.
Signed-off-by: Najumon B.A <najumon.ba@intel.com>
RISC-V Spec requires minimum alignment of trap handling code to be
dependent from MTVEC.BASE field size. Minimum alignment for RISC-V
platforms is 4 bytes, but maximum is platform or application-specific.
Currently there is no common approach to align the trap handling
code for RISC-V and some platforms use custom wrappers to align
_isr_wrapper properly.
This change introduces a generic solution,
CONFIG_RISCV_TRAP_HANDLER_ALIGNMENT configuration option which sets
the alignment of a RISC-V trap handling code.
The existing custom solutions for some platforms remain operational,
since the default alignment is set to minimal possible (4 bytes)
and will be overloaded by potentially larger alignment of custom solutions.
Signed-off-by: Alexander Razinkov <alexander.razinkov@syntacore.com>
Add a property to the native_simulator target, to collect
the libraries we want to link with the runner, instead of
abusing the link options to pass them.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
As per #26393, Local APIC is using Kconfig based option for
the base address. This patch adds DTS binding support in the driver,
just like its conunter part I/O APIC.
Signed-off-by: Umar Nisar <umar.nisar@intel.com>
Disable d-cache on the fvp_baser_aemv8r_aarch32 platform because there
are some issues when d-cache is enabled for now.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This header is private and included only in architecture code, no need for
it to be in the top of the public include directory.
Note: This might move to a more private location later. For now just
cleaning up the obvious issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
During Zephyr boot with SMP enabled,
while z_smp_init is not completed yet and only boot core is running,
incorrect code in broadcast_ipi API will cause following:
1. Generate IPI even if other cores are not booted.
2. Incorrect setting of sgi1r register.
3. All the affinity(1/2/3) value will be incorrect.
Signed-off-by: Chirag Kochar <chirag.kochar@intel.com>
Enhanced arch_start_cpu so if a core is not available based on pm_cpu_on
return value, booting does not halt. Instead the next core in
cpu_node_list will be tried. If the number of CPU nodes described in the
device tree is greater than CONFIG_MP_MAX_NUM_CPUS then the extra cores
will be reserved and used if any previous cores in the cpu_node_list fail
to power on. If the number of cores described in the device tree matches
CONFIG_MP_MAX_NUM_CPUS then no cores are in reserve and booting will
behave as previous, it will halt.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
In some shared-memory use cases between Zephyr and other parallel
running OS, for data coherent, the non-cacheable normal memory
mapping is needed.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
This adds a function arch_xtensa_mmu_post_init() which can
be implemented on the SoC layer to perform additional MMU
initialization steps if necessary.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...VECBASE during MMU initialization. This is to make sure
that we can use the TLB miss handling in the exception
vector after we have moved back the VECBASE during MMU
initialization. Or else we would be forever stuck in ITLB
miss because the exception vectors are not in TLB and we
cannot populate the TLB because those vectors are not in
TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...and move it to xtensa_mmu_priv.h.
This would allow the SoC layer to use the RING number if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds arch_reserved_pages_update() which is called in
k_mem_manage_init() to reserve some physical pages so they
are not re-mapped. This is due to Zephyr's linker scripts
for Xtensa which often puts something before z_mapped_start
(aka .text, for example, vecbase). That space needs to be
reserved or else k_mem_map() would be mapping those that
could result in faults.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of only handling data TLB misses for VECBASE, change it
to handle all data TLB misses in the double exception handler.
It is because we may encounter data TLB misses when trying to
preload page table entries inside user exception handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
If CONFIG_XTENSA_RPO_CACHE is not enabled, it can be assumed
that memory is not double mapped in hardware for cached and
uncached access. So we can specify those regions to have
cache via TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Simply using __data_start and __data_end is not enough as
it leaves out kobject regions which is supposed to be
near .data section. So use _image_ram_start and
_image_ram_end instead to enclose data, bss and various
kobject regions (among others).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
During MMU initialization, we clear TLB way 6 to remove all
identity mapping. Depending on CPU configuration, there are
certain number of entries per way. So use the number from
core-isa.h to clear enough entries instead of hard-coded
number 8. Specifying an entry number outside of permitted
range may result in CPU reacting in weird way so better to
avoid that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the identity map of the first 512MB in TLB way 6.
Or else it would interfere with mapped entries resulting in
double mapped TLB exception.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
MMU needs to be initialized before going in to C, so
z_xtensa_mmu_init() is called in crt1.S before call
to z_cstart(). Note that this is the default case
and crt1.S can be disabled if board and SoC desire
to do so.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Turns out not all MMU enabled Xtensa cores have vaddrstatus,
vaddr0 and vaddr1. And there does not seem to be a way to
determine whether they are available. So remove them from
the exception printout for now.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
To allow reusing the path between boards,
instead of defining it in the board, let's define it in
the arch cmake file.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
CONFIG_COVERAGE has been incorrectly used to
change other kconfig options (stack sizes, etc)
code defaults, as well as some samples behaviour,
which should not have dependend on it.
Instead those should have depended on COVERAGE_GCOV,
which, being the one which adds special code and
temporary RAM storage for embedded targets,
require changes to many features.
When building for the native targets, all this was
unnecessary.
=> Fix the dependency.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The CMSIS module glue code was part of arch/ directory. Move it to
modules/cmsis, and provide a single entry point for it: cmsis_core.h.
This entry header will include the right CMSIS header (M or A/R).
To make this change possible, CMSIS module Kconfig/CMake are declared as
external, allowing us to add a new Zephyr include directory.
All files including CMSIS have been updated.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
With gcc from the zephyr sdk and -Wold-style-declaration is giving this
output:
zephyr/arch/arm/core/aarch32/cortex_a_r/fault.c:101:1: warning:
'inline' is not at beginning of declaration [-Wold-style-declaration]
101 | static void ALWAYS_INLINE
z_arm_fpu_caller_save(struct __fpu_sf *fpu)
| ^~~~~~
I searched to all of the source code to find these further occurances
where inline is not at the beginning of a function declaration.
Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
Move the function prototype before declaration of the function itself.
Maybe the prototype could be removed altogether?
Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
Is rsdp_phys is assigned in the loop below code jumps to found label,
otherwise we return and value rsdp_phys is never used after 0
assignment.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
This enables -Wshadow to warn about shadow variables on
in tree code under arch/, boards/, drivers/, kernel/,
lib/, soc/, and subsys/.
Note that this does not enable it globally because
out-of-tree modules will probably take some time to fix
(or not at all depending on the project), and it would be
great to avoid introduction of any new shadow variables
in the meantime.
Also note that this tries to be done in a minimally
invasive way so it is easy to revert when we enable
-Wshadow globally. Source files under modules/, samples/
and tests/ are currently excluded because there does not
seem to be a trivial way to add -Wshadow there without
going through all CMakeLists.txt to add the option
(as there are 1000+ files to change).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The current mechanism of the MPU region switching configures and
reprograms the regions (including inserting, splitting the dynamic
region, and flushing the regions to the registers) every time during the
context switch. This, not only causes a large usage of the kernel stack
but also a lower performance.
To improve it, move the configuration operations ahead to make sure the
context swtich only flushes the current thread regions to the registers
and does not configure the regions anymore. To achieve this, configure
the regions during any operations related to partitions (partition
add/remove, and domain add/remove thread), flush the sys_dyn_regions if
the current thread is the privileged thread, and flush the thread's own
regions if it's a user thread.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Using BR(background region) during the flushing regions instead of
enabling/disabling the MPU which is a heavy operation.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add support for processing the Fault Status Registers and recoverable
data abort for Armv8-R AArch32.
Based on Arm Architecture Reference Manual Supplement Armv8, for the
Armv8-R AArch32 architecture profile (ARM DDI 0568).
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
Architecturally, Set/Way operations are not guaranteed to affect all
caches prior to the PoC, and may require other IMPLEMENTATION DEFINED
maintenance (e.g. MMIO control of system-level caches).
First of all this patch was designed for Xen domain Zephyr build, set/way
ops are not easily virtualized by Xen. S/W emulation is disabled, because
IP-MMU is active for Dom0. IP-MMU is a IO-MMU made by Renesas, as any good
IO-MMU, it shares page-tables with CPU. Trying to emulate S/W with IP-MMU
active will lead to IO-MMU faults. So if we build Zephyr as a Xen Initial
domain, it won't work with cache management support enabled.
Exposing set/way cache maintenance to a virtual machine is unsafe, not
least because the instructions are not permission-checked, but also
because they are not broadcast between CPUs.
In this commit, VA data invalidate invoked after every mapping instead of
using set/way instructions on init MMU. So, it was easy to delete
sys_cache_data_invd_all from enable MMU function, becase every adding of
a new memory region to xclat tabes will cause invalidating of this memory
and in this way we sure that there are not any stale data inside.
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
The zephyr_library() used to only be called if
CONFIG_GEN_ISR_TABLES is enabled. Which means that any
zephyr_library_*() calls are putting things into another
library if CONFIG_GEN_ISR_TABLES is disabled. So pull
the call to zephyr_library() to outside. Also moves
semihost.c into the library.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When building the 32bit native board targets variants
for x86(-64) hosts, gcc will promote float literals to double
(See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92875 )
This can result in unexpected comparison differences.
This is due to the compiler using the 8087 float mode by
default.
Instead let's tell the compiler to use the SSE float path,
which is the default for 64 bit x86-64 builds.
The assumption that any x86 host used for development
will have SSE support should be safe enough.
For more background see
https://github.com/zephyrproject-rtos/zephyr/issues/61345
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
In print_pmp_entries(), start and end are function arguments
and yet another start and end are declared inside the for
loop. So rename the function arguments to fix shadow variables
warning.
The changes in csr_*() macros are needed to avoid shadowing
__v when nesting those functions together, for example,
csr_write(..., csr_read(...)).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add new option to use thread local storage for stack
canaries. This makes harder to find the canaries location
and value. This is made optional because there is
a performance and size penalty when using it.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Instead of implementing a custom power off API (pm_system_off),
implement the sys_poweroff hook, and indicate power off is supported by
selecting HAS_POWEROFF. Note that according to the PSCI specification
(DEN0022E), the SYSTEM_OFF operation does not return, however, an error
is printed and system is halted in case this occurs.
Note that the pm_system_off has also been deleted, from now on, systems
supporting PSCI should enable CONFIG_POWEROFF and call the standard
sys_poweroff() API.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
In some very rare cases (< 1/1000 runs), in very loaded machines,
a race in the glibc pthread_cancel() seems to be triggered.
In this the cancelled thread cleanup overtakes the pthread_cancel()
code, and frees the pthread structure before pthread_cancel()
has finished, resulting in a dereference into already
free'd memory, and therefore a segfault.
Calling pthread_cancel() during cleanup is not required beyond
preventing a valgrind memory leak report (all threads will be
stopped immediately on exit).
Therefore we stop doing this, to avoid this very rare crashes.
This issue was reproduced in Ubuntu 22.04, with its default
gcc 11.3.0 and glibc 2.35.
The issue may also have been seen very rarely in Zephyr's CI.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add Kconfig RISCV_SOC_HAS_CUSTOM_SYS_IO symbol so that a riscv
SoC can set to specify that it has a custom implementation for
sys_io functions.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Refactor the ESP32 target SOCs together with
all related boards. Most braking changes includes:
- changing the CONFIG_SOC_ESP32* to refer to
the actual soc line (esp32,esp32s2,esp32s3,esp32c3)
- replacing CONFIG_SOC with the CONFIG_SOC_SERIES
- creating CONFIG_SOC_FAMILY_ESP32 to embrace all
the ESP32 across all used architectures
- introducing CONFIG_SOC_PART_NUMBER_* to
provide a SOC model config
- introducing the 'common' folder to hide all
commonly used configs and files.
- updating west.yml to reflect previous changes in hal
Signed-off-by: Marek Matej <marek.matej@espressif.com>
xt-clang likes to remove any consecutive NOPs more than 8. So
we need to force the function to have no optimization to avoid
this behavior and to retain all those NOPs.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a Kconfig to introduce the Xtensa specific
arch_spin_relax() which can do more NOPs. Some Xtensa SoCs
may need more NOPs after failure to lock a spinlock,
especially under SMP. This gives the bus extra time to
propagate the RCW transactions among CPUs.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In GNU LD, the location counter (the 'dot' variable) always refers to
the byte offset from the start of current object as mentioned in
documentation[1]:
```
'.' actually refers to the byte offset from the start of the current
containing object. Normally this is the SECTIONS statement, whose start
address is 0, hence '.' can be used as an absolute address. If '.' is
used inside a section description however, it refers to the byte offset
from the start of that section, not an absolute address.
```
For example, if the section 'rom_start':
rom_start : {
. = 0x400;
_vector_start = ABSOLUTE(.);
} > FLASH
has a starting address of 0x8000000, then _vector_start will be
0x8000400
However, behavior of LLVM LLD is quite different, the value of the
location counter is always absolute (see discussion [2]), so in the
example above, the linker will return error, because it will interpret
'. = 0x400' as an attempt to move the location counter backwards.
It could be fixed by changing line to '. += 0x400' (#54796) which will
move the location counter by 0x400 for both linkers, but it would work
only when we are at the beginning of section. Consider the following
example:
rom_start : {
. = 0x400;
KEEP(*(.boot_hdr.conf))
. = 0x1000;
KEEP(*(.boot_hdr.ivt))
KEEP(*(.boot_hdr.data))
KEEP(*(.boot_hdr.dcd_data))
. = 0x2000;
_vector_start = .;
} > FLASH
In this case, _vector_start will be 0x2000, but if we change
'. = 0x2000' to '. += 0x2000', then the value of _vector_start depends
on size of data in input sections (but it's 0x3000 at least).
Actually, this example comes from final linker script when compiling
firmware for mimxrt1170_evk_cm7 board. This board failed to boot
(#55296) after #54796 was merged.
This patch introduces method compatible with both linkers. We calculate
relative offset from the beginning of the section and use that value to
calculate number of bytes by which we should move the location counter
to get CONFIG_ROM_START_OFFSET.
[1] https://sourceware.org/binutils/docs/ld/Location-Counter.html
[2] https://discourse.llvm.org/t/lld-location-counter-inside-objects
Signed-off-by: Patryk Duda <pdk@semihalf.com>
When the frame-pointer based unwinding is enabled, the stop condition
for the stack backtrace is (FP == NULL).
Set FP to 0 before jumping to C code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The current implementation causes a lockup of the core when the exception
originates from an invalid/unreachable pc. This fix first verifies on
armv6-m and armv8-m.base that pc was in an expected runnable region,
namely:
- .text
- .ramfunc
- .itcm
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Adapt to the reworked zephyr cache API.
Fix build errors when building tests/kernel/cache with CACHE_MANAGEMENT and
CPU_HAS_DCACHE enabled for x86 SoCs
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
The z_arm64_fatal_error should be
extern void z_arm64_fatal_error(unsigned int reason, z_arch_esf_t *esf);
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Clarify why we use target_link_options() instead of
target_link_libraries()
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
To ease writing common drivers, let's make the host trampolines
from the native simulator avaliable to all posix based boards.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
On nRF5340 net core it was observed that when `wfi` instruction was
followed by `pop {r0, lr}` in the `arch_cpu_idle` function,
the value of `lr` sometimes got read as 0 from memory despite
having correct value stored in the memory.
This commit inserts additional `nop` instruction after waking up
to delay access to the memory.
Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
Instead of keeping a replica of the nce code,
now that the native_simulator is in tree, let's use
it directly.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Instead of failing badly later, let's give a clear error
message if the user tries to build in an unsupported platform.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This allows building with embedded libCs in the Zephyr side,
as the POSIX arch bottom is not anymore built in Zephyr context.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
So they depend or select on the right NATIVE_BUILD
instead of NATIVE_APPLICATION.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Zephyr provides a default NMI handler (`z_SysNmiOnReset`), which will
basically call `wfi` endlessly. It is allowed to override such handler
when CONFIG_RUNTIME_NMI=y, via `z_arm_nmi_set_handler`. However,
enabling such option also provided `z_arm_nmi_init` (via `NMI_INIT()`),
which basically sets the handler to `DefaultHandler` (a new handler that
basically printks and reboots). This is strictly not needed, and
independent of the runtime NMI option. As a result, most SoCs were
blindly calling `NMI_INIT()`, probably because of a copy&paste effect.
In the majority of cases, this was a no-op, but most SoCs do IRQ
enable/disable, making this even more convoluted. To make things worse,
the init call is expected to run after console has been initialized (for
printk to work?), but most SoCs just called it in PRE_KERNEL_1+0.
This patch just drops this NMI initializer API, and leaves only the
handler set call when CONFIG_RUNTIME_NMI=y.
NMI_INIT() dummy definition is left in this patch to preserve
bisectability, will be dropped later.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.
Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
The implementation of `z_arm_clear_arm_mpu_config` was compiled for all
ARM cores that declare to have an MPU. However, we only want to compile
it if the MPU is actually enabled.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system.
And the top the only one that interacts or is aware
of the hosted operating system (Zephyr).
The bottom uses the native simulator CPU
start/stop emulation.
By now we replicate its code as a provisional measure,
until the native simulator becomes standard.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system, and does not necessarily
need to know about the Zephyr OS.
This is in preparation for the native simulator,
which which the bottom is also fully Zephy agnostic.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The issue due to which we set the default compiler C std version
to C11 is not specific to any particular POSIX arch board,
but to all. Instead of setting this property for each board,
let's set it at the architecture level.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This adds support for 32 double-precision registers in the context
switching of aarch32 architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
`fpscr` is assigned from `struct __fpu_sf.fpscr` in `vfp_restore`, but it
wasn't saved into `struct __fpu_sf.fpscr` in the svc and isr handler, So
it may be a dirty value.
- Fix it by saving `fpscr` in the svc hand isr handler.
- Jump out if FPU isn't enabled
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.
This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.
This solves #54785.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
This adds a few line use zephyr_syscall_header() to include
headers containing syscall function prototypes.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Some RISCV platforms shipping a CLIC have a peculiar interrupt ID
ordering / mapping.
According to the "Core-Local Interrupt Controller (CLIC) RISC-V
Privileged Architecture Extensions" Version 0.9-draft at paragraph 16.1
one of these ordering recommendations is "CLIC-mode interrupt-map for
systems retaining interrupt ID compatible with CLINT mode" that is
described how:
The CLINT-mode interrupts retain their interrupt ID in CLIC mode.
[...]
The existing CLINT software interrupt bits are primarily intended for
inter-hart interrupt signaling, and so are retained for that purpose.
[...]
CLIC interrupt inputs are allocated IDs beginning at interrupt ID
17. Any fast local interrupts that would have been connected at
interrupt ID 16 and above should now be mapped into corresponding
inputs of the CLIC.
That is a very convoluted way to say that interrupts 0 to 15 are
reserved for internal use and CLIC only controls interrupts reserved for
platform use (16 up to n + 16, where n is the maximum number of
interrupts supported).
Let's now take now into consideration this situation in the DT:
clic: interrupt-controller {
...
};
device0: some-device {
interrupt-parent = <&clic>;
interrupts = <0x1>;
...
};
and in the driver for device0:
IRQ_CONNECT(DT_IRQN(node), ...);
From the hardware prospective:
(1a) device0 is using the first IRQ line of the CLIC
(2a) the interrupt ID / exception code of the `MSTATUS` register
associated to this IRQ is 17, because the IDs 0 to 15 are reserved
From the software / Zephyr prospective:
(1b) Zephyr is installing the IRQ vector into the SW ISR table (and into
the IRQ vector table for DIRECT ISRs in case of CLIC vectored mode)
at index 0x1.
(2b) Zephyr is using the interrupt ID of the `MSTATUS` register to index
into the SW ISR table (or IRQ vector table)
It's now clear how (2a) and (2b) are in contrast with each other.
To fix this problem we have to take into account the offset introduced
by the reserved interrupts. To do so we introduce
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET as hidden option for the
platforms to set.
This Kconfig option is used to shift the interrupt numbers when
installing the IRQ vector into the SW ISR table and/or IRQ vector table.
So for example in the previous case and using
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET == 16, the IRQ vector
associated to the device0 would be correctly installed at index 17 (16 +
1), matching what is reported by the `MSTATUS` register.
CONFIG_NUM_IRQS must be increased accordingly.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Zephyr currently only supports CLINT direct mode and CLINT vectored
mode. Add support for CLIC vectored mode as well.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Before adding support for the CLIC vectored mode, rename
CONFIG_RISCV_MTVEC_VECTORED_MODE to CONFIG_RISCV_VECTORED_MODE to be
more generic and eventually include also the CLIC vectored mode.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
With all the stacks and TSS (etc), the x86_64 arch code can only
support maximum of 4 CPUs at the moment. So add a build assert
if more CPUs are specified via CONFIG_MP_MAX_NUM_CPUS, also
overwrite the range value for CONFIG_MP_MAX_NUM_CPUS.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We are missing setting of switch_handle for the thread which
is aborting due to exception (i.e. in case of k_panic or
__ASSERT triggered). This may cause livelock in SMP code
after a08e23f68e commit ("kernel/sched: Fix SMP
must-wait-for-switch conditions in abort/join").
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently the lazy fpu saving algorithm in arm64 is using the fpu_owner
pointer from the cpu structure to understand the owner of the context
in the cpu and save it in case someone different from the owner is
accessing the fpu.
The semantics for memory consistency across smp systems is quite prone
to errors and reworks on the current code might miss some barriers that
could lead to inconsistent state across cores, so to overcome the issue,
use atomics to hide the complexity and be sure that the code will behave
as intended.
While there, add some isb barriers after writes to cpacr_el1, following
the guidance of ARM ARM specs about writes on system registers.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
With Zephyr now always using `int main(void)`, there's no longer any need
for this definition. The last remaining use which gated the declaration of
_posix_zephyr_main isn't necessary as adding that declaration
unconditionally is harmless.
Signed-off-by: Keith Packard <keithp@keithp.com>
cpu_node_list does not hold the corrent mapping of cpu id and mpid when
core booting sequence does not follow the DTS cpu node sequence. This
will cause an issue that sgi cannot deliver to the right target.
Add the cpu_map array to hold the corrent mapping between cpu id and
mpid.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Each core should init their own stack during the reset when SMP enabled,
but do not touch others. The current init results in each core starting
init the stack from the same address which will break others.
Fix the issue by setting a correct start address.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
LOG system has unalignment access instruction which will cause an
alignment exception before MPU is enabled. Remove the LOG print before
MPU is enabled to avoid this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.
This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.
Signed-off-by: Andy Ross <andyross@google.com>
For secure EL2 to be entered the EEL2 bit in SCR_EL3 must be set. This
should only be set if Zephyr has not been configured for NS mode only,
if the device is currently in secure EL3, and if secure EL2 is supported
via the SEL2 bit in AA64PFRO_EL1. Added logic to enable EEL2 if all
conditions are met.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
This reverts commit f0b458a619.
This is a pointless change that simply increases footprint.
Existing code already supports compilation without multithreading.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Per the ARMv8 architecture document, modification of the system control
register is a context-changing operation. Context-changing operations are
only guaranteed to be seen after a context synchronization event.
An ISB is a context synchronization event. One has been placed after
each SCTLR modification. Issue was found running full speed on target.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
This make MCUboot build as Zephyr application.
Providing optinal 2nd stage bootloader to the
IDF bootloader, which is used by default.
This provides more flexibility when building
and loading multiple images and aims to
brings better DX to users by using the sysbuild.
MCUboot and applications has now separate
linker scripts.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for pending FPU IPI from
the spinlock loop using the arch_spin_relax() hook.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for a pending FPU IPI
from the arch_spin_relax() hook and honor it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The compiler is not able to emit a proper DSB operation for ARM64. Move
to the arch-specific implementation and use assembly code instead.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Enhance for cases when call z_float_enable() with NULL thread.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
This adds code to always map data TLB for VECBASE so that
we would be dealing with fewer data TLB misses during
exception handling. With VECBASE always mapped, there is
no need to pre-load anymore.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the TLB miss handling to the C exception handler.
This also allows us to handle page faults (for example,
unmapped pages) during this time as any more exceptions
handled in the C handler will not trigger the double
exception handler but the same C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of being able to arbitrarily set the PTEVADDR for page
table, this provides choices (currently just one). This is in
preparation to enable handling memory management exception in
C code. For that to work, we will need to pre-load the page
table address (PTEVADDR) for the memory page containing
exception code and data (containing jump addresses), and
various stacks. This is to prempt any TLB misses during handling
the level 1 interrupt code. If a TLB miss is encountered during
handling of level 1 interrupt, we will be thrown into double
exception handling code where we will get stuck in infinite
loop. However, in order to pre-load the page table entries,
PTEVADDR needs to be calculated. This requires the use of
PTEVADDR base which cannot be loaded via l32r, as we may cause
a data TLB miss. So we must be able to grab the PTEVADDR base
address strictly within code, and must be without any data
load. So changing CONFIG_XTENSA_MMU_PTEVADDR to be based on
choice so we can have pre-defined bit shift value for shift
operation. This shift value will be used in exception handling
code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add a build option to tell if memory should be mapped in cached
and uncachedr regions.
If the memory is neither in cached nor uncached region it is not double
mapped.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Initial support for Xtensa MMU version 3. It is using a two level page
table based on fact that the page table is in the virtual space. Only
the top level (page directory) is wired mapped in the TLB to avoid
second level page miss.
The mapped memory is completely fragmented in multiple sections, maybe
we find a better way in future.
The exception handler is where we effectively map the memory, the way it
works is:
1) SW try to access some memory address
2) The address is not mapped, so the MMU will try the auto-refill,
looking the page table
3) The page table contents is not mapped (remember, just the top-level page
is mapped)
4) An exception will be triggered, in the exception we try to read the
portion of the page table that maps the original address
5) The address is not mapped, so the MMU will try again the auto-refill.
This time though, the address is mapped by the top level page that is
properly mapped. (The top-level page maps the page table itself).
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Unlike tracing module mainly for debug usage, this is
to allow runtime profiling IRQ performance data, and
target to enable it in product release since platform
can choose to make it work with low weight protocol.
Enable this option and implement runtime_irq_stats()
in platform code, such as Intel ISH platform implement
with SHMI protocol to allow host profiling irq stats.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Until now iterable sections APIs have been part of the toolchain
(common) headers. They are not strictly related to a toolchain, they
just rely on linker providing support for sections. Most files relied on
indirect includes to access the API, now, it is included as needed.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The commit 434ca63e2f introduced the
Cortex-A and Cortex-R CPU type dependency to `CONFIG_FP16` based on
the reasoning that the hardware half-precision support is only
available on them.
While it is true that the _hardware_ half-precision support is limited
to these targets, the compiler will provide the _software_ emulation
for the targets that lack the hardware half-precision support, as
mentioned in 41fd6e003c (the original
commit that introduced `CONFIG_FP16`).
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
In z_xtensa_backtrace_print the parameter depth is checked for <= 0.
There is no need to check it again later, also, since the variable is
not used after the while loop we can use directly the parameter without
an additional variable.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Enable single-threading support for the riscv architecture.
Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.
The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
Intel ISH SoC can't reboot via RST_CNT register,
so make sys_arch_reboot as weak function to allow
implement different arch reboot in SoC layer.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
The "cross stack call" mechanism has intermediate states where the
stack frames are not valid for our own interrupt entry code, which
causes corruption if an interrupt races at exactly the right time.
Leave interrupts masked until just before the call.
The fix is midly complicated by the fact that we RELY on nested window
exception frames to spill registers from the interruptee, so have to
do the masking with PS.INTLEVEL, which requires a register to save its
contents, which we don't have since everything needs to happen in one
4-register window. But thankfully our Zephyr-reserved EPS register is
guaranteed to be available through this process.
Fixes#57009
Signed-off-by: Andy Ross <andyross@google.com>
With paging config, need to use physical address as
paging is not enabled here.
From IA manual, LDMXCSR instruction description is,
Loads the source operand into the MXCSR control/status
register, the source operand is a 32-bit memory location.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Use the common exit() provided by libc so we get standard behavior
across all architectures. So only implement a special exit when
XT_SIMULATOR is defined.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Local variables in ASM macro works differently for GNU and MWDT
toolchains. In case of GNU toolchain they are local per each macro
instance, but in case of MWDT they are local per file where macro
is used.
To avoid issues when macro is used multiple times in one file let's
align _st32_huge_offset to have same behaviour with GNU & MWDT
toolchains.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
The backtrace requires a valid stack pointer to start
printing backtraces. So if there is no stack pointer
being passed in, skip printing backtraces.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
use helper macros from csr.h instead of inline assembly which results
in cleaner and more maintainable code
Signed-off-by: Manojkumar Subramaniam <manoj@electrolance.com>
Commit 408472673e added inline
assembly to lock interrupt. However, XCC doesn't like the syntax
using STRINGIFY, and also an empty clobber section. So parameterize
the second argument to rsil, and remove the last colon.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds some structs for interrupt stack frames to make it
easier to access individual elements, and ultimately getting
rid of magic array element numbers in the code. Hopefully,
this would aid in debugging where you can view the whole
struct in debugger.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Align the comment with the new int main(void) convention
used in Zephyr.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Some functions such as the ISH power management function needs to
reserve GDT entry place holders which are filled during runtime.
Add config option to define the number of GDT entry place holders
reserved and change the ia32 linker script accordingly.
Signed-off-by: Leifu Zhao <leifu.zhao@intel.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
rename the function that sets the handler for the nmi.
It should be namespaced and not camel-case:
z_NmiHandlerSet to z_arm_nmi_set_handler
Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
In case of recoverable fatal errors the execution should
switch to another thread. This will ensure the current_cpu nested
count is reset when there is a context switch.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
We get the following error when building with arm-clang:
error: non-ASM statement in naked function is not supported
__TZ_WRAP_FUNC(preface, foo1, postface);
^
tests/arch/arm/arm_tz_wrap_func/src/main.c:69:25: note: attribute is here
uint32_t __attribute__((naked)) wrap_foo1(uint32_t arg1, uint32_t arg2,
^
1 error generated.
Remove the do/while wrapper to make this a true naked function.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Revert commit 44628735b8
This commit broke the ability for nxp rt series to
reset except with power cycle
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
Current implementation of cache management APIs for ARM only applies to
Cortex-M, so move it to its own directory.
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
Fixes unneeded chain of includes. Since zefi is built separately
(using python script), any dependency creates include chain with
possibly missing configuration options.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
Update current stack limit on every context switch, including switching
to irq stack and switching back to thread stack.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly enable the safe exception stack including the stack
switch. Init the safe exception stack by calling
z_arm64_safe_exception_stack during the boot stage on every core. Also,
tweaks several files to properly switch the mode with different cases.
1) The same as before, when executing in userspace, SP_EL0 holds the
user stack and SP_EL1 holds the privileged stack, using EL1h mode.
2) When entering exception from EL0 then SP_EL0 will be saved in the
_esf_t structure. SP_EL1 will be the current SP, then retrieves the safe
exception stack to SP_EL0, making sure the always pointing to safe
exception stack as long as the system running in kernel space.
3) When exiting exception from EL1 to EL0 then SP_EL0 will be restored
from the stack value previously saved in the _esf_t structure. Still at
EL1h mode.
4) Either entering or exiting exception from EL1 to EL1, SP_EL0 will
keep holding the safe exception stack unchanged as memtioned above.
5) Do a quick stack check every time entering the exception from EL1 to
EL1. If check fail, set SP_EL1 to safe exception stack, and then handle
the fatal error.
Overall, the exception from user mode will be handled with kernel stack
at the assumption that it is impossible the stackoverflow happens at the
entry of exception from EL0 to EL1. However the exception from kernel
mode will be firstly checked with the safe exception stack to see if the
kernel stack overflows, because the exception might be triggered by
stack invalid accessing.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add safe exception stack init function which does several things:
1) setting current cpu safe exception stack pointer to its corresponding
stack top.
2) init sp_el0 with the above safe exception stack.
That makes sure the sp_el0 points to per-cpu safe_stack in the kernel
space.
3) init the current_stack_limit and corrupted_sp with 0
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
As the preparation for enabling safe exception stack, add a variable in
_esf_t to save the user stack held by sp_el0 at the point of the
exception happening from EL0. The newly added variable in _esf_t is
named sp from which the user stack will be restored when exceptions eret
to EL0.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add three per-cpu variables for the convenience of quickly accessing.
The safe_exception_stack stores the top of safe exception stack pointer.
The current_stack_limit stores the current thread's priv stack limit.
The corrputed_sp stores the priv sp or irq sp for the stack overflow
case, or 0 for the normal case.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce two configs to prepare to enable the safe exception stack for
the kernel space. This is the preparation for enabling hardware stack
guard. Also define the safe exception stack for kernel exception stack
check.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
If so this is most certainly a bug. arch_mem_unmap() should be
used before mapping the same area again.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
First, we have commit 7d27bd0b85 ("arch: arm64: Disable infinite
recursion warning for `discard_table`") that blindly shut up a compiler
warning that did actually highlighted a real bug. Revert that and fix
the bug properly. And yes, mea culpa for having been the first to
approve that commit, or even creating the bug in the first place.
Then let's add proper table usage cound handling for discard_table() to
work properly and avoid leaking table pages.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For RISCV arch, enable FLASH_SIZE and FLASH_BASE_ADDRESS config.
To avoid duplicated work, remove flash config from RISCV soc.
Signed-off-by: Jonas Otto <jonas@jonasotto.com>
The image header is compatible for zImage(32) protocol.
Offset Value Description
0x24 0x016F2818 Magic number to identify ARM Linux zImage
0x28 start address The address the zImage starts at
0x2C end address The address the zImage ends at
As Zephyr can be built with a fixed load address, Xen/Uboot can read
the image header and decide where to copy the Zephyr image.
Also, it is to be noted that for AArch32 A/R, the vector table should
be aligned to 0x20 address. Refer ARM DDI 0487I.a ID081822, G8-9815,
G8.2.168, VBAR, Vector Base Address Register :-
Bits[4:0] = RES0.
For AArch32 M (Refer DDI0553B.v ID16122022, D1.2.269, VTOR, Vector Table
Offset Register), Bits [6:0] = RES0.
As zImage header occupies 0x30 bytes, thus it is necessary to align
the vector table base address to 0x80 (which satisfies both VBAR and
VTOR requirements).
Also, it is to be noted that not all the AArch32 M class have VTOR, thus
ARM_ZIMAGE_HEADER header depends on
CPU_AARCH32_CORTEX_R || CPU_AARCH32_CORTEX_A || CPU_CORTEX_M_HAS_VTOR.
The reason being the processors which does not have VBAR or VTOR, needs
to have exception vector table at a fixed address in the beginning of
ROM (Refer the comment in arch/arm/core/aarch32/cortex_m/CMakeLists.txt)
. They cannot support any headers.
Also, the first instruction in zImage header is to branch to the kernel
start address. This is to support booting in situations where the zImage
header need not be parsed.
In case of Arm v8M, the first two entries in the reset vector should be
"Initial value for the main stack pointer on reset" and "Start address
for the reset handler" (Refer Armv8M DDI0553B.vID16122022, B3.30,
Vector tables).
In case of Armv7M (ARM DDI 0403E. ID021621, B1.5.3 The vector table),
the first entry is "SP_main. This is the reset value of the Main stack
pointer.".
Thus when v7M or v8M starts from reset, it expects to see these values
at the default reset vector location.
See the following text from Armv7M (ARM DDI 0403E. ID021621, B1-526)
"On powerup or reset, the processor uses the entry at offset 0 as the
initial value for SP_main..."
Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Add missing include to prevent `'EINVAL' undeclared` when
using `CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT=y`
Signed-off-by: George Ruinelli <caco3@ruinelli.ch>
FP16 isn't something that is supported on Cortex-M so limit the
Kconfig feature to Cortex-A or Cortex-R.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
The CONFIG_ROM_START_OFFSET is supposed to be added to
the current when linking, instead of having the current
address set to it. So fix that.
Not sure why it worked up to this point, but llvm/clang/lld
complained that it could not move location counter backward.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prevent the CPU from actually
entering sleep by skipping the WFE/WFI instruction.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
Looks like some implementors decided not to implement the full set of
PMP range matching modes. Let's rearrange the code so that any of those
modes can be disabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's honor CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT even for kernel
stacks. This saves one global PMP slot when creating the guard area for
the IRQ stack, and some hw implementations might require that anyway.
With this changes, arch_mem_domain_max_partitions_get() becomes much
more reliable and tests/kernel/mem_protect is more likely to pass even
with the stack guard enabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Additional privileged stack space is used by peripheral emulators when
userspace is enabled. This is largely due to additional function calls and
data structures allocated on the stack. This can potentially lead to stack
smashing if the privileged stack size isn't high enough, causing an
exception.
Increase the privileged stack space when userspace and peripheral emulation
are enabled.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
When CONFIG_SOC_ISR_SW_UNSTACKING is defined, it's up to the custom soc
code to remove the ESF, because the software-managed part of the ESF is
depending on the hardware. Fix this in the ISR code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some implementations may not capture the faulting instruction in mtval
and set it to zero when an illegal instruction fault is raised This is
notably the case with QEMU version 7.0.0 when a CSR instruction is
involved.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The FRCSR, FSCSR, FRRM, FSRM, FSRMI, FRFLAGS, FSFLAGS and FSFLAGSI
are in fact CSR instructions targeting the fcsr, frm and fflags
registers. They should be caught as FPU instructions as well.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
- IRQ state for the interrupted context corresponds to the PIE bit not
the IE bit.
- Restoring the saved FPU state should clear the entire field before
or'ing wanted bits in.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For RISC-V, the reg property of a cpu node in the devicetree describes
the low level unique ID of each hart. Using devicetree macro's, a list
of all cpus with status "okay" can be generated.
Using devicetree overlays, a hart or multiple harts can be marked as
"disabled", thus excluding them from the list. This allows platforms
that have non-zero indexed SMP capable harts to be functionally mapped
to Zephyr's sequential CPU numbering scheme.
On kernel init, if the application has MP_MAX_NUM_CPUS greater than 1,
generate the list of cpu nodes from the device tree with status "okay"
and map the unique hartid's to zephyr cpu's
While we are at it, as the hartid is the value that gets passed to
z_riscv_secondary_cpu_init, use that as the variable name instead of
cpu_num
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
RISC-V multi-hart systems that employ a heterogeneous core complex are
not guaranteed to have the smp capable harts starting with a unique id
of zero, matching Zephyr's sequential zero indexed cpu numbering scheme.
Add option, RV_BOOT_HART to choose the hart to boot from.
On reset, check the current hart equals RV_BOOT_HART: if so, boot first
core. else, loop in the boot secondary core and wait to be brought up.
For multi-hart systems that are not running a Zephyr mp or smp
application, park the non zephyr related harts in a wfi loop.
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
Add an option to generate simplified error codes instead of more
specific architecture specific error codes. Enable this by default in
tests to make exception tests more generic across hardware.
Fixes#54053.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Commit 4f9b547ebd ("riscv: smp: prepare for more than one IPI type")
didn't clear pending IPI flags.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We can leverage the FPU dirty state as an indicator for preemptively
reloading the FPU content when a thread that did use the FPU before
being scheduled out is scheduled back in. This avoids the FPU access
trap overhead when switching between multiple threads with heavy FPU
usage.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
FPU context switching is always performed on demand through the FPU
access exception handler. Actual task switching only grants or denies
FPU access depending on the current FPU owner.
Because RISC-V doesn't have a dedicated FPU access exception, we must
catch the Illegal Instruction exception and look for actual FP opcodes.
There is no longer a need to allocate FPU storage on the stack for every
exception making esf smaller and stack overflows less likely.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Instead of saving/restoring FPU content on every exception and task
switch, this replaces FPU sharing support with a "lazy" (on-demand)
context switching algorithm similar to the one used on ARM64.
Every thread starts with FPU access disabled. On the first access the
FPU trap is invoked to:
- flush the FPU content to the previous thread's memory storage;
- restore the current thread's FPU content from memory.
When a thread loads its data in the FPU, it becomes the FPU owner.
FPU content is preserved across task switching, however FPU access is
either allowed if the new thread is the FPU owner, or denied otherwise.
A thread may claim FPU ownership only through the FPU trap. This way,
threads that don't use the FPU won't force an FPU context switch.
If only one running thread uses the FPU, there will be no FPU context
switching to do at all.
It is possible to do FP accesses in ISRs and syscalls. This is not the
norm though, so the same principle is applied here, although exception
contexts may not own the FPU. When they access the FPU, the FPU content
is flushed and the exception context is granted FPU access for the
duration of the exception. Nested IRQs are disallowed in that case to
dispense with the need to save and restore exception's FPU context data.
This is the core implementation only to ease reviewing. It is not yet
hooked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Right now this is hardcoded to z_sched_ipi(). Make it so that other IPI
services can be added in the future.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
If running under Xtensa simulator, it is good to tell simulator
to stop execution once we reach double exception, as the current
double exception handler is simply an endless loop. If we turn
on tracing in the simulator, the output file would contain
an infinite iteration of this endless loop, and the simulator
needs to be stopped manually before the file size goes out of
control. So we need to tell the simulator to stop once
we reach this point instead of doing an endless loop.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Given the Zephyr CPU number is no longer tied to the hartid, we must
consider the actual hartid when sending an IPI to a given CPU. Since
those hartids can be anything, let's just save them in the cpu structure
as each CPU is brought online.
While at it, throw in some `get_hart_msip()` cleanups.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Currently it is assumed that Zephyr CPU numbers match their hartid
value one for one. This assumption was relied upon to efficiently
retrieve the current CPU's `struct _cpu` pointer.
People are starting to have systems with a mix of different usage for
each CPU and such assumption may no longer be true.
Let's completely decouple the hartid from the Zephyr CPU number by
stuffing each CPU's `struct _cpu` pointer in their respective scratch
register instead. `arch_curr_cpu()` becomes more efficient as well.
Since the scratch register was previously used to store userspace's
exception stack pointer, that is now moved into `struct _cpu_arch`
which implied minor user space entry code cleanup and rationalization.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Ultil now Cortex A/R aarch32 implementation for context
switching expects that interrupts was disabled. This is
true if a context switching happens at thread context.
But if a context switching happens at last action during
interrupt context, this assumption is not true because the
interrupts are still enabled (to allow nesting interrupts).
Disable interrupts at the last interrupt action to ensure
the interrupts are always disabled before context switching
is processed
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
In platforms where the linker is capable of doing global optimizations,
like relaxing address mode and synthesize new instructions, Zephyr has to
disable it when enabling USERSPACE since the build expects that address
don't change after the first stage build.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Return specific fault reasons instead of the generic
`K_ERR_CPU_EXCEPTION`, which provides minimal debugging aid.
Fixes#53093.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
This patchset is fixing two things:
1. The proper sys_* functions are used for cache mainteinance
operations.
2. To check the status of the L1 cache the SCB registers are probed so
the code is assuming a core architecture cache is present, thus make
the code conditionally compiled on CONFIG_ARCH_CACHE.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
arch_dcache_range() function does not exist anymore, nor K_CACHE_WB
macro. Removing it entirely.
arch_dcache_flush_range() signature changed, so relevantly applying
these.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
RISC-V has a modular design. Some hardware with a custom interrupt
controller needs a bit more work to lock / unlock IRQs.
Account for this hardware by introducing a set of new
z_soc_irq_* functions that can override the default behaviour.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
esf_get_sp is used only when CONFIG_THREAD_STACK_INFO is enabled.
Just move this function around to be inside an ifdef guard.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Some RISC-V SoCs implement a mechanism for hardware supported stacking /
unstacking of registers during ISR / exceptions. What happens is that on
ISR / exception entry part of the context is automatically saved by the
hardware on the stack without software intervention, and the same part
of the context is restored by the hardware usually on mret.
This is currently not yet supported by Zephyr, where the full context
must be saved by software in the full fledged ESF. This patcheset is
trying to address exactly this case.
At least three things are needed to support in a general fashion this
problem: (1) a way to store in software only the part of the ESF not
already stacked by hardware, (2) a way to restore in software only the
part of the context that is not going to be restored by hardware and (3)
a way to define a custom ESF.
Point (3) is important because the full ESF frame is now composed by a
custom part depending on the hardware (that can choose which register to
stack / unstack and the order they are saved onto the stack) and a part
defined in software for the remaining part of the context.
In this patch a new CONFIG_RISCV_SOC_HAS_ISR_STACKING is introduced that
enables the code path supporting the three points by the mean of three
macros that must be implemented by the user in a soc_stacking.h file:
SOC_ISR_SW_STACKING, SOC_ISR_SW_UNSTACKING and SOC_ISR_STACKING_ESF
(refer to the symbol help for more details).
This is an example of soc_isr_stacking.h for an hardware that doesn't do
any hardware stacking / unstacking but everything is managed in
software:
#ifndef __SOC_ISR_STACKING
#define __SOC_ISR_STACKING
#if !defined(_ASMLANGUAGE)
#define SOC_ISR_STACKING_ESF_DECLARE \
struct __esf { \
unsigned long ra; \
unsigned long t0; \
unsigned long t1; \
unsigned long t2; \
unsigned long t3; \
unsigned long t4; \
unsigned long t5; \
unsigned long t6; \
unsigned long a0; \
unsigned long a1; \
unsigned long a2; \
unsigned long a3; \
unsigned long a4; \
unsigned long a5; \
unsigned long a6; \
unsigned long a7; \
unsigned long mepc; \
unsigned long mstatus; \
unsigned long s0; \
} __aligned(16)
#else
#define SOC_ISR_SW_STACKING \
addi sp, sp, -__z_arch_esf_t_SIZEOF; \
DO_CALLER_SAVED(sr);
#define SOC_ISR_SW_UNSTACKING \
DO_CALLER_SAVED(lr);
#endif /* _ASMLANGUAGE */
#endif /* __SOC_ISR_STACKING */
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Debug monitor needs to be configured to a low priority in order to be
useful for debugging (to prioritize other interrupts when waiting on a
breakpoint).
Added a config that configures the interrupt this way.
Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
It seems that currently it's impossible to create a custom
implementation for debug monitor exception without updating the vector
table (z_arm_debug_monitor maps to fault).
My proposition is to make this symbol weak, so that it can be overriden.
Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
Save FP user register and FP register file during context switch.
This change enables shared FP registers mode using CONFIG_FPU_SHARING.
Since there is no lazy stacking, the FPU registers will be saved regardless
of whether floating point calculations are performed in the threads when
CONFIG_FPU_SHARING is enabled. This require 72 additional bytes in the
stack memory.
Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
Use interrupts (with dedicated interrupt line) for irq_offload
instead of exception-based implementation.
That allows to implement IRQ_OFFLOAD without adding special code
to interrupt / exception path for IRQ_OFFLOAD handling and,
moreover, test the real interrupt code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
add DSP reg in context switch
add AGU reg in context switch to support XY mem
add thread option and API to dis/enable DSP switch
Signed-off-by: Siyuan Cheng <siyuanc@synopsys.com>
There is the possibility that when reconfiguring the static regions,
some data that must be accessed is temporarily not accesible due to the
change on the MPU regions configuration. Workaround by disabling MPU
when doing the reconfiguration, same as with dynamic regions, until BR
can be enabled.
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
Config NOCACHE_MEMORY depend on ARCH_HAS_NOCACHE_MEMORY_SUPPORT. Enable
ARCH_HAS_NOCACHE_MEMORY_SUPPORT for Cortex-R52 to run NXP S32Z/E with
nocache attibute.
Enable nocache in each driver use it.
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
Support I/D cache for Cortex-R52 to run with cache on NXP S32Z/E.
Make sure no data is present in the D-Cache before initializing mpu
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
The cache operations must be quick, optimized and possibly inlined. The
current API is clunky, functions are not inlined and passing parameters
around that are basically always known at compile time.
In this patch we rework the cache functions to allow us to get rid of
useless parameters and make inlining easier.
In particular this changeset is doing three things:
1. `CONFIG_HAS_ARCH_CACHE` is now `CONFIG_ARCH_CACHE` and
`CONFIG_HAS_EXTERNAL_CACHE` is now `CONFIG_EXTERNAL_CACHE`
2. The cache API has been reworked.
3. Comments are added.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
As we start to use _st32_huge_offset in other places (i.e DSP code)
introduce optimized version with address shifted instruction which
doesn't produce extra instruction.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Control shared interrupts enabling/disabling via IDU.
With that we can easily enable and disable them for all cores
in one place.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Make ARC_MP_PRIMARY_CPU_ID definition public so it can be used in
other ARC code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
In 0.15.2 SDK we specify r26 as default register for TLS cached
pointer, so it isn't used by compiler even if TLS support isn't
enabled. Restore the previous behavior - so if we don't use
TLS in Zephyr the register isn't stay reserved.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
The code in prep_c sets VBAR to relocate vector from 0x0, assuming the
low vector bit in SCTLR to be clear. This isn't the case on all
hardware, so set it explicitly to support those.
Signed-off-by: Théophile Ranquet <theophile.ranquet@gmail.com>
1. this header is no use for asm
2. if use xclib, this header include xclib stdbool, and expand to typedef
Signed-off-by: honglin leng <a909204013@gmail.com>
Arm provides a default address map defining default behaviors for
certain address ranges, which can be overlayed with additional regions
in the MPU. Users may also turn off this background map, so that only
regions explicitly programmed in the MPU are allowed.
This provides a Kconfig so that platforms using a non-standard address
map may disable the background address map and provide their own
explicit MPU regions.
Signed-off-by: Benjamin Gwin <bgwin@google.com>
This is a follow-up to commit f400c94adf.
Fix typos in names of introduced macros (*STR -> *SR) and cast their
values to uint32_t to avoid warnings reported for messages formatted
with %x.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
The compiler defines __ARC_TLS_REGNO__ as the number of the
register used for TLS variables. Use that instead of hard-coding
a specific register.
Signed-off-by: Keith Packard <keithp@keithp.com>
This commit updates the `_posix_zephyr_main` declaration to use the
return type of `int` instead of `void` when `CONFIG_CPP_MAIN=y` (i.e.
C++-compliant main() support is enabled) so that Zephyr applications
defining their main() in a C++ source file can make use of the proper
main() definition of `int main(void)` as required by the C++ standard.
Note that the forward declaration of `_posix_zephyr_main` is required
if and only if the main() is defined in a C++ source file (i.e. when
`CONFIG_CPP_MAIN=y`).
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Besides adding ARCH_HAS_CODE_DATA_RELOCATION, this patch also adds
support for the "sample_controller" SoC (used by qemu_xtensa) as
demonstration.
As Xtensa lacks a common linker script at the arch level, enabling it
for each platform will be a piecemeal effort. This patch adds it to the
`soc/xtensa/sample_controller` SoC. Basically, default RAMABLE_REGION is
set to be called "RAM", and hooks are inserted so that
gen_relocate_app.py can add the relevant linker bits.
Also, `tests/application_developent/code_relocation` was tweaked to
support the qemu_xtensa platform. Basically, add the relevant linker
script and ensure that relevant memory regions have their program header
(PHDR) associated.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
The is code duplication as one is in C, and the other is an assembly
macro. As there is no easy way to find out about this duplication,
adding a comment seems the best way to go.
Signed-off-by: Henri Xavier <datacomos@huawei.com>
In preparation for using CONFIG_*_ENDIAN instead of __BYTE_ORDER__, add
a reflection of CONFIG_BIG_ENDIAN that will allow users to conditionally
compile with #ifdef instead of #ifndef while keeping the default case
(little endian) as the first block in the conditional compilation.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The buffer contents returned from arch_gdb_reg_readone is a counted array
of bytes, not a C string. Use memcpy instead of strcpy for the failure
return path to avoid compiler warning about missing NUL termination.
Signed-off-by: Keith Packard <keithp@keithp.com>
