We need to use the mmu spin lock when invalidating the cache during
tlb shootdown, otherwise it is possible that this happens when another
thread is updating the page tables.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When the target has a cache way size (cache size / cache wasy) bigger
than the page size we have cache aliasing, since the number of bits
required by the cache index is bigger than the number of bits in the page
offset.
To avoid this problem we flush the whole cache on context switch or when
the current page table is changed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Swap page tables at exit of exception handler if we are going to
be restored to another thread context. Or else we would be using
the outgoing thread's page tables which is not going to work
correctly due to mapping and permissions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When adding a thread to a memory domain, we need to also update
the mapped page table if it is the current running thread on
the same CPU. If it's not on the same CPU, we need to notify
the other CPUs in case the thread is running in one of them.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
After changing content of page table(s), it is needed to notify
the other CPUs that the page table(s) have been changed so they
can do the necessary steps to use the updated version. Note that
the actual way to send IPI is SoC specific as Xtensa does not
have a common way to do this at the moment.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When kernel OOPS is raised, we need to actually go through
the process of terminating the offending thread, instead of
simply printing the stack and continue running. This change
employs similar mechanism to xtensa_arch_except() to use
illegal instruction to raise hardware exception, and going
through the fatal exception path.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Trigger exception on Xtensa requires kernel privileges. Add
a new syscall that is used when ARCH_EXCEPT is invoked from userspace.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This extracts the printing of fatal exception information into
its own function to declutter xtensa_excint1_c().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
There are known exceptions which are not fatal, and we need to
handle them properly by returning to the fixup addresses as
indicated. This adds the code necessary in the exception
handler for this situation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
When MMU is enabled, we need some scratch registers to preload
page table entries. So update gen_zsr.py to that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This changes the TLB misses handling back to the assembly
in user exception, and any page faults during TLB misses to be
handled in double exception handler. This should speed up
simple TLB miss handling as we don't have to go all the way to
the C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Userspace support for Xtensa architecture using Xtensa MMU.
Some considerations:
- Syscalls are not inline functions like in other architectures because
some compiler issues when using multiple registers to pass parameters
to the syscall. So here we have a function call so we can use
registers as we need.
- TLS is not supported by xcc in xtensa and reading PS register is
a privileged instruction. So, we have to use threadptr to know if a
thread is an user mode thread.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Simplify the logic around xtensa_mmu_init.
- Do not have a different path to init part of kernel
- Call xtensa_mmu_init from C
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Replace all autorefill helpers with only one that invalidates both,
DTLB and ITLB, since that is what is really needed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This register alias was originally introduced to allow A0 to be used
as a scratch register when handling exceptions from MOVSP
instructions. (It replaced some upstream code from Cadence that
hard-coded EXCSAVE1). Now the MMU code is now using too, and for
exactly the same purpose.
Calling it "ALLOCA" is only confusing. Rename it to make it clear
what it's doing.
Signed-off-by: Andy Ross <andyross@google.com>
TF-M only suports floating point in IPC model, not the SFN model.
Since floating point is a basic feature of the architecture and TF-M
has the limitation it makes more sense for the dependency to exist in
TF-M and and limit the TF-M model choice instead of limiting the
option to enable floating point.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
Some workarounds were introduced for intel cavs2.5 platform bring up.
It is not general so move them to platform code.
Signed-off-by: Rander Wang <rander.wang@intel.com>
Each arch platform may has a general arch_cpu_idle implementation but
each vendor may has a custom one, so this config will be used for vendor
to override it.
Some workarounds were introduced for intel cavs2.5 platform bring up.
It is not general so move them to platform code.
Signed-off-by: Rander Wang <rander.wang@intel.com>
When building without optimizations and with only one core the linker
does not throw away arch_start_cpu and we get an undefined reference to
x86_ap_start
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This moves the k_* memory management functions from sys/ into
kernel/ includes, as there are kernel public APIs. The z_*
functions are further separated into the kernel internal
header directory.
Also made a quick change to doxygen to group sys_mem_* into
the OS Memory Management group so they will appear in doc.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
z_mp_entry has been removed from Xtensa architecture.
So there is no need for a function declaration. Remove it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
NXP SYSMPU is used in other SoCs besides the Kinetis series. For
devices like S32K1xx, its bus interface clock lacks of clock gating
and it's driven by the system clock. Hence, only enable the module
clock for the Kinetis series.
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
This patch changes the section of riscv_cpu_wake_flag variable to
noinit from bss to fix hangup of RISC-V multicore boot if hart0 is
not boot hart (CONFIG_RV_BOOT_HART != 0).
Current boot sequence initializes a riscv_cpu_wake_flag to -1 but
this variable is unintentionally changed to 0 by boot hart.
This is because the variable is placed in bss section so this patch
changes the section of the variable to noinit.
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
This patch fixes hangup of RISC-V multicore boot.
Currently boot sequence uses a riscv_cpu_wake_flag to notify wakeup
request for secondary core(s).
But initial value of riscv_cpu_wake_flag is undefined, so current
mechanism is going to hangup if riscv_cpu_wake_flag and mhartid of
secondary core have the same value.
This is an example situation of this problem:
- hart1: check riscv_cpu_wake_flag (value is 1) and end the loop
- hart1: set riscv_cpu_wake_flag to 0
- hart0: set riscv_cpu_wake_flag to 1
hart0 expects it will be changed to 0 by hart1 but it
has never happened
Note:
- hart0's mhartid is 0, hart1's mhartid is 1
- hart0 is main, hart1 is secondary in this example
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
The FMADD, FMSUB, FNMSUB and FNMADD instructions occupy major opcode
spaces of their own, separate from LOAD-FP/STORE-FP and OP-FP spaces.
Insert code to cover them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
elf_rela_t contains elf_rel_t exactly and contains an additional
field at the end. Therefore pointers of that type can be used for
both types, making the code generic.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Most of the public APIs in `riscv_plic.h`
(except `riscv_plic_get_irq` & `riscv_plic_get_dev`) expect the
`irq` argument to be in Zephyr-encoded format, instead of the
previously `irq_from_level_2`-stripped version. The first level
IRQ is needed by `intc_plic` to differentiate between the
parent interrupt controllers, so that correct ISR offset can be
obtained using the LUT in `sw_isr_common`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Instead of using a macro guard to prevent functions in
`sw_isr_common.c` from getting compiled when
`CONFIG_DYNAMIC_INTERRUPTS` isn't enabled, do that in
`CMakeLists.txt` instead.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Refactor multi-level IRQ related code from `sw_isr_common.c` to
`multilevel_irq.c` to simplify `sw_isr_common` & macrologies.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Relocate new and existing internal software-managed table
access functions from the public `sw_isr_table.h` into a
private header that should only be accessed internally.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Change the internal function to `get_parent_entry`, which
returns the entire entry of table.
Store the parent interrupt controller device in the
`irq_parent_offset` table, and added 2 helper functions to:
1. determine the parent interrupt controller based on the IRQ
2. determine the IRQ of the parent interrupt controller
Declare the `struct _irq_parent_entry` in the header and added
`-` suffix to the struct so that it can be used to test the
functions in testsuites.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
This change adds support for the CONFIG_ARM_CUSTOM_INTERRUPT_CONTROLLER
option on Cortex-M platforms. While all Cortex-M platforms have a
NVIC controller some custom SoCs may have additional IRQ controllers
or custom handling. This change allows those SoCs to modify this
bahaviour without having to place platform specific logic inside
applications or drivers.
Signed-off-by: Corey Wharton <xodus7@cwharton.com>
The Cortex ARM documentation states that the DC IVAC instruction
requires write access permission to the virtual address (VA);
otherwise, it may generate a permission fault.
Therefore, it is needed to avoid invalidating read-only memory
after the memory map operation.
This issue has been produced by commit c9b534c.
This commit resolves the issue #64758.
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
Some platforms already have .bss section zeroed-out externally before the
Zephyr initialization and there is no sence to zero it out the second time
from the SW.
Such boot-time optimization could be critical e.g. for RTL Simulation.
Signed-off-by: Alexander Razinkov <alexander.razinkov@syntacore.com>
This commit introduces SMP support into Cortex-A/R aarch32 architecture.
For now, this only supports multiple core start together and only allow
one CPU initialize system as primary core, others loop at the beginning
as the secondary cores and wait for wake up.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit introduce 'USE_SWITCH' feature into cortex-A/R(aarch32)
architecture
For introducing USE_SWITCH, the exception entry and exit are unified via
`z_arm_cortex_ar_enter_exc` and `z_arm_cortex_ar_exit_exc`. All
exceptions including ISR are using this way to enter and exit exception
handler.
Differentiate exception depth and interrupt depth. Allow doing
context switch when exception depth greater than 1 but not allow doing
this when interrupt depth greater than 1.
Currently, USE_SWITCH doesn't support FPU_SHARING and USERSPACE.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
Store the current CPU's struct _cpu instance into TPIDRURO, so that the
CPU core can get its struct _cpu instance by reading TPIDRURO. This is
useful in the SMP system.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
Replace the TLS base address pointer from TPIDRURO to TPIDRURW.
The difference between them is that TPIDRURO is read-only in user mode
but TPIDRURW isn't. So TPIDRURO is much more suitable for store
the address of _kernel.CPU[n]. For this reason, this commit replaces
the base pointer of the TLS area.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
MMU or MPU unit need to be initialized by its own CPU.
- Primary core initialize MMU or MPU unit in z_arm_prep_c.
- Secondary core initialize MMU or MPU unit in z_arm_secondary_start.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
Use ACPI_MADT_LOCAL_APIC instead of struct acpi_madt_local_apic. In the
same go, switch to IF_ENABLED from ifdef - slightly more readable, and
this keeps some static analyzers happy (e.g. upstream Compliance check).
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
HCR_EL2 is configured to certain value by some
loaders such as Uboot on some arm64 boards(such as roc_rk3568_pc),
When HCR_EL2.TGE, HCR_EL2.AMO and HCR_EL2.IMO bits are
set to 1, some unpredictable behaviors may occur during
zephyr boot. So we clear these bits to avoid it.
Signed-off-by: Charlie Xiong <1981639884@qq.com>
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Locate common mpu code together with other arm / nxp mpu code in the
arch folder where it logically belongs.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Relocate multi-level interrupts APIs out of `irq.h` into
a new file named `irq_multilevel.h` to provide cleaner
separation between typical irq & multilevel ones.
Added preprocessor versions of `irq_to_level_x` as `IRQ_TO_Lx`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
This commit re-implements the SPARC V8 ABI "Flush windows" software
trap. The trap is generated by C++ compilers for exceptions and also by
the C standard library function longjmp().
There were two issues with the previous implementation:
1. It did reads and writes via the stack pointer of the trap window,
which is not defined.
2. It executed with traps enabled but without the processor run-time
state set to safely handle traps. In particular there was no valid
stack for trap processing. Even though interrupt priority was set to
highest level, the behavior at other traps was not deterministic. For
example non-maskable interrupt (15) trap or bus error trap for
instruction fetch.
This new implementation does not store backup copies of CPU registers to
the stack, and it executes with traps disabled.
Fixes#63901
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Add pm cpu ops to call the platform specific implementations for
bringing up secondary cores.
Signed-off-by: Lingutla Chandrasekhar <quic_lingutla@quicinc.com>
HAS_DTS has become a redundant option. All Zephyr architectures now
select this option, meaning devicetree has become a de-facto
requirement. In fact, if any board does not provide a devicetree
source, the build system uses an empty stub, meaning the devicetree
machinery always runs.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
_k_neg_eagain is there for used in assembly where including
errno.h is not possible. However, the usage in ARM was simply
to assign value to swap_return_value in a C file, which is
no need to use _k_neg_eagain as errno.h can be included.
So change that to use -EAGAIN directly. Saves 4 bytes in
built binaries in rodata.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The exclusive load/store instructions don't work well when MMU and cache
are disabled on some cores e.g. Cortex-A72. Change it to voting lock[1]
to select the primary core when multi-cores boot simultaneously.
The voting lock has reasonable but minimal requirements on the memory
system.
[1] https://www.kernel.org/doc/html/next/arch/arm/vlocks.html
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This provides custom memory range check functions as
it gets a bit complicated with cached/uncached regions.
These functions are marked as __weak so SoC or board
can override these if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There were a few small nits that were pointed out in the initial PR.
Fixes the LOG_ macro in the arm elf implementation, replaces a few stray
mentions of modules in llext.h with extensions.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prepare data for upcoming
call to z_arm_on_enter_cpu_idle(). The main difference is that
z_arm_on_enter_cpu_idle_prepare() hook is called before interrupts
are disabled.
Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
Adds the linkable loadable extensions (llext) subsystem which provides
functionality for reading, parsing, and linking ELF encoded executable
code into a managed extension to the running elf base image.
A loader interface, and default buffer loader implementation,
make available to the llext subsystem the elf data. A simple management
API provide the ability to load and unload extensions as needed. A shell
interface for extension loading and unloading makes it easy to try.
Adds initial support for armv7 thumb built elfs with very specific
compiler flags.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Co-authored-by: Chen Peng1 <peng1.chen@intel.com>
Co-authored-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Armv8-m baseline support various instruction carrying exclusive-monitor and
acquire-release semantic. By adding this guard we let armv8-m.baseline
fall-back to arch defined or compiler built-in support for atomic
operations.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Add an ARCH_SUPPORTS_ROM_START kconfig symbol to mark architectures
that support ROM_START as an argument to zephyr_linker_sources.
This was added so that features relying on this feature could
depend on this kconfig symbol.
Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
This adds support for using coredump with Xtensa DC233C core,
which are being used by qemu_xtensa and qemu_xtensa_mmu.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The old CONFIG_UART_NS16550_ACCESS_IOPORT has been used to
indicate whether to access the NS16550 UART via IO port
before device tree is used to describe hardware. Now we have
device tree, and we can specify whether a particular UART
needs to be accessed via IO port using property io-mapped.
Therefore, CONFIG_UART_NS16550_ACCESS_IOPORT is no longer
needed (and thus also CONFIG_UART_NS16550_SIMULT_ACCESS).
Remove these two kconfigs and modify code to use device tree
to figure out how to access the UART hardware.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_switch() is basically an alias to xtensa_switch() so
we can mark arch_switch() as ALWAYS_INLINE to avoid another
function call, especially when no optimization is used when
debugging.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Let's make this official: we use the suffix `_MASK` for the define
carrying the GENMASK for the attributes, and the suffix `_GET(x)` for
the actual macro extracting the attributes.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To make the stack guard works well, clean and refine the MPU code. To
save the MPU regions (the number of MPU regions are limited), we choose
to remove the guard region. Comparing to add an individual region to
guard the stack, removing the guard region can at least save 2 regions
per core.
Similarly with userspace, the stack guard will leverage the dynamic
regions switching mechanism which means we need a region switch during
the context switch. Otherwise, the other option is using stack guard
region, but this is very limited since the number of MPU regions is
limited.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Refactor the stack relevant macros to prepare to introduce the stack
guard. Also add comments about the changes related to stack layout.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add the stack check function z_arm64_stack_corruption_check at
z_arm64_fatal_error to handle the stack overflow triggered by the
hardware region.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce the ARM64_STACK_PROTECTION config. This option leverages the
MMU or MPU to cause a system fatal error if the bounds of the current
process stack are overflowed. This is done by preceding all stack areas
with a fixed guard region. The config depends on MPU for now since MMU
stack protection is not ready.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Clean the thread->arch during the arch_new_thread to avoid unexpected
behavior. If the thread struct is allocated from heap or in stack, the
data in thread->arch might be dirty.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Accessing mem before mmu or mpu init will cause a cache coherence issue.
To avoid such a problem, move the safe exception stack init function
after the mmu or mpu is initiated.
Also change the data section attribute from INNER_SHAREABLE to
OUTER_SHAREABLE. Otherwise there will be a cache coherence issue during
the memory regions switch. Because we are using background region to do
the regions switch, and the default background region is
OUTER_SHAREABLE, if we use INNER_SHAREABLE as the foreground region,
then we have to flush all cache regions to make sure the cached values
are right. However, flushing all regions is too heavy, so we set
OUTER_SHAREABLE to fix this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Dom0less is Xen mode without privileged domain. All guests are created
according to hypervisor device tree configuration on boot. Thus, there
is no Dom0 with console daemon, that usually manages console output
from domains.
Zephyr OS contains 2 serial drivers related to Xen hypervisor: regular
with console shared page and consoleio-based. The first one is for
setups with console daemon and usually was used for Zephyr DomU guests.
The second one previously was used only for Zephyr Dom0 and had
corresponding Kconfig options. But consoleio is also used as interface
for DomU output on Dom0less setups and should be configurable without
XEN_DOM0 option.
Add corresponding XEN_DOM0LESS config to Xen Kconfig files and proper
dependencies in serial drivers.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Co-authored-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This is the final step in making the `zephyr,memory-attr` property
actually useful.
The problem with the current implementation is that `zephyr,memory-attr`
is an enum type, this is making very difficult to use that to actually
describe the memory capabilities. The solution proposed in this PR is to
use the `zephyr,memory-attr` property as an OR-ed bitmask of memory
attributes.
With the change proposed in this PR it is possible in the DeviceTree to
mark the memory regions with a bitmask of attributes by using the
`zephyr,memory-attr` property. This property and the related memory
region can then be retrieved at run-time by leveraging a provided helper
library or the usual DT helpers.
The set of general attributes that can be specified in the property are
defined and explained in
`include/zephyr/dt-bindings/memory-attr/memory-attr.h` (the list can be
extended when needed).
For example, to mark a memory region in the DeviceTree as volatile,
non-cacheable, out-of-order:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_VOLATILE |
DT_MEM_NON_CACHEABLE |
DT_MEM_OOO )>;
};
The `zephyr,memory-attr` property can also be used to set
architecture-specific custom attributes that can be interpreted at run
time. This is leveraged, among other things, to create MPU regions out
of DeviceTree defined memory regions on ARM, for example:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-region = "NOCACHE_REGION";
zephyr,memory-attr = <( DT_ARM_MPU(ATTR_MPU_RAM_NOCACHE) )>;
};
See `include/zephyr/dt-bindings/memory-attr/memory-attr-mpu.h` to see
how an architecture can define its own special memory attributes (in
this case ARM MPU).
The property can also be used to set custom software-specific
attributes. For example we can think of marking a memory region as
available to be used for memory allocation (not yet implemented):
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_NON_CACHEABLE |
DT_MEM_SW_ALLOCATABLE )>;
};
Or maybe we can leverage the property to specify some alignment
requirements for the region:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_CACHEABLE |
DT_MEM_SW_ALIGN(32) )>;
};
The conventional and recommended way to deal and manage with memory
regions marked with attributes is by using the provided `mem-attr`
helper library by enabling `CONFIG_MEM_ATTR` (or by using the usual DT
helpers).
When this option is enabled the list of memory regions and their
attributes are compiled in a user-accessible array and a set of
functions is made available that can be used to query, probe and act on
regions and attributes, see `include/zephyr/mem_mgmt/mem_attr.h`
Note that the `zephyr,memory-attr` property is only a descriptive
property of the capabilities of the associated memory region, but it
does not result in any actual setting for the memory to be set. The
user, code or subsystem willing to use this information to do some work
(for example creating an MPU region out of the property) must use either
the provided `mem-attr` library or the usual DeviceTree helpers to
perform the required work / setting.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Add Xen domctl API implementation for Zephyr as control domain.
Previously Zephyr OS was used as unprivileged Xen domain (Domain-U),
but it also can be used as lightweight Xen control domain (Domain-0).
To implement such fuctionality additional Xen interfaces are needed.
One of them is Xen domain controls (domctls) - it allows to create,
configure and manage Xen domains.
Also, used it as a possibility to update files copyright and licenses
identifiers in touched files.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Xen-related Kconfig options were highly dependand on BOARD/SOC xenvm.
It is not correct because Xen support may be used on any board and SoC.
So, Kconfig structure was refactored, now CONFIG_XEN is located in
arch/ directory (same as in Linux kernel) and can be selected for
any Cortex-A arm64 setup (no other platforms are currently supported).
Also remove confusion in Domain 0 naming: Domain-0, initial domain,
Dom0, privileged domain etc. Now all options related to Xen Domain 0
will be controlled by CONFIG_XEN_DOM0.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
After 79d0bf39b8 was merged, the inclusion
of <zephyr/acpi/acpi.h> with CONFIG_ACPI=n caused a build failure because
<acpica/source/include/acpi.h> could no longer be included due to the
inlcude path not being injected anymore.
Fix this by guarding the header inclusion when CONFIG_ACPI
is not set.
Fixes#62679.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This commit provides the users a way to disconnect dynamic
interrupts. This is needed because we don't want to keep
piling up ISR/arg pairs until the number of registrable
clients is reached.
This feature is only relevant for shared and dynamic interrupts.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
This works by overwriting z_isr_install()'s definition
(possible since the symbol is now weak) with our own definiton.
Whenever trying to register a new ISR/arg pair, z_isr_install()
will check to see if the interrupt line is already in use. If it's
not then it will not share the interrupt and will work exactly
as the old z_isr_install(), meaning it will just write the new
ISR/arg pair to _sw_isr_table.
If the interrupt line is already being used by an ISR/arg pair
then that line will become shared, meaning we'll overwrite
_sw_isr_table with our own (z_shared_isr, z_shared_sw_isr_table[irq])
pair.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
Since the shared IRQ code will also use the same logic to compute
the _sw_isr_table index, move the computing logic to a separate
function: z_get_sw_isr_table_idx(), which can be used by other
modules.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
This commit introduces all the necessary changes for
enabling the usage of shared interrupts.
This works by using a second interrupt table: _shared_sw_isr_table
which keeps track of all of the ISR/arg pairs sharing the same
interrupt line. Whenever a second ISR/arg pair is registered
on the same interrupt line using IRQ_CONNECT(), the entry in
_sw_isr_table will be overwriten by a
(shared_isr, _shared_sw_isr_table[irq]) pair. In turn, shared_isr()
will invoke all of the ISR/arg pairs registered on the same
interrupt line.
This feature only works statically, meaning you can only make use
of shared interrupts using IRQ_CONNECT(). Attempting to dynamically
register a ISR/arg pair will overwrite the hijacked _sw_isr_table
entry.
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
LLVM LLD fills empty spaces (created using ALIGN() or moving the
location counter) in executable segments with TrapInstr pattern,
e.g. for ARM the TrapInstr pattern is 0xd4d4d4d4. GNU LD fills
empty spaces with 0x00 pattern.
We may want to have some section (e.g. rom_start) filled with 0x00,
e.g. because MCU can interpret the pattern as a configuration data.
Signed-off-by: Patryk Duda <pdk@semihalf.com>
This commit separates kernel_arch_func.h into two header file,
'cortex_a_r/kernel_arch_func.h' and 'cortex_m/kernel_arch_func.h', it
also removes some functions which is empty.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit separates cpu_idle.S into two asm files,
'cortex_a_r/cpu_idle.S' and 'cortex_m/cpu_idle.S'.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit separates thread.c into two source files,
'cortex_a_r/thread.c' and 'cortex_m/thread.c, it also introduces some
changes.
1. Migrate 'thread.c' and 'cortex_m/thread.c'.
2. Migrate 'thread.c' and 'cortex_a_r/thread.c'
3. Remove the 'z_arm_mpu_stack_guard_and_fpu_adjust' function as this is
obviously written for Cortex-M architecture.
4. Remove the 'z_arm_prepare_switch_to_main' function as this is only
used by Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit Separate 'prep_c.c' into two file based on the architecture,
one is 'cortex_m/prep_c.c', the other is 'cortex_a_r/prep_c.c'
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
There are too many differences between Cortex-A/R and Cortex-M on irq
code, e.g. Cortex-A/R use GIC and Cortex-M uses NVIC. For reducing
the complexity and easier to maintain, this commit separates irq_manage.c
and isr_wrapper.S into two different parts based on the architecture.
This commit also Removes the part related to the option
'CONFIG_ARM_SECURE_FIRMWARE' in 'cortex_a_r/irq_manage.c' because
this code is written for the Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
There are too many differences between Cortex-A/R and Cortex-M on swap
code. For reducing the complexity and easier to maintain, this commit
introduces the following major changes
1. Separate swap.c and swap_helper.S into two different parts based on
the architecture.
2. Rename 'z_arm_pendsv' to 'z_arm_do_swap' for Cortex-A/R.
3. Removes the part related to the option 'CONFIG_BUILTIN_STACK_GUARD'
in 'cortex_a_r/swap_helper.S' because this code is written for
the Cortex-M architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
'irq_relay.S' is only used for Arm Cortex-M architecture, so it's better
to place it to the 'arch/arm/core/cortex_m' directory.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit follows the parent commit work.
This commit introduces the following major changes.
1. Move all directories and files in 'include/zephyr/arch/arm/aarch32'
to the 'include/zephyr/arch/arm' directory.
2. Change the path string which is influenced by the changement 1.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
It doesn't make sense to keep the aarch32 directory in the
'arch/arm/core' directory as the aarch64 has been moved out.
This commit introduces the following major changes.
1. Move all directories and files in 'arch/arm/core/aarch32' to
'arch/arm/core' and remove the 'arch/arm/core/aarch32' directory.
2. Move all directories and files in 'arch/include/aarch32' to
'arch/include' and remove the 'arch/include/aarch32' directory.
3. Remove the nested including in the 'arch/include/kernel_arch_func.h'
and 'arch/include/offsets_short_arch.h' header files.
4. Change the path string which is influenced by the changement 1
and 2.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
The prep_c.c source doesn't depend on the ACPI library. As ACPI support
now requires the ACPICA module, the extra header breaks projects using
x86 without ACPI support.
Signed-off-by: Keith Short <keithshort@google.com>
The old acpi implimentation is replaced with acpica interface
and updated x86 arch porting with the new interface.
Signed-off-by: Najumon B.A <najumon.ba@intel.com>
RISC-V Spec requires minimum alignment of trap handling code to be
dependent from MTVEC.BASE field size. Minimum alignment for RISC-V
platforms is 4 bytes, but maximum is platform or application-specific.
Currently there is no common approach to align the trap handling
code for RISC-V and some platforms use custom wrappers to align
_isr_wrapper properly.
This change introduces a generic solution,
CONFIG_RISCV_TRAP_HANDLER_ALIGNMENT configuration option which sets
the alignment of a RISC-V trap handling code.
The existing custom solutions for some platforms remain operational,
since the default alignment is set to minimal possible (4 bytes)
and will be overloaded by potentially larger alignment of custom solutions.
Signed-off-by: Alexander Razinkov <alexander.razinkov@syntacore.com>
Add a property to the native_simulator target, to collect
the libraries we want to link with the runner, instead of
abusing the link options to pass them.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
As per #26393, Local APIC is using Kconfig based option for
the base address. This patch adds DTS binding support in the driver,
just like its conunter part I/O APIC.
Signed-off-by: Umar Nisar <umar.nisar@intel.com>
Disable d-cache on the fvp_baser_aemv8r_aarch32 platform because there
are some issues when d-cache is enabled for now.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This header is private and included only in architecture code, no need for
it to be in the top of the public include directory.
Note: This might move to a more private location later. For now just
cleaning up the obvious issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
During Zephyr boot with SMP enabled,
while z_smp_init is not completed yet and only boot core is running,
incorrect code in broadcast_ipi API will cause following:
1. Generate IPI even if other cores are not booted.
2. Incorrect setting of sgi1r register.
3. All the affinity(1/2/3) value will be incorrect.
Signed-off-by: Chirag Kochar <chirag.kochar@intel.com>
Enhanced arch_start_cpu so if a core is not available based on pm_cpu_on
return value, booting does not halt. Instead the next core in
cpu_node_list will be tried. If the number of CPU nodes described in the
device tree is greater than CONFIG_MP_MAX_NUM_CPUS then the extra cores
will be reserved and used if any previous cores in the cpu_node_list fail
to power on. If the number of cores described in the device tree matches
CONFIG_MP_MAX_NUM_CPUS then no cores are in reserve and booting will
behave as previous, it will halt.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
In some shared-memory use cases between Zephyr and other parallel
running OS, for data coherent, the non-cacheable normal memory
mapping is needed.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
This adds a function arch_xtensa_mmu_post_init() which can
be implemented on the SoC layer to perform additional MMU
initialization steps if necessary.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...VECBASE during MMU initialization. This is to make sure
that we can use the TLB miss handling in the exception
vector after we have moved back the VECBASE during MMU
initialization. Or else we would be forever stuck in ITLB
miss because the exception vectors are not in TLB and we
cannot populate the TLB because those vectors are not in
TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...and move it to xtensa_mmu_priv.h.
This would allow the SoC layer to use the RING number if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds arch_reserved_pages_update() which is called in
k_mem_manage_init() to reserve some physical pages so they
are not re-mapped. This is due to Zephyr's linker scripts
for Xtensa which often puts something before z_mapped_start
(aka .text, for example, vecbase). That space needs to be
reserved or else k_mem_map() would be mapping those that
could result in faults.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of only handling data TLB misses for VECBASE, change it
to handle all data TLB misses in the double exception handler.
It is because we may encounter data TLB misses when trying to
preload page table entries inside user exception handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
If CONFIG_XTENSA_RPO_CACHE is not enabled, it can be assumed
that memory is not double mapped in hardware for cached and
uncached access. So we can specify those regions to have
cache via TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Simply using __data_start and __data_end is not enough as
it leaves out kobject regions which is supposed to be
near .data section. So use _image_ram_start and
_image_ram_end instead to enclose data, bss and various
kobject regions (among others).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
During MMU initialization, we clear TLB way 6 to remove all
identity mapping. Depending on CPU configuration, there are
certain number of entries per way. So use the number from
core-isa.h to clear enough entries instead of hard-coded
number 8. Specifying an entry number outside of permitted
range may result in CPU reacting in weird way so better to
avoid that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the identity map of the first 512MB in TLB way 6.
Or else it would interfere with mapped entries resulting in
double mapped TLB exception.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
MMU needs to be initialized before going in to C, so
z_xtensa_mmu_init() is called in crt1.S before call
to z_cstart(). Note that this is the default case
and crt1.S can be disabled if board and SoC desire
to do so.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Turns out not all MMU enabled Xtensa cores have vaddrstatus,
vaddr0 and vaddr1. And there does not seem to be a way to
determine whether they are available. So remove them from
the exception printout for now.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
To allow reusing the path between boards,
instead of defining it in the board, let's define it in
the arch cmake file.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
CONFIG_COVERAGE has been incorrectly used to
change other kconfig options (stack sizes, etc)
code defaults, as well as some samples behaviour,
which should not have dependend on it.
Instead those should have depended on COVERAGE_GCOV,
which, being the one which adds special code and
temporary RAM storage for embedded targets,
require changes to many features.
When building for the native targets, all this was
unnecessary.
=> Fix the dependency.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The CMSIS module glue code was part of arch/ directory. Move it to
modules/cmsis, and provide a single entry point for it: cmsis_core.h.
This entry header will include the right CMSIS header (M or A/R).
To make this change possible, CMSIS module Kconfig/CMake are declared as
external, allowing us to add a new Zephyr include directory.
All files including CMSIS have been updated.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
With gcc from the zephyr sdk and -Wold-style-declaration is giving this
output:
zephyr/arch/arm/core/aarch32/cortex_a_r/fault.c:101:1: warning:
'inline' is not at beginning of declaration [-Wold-style-declaration]
101 | static void ALWAYS_INLINE
z_arm_fpu_caller_save(struct __fpu_sf *fpu)
| ^~~~~~
I searched to all of the source code to find these further occurances
where inline is not at the beginning of a function declaration.
Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
Move the function prototype before declaration of the function itself.
Maybe the prototype could be removed altogether?
Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
Is rsdp_phys is assigned in the loop below code jumps to found label,
otherwise we return and value rsdp_phys is never used after 0
assignment.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
This enables -Wshadow to warn about shadow variables on
in tree code under arch/, boards/, drivers/, kernel/,
lib/, soc/, and subsys/.
Note that this does not enable it globally because
out-of-tree modules will probably take some time to fix
(or not at all depending on the project), and it would be
great to avoid introduction of any new shadow variables
in the meantime.
Also note that this tries to be done in a minimally
invasive way so it is easy to revert when we enable
-Wshadow globally. Source files under modules/, samples/
and tests/ are currently excluded because there does not
seem to be a trivial way to add -Wshadow there without
going through all CMakeLists.txt to add the option
(as there are 1000+ files to change).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The current mechanism of the MPU region switching configures and
reprograms the regions (including inserting, splitting the dynamic
region, and flushing the regions to the registers) every time during the
context switch. This, not only causes a large usage of the kernel stack
but also a lower performance.
To improve it, move the configuration operations ahead to make sure the
context swtich only flushes the current thread regions to the registers
and does not configure the regions anymore. To achieve this, configure
the regions during any operations related to partitions (partition
add/remove, and domain add/remove thread), flush the sys_dyn_regions if
the current thread is the privileged thread, and flush the thread's own
regions if it's a user thread.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Using BR(background region) during the flushing regions instead of
enabling/disabling the MPU which is a heavy operation.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add support for processing the Fault Status Registers and recoverable
data abort for Armv8-R AArch32.
Based on Arm Architecture Reference Manual Supplement Armv8, for the
Armv8-R AArch32 architecture profile (ARM DDI 0568).
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
Architecturally, Set/Way operations are not guaranteed to affect all
caches prior to the PoC, and may require other IMPLEMENTATION DEFINED
maintenance (e.g. MMIO control of system-level caches).
First of all this patch was designed for Xen domain Zephyr build, set/way
ops are not easily virtualized by Xen. S/W emulation is disabled, because
IP-MMU is active for Dom0. IP-MMU is a IO-MMU made by Renesas, as any good
IO-MMU, it shares page-tables with CPU. Trying to emulate S/W with IP-MMU
active will lead to IO-MMU faults. So if we build Zephyr as a Xen Initial
domain, it won't work with cache management support enabled.
Exposing set/way cache maintenance to a virtual machine is unsafe, not
least because the instructions are not permission-checked, but also
because they are not broadcast between CPUs.
In this commit, VA data invalidate invoked after every mapping instead of
using set/way instructions on init MMU. So, it was easy to delete
sys_cache_data_invd_all from enable MMU function, becase every adding of
a new memory region to xclat tabes will cause invalidating of this memory
and in this way we sure that there are not any stale data inside.
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
The zephyr_library() used to only be called if
CONFIG_GEN_ISR_TABLES is enabled. Which means that any
zephyr_library_*() calls are putting things into another
library if CONFIG_GEN_ISR_TABLES is disabled. So pull
the call to zephyr_library() to outside. Also moves
semihost.c into the library.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When building the 32bit native board targets variants
for x86(-64) hosts, gcc will promote float literals to double
(See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92875 )
This can result in unexpected comparison differences.
This is due to the compiler using the 8087 float mode by
default.
Instead let's tell the compiler to use the SSE float path,
which is the default for 64 bit x86-64 builds.
The assumption that any x86 host used for development
will have SSE support should be safe enough.
For more background see
https://github.com/zephyrproject-rtos/zephyr/issues/61345
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
In print_pmp_entries(), start and end are function arguments
and yet another start and end are declared inside the for
loop. So rename the function arguments to fix shadow variables
warning.
The changes in csr_*() macros are needed to avoid shadowing
__v when nesting those functions together, for example,
csr_write(..., csr_read(...)).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add new option to use thread local storage for stack
canaries. This makes harder to find the canaries location
and value. This is made optional because there is
a performance and size penalty when using it.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Instead of implementing a custom power off API (pm_system_off),
implement the sys_poweroff hook, and indicate power off is supported by
selecting HAS_POWEROFF. Note that according to the PSCI specification
(DEN0022E), the SYSTEM_OFF operation does not return, however, an error
is printed and system is halted in case this occurs.
Note that the pm_system_off has also been deleted, from now on, systems
supporting PSCI should enable CONFIG_POWEROFF and call the standard
sys_poweroff() API.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
In some very rare cases (< 1/1000 runs), in very loaded machines,
a race in the glibc pthread_cancel() seems to be triggered.
In this the cancelled thread cleanup overtakes the pthread_cancel()
code, and frees the pthread structure before pthread_cancel()
has finished, resulting in a dereference into already
free'd memory, and therefore a segfault.
Calling pthread_cancel() during cleanup is not required beyond
preventing a valgrind memory leak report (all threads will be
stopped immediately on exit).
Therefore we stop doing this, to avoid this very rare crashes.
This issue was reproduced in Ubuntu 22.04, with its default
gcc 11.3.0 and glibc 2.35.
The issue may also have been seen very rarely in Zephyr's CI.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add Kconfig RISCV_SOC_HAS_CUSTOM_SYS_IO symbol so that a riscv
SoC can set to specify that it has a custom implementation for
sys_io functions.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Refactor the ESP32 target SOCs together with
all related boards. Most braking changes includes:
- changing the CONFIG_SOC_ESP32* to refer to
the actual soc line (esp32,esp32s2,esp32s3,esp32c3)
- replacing CONFIG_SOC with the CONFIG_SOC_SERIES
- creating CONFIG_SOC_FAMILY_ESP32 to embrace all
the ESP32 across all used architectures
- introducing CONFIG_SOC_PART_NUMBER_* to
provide a SOC model config
- introducing the 'common' folder to hide all
commonly used configs and files.
- updating west.yml to reflect previous changes in hal
Signed-off-by: Marek Matej <marek.matej@espressif.com>
xt-clang likes to remove any consecutive NOPs more than 8. So
we need to force the function to have no optimization to avoid
this behavior and to retain all those NOPs.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a Kconfig to introduce the Xtensa specific
arch_spin_relax() which can do more NOPs. Some Xtensa SoCs
may need more NOPs after failure to lock a spinlock,
especially under SMP. This gives the bus extra time to
propagate the RCW transactions among CPUs.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In GNU LD, the location counter (the 'dot' variable) always refers to
the byte offset from the start of current object as mentioned in
documentation[1]:
```
'.' actually refers to the byte offset from the start of the current
containing object. Normally this is the SECTIONS statement, whose start
address is 0, hence '.' can be used as an absolute address. If '.' is
used inside a section description however, it refers to the byte offset
from the start of that section, not an absolute address.
```
For example, if the section 'rom_start':
rom_start : {
. = 0x400;
_vector_start = ABSOLUTE(.);
} > FLASH
has a starting address of 0x8000000, then _vector_start will be
0x8000400
However, behavior of LLVM LLD is quite different, the value of the
location counter is always absolute (see discussion [2]), so in the
example above, the linker will return error, because it will interpret
'. = 0x400' as an attempt to move the location counter backwards.
It could be fixed by changing line to '. += 0x400' (#54796) which will
move the location counter by 0x400 for both linkers, but it would work
only when we are at the beginning of section. Consider the following
example:
rom_start : {
. = 0x400;
KEEP(*(.boot_hdr.conf))
. = 0x1000;
KEEP(*(.boot_hdr.ivt))
KEEP(*(.boot_hdr.data))
KEEP(*(.boot_hdr.dcd_data))
. = 0x2000;
_vector_start = .;
} > FLASH
In this case, _vector_start will be 0x2000, but if we change
'. = 0x2000' to '. += 0x2000', then the value of _vector_start depends
on size of data in input sections (but it's 0x3000 at least).
Actually, this example comes from final linker script when compiling
firmware for mimxrt1170_evk_cm7 board. This board failed to boot
(#55296) after #54796 was merged.
This patch introduces method compatible with both linkers. We calculate
relative offset from the beginning of the section and use that value to
calculate number of bytes by which we should move the location counter
to get CONFIG_ROM_START_OFFSET.
[1] https://sourceware.org/binutils/docs/ld/Location-Counter.html
[2] https://discourse.llvm.org/t/lld-location-counter-inside-objects
Signed-off-by: Patryk Duda <pdk@semihalf.com>
When the frame-pointer based unwinding is enabled, the stop condition
for the stack backtrace is (FP == NULL).
Set FP to 0 before jumping to C code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The current implementation causes a lockup of the core when the exception
originates from an invalid/unreachable pc. This fix first verifies on
armv6-m and armv8-m.base that pc was in an expected runnable region,
namely:
- .text
- .ramfunc
- .itcm
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Adapt to the reworked zephyr cache API.
Fix build errors when building tests/kernel/cache with CACHE_MANAGEMENT and
CPU_HAS_DCACHE enabled for x86 SoCs
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
The z_arm64_fatal_error should be
extern void z_arm64_fatal_error(unsigned int reason, z_arch_esf_t *esf);
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Clarify why we use target_link_options() instead of
target_link_libraries()
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
To ease writing common drivers, let's make the host trampolines
from the native simulator avaliable to all posix based boards.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
On nRF5340 net core it was observed that when `wfi` instruction was
followed by `pop {r0, lr}` in the `arch_cpu_idle` function,
the value of `lr` sometimes got read as 0 from memory despite
having correct value stored in the memory.
This commit inserts additional `nop` instruction after waking up
to delay access to the memory.
Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
Instead of keeping a replica of the nce code,
now that the native_simulator is in tree, let's use
it directly.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Instead of failing badly later, let's give a clear error
message if the user tries to build in an unsupported platform.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This allows building with embedded libCs in the Zephyr side,
as the POSIX arch bottom is not anymore built in Zephyr context.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
So they depend or select on the right NATIVE_BUILD
instead of NATIVE_APPLICATION.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Zephyr provides a default NMI handler (`z_SysNmiOnReset`), which will
basically call `wfi` endlessly. It is allowed to override such handler
when CONFIG_RUNTIME_NMI=y, via `z_arm_nmi_set_handler`. However,
enabling such option also provided `z_arm_nmi_init` (via `NMI_INIT()`),
which basically sets the handler to `DefaultHandler` (a new handler that
basically printks and reboots). This is strictly not needed, and
independent of the runtime NMI option. As a result, most SoCs were
blindly calling `NMI_INIT()`, probably because of a copy&paste effect.
In the majority of cases, this was a no-op, but most SoCs do IRQ
enable/disable, making this even more convoluted. To make things worse,
the init call is expected to run after console has been initialized (for
printk to work?), but most SoCs just called it in PRE_KERNEL_1+0.
This patch just drops this NMI initializer API, and leaves only the
handler set call when CONFIG_RUNTIME_NMI=y.
NMI_INIT() dummy definition is left in this patch to preserve
bisectability, will be dropped later.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.
Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
The implementation of `z_arm_clear_arm_mpu_config` was compiled for all
ARM cores that declare to have an MPU. However, we only want to compile
it if the MPU is actually enabled.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system.
And the top the only one that interacts or is aware
of the hosted operating system (Zephyr).
The bottom uses the native simulator CPU
start/stop emulation.
By now we replicate its code as a provisional measure,
until the native simulator becomes standard.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system, and does not necessarily
need to know about the Zephyr OS.
This is in preparation for the native simulator,
which which the bottom is also fully Zephy agnostic.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The issue due to which we set the default compiler C std version
to C11 is not specific to any particular POSIX arch board,
but to all. Instead of setting this property for each board,
let's set it at the architecture level.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This adds support for 32 double-precision registers in the context
switching of aarch32 architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
`fpscr` is assigned from `struct __fpu_sf.fpscr` in `vfp_restore`, but it
wasn't saved into `struct __fpu_sf.fpscr` in the svc and isr handler, So
it may be a dirty value.
- Fix it by saving `fpscr` in the svc hand isr handler.
- Jump out if FPU isn't enabled
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.
This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.
This solves #54785.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
This adds a few line use zephyr_syscall_header() to include
headers containing syscall function prototypes.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Some RISCV platforms shipping a CLIC have a peculiar interrupt ID
ordering / mapping.
According to the "Core-Local Interrupt Controller (CLIC) RISC-V
Privileged Architecture Extensions" Version 0.9-draft at paragraph 16.1
one of these ordering recommendations is "CLIC-mode interrupt-map for
systems retaining interrupt ID compatible with CLINT mode" that is
described how:
The CLINT-mode interrupts retain their interrupt ID in CLIC mode.
[...]
The existing CLINT software interrupt bits are primarily intended for
inter-hart interrupt signaling, and so are retained for that purpose.
[...]
CLIC interrupt inputs are allocated IDs beginning at interrupt ID
17. Any fast local interrupts that would have been connected at
interrupt ID 16 and above should now be mapped into corresponding
inputs of the CLIC.
That is a very convoluted way to say that interrupts 0 to 15 are
reserved for internal use and CLIC only controls interrupts reserved for
platform use (16 up to n + 16, where n is the maximum number of
interrupts supported).
Let's now take now into consideration this situation in the DT:
clic: interrupt-controller {
...
};
device0: some-device {
interrupt-parent = <&clic>;
interrupts = <0x1>;
...
};
and in the driver for device0:
IRQ_CONNECT(DT_IRQN(node), ...);
From the hardware prospective:
(1a) device0 is using the first IRQ line of the CLIC
(2a) the interrupt ID / exception code of the `MSTATUS` register
associated to this IRQ is 17, because the IDs 0 to 15 are reserved
From the software / Zephyr prospective:
(1b) Zephyr is installing the IRQ vector into the SW ISR table (and into
the IRQ vector table for DIRECT ISRs in case of CLIC vectored mode)
at index 0x1.
(2b) Zephyr is using the interrupt ID of the `MSTATUS` register to index
into the SW ISR table (or IRQ vector table)
It's now clear how (2a) and (2b) are in contrast with each other.
To fix this problem we have to take into account the offset introduced
by the reserved interrupts. To do so we introduce
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET as hidden option for the
platforms to set.
This Kconfig option is used to shift the interrupt numbers when
installing the IRQ vector into the SW ISR table and/or IRQ vector table.
So for example in the previous case and using
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET == 16, the IRQ vector
associated to the device0 would be correctly installed at index 17 (16 +
1), matching what is reported by the `MSTATUS` register.
CONFIG_NUM_IRQS must be increased accordingly.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Zephyr currently only supports CLINT direct mode and CLINT vectored
mode. Add support for CLIC vectored mode as well.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Before adding support for the CLIC vectored mode, rename
CONFIG_RISCV_MTVEC_VECTORED_MODE to CONFIG_RISCV_VECTORED_MODE to be
more generic and eventually include also the CLIC vectored mode.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
With all the stacks and TSS (etc), the x86_64 arch code can only
support maximum of 4 CPUs at the moment. So add a build assert
if more CPUs are specified via CONFIG_MP_MAX_NUM_CPUS, also
overwrite the range value for CONFIG_MP_MAX_NUM_CPUS.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We are missing setting of switch_handle for the thread which
is aborting due to exception (i.e. in case of k_panic or
__ASSERT triggered). This may cause livelock in SMP code
after a08e23f68e commit ("kernel/sched: Fix SMP
must-wait-for-switch conditions in abort/join").
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently the lazy fpu saving algorithm in arm64 is using the fpu_owner
pointer from the cpu structure to understand the owner of the context
in the cpu and save it in case someone different from the owner is
accessing the fpu.
The semantics for memory consistency across smp systems is quite prone
to errors and reworks on the current code might miss some barriers that
could lead to inconsistent state across cores, so to overcome the issue,
use atomics to hide the complexity and be sure that the code will behave
as intended.
While there, add some isb barriers after writes to cpacr_el1, following
the guidance of ARM ARM specs about writes on system registers.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
With Zephyr now always using `int main(void)`, there's no longer any need
for this definition. The last remaining use which gated the declaration of
_posix_zephyr_main isn't necessary as adding that declaration
unconditionally is harmless.
Signed-off-by: Keith Packard <keithp@keithp.com>
cpu_node_list does not hold the corrent mapping of cpu id and mpid when
core booting sequence does not follow the DTS cpu node sequence. This
will cause an issue that sgi cannot deliver to the right target.
Add the cpu_map array to hold the corrent mapping between cpu id and
mpid.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Each core should init their own stack during the reset when SMP enabled,
but do not touch others. The current init results in each core starting
init the stack from the same address which will break others.
Fix the issue by setting a correct start address.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
LOG system has unalignment access instruction which will cause an
alignment exception before MPU is enabled. Remove the LOG print before
MPU is enabled to avoid this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.
This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.
Signed-off-by: Andy Ross <andyross@google.com>
For secure EL2 to be entered the EEL2 bit in SCR_EL3 must be set. This
should only be set if Zephyr has not been configured for NS mode only,
if the device is currently in secure EL3, and if secure EL2 is supported
via the SEL2 bit in AA64PFRO_EL1. Added logic to enable EEL2 if all
conditions are met.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
This reverts commit f0b458a619.
This is a pointless change that simply increases footprint.
Existing code already supports compilation without multithreading.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Per the ARMv8 architecture document, modification of the system control
register is a context-changing operation. Context-changing operations are
only guaranteed to be seen after a context synchronization event.
An ISB is a context synchronization event. One has been placed after
each SCTLR modification. Issue was found running full speed on target.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
This make MCUboot build as Zephyr application.
Providing optinal 2nd stage bootloader to the
IDF bootloader, which is used by default.
This provides more flexibility when building
and loading multiple images and aims to
brings better DX to users by using the sysbuild.
MCUboot and applications has now separate
linker scripts.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for pending FPU IPI from
the spinlock loop using the arch_spin_relax() hook.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for a pending FPU IPI
from the arch_spin_relax() hook and honor it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The compiler is not able to emit a proper DSB operation for ARM64. Move
to the arch-specific implementation and use assembly code instead.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Enhance for cases when call z_float_enable() with NULL thread.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
This adds code to always map data TLB for VECBASE so that
we would be dealing with fewer data TLB misses during
exception handling. With VECBASE always mapped, there is
no need to pre-load anymore.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the TLB miss handling to the C exception handler.
This also allows us to handle page faults (for example,
unmapped pages) during this time as any more exceptions
handled in the C handler will not trigger the double
exception handler but the same C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of being able to arbitrarily set the PTEVADDR for page
table, this provides choices (currently just one). This is in
preparation to enable handling memory management exception in
C code. For that to work, we will need to pre-load the page
table address (PTEVADDR) for the memory page containing
exception code and data (containing jump addresses), and
various stacks. This is to prempt any TLB misses during handling
the level 1 interrupt code. If a TLB miss is encountered during
handling of level 1 interrupt, we will be thrown into double
exception handling code where we will get stuck in infinite
loop. However, in order to pre-load the page table entries,
PTEVADDR needs to be calculated. This requires the use of
PTEVADDR base which cannot be loaded via l32r, as we may cause
a data TLB miss. So we must be able to grab the PTEVADDR base
address strictly within code, and must be without any data
load. So changing CONFIG_XTENSA_MMU_PTEVADDR to be based on
choice so we can have pre-defined bit shift value for shift
operation. This shift value will be used in exception handling
code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add a build option to tell if memory should be mapped in cached
and uncachedr regions.
If the memory is neither in cached nor uncached region it is not double
mapped.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Initial support for Xtensa MMU version 3. It is using a two level page
table based on fact that the page table is in the virtual space. Only
the top level (page directory) is wired mapped in the TLB to avoid
second level page miss.
The mapped memory is completely fragmented in multiple sections, maybe
we find a better way in future.
The exception handler is where we effectively map the memory, the way it
works is:
1) SW try to access some memory address
2) The address is not mapped, so the MMU will try the auto-refill,
looking the page table
3) The page table contents is not mapped (remember, just the top-level page
is mapped)
4) An exception will be triggered, in the exception we try to read the
portion of the page table that maps the original address
5) The address is not mapped, so the MMU will try again the auto-refill.
This time though, the address is mapped by the top level page that is
properly mapped. (The top-level page maps the page table itself).
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Unlike tracing module mainly for debug usage, this is
to allow runtime profiling IRQ performance data, and
target to enable it in product release since platform
can choose to make it work with low weight protocol.
Enable this option and implement runtime_irq_stats()
in platform code, such as Intel ISH platform implement
with SHMI protocol to allow host profiling irq stats.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Until now iterable sections APIs have been part of the toolchain
(common) headers. They are not strictly related to a toolchain, they
just rely on linker providing support for sections. Most files relied on
indirect includes to access the API, now, it is included as needed.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The commit 434ca63e2f introduced the
Cortex-A and Cortex-R CPU type dependency to `CONFIG_FP16` based on
the reasoning that the hardware half-precision support is only
available on them.
While it is true that the _hardware_ half-precision support is limited
to these targets, the compiler will provide the _software_ emulation
for the targets that lack the hardware half-precision support, as
mentioned in 41fd6e003c (the original
commit that introduced `CONFIG_FP16`).
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
In z_xtensa_backtrace_print the parameter depth is checked for <= 0.
There is no need to check it again later, also, since the variable is
not used after the while loop we can use directly the parameter without
an additional variable.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Enable single-threading support for the riscv architecture.
Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.
The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
Intel ISH SoC can't reboot via RST_CNT register,
so make sys_arch_reboot as weak function to allow
implement different arch reboot in SoC layer.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
The "cross stack call" mechanism has intermediate states where the
stack frames are not valid for our own interrupt entry code, which
causes corruption if an interrupt races at exactly the right time.
Leave interrupts masked until just before the call.
The fix is midly complicated by the fact that we RELY on nested window
exception frames to spill registers from the interruptee, so have to
do the masking with PS.INTLEVEL, which requires a register to save its
contents, which we don't have since everything needs to happen in one
4-register window. But thankfully our Zephyr-reserved EPS register is
guaranteed to be available through this process.
Fixes#57009
Signed-off-by: Andy Ross <andyross@google.com>
With paging config, need to use physical address as
paging is not enabled here.
From IA manual, LDMXCSR instruction description is,
Loads the source operand into the MXCSR control/status
register, the source operand is a 32-bit memory location.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Use the common exit() provided by libc so we get standard behavior
across all architectures. So only implement a special exit when
XT_SIMULATOR is defined.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Local variables in ASM macro works differently for GNU and MWDT
toolchains. In case of GNU toolchain they are local per each macro
instance, but in case of MWDT they are local per file where macro
is used.
To avoid issues when macro is used multiple times in one file let's
align _st32_huge_offset to have same behaviour with GNU & MWDT
toolchains.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
The backtrace requires a valid stack pointer to start
printing backtraces. So if there is no stack pointer
being passed in, skip printing backtraces.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
use helper macros from csr.h instead of inline assembly which results
in cleaner and more maintainable code
Signed-off-by: Manojkumar Subramaniam <manoj@electrolance.com>
Commit 408472673e added inline
assembly to lock interrupt. However, XCC doesn't like the syntax
using STRINGIFY, and also an empty clobber section. So parameterize
the second argument to rsil, and remove the last colon.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds some structs for interrupt stack frames to make it
easier to access individual elements, and ultimately getting
rid of magic array element numbers in the code. Hopefully,
this would aid in debugging where you can view the whole
struct in debugger.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Align the comment with the new int main(void) convention
used in Zephyr.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
