There are several subsystems and boards which require a relatively large
system heap (used by k_malloc()) to function properly. This became even
more notable with the recent introduction of the ACPICA library, which
causes ACPI-using boards to require a system heap of up to several
megabytes in size.
Until now, subsystems and boards have tried to solve this by having
Kconfig overlays which modify the default value of HEAP_MEM_POOL_SIZE.
This works ok, except when applications start explicitly setting values
in their prj.conf files:
$ git grep CONFIG_HEAP_MEM_POOL_SIZE= tests samples|wc -l
157
The vast majority of values set by current sample or test applications
is much too small for subsystems like ACPI, which results in the
application not being able to run on such boards.
To solve this situation, we introduce support for subsystems to specify
their own custom system heap size requirement. Subsystems do
this by defining Kconfig options with the prefix HEAP_MEM_POOL_ADD_SIZE_.
The final value of the system heap is the sum of the custom
minimum requirements, or the value existing HEAP_MEM_POOL_SIZE option,
whichever is greater.
We also introduce a new HEAP_MEM_POOL_IGNORE_MIN Kconfig option which
applications can use to force a lower value than what subsystems have
specficied, however this behavior is disabled by default.
Whenever the minimum is greater than the requested value a CMake warning
will be issued in the build output.
This patch ends up modifying several places outside of kernel code,
since the presence of the system heap is no longer detected using a
non-zero CONFIG_HEAP_MEM_POOL_SIZE value, rather it's now detected using
a new K_HEAP_MEM_POOL_SIZE value that's evaluated at build.
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
Fix k_sleep compilation error:
[build] ... syscalls/kernel.h:135: undefined reference to `z_impl_k_sleep'
for single thread applications (CONFIG_MULTITHREADING = n).
The shed.c contains source code which must be present also
in single thread applications.
Signed-off-by: Andrej Butok <andrey.butok@nxp.com>
This separates demand paging related headers into its own file
instead of being stuffed inside the main kernel memory
management header file.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the k_* memory management functions from sys/ into
kernel/ includes, as there are kernel public APIs. The z_*
functions are further separated into the kernel internal
header directory.
Also made a quick change to doxygen to group sys_mem_* into
the OS Memory Management group so they will appear in doc.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This enables -Wshadow to warn about shadow variables on
in tree code under arch/, boards/, drivers/, kernel/,
lib/, soc/, and subsys/.
Note that this does not enable it globally because
out-of-tree modules will probably take some time to fix
(or not at all depending on the project), and it would be
great to avoid introduction of any new shadow variables
in the meantime.
Also note that this tries to be done in a minimally
invasive way so it is easy to revert when we enable
-Wshadow globally. Source files under modules/, samples/
and tests/ are currently excluded because there does not
seem to be a trivial way to add -Wshadow there without
going through all CMakeLists.txt to add the option
(as there are 1000+ files to change).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With some of the recent work to disable unnecessary system
calls, there is a scenario where `z_impl_k_thread_stack_free()`
is not defined and an undefined symbol error occurs.
Safety was very concerned that dynamic thread stack code might
touch other code that does not malloc, so add a separate file
for the stack alloc and free stubs.
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
This adds a few line use zephyr_syscall_header() to include
headers containing syscall function prototypes.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When a cache API function is called from userspace, this results on
ARM64 in an OOPS (bad syscall error). This is due to at least two
different factors:
- the location of the cache handlers is preventing the linker to
actually find the handlers
- specifically for ARM64 and ARC some cache handling functions are not
implemented (when userspace is not used the compiler simply optimizes
out these calls)
Fix the problem by:
- moving the userspace cache handlers to a their logical and proper
location (in the drivers directory)
- adding the missing handlers for ARM64 and ARC
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This new implementation of pipes has a number of advantages over the
previous.
1. The schedule locking is eliminated both making it safer for SMP
and allowing for pipes to be used from ISR context.
2. The code used to be structured to have separate code for copying
to/from a wating thread's buffer and the pipe buffer. This had
unnecessary duplication that has been replaced with a simpler
scatter-gather copy model.
3. The manner in which the "working list" is generated has also been
simplified. It no longer tries to use the thread's queuing node.
Instead, the k_pipe_desc structure (whose instances are on the
part of the k_thread structure) has been extended to contain
additional fields including a node for use with a linked list. As
this impacts the k_thread structure, pipes are now configurable
in the kernel via CONFIG_PIPES.
Fixes#47061
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Moves the CONFIG_SCHED_THREAD_USAGE block of code out of sched.c
into its own file. Not only do they employ their own private
spin lock, but it is expected that additional usage routines will be
added in the future.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Threads may wait on an event object such that any events posted to
that event object may wake a waiting thread if the posting satisfies
the waiting threads' event conditions.
The configuration option CONFIG_EVENTS is used to control the inclusion
of events in a system as their use increases the size of
'struct k_thread'.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
To support arm-ds / armlink it is required that the weak main is located
in an object externally to the object using the weak symbol.
If the weak symbol is inside the object referring to it, then the weak
symbol will be used and this will result in
```
Error: L6200E: Symbol __ARM_use_no_argv multiply defined
(by init.o and main.o).
```
as both the weak and strong symbols are used.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This adds a very primitive logic to allow linking a prebuilt
static library of kernel code instead of building the kernel
from source. Note that the library is built with a specific
set of kconfigs, and they must match when building applications,
or else there would be mysterious crashes.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The cache API currently shipped in Zephyr is assuming that the cache
controller is always on-core thus managed at the arch level. This is not
always the case because many SoCs rely on external cache controllers as
a peripheral external to the core (for example PL310 cache controller
and the L2Cxxx family). In some cases you also want a single driver to
control a whole set of cache controllers.
Rework the cache code introducing support for external cache
controllers.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
smp.c only has to be built if CONFIG_SMP is enabled. Remove
preprocessor checks from the file itself and update cmake rules
instead.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Avoid fetching files which use scheduler. By explicitly avoiding
including RTOS specific files we ensure that it is not fetched
accidently.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
This adds more bits to gather statistics on demand paging,
e.g. clean vs dirty pages evicted, # page faults with
IRQ locked/unlocked, etc.
Also extends this to gather per-thread demand paging
statistics.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Now that the old API has been reimplemented with the new API remove
the old implementation and its tests.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This commit provides a complete reimplementation of the work queue
infrastructure intended to eliminate the race conditions and feature
gaps in the existing implementation.
Both bare and delayable work structures are supported. Items can be
submitted; delayable items can be scheduled for submission at a future
time. Items can be delayed, queued, and running all at the same time.
A running item can also be canceling.
The new implementation:
* replaces "pending" with "busy" which identifies the active states;
* supports canceling delayed and submitted items;
* prevents resubmission of a item being canceled until cancellation
completes;
* supports waiting for cancellation to complete;
* supports flushing a work item (waiting for the last submission to
complete without preventing resubmission);
* supports waiting for a queue to drain (only allows resubmission from
the work thread);
* supports stopping a work queue in conjunction with draining it;
* prevents handler-reentrancy during resubmission.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Attempts to reimplement the existing work API using a new work
implementation failed, primarily due to heavy use of whitebox testing
in validating the original API. Add a temporary Kconfig that will
select between the two implementations so we can use the same
identifiers but select which implementation they reference.
This commit just adds the selection infrastructure and uses it to
conditionalize the existing implementation in anticipation of the new
one in the next commit.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
THIS COMMIT DELIBERATELY BREAKS BISECTABILITY FOR EASE OF REVIEW.
SKIP IF YOU LAND HERE.
Remove the existing implementatoin of k_thread_abort(),
k_thread_join(), and the attendant facilities in the thread subsystem
and idle thread that support them.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Remove the MEM_POOL_HEAP_BACKEND kconfig, treating it as true always.
Now the legacy mem_pool cannot be enabled and all usage uses the
k_heap/sys_heap backend.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Move banner and boot delay handling out of init.c
The code for banner was all over the place in init.c making it
unreadable.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Deprecate the Kconfig option for the time being. Unless a contributor
volunteers to take over the work to maintain the option, it will be
removed after 2 releases.
Relates to #27415.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Adds a kconfig CONFIG_KERNEL_MEM_POOL to decide whether
kernel memory pool related code is compiled. This option
can be disabled to shrink code size. If k_heap is not
being used at all, kheap.c will also not be compiled,
resulting in further smaller code size.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Include directories for ${ARCH} is not specified correctly.
Several places in Zephyr, the include directories are specified as:
${ZEPHYR_BASE}/arch/${ARCH}/include
the correct line is:
${ARCH_DIR}/${ARCH}/include
to correctly support out of tree archs.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
include/cache.h: System calls declaration and implementation
kernel/cache_handlers.c: Defination of verification functions
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
This set of functions seem to be there just because of historical
reasons, stemming from Kbuild. They are non-obvious and prone to errors,
so remove them in favor of the `_ifdef()` ones with an explicit
`CONFIG_` condition.
Script used:
git grep -l _if_kconfig | xargs sed -E -i
"s/_if_kconfig\(\s*(\w*)/_ifdef(CONFIG_\U\1\E \1/g"
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This will be the interface for mapping memory in the kernel's
part of the address space, which is guaranteed to be persistent
regardless of what thread is scheduled.
Further code for specifically managing virtual memory will end up in
kernel/mmu.c.
Further defintions for memory management in general will end up
in sys/mem_manage.h.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add a shim layer implementing the legacy k_mem_pool APIs backed by a
k_heap instead of the original implementation.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds a k_heap data structure, a synchronized wrapper around a
sys_heap memory allocator. As of this patch, it is an alternative
implementation to k_mem_pool() with somewhat better efficiency and
performance and more conventional (and convenient) behavior.
Note that commit involves some header motion to break dependencies.
The declaration for struct k_spinlock moves to kernel_structs.h, and a
bunch of includes were trimmed.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Almost all of the k_mem_pool API is implemented in terms of three
lower level primitives: K_MEM_POOL_DEFINE(), k_mem_pool_alloc() and
k_mem_pool_free_id(). These are themselves implemented on top of the
lower level sys_mem_pool abstraction.
Make this layering explicit by splitting the low level out into its
own files: mempool_sys.c/h.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Before C sources can be compiled any generated header that they
include must be generated. Currently, the target 'offsets_h' happens
to depend directly or indirectly on all generated headers.
This means that to compile safely, one can simply depend on
'offsets_h'. But this is coincidental and might not be true in the
future.
To be able to safely depend on a target that represents all generated
headers being ready we introduce the target
'zephyr_generated_headers'.
Any third-party build scripts can now safely depend on
'zephyr_generated_headers' and be protected from any internal changes
to the build system, like the removal of offsets_h.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
* z_NanoFatalErrorHandler() is now moved to common kernel code
and renamed z_fatal_error(). Arches dump arch-specific info
before calling.
* z_SysFatalErrorHandler() is now moved to common kernel code
and renamed k_sys_fatal_error_handler(). It is now much simpler;
the default policy is simply to lock interrupts and halt the system.
If an implementation of this function returns, then the currently
running thread is aborted.
* New arch-specific APIs introduced:
- z_arch_system_halt() simply powers off or halts the system.
* We now have a standard set of fatal exception reason codes,
namespaced under K_ERR_*
* CONFIG_SIMPLE_FATAL_ERROR_HANDLER deleted
* LOG_PANIC() calls moved to k_sys_fatal_error_handler()
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A k_futex is a lightweight mutual exclusion primitive designed
to minimize kernel involvement. Uncontended operation relies
only on atomic access to shared memory. k_futex structure lives
in application memory. And when using futexes, the majority of
the synchronization operations are performed in user mode. A
user-mode thread employs the futex wait system call only when
it is likely that the program has to block for a longer time
until the condition becomes true. When the condition comes true,
futex wake operation will be used to wake up one or more threads
waiting on that futex.
This patch implements two futex operations: k_futex_wait and
k_futex_wake. For k_futex_wait, the comparison with the expected
value, and starting to sleep are performed atomically to prevent
lost wake-ups. If different context changed futex's value after
the calling use-mode thread decided to block himself based on
the old value, the comparison will help observing the value
change and will not start to sleep. And for k_futex_wake, it
will wake at most num_waiters of the waiters that are sleeping
on that futex. But no guarantees are made on which threads are
woken, that means scheduling priority is not taken into
consideration.
Fixes: #14493.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
We do have a multi-architecture latency benchmark now, this one was x86
only, was never used or compiled in and is out-dated.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
There is an effort underway to make most of the Zephyr build script's
reentrant. Meaning, the build scripts can be executed multiple times
during the same CMake invocation.
Reentrancy enables several use-cases, the motivating one is the
ability to build several Zephyr executables, or images, for instance a
bootloader and an application.
For build scripts to be reentrant they cannot be directly referencing
global variables, like target names, but must instead reference
variables, which can vary from entry to entry.
Therefore, in this patch, we replace global targets with variables.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Now that the API has been fixed up, replace the existing timeout queue
with a much smaller version. The basic algorithm is unchanged:
timeouts are stored in a sorted dlist with each node nolding a delta
time from the previous node in the list; the announce call just walks
this list pulling off the heads as needed. Advantages:
* Properly spinlocked and SMP-aware. The earlier timer implementation
relied on only CPU 0 doing timeout work, and on an irq_lock() being
taken before entry (something that was violated in a few spots).
Now any CPU can wake up for an event (or all of them) and everything
works correctly.
* The *_thread_timeout() API is now expressible as a clean wrapping
(just one liners) around the lower-level interface based on function
pointer callbacks. As a result the timeout objects no longer need
to store backpointers to the thread and wait_q and have shrunk by
33%.
* MUCH smaller, to the tune of hundreds of lines of code removed.
* Future proof, in that all operations on the queue are now fronted by
just two entry points (_add_timeout() and z_clock_announce()) which
can easily be augmented with fancier data structures.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Normally a syscall would check the current privilege level and then
decide to go to _impl_<syscall> directly or go through a
_handler_<syscall>.
__ZEPHYR_SUPERVISOR__ is a compiler optimization flag which will
make all the system calls from the kernel files directly link
to the _impl_<syscall>. Thereby reducing the overhead of checking the
privileges.
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Move posix layer from 'kernel' to 'lib' folder as it is not
a core kernel feature.
Fixed posix header file dependencies as part of the move and
also removed NEWLIBC related macros from posix headers.
Signed-off-by: Ramakrishna Pallala <ramakrishna.pallala@intel.com>
