Commit graph

2904 commits

Author SHA1 Message Date
Anas Nashif 31bc210bbc kernel: sched: remove unused prototype: z_is_thread_time_slicing
A prototype not used anywhere.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-04-01 18:47:36 -04:00
Anas Nashif ebb503ff7b kernel: move thread related helper function kthread.h
Move some helper functions to inernal kthread.h, to offload crowded
sched.c

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-04-01 18:47:36 -04:00
Daniel Leung d34351d994 kernel: align thread stack size declaration
When thread stack is defined as an array, K_THREAD_STACK_LEN()
is used to calculate the size for each stack in the array.
However, standalone thread stack has its size calculated by
Z_THREAD_STACK_SIZE_ADJUST() instead. Depending on the arch
alignment requirement, they may not be the same... which
could cause some confusions. So align them both to use
K_THREAD_STACK_LEN().

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-27 19:27:10 -04:00
Daniel Leung 6cd7936f57 kernel: align kernel stack size declaration
When kernel stack is defined as an array, K_KERNEL_STACK_LEN()
is used to calculate the size for each stack in the array.
However, standalone kernel stack has its size calculated by
Z_KERNEL_STACK_SIZE_ADJUST() instead. Depending on the arch
alignment requirement, they may not be the same... which
could cause some confusions. So align them both to use
K_KERNEL_STACK_LEN().

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-27 19:27:10 -04:00
Daniel Leung efe30749de kernel: rename Z_THREAD_STACK_BUFFER to K_THREAD_STACK_BUFFER
Simple rename to align the kernel naming scheme. This is being
used throughout the tree, especially in the architecture code.
As this is not a private API internal to kernel, prefix it
appropriately with K_.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-27 19:27:10 -04:00
Daniel Leung b69d2486fe kernel: rename Z_KERNEL_STACK_BUFFER to K_KERNEL_STACK_BUFFER
Simple rename to align the kernel naming scheme. This is being
used throughout the tree, especially in the architecture code.
As this is not a private API internal to kernel, prefix it
appropriately with K_.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-27 19:27:10 -04:00
Ederson de Souza 321e395a8f linker: Include libkernel.a in the whole-archive when llext is enabled
Differently from other libraries, which are included whole in the final
Zephyr ELF, libkernel.a itself isn't. Assuming this is intended to
enable optimisations (if it isn't, this patch will break things) - linker
can remove parts of the kernel that are not used by the application.

However, when considering Linkable Loadable Extensions (llext), this
optimisations can be counterproductive: for instance, syscalls that are
not used by the application won't be available for extensions. It won't
matter if someone "EXPORT_SYMBOL" for them, or even try to keep them
using LINKER_KEEP, they'll be gone.

To avoid that, this patches includes, when CONFIG_LLEXT=y, libkernel.a
inside the linker "whole-archive" block. This ends up making it consider
libkernel.a as a library whose all symbols should be kept. Note this
doesn't mean that all symbols will be there - things compiled out via
Kconfig will naturally still be out.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-03-26 19:31:56 -04:00
Simon Hein bcd1d19322 kernel: add closing comments to config endifs
Add a closing comment to the endif with the configuration
information to which the endif belongs too.
To make the code more clearer if the configs need adaptions.

Signed-off-by: Simon Hein <Shein@baumer.com>
2024-03-25 18:03:31 -04:00
Daniel Leung 6ea749de52 arch: rename arch_start_cpu() to arch_cpu_start()
Rename arch_start_cpu() to arch_cpu_start() so it belongs to
the "cpu" namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-25 09:58:35 +00:00
Daniel Leung 3664ed64c3 arch: move arch_interface.h under zephyr/arch
arch_interface.h is for architecture and should not be
under sys/. So move it under include/zephyr/arch/.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-25 09:58:35 +00:00
Flavio Ceolin e2f3840380 kernel: Fix possible overflow in k_mem_map
k_mem_map additionally allocates two guard pages that are not mapped.
These pages are not being accounted when checking the provided size and
when they are added an overflow can happen and the mapped memory is not
correct.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-03-22 15:58:33 -04:00
Ederson de Souza 67bb6db3f8 syscall: Export all emitted syscalls, enabling them for extensions
Linkable loadable extensions can only use syscalls if they are exported
via EXPORT_SYSCALL (or EXPORT_SYMBOL). Instead of enabling used syscalls
one by one, this patch exports all of them automatically via
`gen_syscalls.py`. If CONFIG_LLEXT=n, the section where the exported
symbols live is discarded, so it should be a non-op when llext is not
enabled.

This patch also removes the now redundant EXPORT_SYSCALL macro. Note
that EXPORT_SYMBOL is still useful on different situations (and is
indeed used by the code generated by `gen_syscalls.py`).

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-03-20 16:26:54 +00:00
TaiJu Wu 1f5f0cf838 sched: Remove multi-level queue priority limit
Modified bitmask to  bitmask array, it can make multilevel queue remove
32 bit prioriry limit.

We can scan bitmask array to find which queue have ready thread.

Only need the number of queues as priority because the priority
is checked on create_thread.

Signed-off-by: TaiJu Wu <tjwu1217@gmail.com>
2024-03-12 19:37:40 -04:00
Andy Ross f2280d119d kernel/sched: Don't touch deadline values on queued threads
k_thread_deadline_set() would modify the thread's deadline and then,
if it was in the run queue, requeue it to put it at the right spot.
Sounds right, right?

It's wrong.  The deadline field is part of the thread priority, so
this results in a mis-ordered list.  For dlist backends, that's benign
as the removal works anyway, but if CONFIG_SCHED_SCALABLE=y we've now
broken the sorting order of an in-tree item and corrupted the rbtree!

Fixes #69935

Signed-off-by: Andy Ross <andyross@google.com>
2024-03-11 15:42:26 +01:00
Nicolas Pitre c31d646198 kernel: timeout: optimize z_timeout_expires()
This currently calls timeout_rem() then adds back the result of
elapsed() to cancel out the subtraction of another elapsed() call
within timeout_rem(). Better not to make any calls to elapsed() in
that case, especially given that elapsed() may incur hardware access
overhead through sys_clock_elapsed().

Let's move the elapsed() subtraction to the only user of timeout_rem()
that needs it and remove its invocation from the z_timeout_expires()
path entirely.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2024-03-08 18:05:10 +01:00
TaiJu Wu 39f710136e sched: finalize_cancel_locked can early return
The code `SYS_SLIST_FOR_EACH_CONTAINER_SAFE` just for remove work
from `pending_cancels`.
After removing work successfully, the function can return early.
It is unnecessary to iterate continuously.

Signed-off-by: TaiJu Wu <tjwu1217@gmail.com>
2024-03-07 19:40:51 -05:00
Peter Mitsis 9f7695dda0 kernel: Remove unused z_pend_curr_irqlock()
The routine z_pend_curr_irqlock() is no longer used anywhere.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-03-07 11:51:06 -05:00
Anas Nashif 0d8da5ff93 kernel: rename scheduler spinlock variable and make it private
rename sched_spinlock to _sched_spinglock to maintain it is privacy and
to avoid any misuse.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 868f099d61 kernel: sched: z_set_prio -> z_thread_prio_set
Rename private function to make it clear what priority we are setting
and to be consistent across the code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 477a04a098 kernel: rename h -> heap
Avoid single characker variables that renders code unreadable and might
cause conflicts in maing, similar to t for both timeout and thread in
some places.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 595ff63f00 kernel: thread: use consistent thread parameter
Use thread wherever it makes sense, using 't' in some places can get
confused with 't' used for timeouts for example.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 3dc0c4544b kernel: move float operations out of thread.c
Move float operation out and add missing vrfy hook for enabling float.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 6c003bdbcf kernel: remove unused code in headers
List of functions defined in headers and not being used anywhere.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 9e83413542 kernel: split thread monitor
Move thread monitor related functions, not enabled in most cases outside
of thread.c and cleanup headers.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 5e591c38f1 kernel: do not export z_thread_priority_set
This function is only being used by a test, so instead of reimplementing
a syscall in the test, provide a Kconfig option to provide the
functionality that only works with tests and remove some of the
duplication and extra code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 3ca50f5060 kernel: move z_init_static_threads to where it is being used
Move out of thread and put directly in init.c where it is being used.
Also remove definition from kernel.h, this is an internal function and
should not be in a public header.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif a6ce422b10 kernel: remove cmsis-rtos layering violation
We shouldn't be calling hooks from optional and upper layer subsystems
in the kernel, instead, just call the hook to set thread status in the
API where it is needed.

This now clears related bit in cmsis thread status bitarray when
terminating a thread in the cmsis rtos v1 layer directly and not in the
kenrel code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 077222c975 kernel: split cpu_mask handling into own file
In an effort to cleanup sched.c, move sections of code that can be
compiled in based on options into own files. CPU mask here is managed by
a kconfig and is not widely used (SMP affinity on multicore systems).

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 6e95bdeca6 kernel: reorg Kconfigs and split them
The kernel kconfig is becoming too big and unmanageable with too many
options scattered across the file. Move some areas out and reorg main
Kconfig slightly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 8791012ed1 kernel: move essential flag related routines out
The functions to manipulate the essential flag indeed operate on
threads, but they are misplaced in the thread implementation file. Put
them alongside other routines setting other thread flags and cleanup
headers a bit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif 2c31db4cc8 kernel: split irq_offload ccode into own file
Move irq_offload code out of thread.c into own file. This code is not
related to threads.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif a7d74f80ce kernel: move spinlock validation to own file
Move spin_lock validation outside of thread.c into own file. This code
really has nothing to do with the thread code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Ederson de Souza 4440d6a189 kernel/userspace: Fix dynamic thread stack allocation at userspace
It wasn't saving adjusted stack size at either the private stack or the
k_object, thus failing subsequent checks.

Test added to check for this case and prevent regressions.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-03-06 14:17:53 +01:00
Anas Nashif 46484da502 kernel: move priority queue handling to own file/header
clean up headers under include/ and move handling of priority queue to
own file/header.
No need for the header  include/zephyr/kernel/internal/sched_priq.h
anymore. Move the relevant structures where they are being used in
kernel_structs.h.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-02 15:06:45 +01:00
Peter Mitsis ee9c44fee6 linker: Round TLS sizes up in linker script
Instead of rounding up both __tdata_size and __tbss_size at runtime,
perform the calculation when the image is built.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-02-25 22:28:56 -05:00
Nguyen Minh Thien 8188be57d3 kernel: fix spelling errors
Fix spelling errors found in comment of the kernel source code.

Signed-off-by: Nguyen Minh Thien <nguyenmthien@live.com>
2024-02-25 20:53:37 -05:00
Peter Mitsis 51ae993c12 kernel: Update k_wakeup()
This commit does two things to k_wakeup():

1. It locks the scheduler before marking the thread as not suspended.
As the the clearing of the _THREAD_SUSPENDED bit is not atomic, this
helps ensure that neither another thread nor ISR interrupts this
action (resulting in a corrupted thread_state).

2. The call to flag_ipi() has been removed as it is already being
made within ready_thread().

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-02-25 20:50:03 -05:00
Guennadi Liakhovetski 09cf3e0910 smp: fix a race when starting / resuming multiple CPUs
cpu_start_fn is global, it's used by the initiator CPU to start or
resume secondary CPUs. However it's possible, that the initiator CPU
goes ahead and starts a second secondary CPU before the first one has
finished using the object. Fix this by creating a local copy of the
global object.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-02-16 07:27:04 +01:00
Flavio Ceolin d622533ca7 kernel: thread: Allow stack in coherent memory
When allowing dynamic thread stack allocation the stack may come from
the heap in coherent memory, trying to use cached memory is over
complicated because of heap meta data and cache line sizes.
Also when userspace is enabled, stacks have to be page aligned and the
address of the stack is used to track kernel objects.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-02-09 14:23:14 -05:00
Daniel Leung caacc27d37 kernel: smp: CPU start may result in null pointer access
It is observed that starting up CPU may result in other CPUs
crashing due to de-referencing NULL pointers. Note that this
happened on the up_squared board, but there was no way to
attach debugger to verify. It started working again after
moving z_dummy_thread_init() before smp_timer_init(), which
was the old behavior before commit
eefaeee061 where the issue
first appeared. So mimic the old behavior to workaround
the issue.

Fixes #68115

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-02-04 19:55:14 -06:00
Andy Ross 840db7858e kernel/thread: Detect in-kernel "reserved" stack overflow
Traditionally, k_thread_create() has required that the application
size the stack correctly.  Zephyr doesn't detect or return errors and
treats stack overflow as an application bug (though obviously some
architectures have runtime features to trap on overflows).

At this one spot though, it's possible for the kernel to adjust the
stack for K_THREAD_STACK_RESERVED in such a way that the arch layer's
own stack initialization overflows.  That failure can be seen by
static analysis, so we can't just sweep it under the rug as an
application failure.

Unfortunately there aren't any good options for handling it here (no
way to return failure, can't be a build assert as the size is a
runtime argument).  A panic will have to do.

Fixes: #67106
Fixes: #65584

Signed-off-by: Andy Ross <andyross@google.com>
2024-02-04 10:23:25 -05:00
Krzysztof Chruściński 25173f71cd pm: device_runtime: Extend with synchronous runtime PM
In many cases suspending or resuming of a device is limited to
just a few register writes. Current solution assumes that those
operations may be blocking, asynchronous and take a lot of time.
Due to this assumption runtime PM API cannot be effectively used
from the interrupt context. Zephyr has few driver APIs which
can be used from an interrupt context and now use of runtime PM
is limited in those cases.

Patch introduces a new type of PM device - synchronous PM. If
device is specified as capable of synchronous PM operations then
device runtime getting and putting is executed in the critical
section. In that case, runtime API can be used from an interrupt
context. Additionally, this approach reduces RAM needed for
PM device (104 -> 20 bytes of RAM on ARM Cortex-M).

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2024-02-01 15:03:42 +01:00
Christopher Friedt 9f3d7776ab kernel: dynamic: reduce verbosity in degenerate case
k_thread_stack_free() is designed to be called with any pointer
value. We return -EINVAL when an attempt is made to free an
invalid stack pointer.

This change reduces the verbosity in the degenerate case, when
the pointer is not obtained via k_thread_stack_alloc(), but
otherwise does not affect functionality.

If debug log verbosity is not enabled, we save a few bytes in
.text / .rodata / .strtab.

Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2024-01-26 06:50:11 -05:00
Daniel Leung 2cdd44801e kernel: move z_init_cpu to private kernel headers
z_init_cpu() is a private kernel API so move it under
kernel/include.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung 35a1814c4d kernel: smp: remove z_smp_thread_init/_swap
This removes z_smp_thread_init() and z_smp_thread_swap() as
SOF has been updated to use k_smp_cpu_custom_start() instead.
This removes the need for SOF to mirror part of the SMP
power up code, and thus these two functions are no longer
needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung eefaeee061 kernel: smp: introduce k_smp_cpu_resume
This provides a path to resume a previously suspended CPU.
This differs from k_smp_cpu_start() where per-CPU kernel
structs are not initialized such that execution context
can be saved during suspend and restored during resume.
Though the actual context saving and restoring are
platform specific.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung fe66e35db0 kernel: smp: put comment on SMP code
Adds some comments to the SMP code to, hopefully, make it
a bit more clear to future readers.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung 89b231e7e2 kernel: smp: introduce k_smp_cpu_start
This renames z_smp_cpu_start() to k_smp_cpu_start().
This effectively promotes z_smp_cpu_start() into a public API
which allows out of tree usage. Since this is a new API,
we can afford to change it signature, where it allows
an additional initialization steps to be done before a newly
powered up CPU starts participating in scheduling threads
to run.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung 803e0e452f kernel: amend wording on CONFIG_SMP_BOOT_DELAY
This extends the wording so that not only architecture code can
start secondary CPUs at a later time. Also adds a missing 'to'.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Gerson Fernando Budke b8188e54a4 kernel: Implement k_sleep for Single Thread
The current z_tick_sleep return directly when building kernel for Single
Thread model. This reorganize the code to use k_busy_wait() to be time
coherent since subsystems may depend on it.

In the case of a K_FOREVER timeout is selected the Single Thread the
implementation will invoke k_cpu_idle() and the system will wait for
an interrupt saving power.

Signed-off-by: Gerson Fernando Budke <gerson.budke@ossystems.com.br>
2024-01-10 15:10:16 +01:00