Removes an unnecessary schedule lock/unlock pair from k_mutex_unlock().
Rationale: Given that only the current thread (which would also be the
mutex owner) will be able to modify the mutex object AND that a
recursive unlock ought never trigger any reschedule (as it does not
touch the pend queue), then performing a schedule lock is not needed
prior to testing for a recursive unlock.
Furthermore, even if it is not a recursive unlock, then a schedule lock
is superfluous as the existing spinlock provides sufficient protection.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
When threads are in more than one state at a time, k_thread_state_str()
returns a string that lists each of its states delimited by a '+'.
This in turn necessitates a change to the API that includes both a
pointer to the buffer to use for the string and the size of the buffer.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Add an API that clears cpu mask from a thread and sets it to a specific
CPU.
This is the equivelent of:
k_thread_cpu_mask_clear(&thread);
k_thread_cpu_mask_enable(&thread, cpu_idx);
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Instead of resizing all devices handles, we just resize devices that are
power domains. This means that a power domain has to be declared as
compatbile with "power-domain" in device tree node.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add API to add devices to a power domain in runtime. The number of
devices that can be added is defined in build time.
The script gen_handles.py will check the number defined in
`CONFIG_PM_DEVICE_POWER_DOMAIN_DYNAMIC` to resize the handles vector,
adding empty slots in the supported sector to be used later.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Commit b1182bf83b ("kernel/timeout: Serialize handler callbacks on
SMP") introduced an important fix to timeout handling on
multiprocessor systems, but it did it in a clumsy way by holding a
spinlock across the entire timeout process on all cores (everything
would have to spin until one core finished the list). The lock also
delays any nested interrupts that might otherwise be delivered, which
breaks our nested_irq_offload case on xtensa+SMP (where contra x86,
the "synchronous" interrupt is sensitive to mask state).
Doing this right turns out not to be so hard: take the timeout lock,
check to see if someone is already iterating
(i.e. "announce_remaining" is non-zero), and if so just increment the
ticks to announce and exit. The original cpu will then complete the
full timeout list without blocking any others longer than needed to
check the timeout state.
Fixes#44758
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
On multiprocessor systems, it's routine to enter sys_clock_announce()
in parallel (the driver will generally announce zero ticks on all but
one cpu).
When that happens, each call will independently enter the loop over
the timeout list. The access is correctly synchronized, so the list
handling is correct. But the lock is RELEASED around the invocation
of the callback, which means that the individual callbacks may
interleave between cpus. That means that individual
application-provided callbacks may be executed in parallel, which to
the app is indistinguishable from "out of order".
That's surprising and error-prone. Don't do it. Place a secondary
outer spinlock around the announce loop (but not the timeslicing
handling) to correctly serialize the timeout handling on a single cpu.
(It should be noted that this was discovered not because of a timeout
callback race, but because the resulting simultaneous calls to
sys_clock_set_timeout from separate cores seems to cause extremely
high latency excursions on intel_adsp hardware using the cavs_timer
driver. That hardware issue is still poorly understood, but this fix
is desirable regardless.)
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The idle thread got an index suffix in #23536 to make it easier to
identify different idle threads on different cores. This looks out of
place on single-core devices when the idle thread is listed next to
other kernel threads, such as main.
Remove the idle thread index on single-core platforms, and replace all
references to this format in tests and documentation.
Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
This is an attempt at formally distinguishing and supporting the case
described in 40795 where an architecture doesn't preserve/restore the
complete thread state upon entering/exiting interrupt exception state.
This is mainly about promoting the current behavior from the accepted
workaround to a formal API specification. This workaround is currently
used on ARM64 but RISC-V requires it too.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A reference to the entropy device can be obtained at compile time, so
avoid using device_get_binding().
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
There is an API to get an specific number of random bytes. There is
no need to re-implement this logic here.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Zephyr's timeslice implementation has always been somewhat primitive.
You get a global timeslice that applies broadly to the whole bottom of
the priority space, with no ability (beyond that one priority
threshold) to tune it to work on certain threads, etc...
This adds an (optionally configurable) API that allows timeslicing to
be controlled on a per-thread basis: any thread at any priority can be
set to timeslice, for a configurable per-thread slice time, and at the
end of its slice a callback can be provided that can take action.
This allows the application to implement things like responsiveness
heuristics, "fair" scheduling algorithms, etc... without requiring
that facility in the core kernel.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:
sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Things had gotten a little tangled in there so let's do some cleanup.
Remove the distressingly-special-purpose z_reinit_idle_thread() hook
(which existed to support secondary core bringup when
SMP_BOOT_DELAY=y), and just fold that into a generic z_init_cpu(),
which we can call in obvious and symmetric ways from main
initialization, z_smp_init(), and z_smp_start_cpu() (the now-official
programmatic hook for starting cpus).
Remove the "#if CONFIG_MP_NUM_CPUS > 1" exclusions. These weren't
saving any code size and were propagating themselves into platform
layers trying to avoid build failures.
There are some "special" APIs added for SOF which need to go away in
favor of the newer/generic z_smp_start_cpu(). Collect them in one
place and put them under a "#ifdef CONFIG_SOF" to prevent them from
being used in Zephyr apps.
Move some function declarations that didn't have homes into
<kernel/thread.h>.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds a LOG_DBG() line for z_phys_unmap which mirrors
what is in z_phys_map(). This also fixes a warning from
Clang about a variable being set but never used (addr_offset).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Commit 678b76e4b0 ("kernel/init.c: allow for memset/memcpy
alternatives during early boot") and commit da28829b64 ("kernel:
zero the bss section of OCM memory at boot time") were created
independently and missed changes from each other.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The x86 and xtensa implementations of irq_offload() invoke synchronous
interrupts on the local CPU, and are therefore safe to use from within
an interrupt context. This is a cheap and portable way to exercise
nested interrupts, which are otherwise highly platform-dependent to
test. Add a kconfig to signal the capability.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Zeroing the BSS and copying data to RAM with regular memset/memcpy may
cause problems when those functions are assuming a fully initialized
system for their optimizations to work e.g. some instructions require
an active MMU, but turning the MMU on needs the .bss section to be
cleared first, etc.
Commit c5b898743a ("aarch64: Fix alignment fault on z_bss_zero()")
provides a detailed explanation of such a case.
Replacing z_bss_zero() with an architecture specific one is problematic
as the former may see new sections added to it that would be missed by
the later. The same reasoning goes for z_data_copy().
Let's make maintenance much easier by providing weak versions of
memset/memcpy that can be overridden by architecture-specific safe
versions when needed.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Extracting stack usage calculation from k_thread_stack_space_get to
z_stack_space_get so it can be used also for interrupt stack.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
If a chosen entry exists for a memory area of type OCM, zero the OCM
memory's bss section at boot-time.
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
We can't simply use CLAMP to set the next timeout because
when CONFIG_SYSTEM_CLOCK_SLOPPY_IDLE is set, MAX_WAIT is
a negative number and then CLAMP will be called with
the higher boundary lower the lower boundary.
Fixes#41422
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
So that logging and "satellite" subsystems, such as tracing and object
tracking can count on kernel structs being properly initialised, such
as `_current_cpu`.
Fixes#42061.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Updates sched_cpu_update_usage() such that the CPU runtime stats
only update the its non-idle time when the current thread is not the
idle thread. This is necessary as otherwise the CPUs idle-time will
be double counted in k_thread_runtime_stats.execution_cycles.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Introduce a hidden kconfig CONFIG_KERNEL_VM_SUPPORT which
enables some kconfigs that are required for virtual memory
support. CONFIG_KERNEL_VM_BASE, CONFIG_KERNEL_VM_OFFSET,
and CONFIG_KERNEL_VM_SIZE are moved under this new kconfig
so these can be enabled independent of CONFIG_MMU.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves CONFIG_MMU and its children from arch/Kconfig into
kernel/Kconfig. These are used to enable kernel support of MMU
so they should be under kernel/.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to use conditional compilation for the function
prototypes in the kernel architecture header file. So remove it.
Added bouns is that these functions can appear in documentation
without explicitly enabled in pre-defines during doc build.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add maximum timeout used for conversion to Kconfig. Option is used
to determine which conversion algorithm to use: faster but overflowing
earlier or slower without early overflow.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Commit 3457118 changed the sequence of z_smp_thread_init() and
smp_timer_init() in smp_init_top() subroutine, which initializes
other BPs. In some boards (up_squared, acrn_ehl_crb) it will fail to
work while SMP enabling. If the timer interrupt is enabled before the
first thread was initialized. Now change back to its original order.
Fixes#41835
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This has bitrotten a bit. Early implementations had a synchronous
arch_start_cpu(), but then we started allowing that to be an async
operation. But that means that CPU start now becomes surprisingly
reentrant to the arch layer (cpu 0 can get a call to start cpu 2 while
cpu 1's initialization code is still running). That's just error
prone; we never documented the requirements cleanly (the window is
very small, but not so small to a slow simulator!).
Add an extra flag so we don't issue the next start until the last is
out of the arch layer and running in smp_init_top().
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Adds two routines to flush pipe objects:
k_pipe_flush()
- This routine flushes the entire pipe. That includes both
the pipe's buffer and all pended writers. It is equivalent
to reading everything into a giant temporary buffer which
is then discarded.
k_pipe_buffer_flush()
- This routine flushes only the pipe's buffer (if it exists).
It is equivalent to reading a maximum of "buffer size" bytes
into a temporary buffer which is then discarded.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Fixes a race condition in the k_pipe_cleanup() routine by adding
a spinlock. Additionally, internal counters are now reset after
freeing the buffer as the pipe has now become a bufferless pipe.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Resolves void pointer arithmetic build warnings in k_pipe_put() by
casting the pointer to a uint8_t pointer.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Extends the CPU usage runtime stats to track current, total, peak
and average usage (as bounded by the scheduling of the idle thread).
This permits a developer to obtain more system information if desired
to tune the system.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
When the new Kconfig option CONFIG_SCHED_THREAD_USAGE_ANALYSIS
is enabled, additional timing stats are collected during context
switches. This extra information allows a developer to obtain the
the current, longest, average and total lengths of the time that
a thread has been scheduled to execute.
A developer can in turn use this information to tune their app and/or
alter their scheduling policies.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
This commit does two things to the z_sched_thread_usage(). First,
it updates the API so that it accepts a pointer to the runtime
stats instead of simply returning the usage cycles. This gives it
the flexibility to retrieve additional statistics in the future.
Second, the runtime stats are only updated if the specified thread
is the current thread running on the current core.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Moves the CONFIG_SCHED_THREAD_USAGE block of code out of sched.c
into its own file. Not only do they employ their own private
spin lock, but it is expected that additional usage routines will be
added in the future.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
The functionality provided by device_usable_check is already provided by
device_is_ready. The (z_)device_usable_check APIs have been
re-implemented using the (z_)device_is_ready APIs and have been marked
as deprecated.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Instead of using device_usable_check() syscall, implement a new syscall
for device_is_ready that uses z_device_is_ready underneath.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
