Commit graph

1171 commits

Author SHA1 Message Date
Krzysztof Chruscinski accaebb708 lib: os: mpsc_pbuf: Fix concurrency issues
Fixed issues which were leading to failures when producing
and consuming is preempted at various stages.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-12-29 10:33:29 +01:00
Krzysztof Chruscinski 235ee63233 lib: os: mpsc_pbuf: Use flag for buffer full indication
Use flag instead of word in the buffer. Using this method allows
to dedicate full buffer capacity for data.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-12-29 10:33:29 +01:00
Krzysztof Chruscinski e46efdcd94 lib: os: mpsc_pbuf: Improve debugging
Improved debugging messages.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-12-29 10:33:29 +01:00
Nicholas Lowell 51f83185b0 posix: Kconfig for timer_create wait time
should be able to configure the time spent waiting
for available resources when calling timer_create()
to not cause a hiccup in applications that require
faster response times than the original hard-coded 100 ms.

Signed-off-by: Nicholas Lowell <nlowell@lexmark.com>
2022-12-27 15:34:13 +01:00
Carlo Caione d540cf8877 openamp: Add new Kconfig option to enable dcache
Currently OpenAMP is unconditionally compiled with the options to use
the vrings cache operations disabled.

Add a new CONFIG_OPENAMP_WITH_DCACHE Kconfig option to enable the
support for d-cache operations in OpenAMP when needed.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-12 18:41:01 +01:00
Stephanos Ioannidis 997f5edf1a lib: libc: picolibc: Clean up Kconfig configurations
This commit updates the Picolibc configurations to remove any
unnecessary defaults and dependencies and conform to the de-facto
standard convention across the Zephyr repository.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-08 15:09:55 -05:00
Stephanos Ioannidis c9c40d9465 lib: libc: Move newlib libc configs to lib/libc/newlib/Kconfig
This commit relocates the newlib libc configurations under the top-
level libc Kconfig to a new Kconfig under `lib/libc/newlib` for
improved organisation of the libc configurations.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-08 15:09:55 -05:00
Stephanos Ioannidis 5efc0d51c8 lib: libc: Move minimal libc configs to lib/libc/minimal/Kconfig
This commit relocates the minimal libc configurations under the top-
level libc Kconfig to a new Kconfig under `lib/libc/minimal` for
improved organisation of the libc configurations.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-08 15:09:55 -05:00
Stephanos Ioannidis c3db8dc9e7 lib: libc: Clean up CMake script to use add_subdirectory_ifdef
This commit cleans up the top `CMakeLists.txt` for the libc directory
to use `add_subdirectory_ifdef` in order to make the code more
readable.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-08 15:09:55 -05:00
Radoslaw Koppel 5a55d53242 spsc_pbuf: Fix cache wb in spsc_pbuf_free
This commit properly writes back cached read pointer in spsc buffer
when freeing the memory.

Signed-off-by: Radoslaw Koppel <radoslaw.koppel@nordicsemi.no>
2022-12-07 10:21:33 +00:00
Stephanos Ioannidis 82a902e5ea lib: newlib: Define _ANSI_SOURCE
This commit updates the Newlib integration to define `_ANSI_SOURCE`
in order to prevent Newlib from defining POSIX primitives in its
headers when GNU dialect is used (`-std=gnu*`).

Newlib `features.h` defines `_DEFAULT_SOURCE` when `__STRICT_ANSI__`
is not defined by GCC (i.e. when `-std=gnu*`), which results in the
Newlib headers defining POSIX primitives that are in conflict with the
POSIX primitives defined by Zephyr.

Newlib must not define POSIX primitives unless the feature test macros
such as `_POSIX_SOURCE`, `_GNU_SOURCE` and `_DEFAULT_SOURCE` are
explicitly defined.

Note that `-std=gnu` does not imply `_GNU_SOURCE` or `_DEFAULT_SOURCE`
because it is only supposed to instruct the compiler to use the GNU C
language dialect (i.e. GNU C language extensions).

Refer to the GitHub issue #52739 for more details.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-05 11:02:36 +01:00
Chris Friedt ff3aaa6ef3 posix: getopt: move declarations to unistd.h
Declarations for `getopt()` should be in `<unistd.h>`
according to the spec. The extended versions `getopt_long()`
and `getopt_long_only()` are declared in `<getopt.h>`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-12-04 14:51:52 +01:00
Chris Friedt 4d906d944b Revert "posix: getopt: move declarations to unistd.h"
This seems to have caused build failures in spite of CI being
green in PR 52653.

This reverts commit fc92eb1b37.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-12-02 14:09:37 +09:00
Chris Friedt fc92eb1b37 posix: getopt: move declarations to unistd.h
Declarations for `getopt()` should be in `<unistd.h>`
according to the spec. The extended versions `getopt_long()`
and `getopt_long_only()` are declared in `<getopt.h>`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-12-01 15:52:47 -05:00
Carlo Caione cc427b4bb0 cache: Fix libraries and drivers
Fix the usage to be compliant to the new cache API.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-01 13:40:56 -05:00
Chris Friedt 2e36d1f3eb lib: os: add CRC shell command for integrity verification
There are several Zephyr shell commands that are used for
transferring data over a possibly unreliable connection such as a
UART into either memory or flash. For example, `flash load` and
`devmem load`.

Make the CRC functions available so that they can be used to
verify the integrity of data transferred over possibly
unreliable connections.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-29 09:32:28 +01:00
Chris Friedt 7b95428fa0 lib: posix: update usleep() to follow the POSIX spec
The original implementation of `usleep()` was not compliant
to the POSIX spec in 3 ways.
- calling thread may not be suspended (because `k_busy_wait()`
  was previously used for short durations)
- if `usecs` > 1000000, previously we did not return -1 or set
  `errno` to `EINVAL`
- if interrupted, previously we did not return -1 or set
  `errno` to `EINTR`

This change addresses those issues to make `usleep()` more
POSIX-compliant.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-24 19:31:33 -05:00
Chris Friedt dcfcc6454b lib: posix: sleep() should report unslept time in seconds
In the case that `sleep()` is interrupted, the POSIX spec requires
it to return the number of "unslept" seconds (i.e. the number of
seconds requested minus the number of seconds actually slept).

Since `k_sleep()` already returns the amount of "unslept" time
in ms, we can simply use that.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-24 19:31:33 -05:00
Chris Friedt f1ececc682 posix: key: abstract pthread_key_t as uint32_t
Consistent with the change of `pthread_t`, `pthread_mutex_t`,
and `pthread_cond_t` to `uint32_t`, we can now also abstract
`pthread_key_t` as `uint32_t` and separate the implementation
detail, hidden from POSIX API consumers.

This change introduces `CONFIG_MAX_PTHREAD_KEY_COUNT`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-24 04:19:01 +09:00
Chris Friedt 96b9d7cb6d lib: posix: pthread_key: use spinlock instead of semaphore
None of the operations that `pthread_key_sem` protected were
blocking, so simply make it a spinlock.

Also made the lock static.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-24 04:19:01 +09:00
Chris Friedt 187df899da lib: posix: pthread: rename pthread_key_lock to pthread_once_lock
Since this lock is only used in `pthread_once()` it makes sense
to name it accordingly.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-24 04:19:01 +09:00
Anas Nashif cffe98d9de crc: Make the build of crc function dependent on a Kconfig
Add CONFIG_CRC for building CRC related routines.
CRC routines are now being built for each application, whether used or
not and are add in the build system unconditionally.

Keep CONFIG_CRC enabled by default for now and until all users have
converted to use the new option.

Partial fix for #50654

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-23 13:30:00 +01:00
Chris Friedt 3a56d276e9 posix: cond: abstract pthread_cond_t as uint32_t
Consistent with the change of `pthread_t` from
`struct posix_thread` to `uint32_t`, we can now also abstract
`pthread_cond_t` as `uint32_t` and separate `struct posix_cond`
as an implementation detail, hidden from POSIX API consumers.

This change deprecates `PTHREAD_COND_DEFINE()` in favour of the
(standardized) `PTHREAD_COND_INITIALIZER`.

This change introduces `CONFIG_MAX_PTHREAD_COND_COUNT`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-19 14:16:38 +09:00
Chris Friedt afae448ff2 posix: pthread: take care with pthread cond resources
Previously, `pthread_cond_init()` could not actually fail, and
destroying condition variables was a no-op, and it was missing
in `pthread_exit()`.

However, with the change of `pthread_cond_t` to `uint32_t`, and
since those are embedded inside of `struct posix_thread` for the
time being, the pthread code needs to keep track that it is
relinquishes used condition variables when a thread completes.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-19 14:16:38 +09:00
Chris Friedt 1777a33558 lib: posix: internal: use a more generic INIT mask and inlines
Previously `PTHREAD_MUTEX_MASK_INIT` was used to mark a
`pthread_mutex_t` as initialized.

The same needs to be done for `pthread_cond_t` and likely others.

Rather than copy-pasting that and a number of inlines that
duplicate the same functionality, simply make it more generic.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-19 14:16:38 +09:00
Chris Friedt b0b4c9c3f1 posix: mutex: abstract pthread_mutex_t as uint32_t
Consistent with the change of `pthread_t` from
`struct posix_thread` to `uint32_t`, we can now also abstract
`pthread_mutex_t` as `uint32_t` and separate `struct posix_mutex`
as an implementation detail, hidden from POSIX API consumers.

This change deprecates `PTHREAD_MUTEX_DEFINE()` in favour of the
(standardized) `PTHREAD_MUTEX_INITIALIZER`.

This change introduces `CONFIG_MAX_PTHREAD_MUTEX_COUNT`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-15 05:56:31 +09:00
Chris Friedt 6042acc1a9 posix: pthread: take care with pthread mutex resources
Previously, `pthread_mutex_init()` could not actually fail, and
destroying mutexes was a no-op, so it was missing in a couple of
places.

However, with the change of `pthread_mutex_t` to `uint32_t`, and
since those are embedded inside of `struct posix_thread` for the
time being, the pthread code needs to keep track that it is
relinquishes used mutex resources when a thread completes.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-15 05:56:31 +09:00
Chris Friedt 39b8b3ac8a posix: pthread_create: use spinlock for pthread_pool_lock
The `pthread_create()` function is not a cancellation point and
iterating over / mutating `posix_thread_pool` is not a blocking
operation, so use a spinlock for the internal `pthread_pool_lock`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-08 16:10:26 -05:00
Chris Friedt 660a7db3f2 posix: pthread: use spinlock instead of mutex for cancel lock
The cancel lock will not block so a mutex is overkill.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-08 13:30:02 -05:00
Chris Friedt 4bae4f426f include: posix: move pthread impl detail to posix_internal.h
The `struct pthread` and `enum pthread_state` are actually
implementation details specific to Zephyr.

Let's limit the scope where that level of detail is visible.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-07 09:09:11 -05:00
Chris Friedt 2812f61e4e posix: pthread_once: use k_mutex for pthread_key_lock
In the interest of reducing any layering concerns,
avoid using POSIX locking primitives where necessary.

Note: it is not safe to use a spinlock here, as the
callback function to `pthread_once()` may itself be
a cancellation point.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-07 09:04:01 -05:00
Chris Friedt ad5c3a1ae3 posix: pthread: remove duplicate assignment
The `thread` pointer variable is already assigned within the loop
above.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-07 10:16:09 +00:00
Keith Packard d8d81c4a09 libc/picolibc: Remove unused POSIX-ish hooks
While reviewing the dependency between Picolibc POSIX APIs and Zephyr, I
found that the picolibc libc-hooks code copied a bunch of functions from
the newlib version which weren't needed. This required replacing a few
calls to the (now removed) '_write' hook with printk instead.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-11-02 06:49:32 -04:00
Chris Friedt ce054404a2 posix: use uint32_t instead of void* for pthread_t
Several other widely-used pthread implementations
abstract `pthread_t` as `uint32_t`. The benefit
there is that we avoid passing around a pointer to
an internal structure (implementation detail).

Additionally, this removes the alias from `k_tid_t`
to `pthread_t` inside of `struct pthread_mutex`.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-02 15:39:44 +09:00
Kumar Gala 9be0914885 lib: os: p4wq: Convert CONFIG_MP_NUM_CPUS handling
Move runtime checks to use arch_num_cpus().  This is to allow
runtime determination of the number of CPUs in the future.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-31 17:08:34 +01:00
Chris Friedt f5cfeae152 posix: pthread: remove duplicate assignment in pthread_exit
The `self->retval` field was assigned twice.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-10-31 16:58:47 +01:00
Keith Packard 25c378461e libc/picolibc: Set __LINUX_ERRNO_EXTENSIONS__ for picolibc
This makes sure extra errno values from Linux that Zephyr uses are
available.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-10-31 17:17:42 +09:00
Daniel Leung cf23b312c3 lib: picolib: put stdio variables into libc partition
This adds the necessary modifier to the stdin/stdout/stderr
variables in picolib, and putting into the z_libc_partition.
This allows userspace applications to utilize these variables
for console I/O.

Fixes #51343

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2022-10-20 09:59:42 +02:00
Daniel Leung 6ee7294ac2 lib: picolib: move static to be the first modifier
Compliance check complains about static not being the first
modifier. So move them so there are no more complains
in the future.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2022-10-20 09:59:42 +02:00
Stephanos Ioannidis 28b22b276a lib: libc: newlib: Make newlib nano variant optional
The newlib nano variant is currently enabled by default when
`CONFIG_NEWLIB_LIBC=y` and the selected toolchain-architecture
combination includes the newlib nano variant support, even if
`CONFIG_NEWLIB_LIBC_NANO` is not selected by the user.

When `CONFIG_NEWLIB_LIBC=y`, this results in the newlib nano variant
being selected for some architectures (e.g. ARC, ARM and RISC-V), while
the full variant is selected for the rest of the architectures.

The above behaviour is problematic because there exist functional
differences between the newlib full and nano variants (e.g. C99 format
modifiers such as `hh`, `ll`, `z`, `j` and `t` are not available in the
newlib nano variant), and this effectively leads to different level of
C standard support across different architectures when
`CONFIG_NEWLIB_LIBC=y`.

This commit fixes this problem by making the `CONFIG_NEWLIB_LIBC_NANO`
not `default y` and requiring its user to explicitly set this symbol to
`y` when they want to use the newlib nano variant.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-10-19 16:02:51 +02:00
Anas Nashif a5bd666f4e lib: notify: build sys-notify conditionally.
Add a new Kconfig and build this code conditionally, so we do not end up
with this file being built for each zephyr app.

Partial fix for #50654

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-10-03 10:18:15 +02:00
Anas Nashif a81b322828 lib: onoff: add a config for on-off and build conditionally
Do not build this service unconditionally.

Partial fix of #50654

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-10-03 10:18:15 +02:00
Piotr Pryga e2e06a74c3 libc: minimal: Add C11 aligned_alloc
Extend capabilities of a minimal libc to support C11 capability
to allocate memory with requested alignment.

Signed-off-by: Piotr Pryga <piotr.pryga@nordicsemi.no>
2022-10-03 10:13:25 +02:00
Stephanos Ioannidis 175cc385c4 lib: picolibc: Use smaller default heap size
The picolibc heap size configuration (`CONFIG_PICOLIBC_HEAP_SIZE`) is
used to set the statically allocated malloc heap size when userspace is
enabled.

The current default heap size of 1048576 bytes (MMU) and 65536 bytes
(MPU) is too large for most platforms that Zephyr supports and may
result in the picolibc tests being filtered out due to the increased
memory footprint of the compiled image (i.e. SRAM overflow).

This commit updates the default picolibc heap size to a more reasonable
16384 bytes for MMU platforms and 1024 bytes for MPU platforms.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-09-30 13:52:24 +00:00
Chris Friedt 0217c3952a posix: conditionally compile perror pthread_common and nanosleep
Previously, these files were compiled unconditionally.

Partial fix for #50654

Signed-off-by: Chris Friedt <cfriedt@fb.com>
2022-09-28 14:06:55 +00:00
Jiafei Pan d486bd3cfe lib: libc: newlib: make sure retargetable locking is enabled in toolchain
Add build assert to make sure _RETARGETABLE_LOCKING is enabled in
toolchain, When _RETARGETABLE_LOCKING is enabled, "_LOCK_T" is "__lock"
pointer type, otherwise "_LOCK_T" is "int" type, so there will be the
following compile warnings when toolchain doesn't enable
_RETARGETABLE_LOCKING:

zephyr/lib/libc/newlib/libc-hooks.c:416:13: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
  416 |  k_sem_take((struct k_sem *)lock, K_FOREVER);
      |             ^
zephyr/lib/libc/newlib/libc-hooks.c: In function '__retarget_lock_acquire
_recursive':
zephyr/lib/libc/newlib/libc-hooks.c:423:15: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
  423 |  k_mutex_lock((struct k_mutex *)lock, K_FOREVER);
      |               ^
...

Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
2022-09-21 08:59:24 +00:00
Seppo Takalo c93b4cf307 libc: newlib: Call gettimeofday() also when CONFIG_POSIX_CLOCK
When CONFIG_POSIX_CLOCK is enabled, we should have implementation
of gettimeofday() and therefore time(NULL) should return correct
time, instead of -1.

Signed-off-by: Seppo Takalo <seppo.takalo@nordicsemi.no>
2022-09-20 08:19:23 +00:00
Krzysztof Chruscinski a7224830ce lib: os: cbprintf: Mechanism for detecting %p in static package
Static packaging is using only argument types to build a package. There
is one case where analysing argument type only is not enough to
determine package content. That is %p with (unsigned) char pointer vs
%s. In case of %s a string might need to be appended to the package
and in case of %p it must be avoided. Format string analysis is required
to distinguish those two cases.
In order to speed up the runtime inspection, additional information is
added to a static package. That is index of the string argument (where
first argument has index 0). This information allows quick format string
inspection where nth format specifier is found and checked if it is a
pointer format specifier.
Inspection algorithm is added to cbprintf_package_convert() and if %p
is found then data for that argument is discarded. Additionally, log
warning is printed with suggestion to cast pointer argument to void *
to avoid confusion. It is desired to get rid of this ambiguity because
there are going to be logging configurations where strings are stripped
from a binary and runtime inspection cannot be performed.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-09-19 10:14:23 +00:00
Gerard Marull-Paretas 79e6b0e0f6 includes: prefer <zephyr/kernel.h> over <zephyr/zephyr.h>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.

The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.

NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-05 16:31:47 +02:00
Christopher Friedt a5ee862ff0 posix: clock: make tz non-const in gettimeofday
This `tz` field is not const in the spec.

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-09-03 04:00:37 -04:00