Commit graph

4238 commits

Author SHA1 Message Date
Fabio Baltieri 6d99e38a84 scripts: check_init_priorities: ignore zephyr,cdc-acm-uart
The new stack zephyr,cdc-acm-uart driver has two separate init path, one
of which kicks in before the USB stack to start buffering log messages.

This generates a false positive in the build time priority checking,
adding a check to ignore that compatible entirely.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-07-25 09:10:26 +02:00
Anas Nashif 47102de474 west: build: support additional configuration with --test-item
Also support extra_conf_files, extra_overlay_confs, extra_dtc_overlay_files

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-07-24 11:09:12 -04:00
Anas Nashif 6fdfc91a6c west: build: also parse common section in yaml file
Parse common section and append to cmake args if available when using
--test-item option of west build.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-07-24 11:09:12 -04:00
Yves Vandervennet 58e4df6460 west: runner: add support for NXP's linkserver
Linkserver is a utility for launching and managing GDB servers for NXP
debug probes, which also provides a command-line target flash programming
capabilities. Linkserver can be used with NXP MCUXpresso for Visual Studio
Code.

For more information about LinkServer, please visit the LinkServer web
page (link [1] below).

This commit adds a runner to west, supporting debug and flash commands.

Documentation is also added.

[1] - LinkServer web page:
    https://www.nxp.com/design/software/development-software/mcuxpresso-software-and-tools-/linkserver-for-microcontrollers:LINKERSERVER

Signed-off-by: Yves Vandervennet <yves.vandervennet@nxp.com>
2023-07-21 14:57:25 -05:00
Emil Gydesen cab52d23c4 footprint: Add BT TMAP broadcast samples
Add the BT Telephony and Media Audio Profile broadcast samples.
These reflect devices that control telephone and media audio
of LE Audio.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2023-07-21 10:55:03 +00:00
Lukasz Mrugala a9a6df9b50 scripts: tests: Environment tests update
Implemented style changes proposed by gchwier previously.
Tests updated according to fixes in environment.py in
  * PR #60526
  * commit d72c434
  * commit 7afeefd

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-07-20 16:25:31 +02:00
Jamie McCrae d52b634610 doc: mgmt: mcumgr: Add details on making handlers
Adds a guide on how to make out-of-tree MCUmgr function handlers
and groups, with an example showing a test implementation.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-07-20 16:22:31 +02:00
Aleksander Wasaznik bb8b515ffa native_posix: Add support for west debug
This change adds the convenience of `west debug` to native builds.

Signed-off-by: Aleksander Wasaznik <aleksander.wasaznik@nordicsemi.no>
2023-07-20 12:43:47 +02:00
Jeremy Bettis d72c4344ed twister: Support --coverage-formats on lcov also
When using twister to generate coverage with the coverage tool lcov,
allow using --coverage-formats to pick if you want lcov or html,
defaulting to both. Picking html will also use lcov, since that is
required for geninfo.  This will allow callers to avoid the potentially
slow and disk intensive html reports if they only wanted the lcov info
file.

Signed-off-by: Jeremy Bettis <jbettis@google.com>
2023-07-19 21:00:10 -04:00
Lukasz Mrugala 48fef88038 scripts: pylib: twister: twisterlib: environment: TwisterEnv fixes
Fixes Issue #60522 by guarding member access against None parameters.
Shortens the init by using the if...else assignment.
Fixes two typos.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-07-19 20:47:29 -04:00
Keith Packard 3d94c830a3 scripts/checkpatch: Fix check for libc API defines
Re-ordering the API names moved a trailing '|' causing the pattern to match
*every* #define in the input.

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-07-19 08:57:04 +00:00
Maciej Perkowski 93031ad5a0 west_commands: Fix parsing of extra args for test-item
An arg --test-item makes west loading twister's test configurations
form sample/testcase.yaml. It has to mirror twister's behavior.
It was not the case with "extra_args" section. Quotation  marks were
not removed in west as they were in twister. The quotation marks
have to be removed from the extra_args section but left in
extra_configs. The commit adds differentiation for those.

fixes: #60297

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-07-18 21:33:00 -04:00
Yong Cong Sin 335acf817d posix: signal: implement strsignal
Implementation and ztest for strsignal.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2023-07-18 15:35:42 -04:00
Yong Cong Sin 2037362e16 checkpatch: exclude POSIX headers from typedef rule
The `NEW_TYPEDEFS` rule consistently generates false
positives when implementing POSIX standard APIs.

It makes sense to disable this check for the POSIX haders
rather than requiring merge superpowers constantly. That
way, we can merge as per usual after sufficient approvals
rather than waiting for someone with merge superpowers to
override / manually merge.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2023-07-18 15:35:42 -04:00
Keith Packard b021dece98 scripts/checkpatch: Check for patches adding #defines for libc APIs
All code in the Zephyr core must use only the Zephyr C library API
according to rules A.4 and A.5. Such code is not permitted to request API
extensions from the C library via any of the API request mechanisms.

This addition to checkpatch.pl verifies that patches don't #define
any of these:

	__STRICT_ANSI__
	_POSIX_SOURCE
	_POSIX_C_SOURCE
	_XOPEN_SOURCE
	_ISOC99_SOURCE
	_ISOC11_SOURCE
	_ATFILE_SOURCE
	_GNU_SOURCE
	_BSD_SOURCE
	_SVID_SOURCE
	_DEFAULT_SOURCE

Reference: #49922

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-07-18 19:20:24 +00:00
Lukasz Mrugala d42b2e0a9b scripts: tests: Environment tests update
Implented suggestions proposed by gchwier.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-07-18 16:20:54 +02:00
Lukasz Mrugala 7c270c8488 scripts: tests: Expand environment.py tests
As we aim to enhance our test coverage, environment module
is a target-rich environment. Some errors in the original
module were discovered and tests regarding these cases were
commented out until they are fixed.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-07-18 16:20:54 +02:00
Grzegorz Chwierut 57c27cd736 twister: pytest: Added fixture for mcumgr with unittests
Added mcumgr fixture to pytest-twister-harness. Added unittests for
new fixture.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-07-17 16:23:59 -04:00
Dmitrii Golovanov 7afeefd7dd twister: Fix flashing timeout command line options check
Fix `--device-flash-with-test` command line options check logic
to be effective with `--device-testing`.

Remove `--device-flash-timeout` check for `--device-testing`
presence as marginal.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-07-17 15:57:03 -04:00
Emil Gydesen 8fdb0fb1ca footprint: Add BT TMAP samples
Add the BT Telephony and Media Audio Profile samples.
These reflect devices that control telephone and media audio
of LE Audio.

The central is typically a more resourceful device, and the
peripheral a resource-constrained device.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2023-07-17 13:20:54 +00:00
Lukasz Mrugala a36af82f64 scripts: tests: Twister test expansion - quarantine
Here we achieve 100% coverage for the quarantine module,
thanks to unit tests for QuarantineElement and QuarantineData dataclasses.

Implemented PR suggestions of gchwier.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-07-13 14:18:27 +02:00
Gerard Marull-Paretas 7f9215da0e doc: move requirements to doc/
Zephyr scripts do not require documentation dependencies, so let's
move them from scripts/ to doc/.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-07-12 09:13:49 -04:00
Gerard Marull-Paretas b990a5fdb6 doc: update sphinx/docleaf requirements
Update Sphinx and docleaf. Note that Sphinx 7 is not an option because
rtd_theme requires <= 6 (see
https://github.com/readthedocs/sphinx_rtd_theme/issues/1463).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-07-12 09:13:49 -04:00
Manoel Brunnen 368b0e0d16 twister: Initialise unbound 'hardware' variable
hardware was referenced as return value, without being declared. When the
program goes into the 'except' path, hardware is unbound, but returned,
which throws an error:

`UnboundLocalError`: local variable 'hardware' referenced before assignment

Signed-off-by: Manoel Brunnen <manoel.brunnen@ithinx.io>
2023-07-10 13:57:21 +02:00
Iuliana Prodan dc49a314fd west: sign.py: fix sign when CONFIG_KERNEL_BIN_NAME is used
This fixes the following build error:
```
zephyr/zephyr.elf', needed by 'zephyr/zephyr.ri', missing and no known
rule to make it
```
This appears when CONFIG_KERNEL_BIN_NAME is used.

Therefore, do not use zephyr.elf since some samples might be called
based on CONFIG_KERNEL_BIN_NAME.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
2023-07-10 09:29:33 +02:00
Marc Herbert e1f36c40eb west: sign.py: de-duplicate the bootloader condition
De-duplicate the `if target in ('imx8', 'imx8m'):` copy/paste/diverge
before someone updates one and not the other. Also better to define
bootloader always.

Zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-07-10 09:29:33 +02:00
Alberto Escolar Piedras e9721db183 native_simulator: Align with upstream latest with more trampolines
Upstream SHA: be3eac6931b49291e51da5d5aa1a99ab58f81541

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-07 14:32:41 +02:00
Carlo Caione 935268ee64 devicetree.h: DT_FOREACH_NODE_VARGS, DT_FOREACH_STATUS_OKAY_NODE_VARGS
Add the _VARGS variant of DT_FOREACH_NODE and
DT_FOREACH_STATUS_OKAY_NODE for when we want to do some kind of
operation on all the nodes in the tree.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-07-07 11:41:19 +02:00
Emil Obalski e8a7c6706f scripts: west: Check for sysbuild flag when parsing test item
If build is used with -T option, sysbuild flag was ignored
neverthelesss being present in sample.yaml/testcase.yaml
file.

Signed-off-by: Emil Obalski <Emil.Obalski@nordicsemi.no>
2023-07-07 10:27:23 +02:00
Henrik Brix Andersen c5402548d1 scripts: west: runners: esp32: add support for disabling flasher stub
Add support for disabling loading of the flasher stub and instead only talk
to the ROM bootloader.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2023-07-07 09:19:02 +02:00
Christopher Friedt 6e27b7d566 checkpatch: exclude lib/posix/*.c from negative errno rule
The `USE_NEGATIVE_ERRNO` rule consistently generates false
positives when working with certain areas of POSIX.

It makes sense to disable this check for the POSIX API
rather than requiring merge superpowers constantly. That
way, we can merge as per usual after sufficient approvals
rather than waiting for someone with merge superpowers to
override / manually merge.

Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2023-07-05 21:06:55 -04:00
Alberto Escolar Piedras 04a16f21e4 valgrind: Update suppression file to align with updated POSIX arch
Things have moved around so we need to update the
suppression file so the callstack and types of
leaks valgrind reports still match.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Alberto Escolar Piedras 850fc2f22f Native simulator: Add first version in-tree
Add the first version of the native simulator.
The simultaor is taken as is from
https://github.com/BabbleSim/native_simulator/
sha: 74986abfe088a1780e604dae65f87470b4c2a0eb

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Fabio Baltieri 9de541b9c2 scripts: ci: check_compliance: add a no-modules Kconfig check
Add a variation of the basic Kconfig check that runs with no modules,
catches symbols that are used in the main repository but are defined
only in modules, which are potentially problematic if a downstream
project is not using the specific module.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-07-05 09:11:02 +02:00
Carles Cufi ff2d9cfcb6 scripts: compliance: Fix handling of integer node items
Some node items in Kconfig can be kconfiglib.MENU or kconfiglib.COMMENT.
Those are integers and thus do not contain a node.item.name field.
Handle those separately to avoid hitting the followig exception:

Traceback (most recent call last):
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1307, in main
    n_fails = _main(args)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1242, in _main
    test.run()
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 277, in run
    self.check_no_redefined_in_defconfig(kconf)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 445, in check_no_redefined_in_defconfig
    Kconfig node '{node.item.name}' found with prompt or help in\
    {node.filename}.
AttributeError: 'int' object has no attribute 'name'

Seen in #58454.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-07-04 08:23:02 +02:00
Michael Jones 30c3ce4a92 doc: Update requirements-doc.txt to use docleaf
Instead of breathe.

I'm not sure of the best version comparison to use. There is no
intention to break the API for docleaf but it is also still young
in some ways and there might be adjustments to the configuration values
that would need to be reflected in the version number as it is still
pre-1.0 that would only be a minor version change.

Signed-off-by: Michael Jones <m.pricejones@gmail.com>
2023-07-03 10:05:52 +00:00
Roberto Medina 6622735ea8 arch: arm64: add support for coredump
* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.

Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
2023-07-03 09:32:26 +02:00
Grzegorz Chwierut 4045bab1b7 twister: pytest: Parse report file to get testcases
Extended parsing of report.xml file - produced by pytest-twister-harness
plugin. Now testcases are properly extracted and added to twister
report. Added unit tests.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-06-25 02:44:49 -04:00
Anas Nashif b8fc1c3607 twister: run slow tests only
We have many tests that are marked as slow that do not run in CI and any
known workflow, to isolate them and be able to run them on their own,
add a new option --enable-slow-only which causes only those tests to build
and execute.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-22 17:14:02 -04:00
Lucas Tamborrino eb028ccf55 debug: coredump: xtensa: Add esp32s3
Add coredump support for esp32s3.

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2023-06-21 16:06:06 -04:00
Lucas Tamborrino ba3766a75f debug: coredump: xtensa: add esp32s2
Add coredump support for esp32s2.

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2023-06-21 16:06:06 -04:00
Lucas Tamborrino fbbb82d6ac debug: coredump: esp32: Fix registers index for current toolchain
The current toolchain version for espressif SoCs does not
have the same limitation on registers indexing as the previous
one, enabling sending the correct A0-A15 register values directly

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2023-06-21 16:06:06 -04:00
Gerard Marull-Paretas b6d5d246b7 scripts: build: gen_device_deps: s/--dynamic-handles/--dynamic-deps
Align naming of the CLI option with recent changes in handles naming.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas d93586f831 device: s/__devicehdl_/__devicedeps_/
Rename the device dependencies array variable to use the "deps" name, in
line with latest renamings in device.h

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas 4eb406e093 device: s/DEVICE_HANDLE/DEVICE_DEPS/
Rename multiple internal device macros to use the DEVICE_DEPS naming, so
that it is clear they belong to the device dependencies APIs.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas e5f82cb5dc device: hide DEVICE_HANDLE_SEP/ENDS
These macros are used internally by the device dependencies functions.
There's no need to expose them publicly, so prefix them with Z_ and add
them under INTERNAL_HIDDEN docs section.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas 8bee39edff device: s/device_handles/device_deps/ in linker scripts
Use the "device_deps" naming scheme to emphasize we are storing device
dependencies. The fact we are using device handles to store them is an
implementation detail.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas 92707866d1 device: s/gen_handles/gen_device_deps
Rename the gen_handles script and all of its references/associated files
to gen_device_deps. The new new makes things more clear, because the
script just take care of generating, for each device, an array of device
dependencies. While device handles are used internally to store this
information, it is in reality an implementation detail.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Grzegorz Chwierut 732dd4843d twister: pytest: Simplify interface to pytest-twister-harness
Add DeviceAbstract class to default imports from pytest-twister-harness
package to simplify importing DUT package, when creating tests.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-06-20 14:46:55 -04:00
Björn Stenberg 234fec579a scripts: gen_relocate_app.py: Give sections unique names
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.

This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.

This solves #54785.

Signed-off-by: Björn Stenberg <bjorn@haxx.se>
2023-06-20 16:33:49 +02:00
Krzysztof Chruściński 9c2d1c8e99 scripts: logging: dictionary: Fix database generation
I noticed that not all strings were put into database. That is because
algorithm was searching for any null terminated byte arrays in the
section (e.g. rodata) and then attempting to decode it to string.
However, section may contain other static const variable and if a
string is preceded by a variable that has non-printable bytes and no
zero at the end then algorithm was picking a candidate that started
with non-printable characters followed by the string. Such candidate
was discarded because it was not a valid string.

Algorithm is changed to treat as string candidate any sequences that
contains printable bytes followed by 0.

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2023-06-19 15:03:07 +02:00
Jamie McCrae 12e8de71b1 scripts: build: parse_syscalls: Fix duplicate paths on windows
Fixes an issue on windows where there would be many permutations
of paths with different folder separators by converting to
posix-style path separators.

Issue was introduced by
https://github.com/zephyrproject-rtos/zephyr/pull/58351

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-06-19 14:57:39 +02:00
Marc Herbert 07f2c7a1df west: sign.py: give rimage a key placeholder when there's none
rimage always requires a key argument even when it does not use it.

Some configurations (IMX) use rimage only for "stitching" the image but
not for signing. These were failing to build directly from west with
pretty cryptic stack trace, see below. This has never been a problem for
the SOF project because it has always worked around this rimage
limitation by providing a bogus -k argument - very much like the one
added by this commit.

```
File "zephyrproject/zephyr/scripts/west_commands/sign.py",  in sign
  extra_ri_args += [ '-k', str(sof_src_dir / 'keys' / cmake_default_key) ]
TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType'
ninja: build stopped: subcommand failed.
```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-06-19 08:38:48 +02:00
Daniel Leung 80e78208e6 kernel: syscalls: no need to include all syscalls in binary
The syscall generation phase parses all header files to look
for potential syscalls, and emits all the relevant files to
enable syscalls. However, this results in all the syscall
marshalling functions being included in the final binary.
This is due to these functions being referred to inside
the dispatch list, resulting in ineffective garbage
collection during linking. Previous commits allows each
drivers and subsystems to specify which header files
containing syscalls are relevant. So this commit changes
the syscall generation to only include the syscalls needed
for the build in the syscall dispatch list and removing
various bits related to that. This allows the linker to
garbage collect unused syscall related function, and thus
reducing final binary size.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-06-17 07:57:45 -04:00
Grzegorz Chwierut c37deeb0c4 twister: Use natural sort when generating hardware map
Use the natural sort of list when generating a hardware map. The
list is sorted with a serial port as a key. When more than 10 ports
are active and some of devices use more than one port, the ports
of one device may be listed in wrong sequence, which may cause
futher problems with selecting the right port for listening to
the device.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-06-17 07:54:46 -04:00
Gerard Marull-Paretas e5335f345a device: dynamic device handles were declared as const
When CONFIG_HAS_DYNAMIC_DEVICE_HANDLES is selected, devices handles are
placed in RAM region so that they can be modified at runtime. However,
the gen_handles.py script added the `const` attribute to the device
handle arrays regardless of this setting, leading to faults when running
the application. This may be an indicator that this feature is not
actively used.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-14 13:48:02 -04:00
Anas Nashif 7f0aff9045 ci: testplan: do not start more runners than needed
We have been launching 10 runners up a certain number of tests, although
we only neeed half of that for some scenarios.
Too many runners started that have to execute just a small number of
tests wastes times on setup and blocks the queue. Just start the number
of nodes needed based on initial calculation.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-08 07:33:56 -04:00
Jamie McCrae 65c9058ed4 sysbuild: Fix value propagation of signature type to MCUboot
Fixes an issue whereby the type of the signature was not passed
to MCUboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-06-08 06:51:33 -04:00
Piotr Golyzniak 4f77883043 scripts: twister: add timeout for pytest process
Add protection timeout for pytest subprocess, to avoid situation of
suspending whole Twister in case of internal pytest test problem.

Co-authored-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-06-02 18:55:27 -04:00
Grzegorz Chwierut ab5b48beed twister: pytest: Wrap iter_stdout
Added waiting for prompt instead of calling time.sleep,
to fix issue with longer setting up platforms with TF-M
enabled. Wrapped iter_stdout generator, to avoid issues
when generator is created and called multiple times
in test scenarios.
Fixes: #58747

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-06-02 18:55:27 -04:00
Jordan Yates 86dd23e1eb scripts: build: gen_handles: exit on relocatable files
Print a clean error message when provided a relocatable file, instead
of generating a source file which will cause compilation errors further
away from the source.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2023-06-02 18:53:06 -04:00
Jordan Yates 60aa9e1857 scripts: build: elf_parser: fix relocatable data offsets
Based on investigations with relocatable `.elf` files, the symbol table
data in relocatable files is not shifted by the section address. Remove
the shift from these files to ensure that data can be pulled from the
symbol table.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2023-06-02 18:53:06 -04:00
Jordan Yates 4d181d6395 scripts: build: elf_parser: retrieve section directly
Retrieve the section holding the symbol data directly from the symbol
information, instead of trying to find a section that contains data at
the correct address. The later approach does not work for relocatable
files, which contain multiple sections all containing data in
overlapping temporary memory addresses (usually starting at 0).

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2023-06-02 18:53:06 -04:00
Carles Cufi d2fff07b4a scripts: size_report: Use default encoding to process filenames
The linker generates an .elf file wth encoded strings containing
filenames. If those filenames contain non-ASCII characters we need to
know which encoding was used by the linker to store those filenames. Use
the system's preferred encoding for that purpose.

Fixes #55148.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-06-02 15:44:36 -04:00
Carles Cufi fde1a239c9 scripts: snippets: Use UTF-8 to write CMake files
UTF-8 is not always the default encoding on some platforms, but it's the
only reliable way to store encoded strings for filenames using
non-ASCII characters.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-06-02 15:44:36 -04:00
Fabio Baltieri 44e691e20e scripts: check_init_priorities: add extra verbose output
Add support for more verbose output enabling debugging output and add a
first debug print to output the file to device path mapping. Found this
to be extremely useful to find what file is instantiating a specific
device for non obvious cases.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-06-02 04:47:24 -04:00
Anas Nashif 432eefca05 ci: tags: add mcumgr to tags file
Filter based on changes in pull request and do not build mcumgr on
unrelated changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif 9cef0451ba ci: tags: add networking to tags file
If we are not changing the network subsystem, do not build samples and
tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif cec8fd1c6e tests: ztest: fix tags for testsuite and unify them
Use test_framework as unified tag so we can filter tests in a consistent
way.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif 4e8cdb8bc3 ci: manage when to build testsuite tests
Do not build test framework tests on every change, use tag filtering to
improve CI performance.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif 522624d6c6 west: build: keep kconfig quotes from testcase.yaml
When parsing extra configs from the yaml file, keep quotes.

Fixes #56248

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-05-31 10:02:28 -04:00
Carles Cufi 462dc9746b scripts: footprint: Fix size_report on freestanding apps
os.path.commonpath() throws a ValueError exception when paths in the
list provided share nothing in common (like for example two Windows
paths on different drive letters). Handle that special case properly
instead of letting the exception propagate up.

Fixes #51549.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-05-30 15:00:52 -04:00
Anas Nashif dae79cefaa twister: coverage: set coverage platforms default to an empty list
When calling twister with --coverage and no platform specified on the
command line, set the the value of coverage platforms to an empty list.

Fixes #57534

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-05-30 06:30:39 -04:00
Lukasz Mrugala 164c26ff18 scripts: Typo in the word 'platform' fix
testinstance.py contains a string literal comparison to the
word 'platform', which has been misspelled as 'plaform'.

The change is verified by a test expansion.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-05-29 14:34:33 -04:00
Keith Short dfa992c596 twister: Remove newline suffix in BinaryHandler
Update the code that removes newline suffix in BinaryHandler for
compatibility with python 3.8.

Fixes: https://github.com/zephyrproject-rtos/zephyr/issues/58335

Signed-off-by: Keith Short <keithshort@google.com>
2023-05-26 14:59:03 -04:00
Madhurima Paruchuri a19d905cc4 USB-C: genVIF: Cleanup and add support to pick static data from input
Removed few VIF properties which are being hardcoded
Updated the script to parse source VIF XML and add information to
the output
Added optional Kconfig option to configure custom source VIF XML path
Cleaned up the code

Signed-off-by: Madhurima Paruchuri <mparuchuri@google.com>
2023-05-26 13:54:43 -04:00
Grzegorz Chwierut f1f305f4ae scripts: twister: adaptation for pytest plugin
Making the necessary changes to enable the new pytest plugin.
By default Twister should work without the pytest-twister-harness
plugin installed. To achieve this, each time Twister calls pytest,
the PYTHONPATH environment variable is expanded and the
`-p twister_harness.plugin` option is added to the pytest command.

Co-authored-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-05-26 13:52:15 -04:00
Piotr Golyzniak 8c4bfcf324 scripts: add pytest plugin
Adding pytest plugin dedicated to running pytest tests in Zephyr
project. This plugin provides a dut fixture which allows to handle
bidirectional communication with the device under test. This version
of plugin can be used for tests dedicated to real hardware, QEMU and
native_posix simulator.

Co-authored-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Co-authored-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Co-authored-by: Katarzyna Giadla <katarzyna.giadla@nordicsemi.no>
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-05-26 13:52:15 -04:00
Yuval Peress ca5bf10143 sensor_shell: Update to new sensor_read API
Update the sensor shell logic to use the new sensor_read() APIs and
make triggers an option of the sensor_shell sample (this avoids the
trigger stealing the interrupt status from one-shot reads).

Signed-off-by: Yuval Peress <peress@google.com>
2023-05-26 11:04:54 -05:00
Yuval Peress ed380de152 sensors: Add new async one-shot reading API
Add a new async API based on the RTIO subsystem. This new API allows:
1. Users to create sampling configs (telling the sensor which channels
   they want to sample together).
2. Sample data in an asynchronous manner which provides greater control
   over the data processing priority.
3. Fully backwards compatible API with no driver changes needed for
   functionality (they are needed to improve performance).
4. Helper functions for processing loop.

Signed-off-by: Yuval Peress <peress@google.com>
2023-05-26 11:04:54 -05:00
Michał Szprejda bdf02ff5d6 Twister: Add integration with renode-test
Add support for calling the `renode-test` command from west and twister.
Enable running Robot Framework tests suites in Renode.

Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Signed-off-by: Mateusz Hołenko <mholenko@antmicro.com>
2023-05-26 09:43:49 -04:00
Niek Ilmer abf9e4d1f4 scripts: runners: ezflashcli: Add support to flash images for MCUboot
Internal bootloader will only run application image if valid product
header is present on flash. This means product header is required for
application that are not linked to code partition. Other applications
that are linked to code partition are meant to be run by MCUboot and
should not touch product header, with the exception of MCUboot itself
which is also run by internal bootloader so requires product header.

Default flash load offset for applications not linked to code partition
is set to 0x2400 as this is where internal bootloader looks for an
application image to run based on product header written by flasher.

Flash load offset for MCUboot is set from boot partition.

Valid product header is added by ezFlashCLI when using "flash_image"
command.

Co-authored-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Niek Ilmer <niek.ilmer.aj@renesas.com>
2023-05-26 05:53:02 -04:00
Lukasz Mrugala 6ce37948a0 scripts: Twister ConfigurationError Fix
Current implementation of ConfigurationError causes
an exception to be thrown.
It is because a PosixPath is being concatenated with an str with a '+'
operator, which is not permitted.
The path in question is now cast to an str.
Additional test has been added to test_twister.py to track regressions.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-05-26 05:50:20 -04:00
Kumar Gala 0c5ff8ce43 twister: set default supported toolchains in twister
For the arch's that have standard toolchains thare are supported
across the arch set them in twister rather than in <BOARD>.yaml.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-05-26 05:48:59 -04:00
Stephanos Ioannidis 4f5cb1bb1e scripts: snippets: Preserve snippet processing order
This commit updates the `snippets.py` script to process the snippets
in the order that the `--snippet` arguments are specified, which
corresponds to the order of snippets listed in the `SNIPPET` CMake
variable.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2023-05-25 21:51:26 +02:00
Marek Matej 8e40db577a west: esp32: Fix target flashing without the IDF boot
Make use of the IDF partition table only if asked to.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-05-25 16:15:54 +02:00
Fabio Baltieri 8412cb32ce scripts: build: add a test suite for check_init_priorities
Add a bunch of unit tests for the various check_init_priorities classes.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-05-25 15:40:07 +02:00
Fabio Baltieri ef1bd08823 scripts: build: add a script for build time init priority validation
Add a check_init_priorities.py scripts. This goes through all the object
files looking for known initialization sections and building a map of
device ordinals and their effective initialization priority. Then
compares that with the list of dependencies generated based on the
device tree and reports any warning (dependent devices, same priority)
or error (incorrect priority).

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-05-25 15:40:07 +02:00
Torsten Rasmussen ba48dd8763 cmake: support snippets scope for zephyr_get()
Fixes: #57139

Snippets provides the possibility of defining EXTRA_DTC_OVERLAY_FILE and
EXTRA_CONF_FILE in snippets.
Snippets must co-exist with existing infrastructure of
EXTRA_DTC_OVERLAY_FILE and EXTRA_CONF_FILE, and a user specifying a
snippet must be able to specify extra files for adjusting the snippet.

This means that if the following is specified:
`-DSNIPPET=some_snippet -DEXTRA_CONF_FILE=extra.conf`
then `extra.conf` may contain adjustments to the snippet.
Similar to sysbuild. Imagine a sysbuild controlled image uses a default
snippet for building, by ensuring that any extra
`-D<image>_EXTRA_CONF_FILE=extra.conf` arguments takes precedence over
the snippet we allow users to make adjustments if they need.

This commit introduces a snippets scope where snippet scoped variables
can be set with `zephyr_set()` and then `zephyr_get()` will take the
snippet scoped variables into consideration before returning.

Adjust calls to `zephyr_get(EXTRA_DTC_OVERLAY_FILE)` and
`zephyr_get(EXTRA_CONF_FILE)` to use `MERGE` to ensure all scopes are
considered.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 13:31:53 +00:00
Torsten Rasmussen 1561a0705f snippets: support for EXTRA_DTC_OVERLAY_FILE and EXTRA_CONF_FILE
This commit adds change snippets to use the new EXTRA_DTC_OVERLAY_FILE
and EXTRA_CONF_FILE Zephyr build system variables instead of
DTC_OVERLAY_FILE and OVERLAY_CONFIG.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 13:31:53 +00:00
Torsten Rasmussen db89e7699d sysbuild: generate .config.sysbuild for controlling build settings
Sysbuild now generates a .config.sysbuild config file which specifies
settings controlled by sysbuild.
Any setting specified in this .config will overrule user provided
setting, and a warning will be raised if the sysbuild controlled value
is different from the value specified by the user.

This has the following benefits:
- Allow sysbuild to control any build setting without adjustments to
  the existing Kconfig tree
- Allow sysbuild to adjust settings based on knowledge regarding enabled
  images / bootloaders.
- Cleanup CMake code, as settings in sysbuild no longer needs to be
  propagated using CMake cache variables.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 14:57:41 +02:00
Torsten Rasmussen 5c71e68607 kconfig: provide an option for enforcing Kconfig settings
This commit introduces an internal FORCE_CONF_FILE CMake setting which
allows higher order build systems to generate a configuration file
which will always take precedence.

This means that in case a user tries to change any setting to be
different than the defined value in the FORCE_CONF_FILE provide file(s),
then a warning will be printed and the setting will be reset to the
value given in the FORCE_CONF_FILE file.

Example of such warning:
   <path>/.config.sysbuild:1: warning: BOOTLOADER_MCUBOOT
   (defined at Kconfig.zephyr:766) set more than once.
   Old value "n", new value "y".

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 14:57:41 +02:00
Anas Nashif 6feacfa243 twister: improve new ztest case detection
Address cases where a suite and its testcases are defined in different
files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-05-25 04:44:11 -04:00
Anas Nashif 164c887ec1 twister: fix testcase detection
Do not reset initial list of testcases in cases where we do not use
ztest.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-05-25 04:44:11 -04:00
Yuval Peress 64c41022e2 twister: Add support for gTest as a harness
Some 3P logic (Pigweed for example) is already heavily invested in
gTest as a testing framework. Adding the `gtest` harness enables
running the existing 3P tests on various Zephyr platforms and configs.

Signed-off-by: Yuval Peress <peress@google.com>
2023-05-24 19:39:10 -04:00
Maureen Helm 6a6d05b941 twister: Remove newline suffix in BinaryHandler
Simics converts newline characters when writing console output to
stdout, so we need to remove them as a suffix string. Otherwise the
console harness fails to match expected PASS/FAIL/RunID string patterns
and twister tests timeout.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2023-05-24 13:14:27 -04:00
Katarzyna Giądła 8bb576b7e8 scripts: requirements: Update protobuf and grpcio-tools requirements
Minimal requirements for nanopb were not accurate
to allow its usage (and causing related tests to build fail).
This change updates requirements for protobuf (>=3.20.3) and
grpcio-tools (>=1.47.0) to use protocol buffer.

Fixes #56103

Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
2023-05-23 08:55:16 +02:00
Grzegorz Chwierut 243a9dfba4 scripts: twister: skip unknown platform from hardware map
After generating a hardware-map a platform field is marked
as 'unknown'. If it is not changed, do not process them
when running --device-testing.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-05-22 11:24:46 -04:00
Piotr Golyzniak e8c3baa4a1 scripts: twister: create build dir for logs
In some cases CMake doesn't create build directory and in those cases
Twister should create this directory by itself.

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-05-22 07:03:10 -04:00
Marc Herbert 4975c94845 west.cmake: make MIN_WEST_VERSION catch up with requirements-base.txt
Also add a comment in each file reminding to keep them the same.

Fixes 251f269e23 ("west: v0.14.0 is required now (and soon, v1.1")

Confusing error message before this commit:

```
-- Found west (found suitable version 0.13.1, minimum required is 0.7.1)
CMake Error at SOF/zephyr/cmake/modules/zephyr_module.cmake:77 (message):
  Traceback (most recent call last):

    File "SOF/zephyr/scripts/zephyr_module.py", line 733, in <module>
      main()
    File "SOF/zephyr/scripts/zephyr_module.py", line 678, in main
      west_projs = west_projects()
                   ^^^^^^^^^^^^^^^
    File "SOF/zephyr/scripts/zephyr_module.py", line 536, in west_projects
      from west.configuration import MalformedConfig

  ImportError: cannot import name 'MalformedConfig'
                      from 'west.configuration'
  (west/src/west/configuration.py)
```

Clearer error message after this commit:

```
CMake Error at SOF/zephyr/cmake/modules/west.cmake:68 (message):
  The detected west version, 0.13.1, is unsupported.

    The minimum supported version is 0.14.0.
    Please upgrade with:
        /usr/bin/python3.11 -m pip install --upgrade west
```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-05-22 10:17:11 +02:00