Commit graph

3643 commits

Author SHA1 Message Date
Carles Cufi 336aa9dc88 modules: Basic binary blob infrastructure
This patch introduces the basic infrastructure to list and fetch binary
blobs. This includes:

- The new 'blobs' extension command
- An implementation of the `west blobs list` command
  with custom formatting
- A very simple mechanism for loading fetchers
- A basic implementation of an HTTP fetcher

In order to ensure consistency among the west extension commands in the
main zephyr tree, we reuse a similar class factory pattern that is present
for ZephyrBinaryRunner instances in the ZephyrBlobFetcher case. This
could be achieved with a simpler mechanism, but opted for consistency
before simplicity.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-16 05:59:02 -07:00
Carles Cufi 57e5cd41ca scripts: modules: Use pathlib to locate the module file
Instead of using strings, use pathlib's functionality to handle the
fact that it can have two different extensions.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-16 05:59:02 -07:00
Christoph A Schnetzler 6fc4bb9486 kernel: Prevent Wundef
prevent `Wundef` warnings from occurring due to
missing CONFIG_ symbols and __cplusplus.

Signed-off-by: Christoph A Schnetzler <Christoph.Schnetzler@husqvarnagroup.com>
2022-08-15 14:36:47 +02:00
Guo Lixin 0ea346009f twister: avoid monitor serial when flash error
Currently, twister keeps on monitoring serial output until timeout
even if we have a flash error. And the error reason will be reported
as 'No Console Output(Timeout)', which is not accurate.
So add a flash_error flag to control if we need to monitor the output.

Signed-off-by: Guo Lixin <lixinx.guo@intel.com>
2022-08-13 08:52:03 -04:00
Anas Nashif f28a8d2240 twister: fix bug retrieving subtests
During the recent overhaul, some code was left retrieving cases the old
way.

Fixes #48897

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-08-12 17:39:35 +02:00
Torsten Rasmussen 5fa4156462 doc: sysbuild documentation guide
This is this initial sysbuild guide on what sysbuild is and how to build
a sample using sysbuild.

It provides an architectural overview of sysbuild.
Descriptions on how to use `west build` or `cmake` + `ninja` to build
projects with the sysbuild infrastructure.

Flashing is described through the use of `west flash`.

Extending sysbuild with additional Zephyr based applications are
described, and reference to CMake documentation for including non-Zephyr
based applications are provided.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-12 13:56:55 +02:00
Gerard Marull-Paretas d9ed09357e doc: update requirements
Update to Sphinx 5.x, also update breathe since version 4.34 is the
first to officially support Sphinx 5.x.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-12 11:04:26 +02:00
Fabio Baltieri 0226e53e23 Restore "cmake: remove device_extern.h logic"
This reverts commit 87c6789355, restoring
commit 3b341085a2.

Restore the original change now that the underlying issue has been fixed
in 6cfb18686e.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-08-12 07:45:01 +01:00
Fabio Baltieri 87c6789355 Revert "cmake: remove device_extern.h logic"
This reverts commit 3b341085a2.

Seems to be breaking CI for few qemu platforms.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-08-11 17:21:46 -05:00
Emil Gydesen 5cff937b3a footprint: Add Hearing Access Hearing Aid BT Audio sample
Add the hearing aid sample to the footprint tracking for the
nrf52840dk_nrf52840 and nrf5340dk_nrf5340_cpuapp.

The sample should represent a simple hearing aid implementation,
and is thus a very good candidate for tracking the footprint
of LE Audio in Zephyr.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2022-08-10 13:08:18 +02:00
Carles Cufi a580b3d174 west: Fix handling of modules in the boards command
The boards command was not properly using the zephyr_module
functionality to obtain the board roots of all modules. Fix that by
moving the functionality required to the core zephyr_module file and
reuse it from external scripts.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-09 16:07:51 +02:00
Chen Peng1 7f105cea87 twister: support board@revision in platform filter
support to use board@revision as platform filter when running
twister, like "twister -p nucleo_f030r8@1 ...".

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-08-09 08:45:55 -04:00
Martí Bolívar 3b341085a2 cmake: remove device_extern.h logic
This file is no longer used by device.h, so let's avoid spending time
generating the content formerly in it.

In order to leave a pointer in place for users who are expecting to
see it or are pulling it into their own builds, however, replace its
contents with an #error directive that tells them what happened. This
can be removed later on when we expect people are used to the
transition.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-08 10:44:41 +02:00
Martí Bolívar f0d11f780c gen_defines: add helpers for iterating over the entire tree
Add two helper macros:

- DT_FOREACH_HELPER
- DT_FOREACH_OKAY_HELPER

These are internal generated macros which will be used to define
devicetree.h APIs for iterating over every node in the tree, and every
node in the tree with status "okay", respectively.

We can use these new APIs to implement the functionality in
device_extern.h natively from device.h without requiring a generated
header. It will also be useful for other purposes later on.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-08 10:44:41 +02:00
Martí Bolívar 190197ec97 gen_defines: trivial rename
Generalize a function name as prep for extendings its scope.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-08 10:44:41 +02:00
Kumar Gala 57538262e1 scripts: kconfig: tweak dt_chosen_label
Now that we've removed the label property from most devicetree nodes
the dt_chosen_label will end up returning "" in most cases.  For the
small handful of cases that the function is used, return the node.name
instead as this matches what DEVICE_DT_NAME will do.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-08 10:41:34 +02:00
Keith Short f896fc2306 scripts: gen_handles: Sort the device handles
Replicate the devicetree dependencies into a sorted list. This ensures
that the structures added to the .__device_handles_pass2 section are
reproducible from build to build.

Tested with: west build -b native_posix tests/drivers/build_all/sensor

Without this change, two consecutive builds do not compare.

Signed-off-by: Keith Short <keithshort@google.com>
2022-08-05 13:00:35 +02:00
Gerard Marull-Paretas a57001347f scripts: west_commands: runners: jlink: support pylink >= 0.14
pylink 0.14.0 changed the class variable where JLink DLL library name
(libjlinkarm) is stored. This patch adds support for new pylink
libraries while keeping backwards compatibility.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-05 12:58:23 +02:00
Gerard Marull-Paretas 7007451955 scripts: build: elf_parser: use node name for Graphviz diagrams
The elf_parser library now generates a dot file with device dependencies
that can be later rendered using Graphviz. Each node in the diagram
contains the device label (taken from DT node). In some cases the label
property can be None, leading to build failures like:

```
line 273, in device_dependency_graph
text = '{:s}\\nOrdinal: {:d} | Handle: {:d}\\n{:s}'.format(
TypeError: unsupported format string passed to NoneType.__format__
```

This patch switches to node name instead, which will always be set to
some value. This value is actually what devices get now as a name if
they do not have a label set.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-04 18:36:35 +02:00
Torsten Rasmussen 69f4fa6a4e west: adding --domain to west build synopsis
Adding `--domain DOMAIN` to the west build synopsis.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-08-04 13:06:13 +02:00
Henri Xavier b187477fe7 scripts: Upgrade to junitparser v2+
There is a breaking change in Junitparser 'TestCase.result' between
v1.x and v2.x.

Update check_compliance.py minimally to take into account this change.

This avoids relying on an outdated package.

Signed-off-by: Henri Xavier <datacomos@huawei.com>
2022-08-03 14:04:36 -04:00
Torsten Rasmussen 8408af6d7c scripts: west commands to support --domain
This commit extends the west commands build, flash, and debug to support
--domain when having multiple domains (images) defined in a domains.yaml
build file.

The domains.yaml uses the following yaml format to specify the
build directory of each domain in the multi image build:
> default: <domain-n>
> domains:
>   <domain-1>:
>     build_dir: <build_dir-domain-1>
>   <domain-2>:
>     build_dir: <build_dir-domain-2>
>   ...

`west <build|flash|debug>` has been extended to support
`--domain <domain>`.

`west build` calls CMake to create the build system, and if `--domain`
is given, then the build tool will be invoked afterwards for the
specified domain.

`west flash` will default flash all domains, but `--domain <domain>`
argument can be used to select a specific domain to flash, for example:
> west flash --domain mcuboot

`west debug` only a single domain can be debugged at any given time.
If `--domain` is not specified, then the default domain specified in the
domains.yml file will be used.
Users can still select a different domain, for example with:
> west debug --domain mcuboot

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-08-03 16:05:07 +02:00
Torsten Rasmussen 5fe5d6b43d scripts: extend west build command to support sysbuild CMake project
west build now support the sysbuild CMake project which allows users
to easily enable and build a bootloader together with any sample.

This will allow for cleaning up samples and boards which already does
custom multi image in a non-generic way, for example:
- mps2_an521 which includes a build for remote board
- samples/ipc/ which includes a samples for remote board
- esp32 which includes custom bootloaders

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-08-03 16:05:07 +02:00
Martí Bolívar 65e3593bf9 west_commands: runners: reserve -O for --tool-opt
The --tool-opt runner option is the recommended practice for allowing
runners to take additional arguments that are passed on to the
underlying tool. It exists because we don't want to add one runner
option for every single tool option that users might want to tweak --
that would be a nightmare.

Enough runners are using this option that it's time to promote it to a
common runner capability with consistent behavior, the same way we did
for the --dev-id option in the past. This removes boilerplate from
individual runner files and ensures consistent argument handling for
this option when it is supported.

Since --tool-opt is a bit long to type, and we've had some complaints
about that, take this as an opportunity to standardize on -O as a
short option equivalent for it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-03 05:06:53 +01:00
Anas Nashif 0200da97c2 ci: fix pattern for skipping boards in testplan.py
boards have their own unique pattern, use that for proper skip.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-08-02 13:05:39 -04:00
Reto Schneider 80270727d0 scripts: requirements: extras: Suitable clang-format version
Even when BreakBeforeBraces is set to Linux, implying BraceWrapping for
AfterEnum to be true, clang-format version 12 puts the opening brace of
an enum declaration on a new line. [1]

One possible workaround would be to change AllowShortEnumsOnASingleLine
from false to true, but that would yield a different kind of unwanted
formatting.

clang-format version 13 and newer have this issue fixed, therefore
bumping the minimal version.

[1] https://github.com/llvm/llvm-project/issues/48983

Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
2022-08-02 15:17:18 +02:00
Piotr Golyzniak 4484f68d02 twister: verify platform names from quarantine
Add verification of platform name from quarantine list.

Fixes: #43450

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2022-08-01 09:29:03 -04:00
Jordan Yates 13f2494f4c device: symetric injected dependencies
If a device manually specifies that it depends on a second DT device,
add the first device to the second devices list of supported devices.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates 29942475c5 scripts: gen_handles: output dependency graph
Output the final dependency graph as a `.dot` file, which when rendered
by graphviz can be easier to comprehend than the text descriptions.

This output is optional in that it will not be generated if `graphviz`
is not installed.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates 8d17e857a1 scripts: gen_handles: use elf_parser.py
Use the new elf_parser module to simplify the process of generating the
final device handle arrays.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates 06b0d5a4fb scripts: elf_parser: added
Adds a python module intended to simplify other scripts that need to
work with device information compiled into the first pass `.elf` file.

Scripts utilizing this module can focus on iterating over dependency
graphs to generate the desired output, instead of also needing to
extract and build the graphs in the first place.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates cfb8c5e70f pm: device: rename placeholder variable
Rename the placeholder variable generated for PM slots so that the
prefix doesn't colide with the PM structs declared by devices. This
simplifies the process of searching for symbols in `.elf` files.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Anas Nashif c0317fba1f twister: handler quotes in extra_args
Escape quotes provided in extra arguments on the command line.

example: ... -x "CONFIG_COMPILER_OPT=\"-fanalyzer\""

Fixes #46382

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-29 05:58:15 -04:00
Tristan Honscheid 4a8ffee1d8 twister: Don't run coverage report when --build-only is used
When doing a split build and test with coverage enabled, (i.e.
`twister --coverage --build-only` followed by `twister --coverage
--test-only`), Twister attempts to run coverage reports immediately
after building, which fails because tests haven't actually run yet
and causes several error messages to print in to the log.

This change causes twister to skip the call to `run_coverage()` if
`--build-only` is set and print an info message. This suppresses the
error messages from the coverage report tool complaining about missing
coverage files, but still instruments the built code for coverage data
collection. When twister is invoked again, but with `--test-only`
instead, the coverage files will be emitted and the reports
successfully generated.

 #### Testing

I ran...

```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --build-only
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --test-only
```

... followed by a combined build and test ...

```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix
```

... as a control and diffed the HTML coverage reports, which were
identical other than a timestamp.

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2022-07-28 05:58:52 -04:00
Piotr Golyzniak 7ce914b7f4 twister: verify all platform names
Fix bug in verify_platforms_existence method - make it possible to
verify all platform names from list - not only first.

Fixes: #48321

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2022-07-26 20:42:20 -04:00
Stephanos Ioannidis 0f9588e51f scripts: twisterlib: Fix default value for --ninja argument
This commit makes the `--ninja` argument default to `true` when neither
`-k` nor `--make` is specified (i.e. when Make generator is not
selected).

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Stephanos Ioannidis 1f81d2cd97 scripts: twisterlib: Fix --short-build-path argument handling
When `--short-build-path` argument is specified to the twister, the
following error message is displayed because the Python argparse module
does not allow specifying a mutually exclusive argument as required:

  ValueError: mutually exclusive arguments must be optional

This commit removes the `required` parameter when adding the `--ninja`
argument, which is a mutually exclusive argument, and adds a manual
check to validate this condition instead.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Anas Nashif 5432cac806 ci: testplan: still run default tests with global changes
In cases of global changes where 100s of nodes are launched, i.e. on
samples and tests (more than 20 tests/samples changed), do a full
covrage run.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-22 11:00:23 -04:00
Kumar Gala ccb9b6b5ba scripts: check_compliance: generate Kconfig.dts
Generate a Kconfig.dts to allow check_compliance to run.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-07-19 09:39:49 -07:00
Kumar Gala b109a0a9f7 dts: Add simple script to generate Kconfig.dts based on bindings
Given binding dirs the script will generate a Kconfig.dts of the
form:

    DT_COMPAT_ADI_ADT7420 := adi,adt7420

    config DT_HAS_ADI_ADT7420_ENABLED
            depends on DTS_HAS_ADI_ADT7420
            def_bool $(dt_compat_enabled,$(DT_COMPAT_ADI_ADT7420))

Than a driver Kconfig can use these Kconfig symbols as follows:

    menuconfig ADT7420
            bool "ADT7420 Temperature Sensor"
            default y
            depends on DT_HAS_ADI_ADT7420_ENABLED
            ...

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-07-19 09:39:49 -07:00
Bartosz Bilas 3322aad03b scripts: pylib: twister: twisterlib: fix typo
s/enviornment/environment

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2022-07-19 11:41:54 -04:00
Alex Kolosov f59e9dd857 scripts: runners: improvements for spi_burn west runner
Following changes has been made by this commit:
    - fix issue when a binary is not able to be found by relative path
    - implement "west debug" command
    - implement --dt-flash option

Signed-off-by: Alex Kolosov <rikorsev@gmail.com>
2022-07-19 12:23:47 +00:00
Anas Nashif 6027152408 scripts: set_assignee: handle exceptions
Deal with exceptions that happen when a user does not exist on GH.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-18 21:41:22 -04:00
Martí Bolívar d6f68f0bbb scripts: gen_defines: fix tokenization of array elements
This fixes commit
3d5cc38cf6.

That commit only works if removing the quotes from e.g. a string in an
array actually results in a token. If the string's value is something
like

  "foo,bar"

though, it will result in the "token"

  foo,bar

in the generated output.

This is wrong; fix it using the new edtlib.str_as_token() API
introduced to allow callers to perform the same procedure as that
library for fixing up strings.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar 7b2a728584 scripts: gen_defines: add a missing comment
Adding a comment at this point in the header file splits these macros
into their own section. They are presently grouped together with the
previous section, which is unrelated. This is confusing; fix it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar 4ef8c8e447 edtlib: expose str_as_token() API
Some callers need to be able to convert strings to tokens in the same
way edtlib does. Make this possible by exposing the internal helper
function used to do that under a suitable name.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar 19bb6b330c edtlib: fix error handling in an internal helper
The error message emitted by _interrupt_parent() is wrong; it
mistakenly says:

  node None has an 'interrupts' property, but [...]

This 'None' is appearing because the same routine overwrites the
'node' argument that the caller is asking about with node parents
until it hits the root, at which point root.parent is None.

Fix it by caching the original node and using that in the error
message instead.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Jason Wright c6c7dde969 scripts: better logging for --coverage-formats option
Adds additional log messages for coverage reports
beyond HTML format.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright 60c4d20548 scripts: restrict --coverage-formats option to gcovr
Generates an error if --coverage-formats option is supplied without
coverage tool being set to gcovr.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright 786f28f162 scripts: add --coverage-formats option for use with gcovr
Adds optional twister flag to support gcovr output report formats.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00