Commit graph

4369 commits

Author SHA1 Message Date
Flavio Ceolin 2e5c37142c scripts: kconfig: Always generate ZEPHYR_*_MODULE
Define the ZEPHYR_{MODULE_NAM}_MODULE option for all available modules.
Right now only modules defined as 'kconfig-ext' has this symbol defined
which may cause problem if the build wants to check if a module not
using this options is available.

This allow us to do something like:

comment "mbedTLS module not available."
	depends on !ZEPHYR_MBEDTLS_MODULE

or make a feature depends on the module be present.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-10-24 10:53:24 +02:00
Maciej Perkowski 0752d60443 scripts: Add arg to test_plan.py for alternative test locations
The arg --testsuite-root was copied from twister. When it is used
for test_plan.py it will be propagated to twister calls. This allows
to make alternative test locations (e.g. from another repo) to work
with test_plan.py

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski 7692610004 scripts: Add arg to set_plan.py for alternative tag relation list
Add --alt-tag arg for test_plan.py script. User can use it and point
to an alternative file with tag-directories relations. If so, such
file will be used instead of the default one.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski 47405a114a scripts: Allow using alternative ignore-patters in test_plan.py
Expand test_plan.py args with --ignore-path. This allows to provide an
alternative lists of patterns for the script.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski f7c4d5f9d4 scripts: Allow test_plan.py to work with other than zephyr repos
The test_plan.py script has a path to repository to be scanned for
changes hard coded to zephyr. This patch separates zephyr path from
such repository's path and adds an arg to pass repo to scan

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski f9178926a9 scripts: Add --no-detailed-test-id arg to test_plan.py script
An option --no-detailed-test-id was added to twister to help align
names for test outside of zephyr tree. This commit add this arg
to test_plan.py script which is then propagated to twister.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Anas Nashif b643692fa5 ci: test_plan: use integration mode for architectures
Reduce the scope to integration only.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-23 11:20:57 -04:00
Anas Nashif 99bb4b9581 ci: testplan: fix include path
Add zephyr to the include path detection.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-23 11:20:57 -04:00
Alberto Escolar Piedras bc5da1a158 native simulator: Get latest from upstream
Align with native_simulator's upstream main
d32b2504ad2b6d6d541bc71a0f9b16ab7b6a3831

Which includes:
* d32b250 Minor format fix
* 7b4f35b native HW counter: Provide new API to reset and control
* 4f815cb INT CNTLR: Bugfix for more than 32 interrupts
* 1d36254 Provide new 64 version of nsi_find_lsb_set

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-10-20 15:20:43 +02:00
Anas Nashif 2ab8c1b779 ci: do not run twister on set_assignees.py changes
No need to run twister when changing this script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:10:11 +02:00
Anas Nashif d9a300e4ae ci: various set_assignee fixes
Cleanup of script and change how we assign PR in general:

Make sure that when multiple areas are changed, including platform or
driver changes, we set assignee of the general areas and not the
platform maintainers. For example, if I change 100 SoC files and touch
the kernel file, the assignee is set to the kernel maintainer.

First, we should as much possible try and split such changes and not
have common area changes with platform changes at the same time. Second,
it is important that such changes which affect everyone are reviewed and
approved by the maintainer of the general areas rather than leave them
to the platform maintainer.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:10:11 +02:00
Anas Nashif 719fee2bbc doc: ztest: adapt ztest docs
Remove remaining usage and documenation of dropped kconfig that is now
the default.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:04:29 +02:00
Anas Nashif 996c8457d7 ztest: remove old ztest api
Remove old Ztest API entirely. New API is the now the default.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:04:29 +02:00
Radoslaw Koppel 48b93ee0aa scripts: build: gen_isr_tables: Cleanup access to ".intList"
This commit removes the needs or generating isrList.bin temporary file.
Now gen_isr_tables.py script access the required section directly in
elf file, that was accessed by the script anyway.
It simplifies the building removing one step.

Signed-off-by: Radosław Koppel <radoslaw.koppel@nordicsemi.no>
2023-10-20 14:58:55 +02:00
Øyvind Rønningstad bb1cfbad19 twister: Add option for generating rom.json and ram.json
Generate the files via the 'footprint' target for each project.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2023-10-20 14:54:54 +02:00
Alberto Escolar Piedras ec273d1357 twister: Fix description of relationship between filter and allow/exclude
The current description of the relationship between the tests filter
and the arch/platform allow/exclude was actually incorrect.

Let's fix it.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-10-17 18:55:35 +03:00
Anas Nashif 38e86c8e86 twister: runner: change log message from info to debug
Very verbose log message without too much context that should be a debug
message rather than a info().

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-12 20:40:56 +03:00
Maciej Perkowski 4cd7df2eec twister: Fix error for --device-testing with not runnable integration
An urforseen error was introduce with #62713. When "--device-testing"
or "--filter runnable" is used in CLI, twister will skip tests which
cannot be executed (not just built) on a given platform. If a given
platform is among integration platforms it will cause an error.
However, it shouldn't be the case. Such skip is intentional and
shouldn't be an error. Fixed by assigning an existing Filter.CMD_LINE,
which is exempted from being turned to error.

fixes: #63844

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-12 18:22:23 +03:00
Dmitrii Golovanov 496509cd65 twister: Reset TestInstance status before run
Reset TestInstance status when ProjectBuilder.run() is called
to run the test after its successful build, so the build status
will not propagate as the run status if the run fails.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-10-12 14:43:38 +03:00
Dmitrii Golovanov 9cdb0aaa14 twister: harness: Fix Console pattern matching for ztest
Fix the Twister Console harness ordered 'multi_line' and 'one_line'
pattern matching to treat the ztest as failed when not all of the
expected patterns were found in the console output, but the ztest
application itself reports 'PROJECT EXECUTION SUCCESSFUL'.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-10-12 14:43:38 +03:00
Gerard Marull-Paretas 7772cec6bd edtlib: always insert root node to the graph
When we have an empty Devicetree, ie,

```
/dts-v1/;

/ {

};
```

The node's dep_ordinal is never initialized because the node graph is
empty. This ends up with invalid ordinal tokens (-1) in
devicetree_generated.h which in turn produce some cryptic compiler
errors, see e.g.

```
error: pasting "dts_ord_" and "-" does not give a valid preprocessing
token
   95 | #define Z_DEVICE_DT_DEV_ID(node_id) _CONCAT(dts_ord_,
      DT_DEP_ORD(node_id))

...

include/zephyr/devicetree.h:2498:41:
note: in expansion of macro 'DT_FOREACH_OKAY_HELPER'
 2498 | #define DT_FOREACH_STATUS_OKAY_NODE(fn)
      DT_FOREACH_OKAY_HELPER(fn)
            |
	    ^~~~~~~~~~~~~~~~~~~~~~
include/zephyr/device.h:1022:1:
note: in expansion of macro 'DT_FOREACH_STATUS_OKAY_NODE'
     1022 |
	  DT_FOREACH_STATUS_OKAY_NODE(Z_MAYBE_DEVICE_DECLARE_INTERNAL)

```

(devicetree_generated.h)

```
...
 #define DT_N_ORD -1
 #define DT_N_ORD_STR_SORTABLE 000-1
...
```

This patch makes sure root node is always inserted (without any target)
so that it gets initialized later.

Discovered as part of
https://github.com/zephyrproject-rtos/zephyr/pull/63696

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2023-10-11 18:28:01 +03:00
Grzegorz Swiderski ce1b8d6d16 scripts: kconfig: Make DEPRECATED/EXPERIMENTAL symbols optional
Make `kconfig.py` not crash when any one of these settings is undefined:

  * DEPRECATED
  * EXPERIMENTAL
  * WARN_DEPRECATED
  * WARN_EXPERIMENTAL

While these will continue to exist in Zephyr, they shouldn't be strictly
required for the script to work. One situation in which they could go
missing is when a user creates an application-specific Kconfig root, but
forgets to source "Kconfig.zephyr" inside it - not that this is expected
to work anyway, but it could at least raise a more intelligible error.

Note that when WARN_DEPRECATED is undefined, selecting DEPRECATED will
always produce warnings. Same with WARN_EXPERIMENTAL and EXPERIMENTAL.
This matches the fact that the WARN_* symbols normally have `default y`.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2023-10-11 18:25:57 +03:00
Benjamin Cabé 4773f42c27 west: bossac: handle stty from coreutils on macOS
There might be situations where people are running the coreutils version
of stty on macOS, hence the need for being smarter at detecting when
that might be the case

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2023-10-11 11:05:48 +01:00
Alberto Escolar Piedras 07e88a9fba west native_gdb runner: Fix to use exe instead of elf
The runnable output from the build system is the
exe file. In native_posix the elf happend to also be
runnable, but this is not the case in general,
and not for native_sim or the nrf5*bsim boards.

So lets use the exe instead.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-10-10 15:26:09 +02:00
Alberto Escolar Piedras f8455b410e west runner: Add exe file to configuration
Add the exe fle to the runnerconfiguration class,
so we can use it from runners which will need it.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-10-10 15:26:09 +02:00
Anas Nashif 22042e7b4e ci: tags: include wifi/net drivers in the net group
Changes to networking and wifi drivers should trigger networking tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-10 16:22:54 +03:00
Lukasz Mrugala eeb142d409 .github: workflows: Workflow for build_helpers
PR #63195 has managed to change code under test without
triggering those tests.

This change should remedy that by creating a new workflow,
based on twister_tests.yaml, that runs tests in
scripts/tests/build_helpers if build_helpers are modified.

Such a workflow should be easily extendable if we ever have more
non-Twister things to test in scripts/pylib.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-10-10 15:55:58 +03:00
Lukasz Mrugala 76f9a024c3 scripts: tests: twister: Domains tests fix
PR #63195 has changed how Domains work without
triggering necessary tests, thus breaking them.
This commit fixes the domains tests.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-10-10 15:55:58 +03:00
Lukasz Mrugala 2fcf4e3499 scripts: pylib: build_helpers: flash_order fix
If flash_order were to be missing from the YAML, the Domains init
would crash instead of substituting an empty list.
This commit fixes that.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-10-10 15:55:58 +03:00
Seppo Takalo 1ecd4fda1a net: script: Fix the net-tools path
Fix the correct path to net-tools directory.
Also default to Bash instead of /bin/sh as we use
Bash syntax.

Signed-off-by: Seppo Takalo <seppo.takalo@nordicsemi.no>
2023-10-06 16:14:39 +03:00
Yong Cong Sin 0722c621c8 util: increase several macros limit from 255 to 4095
Currently, the following macros will only work for 0 - 255:
- `IS_EQ`
- `UTIL_X2`
- `UTIL_INC`
- `UTIL_DEC`
- `LISTIFY`

This patch increases their limit to 4095.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2023-10-05 06:10:06 -04:00
Grzegorz Swiderski 9bbf7e5a48 scripts: domains: Convert Domain to a dataclass
For conciseness.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2023-10-04 18:04:18 +02:00
Grzegorz Swiderski 652b0a8a98 scripts: domains: Support initialization only from YAML
By requiring a full-on YAML fragment, the Domains class can rely on
pykwalify to do the heavy lifting with validating types and structure.
Remove support for loading domains from a dictionary, so that we don't
ever have to replicate the same checks on the input, and we can instead
have a single entry point.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2023-10-04 18:04:18 +02:00
Grzegorz Swiderski f82e2ef043 scripts: domains: Tighten up initialization
Do extra checks to reject inconsistent `domains.yaml` contents early:

   default: app
   build_dir: /path/to/build
   domains:
     - name: image_1
       build_dir: /path/to/build/image_1
     - name: image_2
       build_dir: /path/to/build/image_2
   flash_order:
     - image_1
     - image_2
     - unknown

This example lists "image_1" and "image_2" as known domains, but an
entry for the default "app" is missing, which should never be allowed.
A valid `domains.yaml` file shall also not contain extra domains under
`flash_order`, such as "unknown", which doesn't appear under `domains`.
Either of these cases can now be caught while initializing an instance
of the Domains class.

It follows that a valid `domains.yaml` file must always contain a list
of domains with at least one entry - the default image. Since `default`
is a required key in the YAML schema, `domains` should be required too.
Thus, empty lists will be rejected by pykwalify, so the questionable
warning for "no domains defined; this probably won't work" can be axed.

Fixes #63166

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2023-10-04 18:04:18 +02:00
Grzegorz Chwierut fd6f3c5a2a twister: Fix quarantine performance issue
When using a quarantine file with more than 512 unique
entries, than time of matching quarantine increases
significantly. This is because regexp cache size is
512. Add precompiled regexp entries to the quarantine
as a fix.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-10-04 14:49:38 +01:00
Øyvind Rønningstad fea8caccea twister: Make string stripping consistent across handlers
All handlers except QEMU were right-stripping console lines before
checking them. In certain cases where the regex is expecting the
whitespace, this could make the test fail on qemu when it is passing
on something else. Change QEMUHandler to us .rstrip instead of .strip()

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2023-10-03 16:38:02 +01:00
Lukasz Mrugala b892a4bc49 scripts: tests: twister: Add log_helper tests
Log helper is a small module that this commit
covers with an explicit test.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-10-03 15:22:08 +01:00
Carles Cufi 2332a1db39 west: runners: nrf: Document why the network core is recovered first
The network core needs to be recovered first due to the fact that
recovering it erases both the network and application cores' flash
memory. This means that the little flash image that is programmed during
the recover operation would be lost for the app core if we inverted the
order.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-10-03 15:11:55 +01:00
Grzegorz Chwierut 0badef8091 twister: Fix missing timeout in handler class
In PR #62947 a timeout variable was replaced with a method,
but this timeout was used in another place to start
pytest subprogram. This commit provides a fix.

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-10-02 16:40:11 +01:00
Anas Nashif 9d9f3b5c67 twister: do not error on module filters
If a module is not available, then it is optional, so do not error in
--integration mode.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-01 09:21:50 +03:00
Martí Bolívar 23ed21c0a0 scripts: runners: rework RunnerCaps implementation
This provides miscellaneous quality of life improvements:

- We couldn't use dataclasses when this class was originally written.
  We can now, so move to dataclass to avoid having to write
  __repr__().

- Add missing validation for the advertised commands

- Add missing documentation for the 'file' capability

Signed-off-by: Martí Bolívar <mbolivar@amperecomputing.com>
2023-09-29 16:30:07 +02:00
Maciej Perkowski 066cc2c9d2 twister: Add options deciding if paths be included in tests' names
Test suites names are not being handled uniformly for tests not
in zephyr tree. Their names depend on -T arg used in twister's CLI.
The newly added options allow to select if twister should add paths
to suite names. This is needed if test plans are to be used for tests
outside of zephyr tree.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-09-29 16:29:16 +02:00
Maciej Perkowski 4e4b743fe1 twister: Add "path" entry to json test report
"Path" is a path (relative to zephyr) to a directory with test suite's
definition. Such entry helps to locate a given test.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-09-29 16:29:16 +02:00
Lukasz Mrugala 66730c961e scripts: tests: twister: Domain module testing
Domain module was not yet covered by explicit unit tests.
This commit adds a new test file to cover that gap.
It achieves 100% coverage at the time of creation.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-09-29 16:26:35 +02:00
Manoel Brunnen ae924b2afc twister: Fix test-only tests with hardware map fixtures
In test-only mode with a hardware map, all tests with fixtures were
skipped, even when the hardware map has appropriate fixtures defined.

TestPlan.load_from_file had the check for fixtures missing, but actually
best is that the TestInstance checks the fixtures in the hardware map.

Signed-off-by: Manoel Brunnen <manoel.brunnen@ithinx.io>
2023-09-29 16:25:47 +02:00
Evgeniy Paltsev 80d2872a41 twister: introduce twister-level timeout multiplier
Twister allows us to control maximum execution time for each test
with timeout value in test's .yaml configuration and
platform level timeout multiplier which allows us to tweak
timeout value for specific platform.

However, sometimes we want to additionally adjust tests timeouts
when running twister. This is especially useful in case of
simulation platform as simulation time may depend on the host
speed & load, we may select different (i.e. cycle accurate but
slower one) simulation method, etc...

Let's introduce global (twister-level) timeout multiplier option.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2023-09-29 12:59:31 +02:00
Evgeniy Paltsev c2de739f1f twister: tests: handler: fix incorrect timeout mock
There is clearly an issue with current timeout value
timeout into mocked_instance as this value is used only
once before this mock happen, so this mock is actually useless
and we always use default value. Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2023-09-29 12:59:31 +02:00
Jukka Rissanen 6661f05daf net: ppp: Mark the PPP L2 as non-experimental
The experimental status of the PPP L2 is long overdue so
it can be removed as the component is working fine.

Signed-off-by: Jukka Rissanen <jukka.rissanen@nordicsemi.no>
2023-09-29 10:33:50 +02:00
Yonatan Schachter 66c4cef240 west_commands: bindesc: Fixed crash when no sub-command is given
The west bindesc command currently crashes when no subcommand is
given. This is because the subcommand was not marked as required.
This commit fixes the issue.

Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
2023-09-28 23:04:47 +03:00
Carles Cufi 2d38c095a6 west: runners: Add support for a common --reset argument
Some of the runners in the tree have been adding their own,
class-specific versions of a switch to instruct the runner to reset or
not the device after flashing.

In order to better support multi-image builds that require more than one
flash operation, introduce a new --reset,--no-reset command-line
parameter that is part of the RunnerCaps so taht this functionality can
be accessed in a standardized manner.

Implementations for the new parameter are provided for the runner
classes that were already configurable in this regard.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-09-28 11:08:25 -07:00