Commit graph

3523 commits

Author SHA1 Message Date
Anas Nashif d5424d5116 twister: split report classes out
Move report classes out into own file and split out of testplan class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif d8d217d2ea twister: move testsuite, testcase classes out
Move testsuite/testcase and supporting classes into own files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif 521ce42161 twister: move cmake cache class
Move cmake cache classes into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif e01cbbe31f twister: add environment class
Add environment class to discover toolchain, zephyr version and other
items.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif acda94be2d twister: optimize file scanning
We have been scanning code for tests for every scenario defined in the
yaml file. This needs to only be parsed once and not more. We are
dealing with the same files for each scenario.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Bartosz Bilas 888ce37799 runners: jlink: allow passing file path with spaces
Put file path within the " " marks to allow
passing file path with spaces.

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2022-07-05 13:20:17 +02:00
Christopher Friedt 4cc443705d libc: minimal: add strerror and strerror_r function
Add simple strerror() and strerror_r() implementations.

Fixes #46099

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-07-04 22:53:36 +02:00
Enjia Mai ce18d3c4ff requirements: add netifaces, needed by adsp runner
Needed for running tests on intel_adsp_cavs boards, the cavstool.py
are using it to get the host ip address automatically.

Signed-off-by: Enjia Mai <enjia.mai@intel.com>
2022-07-04 16:33:03 +02:00
Reto Schneider 35f9a32781 twister: Remove testcase-root reference
--testcase-root got renamed to --testsuite-root

Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
2022-07-02 19:24:58 -05:00
Sylvio Alves 60417bad79 scripts: runners: esp32: add erase option
Adds option for erasing esp32 SoC.

Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2022-07-01 16:25:34 +02:00
Henrik Brix Andersen e12d06690e scripts: west: openocd: respect pre-load and post-verify cmds for elf
Respect --cmd-pre-load and --cmd-post-verify when flashing ELF files
(--use-elf).

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2022-07-01 16:12:52 +02:00
Anas Nashif 33dd2146bb twister: use original reason with qemu failures
If we already have recorded the reason during the qemu run, do not
override it at the final stage of status processing.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-01 07:31:08 -04:00
Anas Nashif 1cdada84ff twister: also paese ZTEST_USER and ZTEST_USER_F cases
Parse additional ztest types with new API that were missed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-01 06:28:35 -04:00
Hake Huang 5ae7d41111 twister: test: add default harness check
below command line need has fixture configured in map.yml,
if not, it shall be skipped
twister -p frdm_k64f --device-testing \
--hardware-map map.yml -T tests/subsys/modbus/ -vv

add harness check with testcase harness by default
will fix this issue

Fixing: #46635

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2022-07-01 11:36:10 +02:00
Abramo Bagnara 795500bbe3 coding guidelines: comply with MISRA C:2012 Rule 9.3
MISRA C:2012 Rule 9.3 (Arrays shall not be partially initialized.)

Systematically use `{0}' to specify full 0 initialization
(not `{}', not `{0U}').

Signed-off-by: Abramo Bagnara <abramo.bagnara@bugseng.com>
Signed-off-by: Simon Hein <SHein@baumer.com>
2022-06-30 17:34:05 -04:00
Carles Cufi f9739ab41d python: Fix the requirement for libmagic to be installed
The Python python-magic module that we install for compliance checks
requires libmagic to be installed on the system to work. This requires
extra OS packages on Ubuntu and macOS, and an extra wheel on Windows.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-06-30 13:34:26 +02:00
Marc Herbert 9782b2cc05 scripts: intel_adsp: always invoke west sign
west flash invokes west build and we have no way to tell whether there
was any code change.

Also remove the rename() trick that achieved the same effect in a
a confusing way; just use copy() instead.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-06-30 10:34:30 +02:00
Marc Herbert 739aa951cd scripts: intel_adsp: don't invoke west sign with --tool-path None
When there is no rimage found in the path, don't invoke west sign with
"-tool-path None". This enhances the error message from the somewhat
cryptic:

  ERROR: --tool-path None: not an executable

... to the user friendlier:

  FATAL ERROR: rimage not found; either install it or provide --tool-path

Also log the complete west sign command at the "info" level before
running it to show that no signed firmware was found, to show which
rimage is used and in case other rimage problems arise. Invoking an
external command is an important "checkpoint".

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-06-30 10:33:53 +02:00
Henrik Brix Andersen 3d5cc38cf6 scripts: dts: gen_defines: define _TOKEN + _UPPER_TOKEN for string arrays
Generate definitions for *_IDX_n_TOKEN and *_IDX_n_UPPER_TOKEN for all
string-array type properties.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2022-06-28 20:46:11 +02:00
Henrik Brix Andersen 27820e9dcb scripts: requirements: extras: add clang-format
Add clang-format to the requirements-extras.txt and require a version
compatible with the configuration options present in the .clang-format
file.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2022-06-28 11:09:14 +02:00
Andrzej Głąbek fb3e8138a2 west_commands: nrfjprog: Add handling of data targeting the XIP region
When a hex file contains some data that needs to be programmed using
the QSPI peripheral (because it targets the XIP region), nrfjprog
needs to be provided with an option that specifies the QSPI erase
method to be used (otherwise, reprogramming will fail).
This commit adds the "--qspisectorerase" option when needed and it
also fixes the programming routine for nRF53, so that data for the
XIP region is not incorrectly interpreted as intended for the network
core.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2022-06-24 20:29:00 +02:00
Lauren Murphy 318e6db239 debug: coredump: add xtensa intel adsp, support toolchains
Adds compatibility with Intel ADSP GDB from Zephyr SDK and
from Cadence toolchain to coredump_gdbserver.py.

Adds CAVS 15-25 (APL) register definitions. Implements
handle_register_single_read_packet to serve ADSP GDB
p packets.

Prevents BSA from changing between stack dump printout
and coredump by taking lock. Observed to be necessary for
accurate results on slower simulated platforms.

Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
2022-06-23 15:44:45 -04:00
Maciej Perkowski dfffc27039 twister: Delete less files during artifact cleanup
Option --runtime-artifact-cleanup/-M allows to clean build artifacts
during runtime to save space usage. However, if one wanted to combine
it with --build-only and then --test-only, they would get an error.
This commit expands the list of files stored with -M with files
required to made --test-only also to work afterwards. Such change can
be useful for setups where building is done on one node and then
minimal amount of artifacts are transfered to another one.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2022-06-23 09:11:40 +02:00
Jeremy Bettis 3b2d3129a5 west: Fix --openocd-search argument
Running west flash with the --openocd-search arg will result in the
openocd command being run with a -s param added for every character in
the passed DIR.

Fix the argparser to append all openocd-search DIRs into a list, instead
of just passing a string to the subcommand.

Signed-off-by: Jeremy Bettis <jbettis@google.com>
2022-06-20 10:26:23 +02:00
Anas Nashif d63c2c44b5 ci: do not add assignees if already set
If PR already has an assignee, do not add additional assignees to the
PR. Each PR should only have one assignee (in most cases)

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-17 11:56:07 -04:00
Gerard Marull-Paretas abaa5b8dfa scripts: checkpatch: allow leading spaces in multi-lines macros
With the current clang-format rules, we have that multi-line macros that
contain empty lines (usual in driver definition macros) are formatted
with spaces only, no leading tabs. For example:

```c
 #define MYDRIVER_DEFINE(i)                 \
 	 struct mydriver_data data##i;      \
                                            \ /* starts with spaces */
	 struct mydriver_config config##i;  \
	 ...
```

This patch makes checkpatch ignore such cases, so that clang-format can
be used in tree while keeping checkpatch happy.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-06-17 10:18:37 +02:00
Gerard Marull-Paretas d89dc22665 uncrustify: remove configuration file
Remove uncrustify configuration file in favor of using clang-format, a
tool that is nowadays more popular/powerful for code formatting.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-06-15 09:37:30 -04:00
Benjamin Gwin 3f8d5c49b3 twister: Exit with a bad status if there were build errors
Previously, twister would exit with a 0 status code if there were build
failures but no execution failures. This makes it easier to catch
build breakages when using twister in CI workflows.

Signed-off-by: Benjamin Gwin <bgwin@google.com>
2022-06-14 21:56:51 -04:00
Anas Nashif 071f62cb75 west: remove intel_s1000_crb runners
Remove runners specific to the intel_s1000_crb board. it is no longer
available or supported in the zephyr tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-13 16:19:51 -04:00
Adrian Bonislawski def0127f34 requirements: bump pyelftools to >=0.27
The minimum version of pyelftools is 0.27 to make it working
with databse_gen.py

Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
2022-06-10 09:46:31 +02:00
Anas Nashif 2e1814263e twister: rename add_missing_testscases
add_missing_case_status is more appropriate for this function.
We are adding missing status, not cases.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-09 12:15:28 -04:00
Anas Nashif 87be2f591f twister: rework how we discard tests
Add a filter type to distinguish between testsuite filters that are
maintained in the testcase.yaml file and those provided on the command
line.

The issue was that when something is excluded deliberately on the command
line twister would report an error if that filter would filter out an
integration platform. We do not want that, because the filtering is
being done by the caller knowing that some tests would be not run, even
if those are integration platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-09 12:15:28 -04:00
Ming Shao 0e79dc393c twisterlib: drain the serial leftover before new test
Some tests can cause serial leftover logs buffered on board.
A simple ser.flush() is not enough. So add explicit readline()
to drain such logs which prepares a clean serial context for
the case that follows.

Signed-off-by: Ming Shao <ming.shao@intel.com>
2022-06-09 06:35:48 -04:00
Anas Nashif 20483161b2 scripts: introduce script to set labels, assignees and reviewers
Use MAINTAINERS.yml file to set lables, assignees and reviewers for
specific PRs or all unassigned PRs since a given date.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-06-08 22:49:13 +09:00
Anas Nashif c62ee3be4e twister: remove fallback testcase if cases added at runtime
The fallback testcase, which is the scenario name from the yaml file is
added if we did not discover any testcases via parsing. It can be
removed if we discover testcaes at runtime or otherwise it is added as
skipped...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-07 06:02:00 -04:00
Anas Nashif bcf4e23562 twister: do not set case status on flash timeout
Status was set for testcases after a timeout on flash, which does not
abort the process and continues with test execution and eventually
passes the test.

Setting status on the testcases was done too early and before we have
executed the tests.

Also increase the flash timeout to 60s.

Fixes #45845

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-07 06:02:00 -04:00
Anas Nashif eeffdc373e twister: do not mark tests as blocked if a testsuite has passed
Do not set status as blocked on testsuites which passed. The sub cases
with no status will get a processed later.

Fixes #45845

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-06-07 06:02:00 -04:00
Martí Bolívar f6a6843471 python-devicetree: CI hotfix
Pin the types-PyYAML version to 6.0.7. Version 6.0.8 is causing CI
errors for other pull requests, so we need this in to get other PRs
moving.

Fixes: #46286

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-06-06 09:09:27 -07:00
Jordan Yates 57fa9121a1 scripts: gen_handles: injected device handles
Update the device handle array for injected devices to output device
handles, not device ordinals. This requires that the extra ordinals
provided to `DEVICE_DT_INST` map to an existing device in order to
appear in the final handle array.

This is required for the injected handles to be useful, no meaningful
operations can be done on device ordinals.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-06-05 14:26:39 +02:00
Jordan Yates 2e014ec1cb scripts: gen_handles: fix extra DEVICE_HANDLE_SEP
Remove an extra `DEVICE_HANDLE_SEP` that was being inserted into the
handle array when injected dependencies were present.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-06-05 14:26:39 +02:00
Enjia Mai 2674c44a59 twister: passing a pty pararmeter to intel_adsp runner
Add a pty parameter from twister to intel_adsp when west-flash,
to tell the intel_adsp runner to change the way it outputs the
log.

Signed-off-by: Enjia Mai <enjia.mai@intel.com>
2022-06-05 14:13:57 +02:00
Enjia Mai 12c44aaec7 scripts: runner: add a intel_adsp west runner
Add a intel_adsp west runner instead of using misc-flasher for
intel_adsp boards. Now running the test on intel_adsp_cavs by:

  west build -b intel_adsp cavs25 ...
  west flash --remote-host [remote hostname or ip addr] \
             --rimage-tool [path to rimage tool] \
             --config-dir [path to dir of .toml config file] \
             --key [path to signing key]

The intel_adsp west depends on the cavstool_server.py running
on the remote host machine.

Signed-off-by: Enjia Mai <enjia.mai@intel.com>
2022-06-05 14:13:57 +02:00
Martí Bolívar b964f0b156 scripts: add dump_bugs_pickle.py
This prints the contents of a zephyr-bugs-$DATE.pickle file in a
format similar to the output of list_issues.py.

It will be useful to have this in tree so that we can better record
the known issues at the time of a particular zephyr release.

The pickle file itself should be created using the bug snapshot
workflow defined in .github/workflows/bug_snapshot.yaml, which can be
triggered manually from this URL:

https://github.com/zephyrproject-rtos/zephyr/actions/workflows/bug_snapshot.yaml

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-06-03 19:19:25 -05:00
Evgeniy Paltsev 554aa9cde3 ARC: west: mdb runner: change folder to run MDB
Make the build folder the place where MDB is run. Otherwise mdb
will store the .sc.project folder in the place where twister is
launched (so .sc.project folder will be shared across the runs)

For that we add **kwargs argument to popen_ignore_int method
so we can pass a cwd argument to the Popen constructor.

Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2022-06-02 13:09:51 +02:00
Ming Shao 33795e3776 twisterlib: fix the execution time for device testing
At the end of DeviceHandler.handle(), the execution time
should be recorded no matter what the harness.state is.
This reflects how much time has been spent on the device.
And it is also used later to determine if a test has been
attempted on a real device or just build-only.

Signed-off-by: Ming Shao <ming.shao@intel.com>
2022-05-30 10:30:00 +02:00
Fabio Baltieri 93f20d7a7a include: add zephyr/ on script generated #include
Fix few script generated #include that needed the zephyr/ prefix.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-05-27 15:20:27 -07:00
Hake Huang 75845f5f4e test: twister: fix false negative in device test
in device test when harness is Test, if there is no console output,
the self.state is None, as at this time self.instance.state
is "pass", as the last step cmake is "pass". So it will report "pass",
but actually there is no console output issue

you can easily reproduce this issue by set
line='' at the begining of the Test.handle function

fixing: #45942

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2022-05-26 11:39:14 +02:00
Stephanos Ioannidis 39956ec835 twister: Add option to always force color output
Colorama, by default, strips out the color escape characters when the
output is redirected, and this may have an unintended consequence of
disabling color outputs when using a utility such as `tee` and in the
CI runners that redirect the stdout and stderr console outputs.

This commit adds a new command line option called `--force-color` to
always force the ANSI color escape sequence output even when the output
is redirected.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-05-25 12:48:31 +09:00
Yuval Peress 86cadf9283 ztest: Fix userspace ztests in new API
Update the new API to use K_USER as the flags for both
CONFIG_USERSPACE and CONFIG_TEST_USERSPACE. Also, fix the linker
script to properly include the suites, tests, and rules.

Fixes #44108

Signed-off-by: Yuval Peress <peress@google.com>
2022-05-25 11:20:13 +09:00
Martí Bolívar 78edc12c1b scripts: add make_bugs_pickle.py
This is a simple script which snapshots open bugs in the
zephyrproject-rtos/zephyr repository using the GitHub REST API.

It relies on the 'github' module, which is a de-facto standard for
accessing this API in Python:

  https://pygithub.readthedocs.io

We are already using this package in scripts/release/bug_bash.py, so
this is not a new dependency, even though it's a third party package
not available in the standard library.

The resulting bugs are stored in the standard library's 'pickle'
format, as a list of github.Issue.Issue objects. For more on pickle,
see:

  https://docs.python.org/3/library/pickle.html

I am choosing pickle because it is standard, easy to use, and we are
already using it in the build system to store the edtlib.EDT object
created by gen_defines.py. This is also therefore not a new
dependency (and even if it were, it's in the standard library).

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-05-24 18:48:03 +02:00