Commit graph

4074 commits

Author SHA1 Message Date
Flavio Ceolin 02ac343c18 security: hardening: Update general recommendations
Update security recommendations. That is not a
thorough list though.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-13 02:28:29 +09:00
Flavio Ceolin 8276658ac7 security: hardening: Update debug susbsys options
Update information for debug susbys options.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-13 02:28:29 +09:00
Flavio Ceolin d74e77147b security: hardening: Update experimental features
Update all experimental features on Zephyr. Remove outadated and
added new ones.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-13 02:28:29 +09:00
Flavio Ceolin a06083c3c4 security: hardening: Organize hardened file
Separate debug and experimental options in hardened.csv
file for easier maintenance.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-13 02:28:29 +09:00
Dmitrii Golovanov 7617046c67 twister: Add expected and detected test case names to debug log
Log expected and detected test case names for the test suite name
check debug.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-05-12 10:03:52 +02:00
Dmitrii Golovanov 8ee23dcdb8 twister: Don't override test name check with RunID check
Don't override a test case name check failed result by the following
RunID check failed result to report clearly about test case mismatch
situation as well as more specifically when the same test case appears
with unexpected RunID.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-05-12 10:03:52 +02:00
Dmitrii Golovanov 14ba39c50e twister: Fix suite-name-check bypass on flash error
Fix suite-name-check bypassed at DeviceHandler on flash errors.

If the flash timeout is smaller than needed, but test images are
actually flashed successfully and appropriate tests executed,
then results from the previous test case were taken because
the flash error status makes the suite-name-check bypassed.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-05-12 10:03:52 +02:00
Gerard Marull-Paretas 76b91c700d twister: fix handling of mixed str/list configurations
We can now have situations like this when it comes to lists in Twister
config:

```yaml
common:
  platform_exclude: foo
tests:
  my.test:
    platform_exclude: bar
```

```yaml
common:
  platform_exclude: foo
tests:
  my.test:
    platform_exclude:
      - bar
      - baz
```

```yaml
common:
  platform_exclude:
    - foo
    - bar
tests:
  my.test:
    platform_exclude: baz
```

```yaml
common:
  platform_exclude:
    - foo
    - bar
tests:
  my.test:
    platform_exclude:
      - baz
      - bazz
```

This patch fixes handling of cases (2), (3) and (4).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-11 09:17:44 -04:00
Jamie McCrae 2c2d1c72db requirements: Update pyOCD requirement to 0.35.0
Updates the pyOCD requirement to 0.35.0 to support a new flasher
option of flashing and preventing reset of the device.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-05-10 10:15:52 +02:00
Gerard Marull-Paretas a91620f5bb twister: deprecate space-separated lists
Deprecate in favor of native YAML lists.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Gerard Marull-Paretas 47c2feca20 scripts: utils: add twister_to_list tool
Add a new tool that convers string-based lists on Twister config files
to native YAML lists.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Gerard Marull-Paretas 11218baffc twister: set 'any' type for set/list fields
This allows to write both, lists or strings in the YAML file.
Unfortunately pykwalify schema language does not support multi-type
fields. JSON schema or Marshmallow could be used instead.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Gerard Marull-Paretas 0db74702c3 twister: handle mixed common/tests types
This patch handles mixed types between common/tests sections in the
configuration, e.g. str+list, list+str, str+str.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Gerard Marull-Paretas d3b8b8d9db twister: improve diagnostics for 'any' set/list types
Since pykwalify does not support multi-type fields, we are forced to use
the 'any' type. This is used for fields that allow sets/lists written as
either YAML lists or space-separated strings. This patch checks for
str/list Python type, and raises a configuration error if not true so
that user obtains better error messages in case the configuration file
contains invalid format.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Gerard Marull-Paretas 560a72171e twister: allow list input for set config fields
Twister configuration files have a few fields defined with the "set"
type. The set needs to be specified as a space-separated list of
entries, for example:

```yaml
plaform_allow: boardx boardy
```

This patch allows users to write these fields as YAML lists, for
example:

```yaml
platform_allow:
  - boardx
  - boardy
```

The list gets converted to a Python set, therfore the same exact
behavior is kept in Twister.

In some cases like in the example above, using lists allows to have more
readable configuration files, specially if the list is long. It also
helps on diffs, provided we have one item per line.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-10 09:52:37 +02:00
Maciej Perkowski 3bf7f83f21 twister: Bring back scope selection rule using platform_allow
A change to the scope selection rules was introduced by #52715
but it doesn't comply with the description in the code. What's more,
the change introduces major issues i.e. non deterministic scope
selection, especially in a CI environment. More context at #57595

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-05-08 11:18:44 +02:00
Kumar Gala bae0a5b8b6 logging: Use TYPE_SECTION macros for log const
Clean up log_const to utilize macros for handling sections.

Update database_gen.py to match naming convention change.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-05-03 10:43:31 +02:00
Gerard Marull-Paretas baeee77ba6 twister: automatically add module board roots
Twister does not handle out-of-tree boards automatically. Modules like
e.g. example-application that define their own board root folder, are
required to inform twister via `--board-root`. In contrast, west is able
to to this automatically. This patch makes twister a bit smarter by
automatically filling the board root list with module paths as well.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-02 13:25:12 -04:00
Piotr Kosycarz 7892f40a52 twister: add options to shuffle tests across subsets
Add option to shuffle tests (randomly) before they are split into subsets.
This allow to fairly distribute tests across subsets,
so that long tests are not crowded in a single group.

This also allows to detect unwanted dependencies in test execution order.

Also, added option to provide custom seed to random generator
used to shuffle tests.
This will allow to reproduce certain test order if needed.
Used seed is printed at console.

Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
2023-05-01 06:12:59 -04:00
Rihards Skuja 9a6f7efd00 scripts: fix 'compatible' property of generated thermistor tables
The generated 'compatible' property doesn't match the associated devicetree
binding.

Signed-off-by: Rihards Skuja <rihards.s@origin-robotics.com>
2023-04-27 17:38:31 +09:00
Carles Cufi 0ebc37a972 scripts: runners: nrf: Fix the formatting of the old --nrf-family
The now legacy --nrf-family parameter takes the "raw" family name as
nrfjprog accepts it. But now we use an intermediate representation with
{FAMILY}_FAMILY as the format, so covert to it as required.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-27 09:50:48 +02:00
Aurelien Jarno bdb3fd0ae9 scripts: gen_isr_tables: fix pylint issues
pylint reports the following issues on the
scripts/build/gen_isr_tables.py:

 C0325:Unnecessary parens after '=' keyword (superfluous-parens)
File:scripts/build/gen_isr_tables.py
Line:16
Column:0
 C0325:Unnecessary parens after '=' keyword (superfluous-parens)
File:scripts/build/gen_isr_tables.py
Line:316
Column:0

Fix that so that unrelated PR touching the same file have a chance to
pass the compliance checks.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2023-04-26 12:58:13 +02:00
Gerard Marull-Paretas a01d40f604 scripts: utils: migrate_sys_init: handle empty line after ARG_UNUSED
The script left empty lines below ARG_UNUSED. After this patch,

```c
static int f(const struct device *dev)
{
	ARG_UNUSED(dev);

	/* code */
	...
}
```

will become:

```c
static int f(void)
{
	/* code */
	...
}
```

instead of:

```c
static int f(void)
{

	/* code */
	...
}
```

Fixes #56954

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-04-25 19:57:00 +02:00
Gerard Marull-Paretas 2dbbfaec6b scripts: utils: migrate_sys_init: remove dead ZEPHYR_BASE
ZEPHYR_BASE is not used, so delete it.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-04-25 19:57:00 +02:00
Roman Dobrodii 2e0820f08c scripts: west: add silabs_commander runner
- Runs Silabs Commanders which is useful for SiLabs boards that
can not use J-Link for various reasons.
- Flash support only, no debug.

Signed-off-by: Roman Dobrodii <rdobrodii@antmicro.com>
2023-04-25 10:34:16 -07:00
Stephanos Ioannidis 428f4a6466 scripts: get_maintainer: Fix glob pattern directory match error logic
The `get_maintainer` script currently generates an error when a non-
slash-suffixed glob pattern matches any directories -- this is
incorrect because the glob pattern may be used to match files.

This commit updates the script to only generate an error when a
non-slash suffixed _only_ matches directories.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2023-04-26 00:29:25 +09:00
Marc Herbert 16880cc381 scripts: gen_handles: silence "should it be static?" warnings
Some compilers and static analyzers warn when non-static (globals) are
not declared in some .h file. Tweak the generated code and add an
`extern` declaration before each device handle definition to make it
look it was declared in a (non-existent) .h file.

This makes the code a bit longer but it is already generated and very
repetitive anyway.

In the typical SOF configuration this silences about 150 sparse
warnings; as many as device handles.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-25 12:19:33 +02:00
Georgij Cernysiov d641be7fc8 runners: stm32cubeprogrammer: fix program files path for win x64
The '%ProgramFiles%' is not guaranteed to be 'C:\Program Files' on
Windows x64.

The value will be 'C:\Program Files (x86)' if the west is executed by
32-bit Python.

To correct the issue '%ProgramW6432%' must be used.

Signed-off-by: Georgij Cernysiov <geo.cgv@gmail.com>
2023-04-24 13:35:26 +02:00
Jamie McCrae 96a9b7e1ff scripts: ci: check_compliance: Ignore mcuboot Kconfigs
Adds mcuboot Kconfig options to the ignore list as these are
defined in mcuboot but used in documentation and samples which
utilise sysbuild to build mcuboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-04-24 13:27:53 +02:00
Jamie McCrae 71dfaff583 scripts: utils: migrate_mcumgr_kconfigs: Remove IMG dummy header
Removes converting the dummpy IMG MGMT group Kconfig as this has
been removed from the tree and users would not have used it
anyway.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-04-21 16:33:30 +02:00
Carlo Caione de90dfccbc syscall: Introduce __syscall_always_inline
Sometimes we want to force the inlining of a __syscall. Introduce a new
__syscall_always_inline symbol to do that.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-04-20 14:56:09 -04:00
Kumar Gala 47cf00058e fb: cfb: Use TYPE_SECTION macros for cfb_fonts
Clean up cfb_fonts to utilize TYPE_SECTION macros for handling sections.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-19 17:14:54 +02:00
Kumar Gala 5b5fbc4c11 shell: remove dead linker section for shell
The initshell sections in the linker scripts where associated with theo
old shell code.  The old shell code has been removed for some time so
remove references to initshell in the linker scripts and size_calc.py

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-19 17:14:37 +02:00
Andrei Emeltchenko 29dc4b68ec twister: Cleanup redundant if / else
Convert following construction:

elif self.options.device_serial or self.options.device_serial_pty:
 if self.options.device_serial:
   ...
 else
   ...

to the construction:

elif self.options.device_serial:
  ...
else self.options.device_serial_pty:
  ...

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-04-19 03:27:08 -04:00
Andrei Emeltchenko 186d706c73 twister: Use flash options also for device_serial_pty
Use flash_timeout and flash_with_test options not only for
serial console but also for device_serial_pty.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-04-19 03:27:08 -04:00
Kumar Gala 63ed1d977a twister: size_calc: Fix compliance warning in script
compliance script runs pylink on size_calc.py and reports:

C0325:Unnecessary parens after '=' keyword (superfluous-parens)

Fix warning by remove unnecessary parens.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-19 02:38:46 -04:00
Carles Cufi a7a9cb018a scripts: runners: tests: nrf: Refactor and add nrfutil runner tests
Refactor the existing nrfjprog tests to make them compatible with the
new nrfutil runner implementation, and extend them so that they are
exectued for both tools.

There is missing verification of the JSON generated by the nrfutil
runner, whcih will be added in a follow-up commit.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-17 09:29:04 -07:00
Carles Cufi a5e0cbc4fb scripts: runners: tests: nrf: Minor tweaks
- Rename the test file to test_nrf.py in preparation for future tests
  covering nrfutil as well
- Rename the test folder from nrfjprog/ to nrf/
- Change the id_fn returned string formatting so that it's compatible
  with executing a single test case via -k as a parameter (no spaces or
  commas)

You can now execute:
python -m pytest tests/test_nrf.py -k \
'test_nrfjprog_init[NRF51-pin_reset-override_snr-sector[anduicr]\
_erase-no_recover-no_tool_opt]'

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-17 09:29:04 -07:00
Carles Cufi b022835146 scripts: runners: nrf: Add support for the new nRF Util tool
Add support for Nordic's new nRF Util tool, aka nrfutil. Via its
"device" command, nrfutil now supports most of the functionality offered
by nrfjprog.

The tool itself can be found here:

https://www.nordicsemi.com/Products/Development-tools/nrf-util

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-17 09:29:04 -07:00
Carles Cufi 480b6fb7c0 scripts: runners: nrf: Do not force-flush until the end
In order to better support the ability of the upcoming nrfutil runner to
defer all operations and execute them all at once, defer flushing right
until the end, when everything has been queued up (recover, program and
reset) instead of forcing execution at the program stage.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-17 09:29:04 -07:00
Carles Cufi ff9c38316e scripts: runners: nrf: Use nrfutil error codes
Standardize to nrfutil error codes instead of nrfjprog ones. This is
because nrfutil has a wider range of functionality and will eventually
be the default runner for all nRF devices. It is also consistent with
the internal operation representation, which is now using the nrfutil
format already.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-04-17 09:29:04 -07:00
Martí Bolívar 5847890a18 edtlib: finish adding type annotations
This concludes the type annotations for the public API for the module,
along with the relevant internal state. It's not worth type annotating
the internal backwards compatibility shim for !include.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar 3318380eaf edtlib: type annotate EDT
Incremental progress towards type annotating the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar d89f974760 edtlib: type annotate Node
This requires adding a private constructor so that mypy
can tell what all the final instance state is going to be.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar cae8b6567d edtlib: make PinCtrl a type-annotated dataclass
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar 83b6db2ec1 edtlib: make ControllerAndData a type-annotated dataclass
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar 3a43e1b643 edtlib: make Range a type-annotated dataclass
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar b07f3ddd9f edtlib: clean up Range docstring
Fix grammar issues and typos.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar 49c0d72234 edtlib: make Register a type-annotated dataclass
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar 4415a29af2 edtlib: make Property a type-annotated dataclass
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00