Commit graph

15 commits

Author SHA1 Message Date
Patryk Kuniecki d8b368a160 scripts: tests: twister_blackbox: Add test_ignore_platform_key
Add new tests to test_filter.py in blackbox twister testing

Signed-off-by: Patryk Kuniecki <patryk.kuniecki@intel.com>
2024-04-03 10:34:35 +02:00
Lukasz Mrugala 2f97795360 scripts: tests: Blackbox test expansion - platform
Adds tests related to the Twister's platform selection:
* -A, --board-root
* -K, --force-platform
* -p, --platform

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-31 17:37:09 -04:00
Kamil Paszkiet 717fccaa09 scripts: tests: Blackbox test expansion - device
Adds tests related to device flags:
--seed

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-29 15:07:38 +00:00
Artur Wilczak 6d8132c445 scripts: tests: Blackbox test expension - disable
Add test related to disable-suite-name-check flag

This flag disabling extended test suite name verification at
the beginning of Ztest test. This option could be useful for tests
or platforms, which from some reasons cannot print early logs.

Add test related disable-warnings-as-errors

Do not treat warning conditions as errors.

Signed-off-by: Artur Wilczak <arturx.wilczak@intel.com>
2024-03-29 15:07:19 +00:00
Kamil Paszkiet fff833eda4 scripts: tests: Blackbox test expansion - filter
Adds tests related to filter flags:
--arch
--vendor

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-27 14:31:43 +00:00
Lukasz Mrugala 3fb11e260a scripts: tests: Blackbox test expansion - addon
Adds tests related to the addon flags:
*     --enable-ubsan
*     --enable-lsan
*     --enable-asan
*     --enable-valgrind
*     --allow-installed-plugin
*     --pytest-args
* -x, --extra-args
* extra test args for the binary

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-26 18:08:31 -05:00
Kamil Paszkiet 2a4332ba5f scripts: tests: Blackbox test expansion - error
Adds tests related to filter flags:
--overflow-as-errors

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-26 11:10:02 -05:00
Alberto Escolar Piedras a4b0e8a706 twister tests: Let's use native_sim instead of native_posix
native_posix is being replaced with native_sim, let's
use native_sim instead in twister tests.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-03-19 13:15:48 -05:00
Lukasz Mrugala 93a377e674 scripts: tests: Blackbox test expansion - outfile
Adds tests related to the changing of the output files:
* -c, --clobber-output
* -n, --no-clean
* -M, --runtime-artifact-cleanup
*     --short-build-path
*     --prep-artifacts-for-testing

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-05 15:56:04 +01:00
Lukasz Mrugala a815f70177 scripts: tests: Blackbox test expansion - config
Adds tests related to the flags concerned with config files:
*     --alt-config-root

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-05 07:44:51 -05:00
Paszkiet Kamil 1799a2bf0c scripts: tests: twister_blackbox: Add test test_runner.py
- change name test_qemu.py to test_runner.py
- add test to test_runner.py:
  - build_only
  - runtest_only
  - ninja (dummy test)
  - dry_run (dummy test)
  - any_platform (dummy test)
  - cmake_only
  - pre_script (dummy test)
  - exclude_platform
  - device_flash_timeout (dummy test)
  - retry_failed
  - retry_interval
  - timeout_multiplier
  - quarantine_list
  - tag
  - only_failed
  - retry_build_errors

Made required changes to test_testplan.py tests

Signed-off-by: Paszkiet Kamil <kamilx.paszkiet@intel.com>
2024-01-29 17:43:59 -05:00
Lukasz Mrugala c405d8d68f scripts: tests: Blackbox test expansion
Adds five new tests, covering six more flags:
* -s, --test
*     --sub-test
* -G, --integration
*     --test-config
*     --level
*     --filter

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-01-19 09:53:36 +01:00
Alberto Escolar Piedras 188cdd95c5 twister tests: Replace native_posix with native_sim
As native_sim is replacing native_posix overall and
becoming the default test platform.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-11-30 09:42:25 -05:00
Anas Nashif 719fee2bbc doc: ztest: adapt ztest docs
Remove remaining usage and documenation of dropped kconfig that is now
the default.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:04:29 +02:00
Lukasz Mrugala 4d467af7f4 scripts: tests: twister: Black box testing
In addition to the white-box testing and unit tests,
we would like to have some black-box testing for the
console commands of twister.

This serves as a Proof of Concept for future expansion.

Special dummy tests are made for blackbox testing.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-08-28 15:57:06 +02:00