Instead of relying on runtime filter to limit scope to emulation
platforms, use the type attribute for each platform and do the filtering
very early on. This will speed things up for tests where we only run on
emulation platforms.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some complex testsuite use macros and other mean to define testcases
which makes them unparseable. This ends up in a descripency between the
testplan and the test results.
Allow adding the testcases into the yaml and use those instead of
parsing the C files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
TestCase in the old terminology is the test application, but we have
just freed up TestSuite, so use TestSuite here to signify the test
application which consists of many test scenarios and testcases.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The hardware map feature can be used with serial pty mode in
this change. And also add an runner_params option for passing
more parameters from hardware map file to west runner. Note
that you need to create this map file manually because it
cannot be scanned out correctly due to pty is not a physical
HW device existing in system.
And also update doc/develop/test/twister.rst for the usage.
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Twister allows us to control maximum execution time for each
test with timeout value in test .yaml configuration. This
helps us to prevent slow tests from stopping by timeout.
However it's hard to choose test timeout for all platforms
as some platforms are naturally slow. It could be a HW board with
power-efficient but slow CPU or simulation platform which
can perform instruction accurate simulation but does it slowly.
As we don't want to increase test timeout infinitely to meet
the needs of the slowest platform let's introduce
platform-specific test timeout management. It's implemented as
an optional 'timeout_multiplier' field in board .yaml
configuration. Setting it to some value multiplies each test
timeout by this value. By that we can increase timeouts only
for the platforms where it's required.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Support new keyword in the testcase.yaml to allow filtering based on
enabled modules. This is useful when you only whitelist a limited set of
modules in your project and want to only run those samples that have
their dependencies satisfied and those modules available.
The option allows adding multiple modules, all of which need to be
enabled in the manifest.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Each board needs to have a maintainer to allow automatic assignment and
adding reviewers when pull requests concerning the platform are being
submitted.
This is now a placeholder so we can start adding maintainers early, the
data is not being used anywhere yet.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Change the "yes/no" value to "true/false".
Otherwise pykwalify 1.8.0 will report below errors:
- Value: 'no' for required keyword must be a boolean:
Path: '/mapping/common'>
- Value: 'yes' for required keyword must be a boolean:
Path: '/mapping/sample/mapping/name'>
Signed-off-by: Shao Ming <ming.shao@intel.com>
Fixes#38046
Document the changes in twister.rst
Add baud rate to debug message in twister
Add baud parameter to twister's add_device function
Set the twister baud rate from input parameters
Use 115200 as the default baud rate if not specified
Add baud to the hwmap-schema.yaml file
Add --device-serial-baud to twister arguments
Fix compliance issues
Fix mistake in parameter name from device-baud to device-serial-baud
Refactoring of the code in orded to simplify the logic
and clear multiple definitions of the default baud rate.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Signed-off-by: Dennis Ruffer <daruffer@gmail.com>
1. Add a class of Pytest(Harness) which can handle pytest script
in harness.py
2. Use running_dir to store current test directory which be
used in pytest scrpt.
3. Add usage of this feature into zephyr doc.
Signed-off-by: YouhuaX Zhu <youhuax.zhu@intel.com>
Add arm fvp emulator in order to use ninja run or west build -t run.
Add armfvp in order to run twister.
Set env ARMFVP_BIN_PATH before using it,
e.g. export ARMFVP_BIN_PATH=<path/to/fvp/dir>
NOTE: ARMFVP_BIN_PATH is the dir path.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Adds feature allowing to use yaml file with dictionaries defining
tests to be quarantined (extra arg "--quarantine-list FILENAME").
The dictionaries are validated according to the proper schema
and loaded.
A flat list is created containing quarantined configurations
(configuration = platform + scenario). Configurations under quarantine
are skipped and get "Quarantine" as a reason in the results reports.
A "comment" can be added to a quarantine entry in the quarantine yaml
with more details (e.g. issue #) and it will be also added to
the report.
The status of tests under quarantine can be verify if
`--quarantine-verify` is used in addition to
"--quarantine-list FILENAME". Using these args will make twister skip
all tests which are not on the quarantine list.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Move yaml schemas into one single location. We have quite a few schema
used across the tree, one common place will make them easier to find and
reference.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
