patrick/zephyr
1
0
Fork 0
Code Issues Pull requests Actions 8 Packages Projects Releases Wiki Activity

twister: deal with predicates and skipped tests

Browse source 
Using ztest predicate feature, testcases are skipped, however, the skip
is only reported as part of the summary and not through normal
execution. Until now such tests were reported as blocked or had a null
status in the twister json output.

This changes will look into summary results and use them to confirm
parsed results and add any missing results that either were not reported
or not captured through the serial console.

The issue can be observed with the drivers.can.api test for example on
the frdm_k64f board.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit is contained in:
Anas Nashif 2024-01-30 11:12:31 -05:00 committed by Maureen Helm
parent 2a260a84c8
commit b64af4008f
1 changed files with 20 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
scripts/pylib/twister/twisterlib/harness.py
View file

@ -624,6 +624,12 @@ class Test(Harness):
self._match = True
result_match = result_re.match(line)
# some testcases are skipped based on predicates and do not show up
# during test execution, however they are listed in the summary. Parse
# the summary for status and use that status instead.
summary_re = re.compile(r"- (PASS|FAIL|SKIP) - \[([^\.]*).(test_)?(\S*)\] duration = (\d*[.,]?\d*) seconds")
summary_match = summary_re.match(line)
if result_match:
matched_status = result_match.group(1)
@ -638,6 +644,20 @@ class Test(Harness):
self.testcase_output = ""
self._match = False
self.ztest = True
elif summary_match:
matched_status = summary_match.group(1)
self.detected_suite_names.append(summary_match.group(2))
name = "{}.{}".format(self.id, summary_match.group(4))
tc = self.instance.get_case_or_create(name)
tc.status = self.ztest_to_status[matched_status]
if tc.status == "skipped":
tc.reason = "ztest skip"
tc.duration = float(summary_match.group(5))
if tc.status == "failed":
tc.output = self.testcase_output
self.testcase_output = ""
self._match = False
self.ztest = True
self.process_test(line)