Commit graph

2282 commits

Author SHA1 Message Date
Anas Nashif e7656ea831 sanitycheck: do not log stderr from BinaryHandler
stderr from the binary handler (native_posix for example) was redirected
to the logger as errors, this is confusing the console and users, so
remove this.

Fixes #21784

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-09 14:28:32 -05:00
Andrew Boie 9b34ecd0c8 userspace: fix incorrect linker routing
The _thread_idx_map bitfield which has '1' set for free
thread indexes really needs to live in the
data section reserved for kernel object metadata, as this
is a part of memory that is allowed to shift addresses
between zephyr_prebuilt.elf and zephyr.elf.

However, if an application defines enough static threads
that there are no free indexes, the entire bitfield will
be zeroed and the bitfield will end up in the main BSS
section.

Force this data to always be in the .kobject_data.data
section regardless of its contents.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-09 09:13:23 -08:00
Ulf Magnusson 5372dedf2f scripts: kconfig: lint.py: Improve error reporting
If the working directory for a command was missing (usually due to
forgetting to run 'west update'), you'd get a FileNotFoundError
exception along with a cryptic error like

    'git' not found

Only catch OSError instead (which is a base class of FileNotFoundError),
and always show the exception message. It makes it clear that it's the
working directory that's missing.

Add some other misc. improvements too:

 - Turn stderr output from external commands into a warning instead of
   an error

 - Add err() and warn() helpers

 - Include the command name in messages

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-09 11:19:57 -05:00
Ulf Magnusson 23a5b4963b dts: edtlib: Add 'type: path' for path references
Add binding support for a 'path' property type, for properties that are
assigned node paths. Usually, paths are assigned with path references
like 'foo = &label' (common in /chosen), but plain strings are accepted
as well as long as they're valid paths.

The 'path' type is mostly for completeness at this point, but might be
useful for https://github.com/zephyrproject-rtos/zephyr/issues/21623.
The support is there already in dtlib.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-08 08:02:00 -06:00
Olof Johansson 8067aca7e9 gen_defines: close files before exit
In some cases, we've seen the output files be truncated when the
python script has been rebuilt into a .pex before running it --
likely due to buffering.

Closing files explicitly is the right thing to do anyway, so let's
do it.

Signed-off-by: Olof Johansson <olof@lixom.net>
2020-01-07 21:06:58 +01:00
Ulf Magnusson def1f0e2d5 devicetree: Remove DT_SRAM_{BASE_ADDRESS,SIZE}, use CONFIG_* versions
The SRAM address and size are currently available as both
DT_SRAM_{BASE_ADDRESS,SIZE} and as CONFIG_SRAM_{BASE_ADDRESS,SIZE} (via
the Kconfig preprocessor).

Use the CONFIG_SRAM_* versions everywhere, and remove generation of the
DT_SRAM_* versions from gen_defines.py.

The Kconfig symbols currently depend on 'ARC || ARM || NIOS2 || X86'.
Not sure why, so I removed it.

It looks like no configuration files set CONFIG_SRAM_* at the moment, so
another option might be to use the DT_* symbols everywhere instead. Some
Kconfig.defconfig.series files add defaults to them though.

Also improve the help texts for CONFIG_SRAM_* to say that they normally
come from devicetree rather than configuration files.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-07 17:19:36 +01:00
Ulf Magnusson 95c18f1e6b dts: gen_defines.py: Clean up 'reg' code
Reorganize write_regs() to reuse more code for *_BASE_ADDRESS and
*_SIZE, and get rid of reg_addr_ident() and reg_size_ident().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-07 16:42:33 +01:00
Anas Nashif e9450065de sanitycheck: additional scripting hook after flashing
Support custom hooks after flashing is completed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 4522e10d65 sanitycheck: report overflow on SRAM overflow
Instead of failing, report overflow when the SRAM overflows.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif bdffb88950 sanitycheck: close pipe file handles
Make sure we close the pipe file handles after we are done, otherwise we
will end up with too many open file descriptors and crash...

Fixes #20974
Fixes #21637

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 64c84b15ce sanitycheck: more debug messages
More debug messages to help locate issues and debug bugs.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 1b1a7e242b sanitycheck: run pre/post script using dedicated function
Reduce duplicated code by introducing a dedicated function for running
custom scripts.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 32e924459d sanitycheck: log errors when executing binaries
Add some verbosity into the log file to help with debugging.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 52b66c6ad1 sanitycheck: when timing out in handler, show correct log file
Fix issue where we timeout in handler and show the device.log file
instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 813deb7967 sanitycheck: use Unknown instead of N/A for fail reason
Instead of N/A, use Unknown which is more appropriate, N/A does not
really apply here.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 5c331f95ad sanitycheck: add Digilent to device discovery
Some platforms use this USB device for serial, detect it when generating
hardware map.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Andrew Boie bd13710524 sanitycheck: use abs path for logs
Makes it much easier to inspect them in a different terminal
from where the test was run. These paths tend to be long
anyway, even if relative.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 114c01b3b9 sanitycheck: allow outdir to be overwritten
Sanitycheck used to delete the output directory if -n
wasn't used, but now it renamed it. Add a new flag to
enable the old behavior.

Do not use logging for these early messages, logger
hasn't been set up yet.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 2d8d4c5562 sanitycheck: fix early messages
When managing build artifacts the logger hasn't been
initialized. Use print() here instead.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 1fe1f3a23d sanitycheck: clean up TTY on exit
The code has a lot of sys.exit() calls, but internally these
just raise a special exception. Add a try...finally block
to ensure 'stty sane' is run before leaving the script.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Håkon Øye Amundsen c9a2a5e7fb scripts: cmake: move print from python to cmake
Prior to this all hex files included in a merge would
be printed on every invokation of cmake.

Allow the user to explicitly require this information
by moving the print to a VERBOSE-only cmake message.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2020-01-03 08:20:08 -05:00
David B. Kinder 17299f0734 doc: document python build scripts
We have a collection of python scripts that are part of our build
system.  This PR collects docstring comments added to these scripts into
a summary document.  Previous references to just the script name in
other documentation are updated to point to this build tool
documentation.

Some of the scripts needed an update to be processed (via include
directives) consistently.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-02 17:09:29 -05:00
Erwan Gouriou 17537c77c8 script/Kconfig: kconfig functions: Add shields_list_contains function
Add shields_list_contains Kconfig function which return bool based on
check of shield presence in cmake SHIELD_AS_LIST.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-01-02 17:02:41 -05:00
Ulf Magnusson e181e1b773 kconfiglib: Update to hide tracebacks for expected errors
Update Kconfiglib to upstream revision 9c0b562c94 to get this commit in:

    Add Kconfig.__init__() helper flag for suppressing tracebacks

    Tools that don't use standard_kconfig() currently generate spammy
    tracebacks for e.g. syntax errors.

    Add a suppress_traceback flag to Kconfig.__init__() for catching
    "expected" exceptions and printing them to stderr and exiting with
    status 1. Use it to make all tools consistently hide tracebacks.

Use the new flag to hide tracebacks for expected exceptions in
kconfig.py, lint.py, and genrest.py.

Some menuconfig robustness tweaks for wonky terminals are included as
well, and a new feature for customizing .config and autoconf.h header
comments via environment variables.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-20 19:48:09 -05:00
Anas Nashif 2a5d61d3ea sanitycheck: define schema for hardware map
Add schema for hardware map and verify it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-19 22:55:16 -05:00
Anas Nashif fc85ff052f sanitycheck: support pre/post flash scripts
This is needed when a board needs to be reset using an external commands
or tools that are not part of the flash command. For example, power
reset or by poking a GPIO header on the board using external wiring.

add post_flash/pre_flash to the platform section in the hardware map.
For example:

- available: true
  connected: true
  id: OSHW000032254e4500128002ab98002784d1000097969900
  platform: reel_board
  post_script: /tmp/post_flash.sh
  pre_script: /tmp/pre_flash.sh
  product: DAPLink CMSIS-DAP
  runner: pyocd
  serial: /dev/ttyACM11

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-19 22:55:16 -05:00
Sebastian Bøe bb95dce98c west: Fix defining modules without a module.yml
It is supported to have a zephyr module that does not have a
module.yml, but zephyr_module.py does not support it and will drop
such modules.

To fix this we add support in zephyr_module.py.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-12-19 15:52:44 -05:00
Martí Bolívar 150eb6575c scripts: west build: slight --help tweak
We have an open request to make the help for the -t option a little
easier. Try to do that without adding too much length to the short
help.

Fixes: #16202
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2019-12-19 11:29:39 +01:00
Ulf Magnusson 5e55eda30e scripts: edtlib: Support nested nodes on buses
For the following devicetree, view 'nested' as being on the bus.
Previously, only 'node' was considered to be on the bus.

    some-bus {
    	compatible = "foo,bus-controller";
    	node {
    		nested {
    			compatible = "foo,device-on-bus";
    		};
    	};
    };

In practice, this means that a 'bus:' key in the binding for
'foo,bus-controller' will now get matched up to an 'on-bus:' key in the
binding for 'foo,device-on-bus'.

Change the meaning of Node.bus and add two new attributes Node.on_bus
and Node.bus_node, with these meanings:

    Node.bus:
      The bus type (as a string) if the node is a bus controller, and
      None otherwise

    Node.on_bus:
      The bus type (as a string) if the node appears on a bus, and None
      otherwise. The bus type is determined from the closest parent
      that's a bus controller.

    Node.bus_node:
      The node for the bus controller if the node appears on a bus, and
      None otherwise

It's a bit redundant to have both Node.bus_node and Node.on_bus, since
Node.on_bus is the same as Node.bus_node.bus, but Node.on_bus is pretty
handy to save some None checks.

Also update gen_defines.py to use Node.on_bus and Node.bus_node instead
of Node.parent wherever the code deals with buses.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-19 11:02:28 +01:00
Ulf Magnusson 379145ffce scripts: edtlib: Rename 'child-bus:'/'parent-bus:' to 'bus:'/'on-bus:'
I keep mixing these up, so that's probably a sign that the names are
bad. The root of the problem is that "parent-bus" can be read as both
"this is the parent bus" and as "the parent bus is this".

Use 'bus:' for the bus "provider" and 'on-bus:' for nodes on the bus
instead, which is less confusing.

Support the old keys for backwards compatibility, along with a
deprecation warning.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-19 11:02:28 +01:00
Anas Nashif 0c69c285ec sanitycheck: cleanup hardware map display
Add a static method for dumping hardware map and reuse it across the
script reducing duplicated code.

Fixes #21475

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 21:52:02 +01:00
Anas Nashif 81b2f37a9d ci: run sanitycheck in verbose mode
Call sanitycheck in verbose mode

Fixes #21483

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 12:07:49 -05:00
Andrew Boie 13b8b41676 drivers: watchdog: add system calls
wdt_install_timeout() was skipped as it installs an ISR-context
callback handler function. The rest are simple wrappers.

Added myself as the maintainer of the syscall handlers. WDT
subsystem appears to not currently have an owner.

Fixes: #21432

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 09:28:52 -05:00
Maksim Masalski 1b4b9ef535 sanitycheck: hardware map added support of the microchip board
Added support of the Microchip with FTDI serial devices to be able
create a hardware map for them. In the future that hardware map
for the Microchip board will let run automatic Sanitycheck tests on it.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2019-12-18 08:05:05 -05:00
Ulf Magnusson 36b7ca44b8 scripts: edtlib.py: Deprecate 'title:'
Most bindings look something like this:

    title: Foo

    description: This binding provides a base representation of Foo

That kind of description doesn't add any useful information, as it's
just the title along with some copy-pasted text. I'm not sure what "base
representation" was supposed to mean originally either.

Many bindings also put something that's closer to a description in the
title, because it's not clear what's expected or how the title is used.
In reality, the title isn't used anywhere. 'description:' on the other
hand shows up as a comment in the generated header.

Deprecate 'title:' and generate a long informative warning if it shows
up in a binding.

Next commits will clean up the 'description:' strings (bringing them
closer to 'title:' in most cases) and remove 'title:' from all bindings.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-18 11:52:45 +01:00
Anas Nashif 17d066b9dc sanitycheck: fix reporting of timeouts
timeouts were not reported correctly and we were getting the default
'N/A' in case of a timeout.

Fixes #21438

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 3cfc608934 sanitycheck: remove commented out code
Remove commented out debug code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 9744568350 sanitycheck: use colorama module for colors
Use colorama module for colors instead of custom codes.

Fixes #21409

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif ebf8dae241 sanitycheck: fix legacy device testing support
When testing with one device and without a hardware map, make sure we
load the temporary map into the suite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif d988238970 sanitycheck: pylint issues
run through pylint and fix.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 7148021627 sanitycheck: remove unused globals
A few globals that were never used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif c5ee3951fd sanitycheck: get rid of more global options
More global option cleanup.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif e9eb00945d sanitycheck: move log_info_* to ProjectBuilder class
Move this to where it is being used and called.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 56656848f5 sanitycheck: do not use gloabl options
More changes moving away from using global options and instead using
class variables and parameters.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif fa8085eb7b sanitycheck: all output from flasher goes to logger.debug
Redirect all output from flasher to logger.debug().

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif f16ed8e0b8 sanitycheck: move global info() into Testsuite class
This function is not used anywhere else now that we use logging module,
so push this into the class where it is being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 67f9ecb79b sanitycheck: consistent output from logger
Make all output consistent with logger and report errors correctly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 6d66a7a933 sanitycheck: use logger instead of print
Use logger instead of print for consistency.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif d4cef07200 sanitycheck: reenable --timestamps option with loggging module
--timestamps now just enables a different formatter with asctime.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif f5e5ef0bd8 sanitycheck: reduce usage of global options
The use of a global options variable is making it very difficult to
create a testsuite for this script, so reduce and push options into
arguments instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00