Commit graph

2212 commits

Author SHA1 Message Date
Ulf Magnusson 16041cda0c west: zcmake.py: Simplify test with 'in'
Getting slightly subjective, but fixes this pylint warning:

    scripts/west_commands/zcmake.py:186:13: R1714: Consider merging
    these comparisons with "in" to "type_ in ('STRING', 'INTERNAL')"
    (consider-using-in)

Use a set literal instead of a tuple literal, as recent Python 3
versions optimize set literals with constant keys nicely.

Getting rid of pylint warnings for a CI check. I could disable any
controversial ones (it's already a list of warnings to enable anyway).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:12:09 -04:00
Ulf Magnusson f5d8ece3b7 west: nrfjprog.py: Remove len() from conditional to fix pylint warning
Empty sequences are falsy in Python, so len() can be skipped.

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:11:44 -04:00
Ulf Magnusson 5d307c98fe west: build.py: Remove len() from conditional to fix pylint warning
Non-empty sequences are truthy in Python, so len() can be skipped.

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:11:02 -04:00
Ulf Magnusson 1f0a8e5a3f west: completion.py: Fix bad indentation and blank lines at end of file
Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:09:56 -04:00
Ulf Magnusson 22274696a9 west: boards: Remove accidental comma to fix pylint warning
Remove a trailing comma that generated a single-element tuple and made
pylint warn:

    scripts/west_commands/boards.py:50:8: W0106: Expression
    "(parser.add_argument(...), )" is assigned to nothing
    (expression-not-assigned)

No functional change.

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:08:48 -04:00
Ulf Magnusson 3206e42935 scripts: gen_kobject_list.py: Simplify test with chained comparison
'a < b and b < c' can be simplified to 'a < b < c' in Python.

Fixes this pylint warning:

    scripts/gen_kobject_list.py:198:22: R1716: Simplify chained
    comparison between the operands (chained-comparison)

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:07:36 -04:00
Ulf Magnusson d2d5fae838 west: runners: core.py: Remove no-op try-excepts
Removing these doesn't change behavior, since the
subprocess.CalledProcessError is just immediately re-raised when caught.

Fixes this pylint warning:

    W0706: The except handler raises immediately (try-except-raise)

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:06:51 -04:00
Ulf Magnusson 4094ee686e west: runners: Remove unnecessary constructors
These just pass their arguments through to the base class constructor.
Removing them means the base class constructor gets called directly
instead.

Fixes this pylint warning:

    W0235: Useless super delegation in method '__init__'
    (useless-super-delegation)

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:06:51 -04:00
Ulf Magnusson bf128d09b3 west: runners: Fix typo'd log.wrn() call
Should be wrn() instead of warn(). Reported by pylint.

Also remove a {} from the message. It's not being formatted.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:05:57 -04:00
Ulf Magnusson 11d98ae9de scripts: gen_relocate_app.py: Simplify test with 'in'
Getting slightly subjective, but fixes this pylint warning:

    scripts/gen_relocate_app.py:228:38: R1714: Consider merging these
    comparisons with "in" to "region in ('data', 'bss')"
    (consider-using-in)

Use a set literal instead of a tuple literal, as recent Python 3
versions optimize set literals with constant keys nicely.

Getting rid of pylint warnings for a CI check. I could disable any
controversial ones (it's already a list of warnings to enable anyway).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:05:22 -04:00
Ulf Magnusson bb634167ae west: commands: Make functions that don't use 'self' static
Fixes this pylint warning:

    R0201: Method could be a function (no-self-use)

Another option would be to turn them into regular functions, but that'd
be a bigger change.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:04:58 -04:00
Ulf Magnusson 4cac371d43 scripts: Suppress/fix undefined variable pylint warnings
dtlib.py and guiconfig.py do some hackery to build token and image
variable names, which triggers spurious pylint warnings like

    scripts/dts/dtlib.py:243:28: E0602: Undefined variable '_T_LABEL'
    (undefined-variable)

Suppress the warning for those files. The generated names get used in
lots of places.

Also suppress some warnings in doc/conf.py ('tags' is from Sphinx), and
fix a legitimate issue in scripts/dts/testdtlib.py.

This pylint check is useful enough to want enabled in the upcoming CI
check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:03:04 -04:00
Ulf Magnusson 3feb8f96f6 scripts: gen_kobject_list.py: Simplify test with 'in'
Getting slightly subjective, but fixes this pylint warning:

    scripts/gen_kobject_list.py:308:11: R1714: Consider merging these
    comparisons with "in" to "kobj in ('device',
    '_k_thread_stack_element')" (consider-using-in)

Use a set literal instead of a tuple literal, as recent Python 3
versions optimize set literals with constant keys nicely.

Getting rid of pylint warnings for a CI check. I could disable any
controversial ones (it's already a list of warnings to enable anyway).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:01:31 -04:00
Ulf Magnusson 89efaeda74 west: build.py: Rename unused variable from unpacking to _
This is a common Python idiom, and it's easy to look up what the unused
value is in this case if you need to. Fixes this pylint warning:

    scripts/west_commands/build.py:227:15: W0612: Unused variable
    'origin' (unused-variable)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:00:57 -04:00
Ulf Magnusson 883b53101c menuconfig: Suppress spurious pylint tuple unpacking warning
pylint might be afraid that there'd be less than three elements in
'args', but there never is. Fixes this warning:

    scripts/kconfig/menuconfig.py:3184:8: W0632: Possible unbalanced
    tuple unpacking with sequence: left side has 3 label(s), right side
    has 0 value(s) (unbalanced-tuple-unpacking)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:00:30 -04:00
Ulf Magnusson 47ef9ba6c2 scripts: sanitycheck: Remove redundant ifs
Fixes these pylint warnings:

    scripts/sanity_chk/ini2yaml.py:25:22: R1719: The if expression can
    be replaced with 'test' (simplifiable-if-expression)

    scripts/sanity_chk/expr_parser.py:208:15: R1719: The if expression
    can be replaced with 'bool(test)' (simplifiable-if-expression)

    scripts/sanity_chk/expr_parser.py:210:15: R1719: The if expression
    can be replaced with 'bool(test)' (simplifiable-if-expression)

Also replace a redundant re.compile().match() with re.match(). compile()
doesn't help when re-compiling the regular expression each time through.

(compile() often doesn't help much in general, because the 're' module
caches compiled regexes.)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 07:59:53 -04:00
Ulf Magnusson 16b548fcc7 gen_gcov_files.py: Replace bare 'except' with 'except Exception'
A bare 'except:' can do some annoying stuff like eat Ctrl-C (because it
catches KeyboardInterrupt). Better to use 'except Exception:' as a
catch-all.

Even better might be to catch some more specific exception, because even
'except Exception:' eats stuff like misspelled identifiers.

Fixes this pylint warning:

    scripts/gen_gcov_files.py:54:12: W0702: No exception type(s)
    specified (bare-except)

Piggyback a formatting nit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 07:58:16 -04:00
Ulf Magnusson 4699375162 west: run_common.py: Remove redundant 'if'
Fixes this pylint warning:

    scripts/west_commands/run_common.py:175:12: R1719: The if expression
    can be replaced with 'test' (simplifiable-if-expression)

Fixing pylint warnings for a CI check.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 07:57:46 -04:00
Ulf Magnusson 0d39a10fbb scripts: Fix random typo'd whitespace
Reported by pylint's 'bad-whitespace' warning.

Not gonna enable this warning in the CI check, because it flags stuff
like deliberately aligning assignments and gets too cultish. Just a
cleanup pass.

For whatever reason, the common convention in Python is to skip spaces
around '=' when passing keyword arguments and giving default arguments:

    f(x=3, y=4)
    def f(x, y=8):
        ...

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 07:54:17 -04:00
David B. Kinder 02ee3754c6 west: update requirements.txt to avoid west 0.6.1
Update scripts/requirements.txt to use 0.6.2 or later (avoiding 0.6.1
that has an known issue)

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-09-06 19:12:33 +02:00
Anas Nashif 053ab4ac41 release: update sanitycheck release file
Update release file used by sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-09-06 13:31:04 +02:00
Wentong Wu ed6ce291ba scripts: elf_helper.py: fix stack declared K_THREAD_STACK_EXTERN
If a stack is declared with K_THREAD_STACK_EXTERN first, analyze array
in elf_helper.py will ignore this declaration which will be referenced
by the actual instances via the tag DW_AT_specification, so that this
stack can't be detected by the kernel object detection mechanism, and
this will cause userspace not work.

Fixes: #16760.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-08-29 08:18:54 -04:00
Wentong Wu 9ca0d5e18b scripts: elf_helper.py: add analyze for DW_TAG_typedef
add analyze for DW_TAG_typedef in order to catch all the
kernel objects.

Fixes: #16760.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-08-29 08:18:54 -04:00
David B. Kinder 2c93d1f1f2 doc: add sphinx-tabs extension
The sphinx-tabs extension provides a tabbed interface within documents
that will let us dynamically display information based on a user
selection, for example, for instruction variations based on the user's
development OS (Linux, macOS, Windows).

I'm adding this early so it can get integrated into the CI doc build
before subsequent PRs using this extension are submitted.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-08-23 19:38:44 -04:00
Anas Nashif 35d4177a0b requirements: add west>-0.6.0
west is yet another python package, so lets add it here. This will
provide us with one single file with all python requirements that can be
used during setup and for example for docker images.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-08-23 13:50:16 -04:00
Anas Nashif dd3776cfa5 requirements: sort alphabetically
Sort packages alphabetically.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-08-23 13:50:16 -04:00
Kumar Gala a0a576d384 scripts/dts/gen_defines.py: Fix generation for multiple IO Channels
If there is more than one IO Channel than generate a define with a
trailing index for the IO Channel.  This matches what we do for GPIOs
and PWMs.

So something like:
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_0
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_1
  ...
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_<N>

Fixes #18352

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-19 14:48:44 -05:00
Håkon Øye Amundsen 208c4ecacb scripts: kconfig: normalize paths to avoid duplicate
There is a bug where non-normalized paths can introduce multiple entries
for the same file. E.g. '/1/2/../2.file' and '/1/2/3/../../2.file' would
both be listed, and end in a ninja error because multiple targets
generate the same file.

This commit fixes this issue by normalizing the paths.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2019-08-19 16:26:31 +02:00
Marti Bolivar f7a4275328 scripts: west flash: fix --board-id for pyocd
Newer pyocd versions (specifically the 0.21.0 we have in our
requirements.txt) no longer support -b and have moved the same option
to -u. Keep up.

Fixes: #17554
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-19 11:03:03 +02:00
Marti Bolivar 08e0d1ff46 scripts: runners: improve hex file detection and output
This is a band-aid to make it more obvious to potential users of 'west
sign' and 'west flash' which hex file they are flashing, when they are
falling back on a binary file, and erroring out when a hex file does
not exist.

Fixes: #18201
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-16 18:42:41 +02:00
Kumar Gala c484935fe5 scripts: openocd: Add error if hex file is missing when flashing
Add a check to make sure the hex file exists as that is what we utilize
in openocd to flash.  If its missing we report that its likely due to
not having CONFIG_BUILD_OUTPUT_HEX set.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-14 07:55:10 -05:00
Ulf Magnusson c2d702b961 dts: dtlib: Fix parsing of character literals
This was allowed due to a misunderstanding:

    foo = 'x';

In reality, 'x' works like an integer literal, and is used like this:

    foo = < 'x' >;

Fix character literal parsing to match the C tools.

Also fix backslash escape parsing to match the C tools exactly
(get_escape_char() in util.c): \<char> should be turned into <char> if
<char> isn't recognized as a special escape character, instead of being
left alone. This fixes parsing of e.g. '\'' (a character literal with a
single quote in it).

Piggyback some more tests for weird property/node names.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson 7168118290 dts: edtlib: Fix outdated doc comment for _init_devices()
No longer takes the DT instance as a parameter.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson 06b746cc58 dts: dtlib/edtlib: Add a syntax-based type-checking system
Property type-checking has been pretty rudimentary until now, only
checking things like the length being divisible by 4 for 'type: array',
and strings being null-terminated. In particular, no checking was done
for 'type: uint8-array', letting

  jedec-id = < 0xc8 0x28 0x17 >;

slip through when

  jedec-id = [ 0xc8 0x28 0x17 ];

was intended.

Fix it by adding a syntax-based type checker:

  1. Add Property.type, which gives a high-level type for the property,
     derived from the markers added in the previous commit.

     This includes types like TYPE_EMPTY ('foo;'),
     TYPE_NUM ('foo = < 3 >;'), TYPE_BYTES ('foo = [ 01 02 ];'),
     TYPE_STRINGS ('foo = "bar", "baz"'),
     TYPE_PHANDLE ('foo = < &bar >;'), and TYPE_COMPOUND (everything not
     recognized).

     See the Property.type docstring in dtlib for more info.

  2. Use the high-level type in
     Property.to_num()/to_string()/to_node()/etc. to verify that the
     property was assigned in an expected way for the type.

     If the assignment looks bad, give a helpful error:

       expected property 'nums' on /foo/bar in some.dts to be assigned
       with 'nums = < (number) (number) ... >', not 'nums = "oops";'

Some other related changes are included as well:

  - There's a new Property.to_bytes() function that works like accessing
    Property.bytes, except with an added check for the value being
    assigned like 'foo = [ ... ]'.

    This function solves problems like the jedec-id one.

  - There's a new Property.to_path() function for fetching the
    referenced node for assignments like 'foo = &node;', with type
    checking. (Strings are accepted too, as long as they give the path
    to an existing node.)

    This function is used for /chosen and /aliases.

  - A new 'type: phandle' type can now be given in bindings, for
    properties that are assigned like 'foo = < &node >;'.

  - Property.__str__() now displays phandles and path references as they
    were written (e.g. '< &foo >' instead of '< 0x1 >', if the
    allocated phandle happened to be 1).

  - Property.to_num() and Property.to_nums() no longer take a 'length'
    parameter, because it makes no sense with the type checking.

  - The global dtlib.to_string() and dtlib.to_strings() functions were
    removed, because they're not that useful.

  - More tests were added, along with misc. minor cleanup in various
    places.

  - Probably other stuff I forgot.

The more strict type checking in dtlib indirectly makes some parts of
edtlib more strict as well (wherever Property.to_*() is used).

Fixes: #18131

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson c9ac5e60e3 dts: dtlib: Remember the format of assignments
Previously, dtlib just stored the raw 'bytes' value for each property,
along with some markers in Property._markers for phandle and path
references.

Extend Property._markers to also remember where different data blocks
start, so that e.g.

    foo = <1 2 3>, "bar", [00 01];

can be reproduced as written.

Use the new information to reproduce properties as written in
Property.__str__(). This gives good test coverage as well, since the
test suite checks literal __str__() output.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Kumar Gala b2ba44c1d7 scripts/dts/gen_defines.py: Fix generation for multiple PWMs
If there is more than one PWM than generate a define with a trailing
index for the PWM.  This matches what we do for GPIOs.

So something like:
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_0
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_1
  ...
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_<N>

Fixes #18171

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-12 17:42:15 +02:00
Marcin Sloniewski b78bf8d6c1 requirements.txt: change package name "hub" to "git-spindle"
Package "hub" has been renamed some time ago to "git-spindle".
Recently package with the same name "hub" has been released on PyPi
page, which causes installing pip requirements.txt to fail since
it is trying to install different package.

Signed-off-by: Marcin Sloniewski <marcin.sloniewski@gmail.com>
2019-08-12 11:53:10 +02:00
Kumar Gala 480dcc1634 scripts/dts/gen_defines.py: Ignore 'gpio-controller' prop
We don't want any defines generated for the boolean
'gpio-controller'.  So skip it in write_props if we see it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 14:10:40 -05:00
Kumar Gala b0a6b3c0be scripts/dts/gen_defines.py: Ignore 'interrupt-controller' prop
We don't want any defines generated for the boolean
'interrupt-controller'.  So skip it in write_props if we see it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 14:10:40 -05:00
Jim Paris 998d7a01dd scripts/dts: add basic test for IO channel property
Test the new io-channel support

Signed-off-by: Jim Paris <jim@jtan.com>
2019-08-09 13:16:09 -05:00
Jim Paris 67f53ba18f scripts/dts: Support 'io-channels' property just like 'pwms'
This is a direct search-and-replace copy of the PWM code.
The name is chosen to match Linux's iio-bindings.txt.

Signed-off-by: Jim Paris <jim@jtan.com>
2019-08-09 13:16:09 -05:00
Kumar Gala 5dd715b9c1 scripts/dts/edtlib.py: error check # and -map properties
Move when we early out for properties that start with # like
"#address-cells" or end with -map like "interrupt-map" to after we do
some error checking.  This allows us to check those properties at least
exist if they are required.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 11:49:16 -05:00
Ulf Magnusson 75391c4f2c dts: edtlib: Shorten some license headers
One line instead of three.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson 4985c214ef dts: edtlib: Shorten _init_{interrupts,gpios}() a bit
Also makes them consistent with _simple_phandle_val_list().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson f7f0e0adf9 dts: edtlib: Simplify 'category: optional' code in Device._prop_val()
Reuse Device.enabled().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson c0c8dd1b85 dts: edtlib: Improve naming in _check_binding()
'keys' is really a dictionary of options (like {"type": "int", ...}) for
a property. Calling it 'options' makes it clearer.

Also s/prop/prop_name/.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Kumar Gala e328e56b30 dts/gen_defines.py: Use err() instead of _err()
_err comes from edtlib and we should be using err() inside of
gen_defines.py.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-08 23:53:22 -05:00
Marcin Niestroj b08adcdd7e scripts: openocd: use hex file for flash command
Use hex file for flash command, instead of elf file. This allows to
flash signed firmware, which is not available in elf format, by
specifying --hex-file command line argument.

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2019-08-08 22:16:50 +02:00
Marcin Niestroj 6778468c73 scripts: openocd: allow to overwrite elf file used to flash device
So far zephyr.elf file was hardcoded in cmake files. Remove it from
there and use cfg.elf_file from python, which can be overwritten by
specifying --elf-file command line option.

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2019-08-08 22:16:50 +02:00
Ulf Magnusson acf276f1df edtlib: Check that 'status' has one of the values from the DT spec.
dtlib is meant to be general and anything-goes re. property values, but
edtlib can be pickier. Check that all 'status' properties have one of
the values listed in the devicetree specification
(https://www.devicetree.org/specifications/), and error out otherwise.

This will get rid of the 'status = "ok"'s (should be "okay") that keep
cropping up.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-08 14:03:25 -05:00
Kumar Gala 8622c3415d scripts/dts: Fix detection of a GPIO specifier
We should only assume a GPIO specifier is either named <FOO>-gpios or
gpios.  Any other form like ngpios should not be considered a GPIO
specifier.  especially since 'ngpios' is the standard property name for
the number of gpio's that are implemented.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-07 13:15:51 -04:00
Kumar Gala 99769990ed scripts/dts/gen_defines: Cleanup write_prop for which props we process
We don't create props for compound types so we dont need to check for
'pwms', 'gpios', 'clocks'.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-07 13:15:51 -04:00
Ulf Magnusson 95deec1d24 scripts: edtlib: Reduce code duplication in phandle/value list parsing
Most of the logic for initializing 'clocks' and 'pwms' is the same and
can be shared. Add an EDT._simple_phandle_val_list() helper function for
initializing 'clocks', 'pwms', and any other properties on the form

    <foo>s = <phandle value phandle value ...>

where the nodes pointed at by the phandles are expected to have a
'#<foo>-cells' property.

This should make it easier to add similar properties.

There's still some code duplication in the classes (PWM, Clock, etc.),
but also some differences, so I'm wondering if requiring a class for
each might be okay. Maybe some more class-related stuff could be
factored out later.

Piggyback some related cleanup:

 - Have _phandle_val_list() take the name of the #foo-cells property
   instead of a function for fetching the value. The pattern is always
   the same.

 - Have _add_names() just take the "foo" part of "foo-names". Same
   pattern everywhere.

 - Clean up some redundant comments for stuff that's already documented
   in docstrings

 - Fix error messages printed by _named_cells() ("GPIOS controller" ->
   "GPIO controller", etc.)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-07 11:48:53 -04:00
Piotr Zierhoffer a3ddc09abf build: west: Update the error message when not in west dir
Zephyr verifies if it is built from a west-initialized directory.
The message changed in west@c08061cef1c7b0e19a58a82db731098e2f4bba4a.

Closes #18034.

Signed-off-by: Piotr Zierhoffer <pzierhoffer@antmicro.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-08-06 19:13:24 +02:00
Paul Sokolovsky 04da7eaf07 scripts: dts: Consistently open text files with utf-8.
Zephyr codebase standardizes in UTF-8 as file encoding. To
accommodate this, we explicitly pass encoding="utf-8" to Python's
open() function, to be independent of any locale setting of a
particular system (e.g., CI/build systems oftentimes have "C",
i.e. ASCII-only, locale). In a few places, we lacked this parameter,
so add it consistently.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-08-05 17:47:57 -04:00
Marti Bolivar 7118d084c9 scripts: un-break test_nrfjprog.py
The pytest.raises context manager is now returning an ExceptionInfo
whose str() doesn't contain the str() of the underlying exception
object. Take str(e.value) directly to make sure we're looking at the
exception string.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-05 15:17:45 +02:00
Marti Bolivar 2b3d9df1d7 west: require v0.6.0 or higher
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.

Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes.  Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.

(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)

This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-05 15:17:45 +02:00
Ulf Magnusson fe2d858b5f scripts: dts: Add test for multiple binding directories
Add two bindings

    test-bindings/multidir.yaml
    test-bindings-2/multidir.yaml

and a new test-multidir.dts with two nodes that use them.

Verify that the two bindings were found by checking the
Device.binding_path attribute for the two device nodes.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-03 14:17:49 -04:00
Michael Scott b8909439bf scripts: dts: support multiple binding dirs in new scripts
gen_defines.py and edtlib.py were recently added in
commit 62d5741476 ("dts: Add new DTS/binding parser").

The old extract_dts_includes.py script allowed for multiple
dts bindings dirs.  Let's add that functionality to the new
scripts as well.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-08-03 14:17:49 -04:00
Ulf Magnusson 5d0db517b9 dts: riscv: Add sifive,plic-1.0.0 binding and fix riscv,ndev values
Add a new sifive,plic-1.0.0 binding that inherits from the riscv,plic0
binding. The new binding adds a required riscv,ndev property, which
gives the number of external interrupts supported.

Use the new binding for microsemi-miv.dtsi (with a value of 31 for
riscv,ndev, from http://www.actel.com/ipdocs/MiV_RV32IMAF_L1_AHB_HB.pdf)
and riscv32-fe310.dtsi (which already assigns riscv,ndev).

Also remove a spurious riscv,ndev assignment from
riscv32-litex-vexriscv.dtsi.

Also make edtlib and the old scripts/dts/ scripts replace '.' in
compatible strings with '_' when generating identifiers.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-02 11:44:09 +02:00
Kumar Gala 11e91a83c1 scripts/sanitycheck: optimize disk usage for --save-tests
If we are doing a nightly build we can utilize a large (over 50G) of
disk space just to generate the list of tests to build.  We need to
optimize this so as we finish building the initial pass we clean up
as we go and only keep around the files we need (like .config,
generated_dts_board.conf, CMakeCache.txt, etc).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-31 18:46:32 -04:00
Peter A. Bigot 9a0d9e3af0 scripts: gen_cfb_font_header: modify to replicate cfb fonts
The content of subsys/fb/cfb_fonts cannot be replicated by the existing
script due to lack of positioning options and use of a full-color frame
buffer, which affects the generated bitmap.  Switch to the solution used
in the original script, add the required options, and document the
process of regenerating the fonts.

This commit also determines the required bounding box for the glyphs to
be sure that the user-provided value is sufficient to avoid partial
characters.  Ideally the calculated width and height would be used for
font characters, but this would require significant restructuring of the
script to make calculated values available at the point where the
arguments are used to produce output.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-31 05:43:50 -04:00
Peter A. Bigot f14b19b00a scripts: gen_cfb_font_header: remove guard and make font data private
The output of this script is intended to be put into an implementation
file, where the font data is accessed by index to an array maintained by
the linker script.  There is no need for protection against multiple
includes, and the font data array should not be a global symbol.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-31 05:43:50 -04:00
Kumar Gala 9d9b108dab scripts/dts: Sort output of DT_COMPAT_ defines
Sort the DT_COMPAT_ defines so comparing them from different builds is
easier.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-31 05:33:41 -04:00
Kumar Gala 4e7863dc41 dts: Make instance defines consistent
We generated a define for each instance to convey its existance of the
form:
	#define DT_<COMPAT>_<INSTANCE> 1

However we renamed all other instance defines to be of the form
DT_INST_<INSTANCE>_<FOO>.  To make things consistent we now generate a
define of the form:

	#define DT_INST_<INSTANCE>_<COMPAT> 1

We also now deprecate the DT_<COMPAT>_<INSTANCE> form and fixup all uses
to use the new form.

Fixes: #17650

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-30 17:10:31 -05:00
Anas Nashif 578ae40761 boards: remove quarl_se_c1000
This board and SoC was discontinued some time ago and is currently not
maintained in the zephyr tree.
Remove all associated configurations and variants from the tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-07-29 21:30:25 -07:00
Ulf Magnusson 7de2f4da14 scripts/dts: Add deprecation comments to old scripts
Makes it easier to understand what's going on.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson 62d5741476 dts: Add new DTS/binding parser
Add a new DTS/binding parser to scripts/dts/ for generating
generated_dts_board.conf and generated_dts_board_unfixed.h.

The old code is kept to generate some deprecated defines, using the
--deprecated-only flag. It will be removed later.

The new parser is implemented in three files in scripts/dts/:

dtlib.py:
  A low-level .dts parsing library. This is similar to devicetree.py in
  the old code, but is a general robust DTS parser that doesn't rely on
  preprocessing.

edtlib.py (e for extended):
  A library built on top of dtlib.py that brings together data from DTS
  files and bindings and creates Device instances with all the data for
  a device.

gen_defines.py:
  A script that uses edtlib.py to generate generated_dts_board.conf and
  generated_dts_board_unfixed.h. Corresponds to extract_dts_includes.py
  and the files in extract/ in the old code.

testdtlib.py:
  Test suite for dtlib.py. Can be run directly as a script.

testedtlib.py (uses test.dts and test-bindings/):
  Test suite for edtlib.py. Can be run directly as a script.

The test suites will be run automatically in CI.

The new code turns some things that were warnings (or not checked) in
the old code into errors, like missing properties that are specified
with 'category: required' in the binding for the node.

The code includes lots of documentation and tries to give helpful error
messages instead of Python errors.

Co-authored-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson a72e451624 scripts/dts: Rename --deprecate-only to --deprecated-only
--deprecate-only sounded like a command to "only deprecate (something)"
to me at first. --deprecated-only might make it clearer that it's about
only generating deprecated stuff.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Jukka Rissanen f95938da0f net: ppp: Initial support for point-to-point protocol
This implements ppp L2 component, LCP and IPCP modules.

Fixes #14034

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-07-29 10:24:46 +03:00
Andrew Boie 81ef42d2bc sanitycheck: simplify fault detection
Any fatal error will print "ZEPHYR FATAL ERROR" now, so
we don't have to maintain a set of strings in the
sanitycheck harness.py

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie 96571a8c40 kernel: rename NANO_ESF
This is now called z_arch_esf_t, conforming to our naming
convention.

This needs to remain a typedef due to how our offset generation
header mechanism works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie 7875707a98 userspace: add kobject flag for drivers
This new flag will indicate that the kernel object represents
an instance of a device driver object.

Fixes: #14037

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-24 12:33:34 +02:00
Ulf Magnusson 0ec0c84808 dts: bindings: Remove unused 'version' field
No binding has anything but 'version: 0.1', and the code in scripts/dts/
never does anything with it except print a warning if it isn't there.
It's undocumented what it means.

I suspect it's overkill if it's meant to be the binding format version.
If we'd need to tell different versions from each other, we could change
some other minor thing in the format, and it probably won't be needed.

Remove the 'version' fields from the bindings and the warning from the
scripts/dts/ scripts.

The new device tree script will give an error when unknown fields appear
in bindings.

The deletion was done with

    git ls-files 'dts/bindings/*.yaml' | xargs sed -i '/^\s*version: /d'

Some blank lines at the beginning of bindings were removed as well.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-22 09:28:07 -04:00
Alexander Wachter b4c5f4b32b linker: Add dtcm section for Cortex M7 MCUs
This commit adds a DTCM (Device Tightly Coupled Memory) section for
Cortex F7 MCUs. The Address and length is defined in the corresponding
device tree file.

Signed-off-by: Alexander Wachter <alexander.wachter@student.tugraz.at>
2019-07-19 10:05:46 +02:00
Wentong Wu fa7ce3ff70 scripts: make mpu align in sections for code reloation
make mpu align inside sections instead of outside to avoid
overlap for code relocation feature.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-18 11:09:48 -04:00
Carles Cufi 9754201e04 west: Load CMakeCache.txt as a UTF-8 encoded file
CMake has been using UTF-8 as an encoding for CMakeCache.txt since 3.2.

Fixes #17635

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Carles Cufi 3ad1f27efc cmake: Encode and decode in UTF-8 in build system
In order to make sure that the build works in folders that require a UTF
encoding, make sure that both CMake and the various Python scripts that
interact with each other on files use the same encoding, in this case
UTF-8.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Kumar Gala 1c6952c4f6 dts/extract: Mark DT_FLASH_AREA_*_LABEL defines as deprecated
We don't use the DT_FLASH_AREA_*_LABEL defines today so lets mark them
deprecated until we actually need something.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:33 -04:00
Kumar Gala 4d2625cd79 scripts/dts: Add option to generate only deprecated defines
Add a --deprecate-only flag to only generate the deprecated defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:07 -04:00
Andy Ross 50b78f08ea cmake: Remove check_link_map.py
This script is looking for a hyperspecific error (mismatched padding
when linking into two simultaneous output sections) that bit us once,
in an era where the linker scripts were less unified.  We haven't seen
it crop up since, and multiple platforms have changed the way they do
this anyway.

It's needless complexity.  Junk it.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-07-14 04:54:29 -07:00
Alberto Escolar Piedras f8908cd0d8 sanitycheck: Add missing section to whitelist
In a3bea8872b (PR #16352)
_bt_br_channels_area was added to the code but not the
sanitycheck sections whitelist => Add it

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 17:08:25 -07:00
Alberto Escolar Piedras 010c609bba sanitycheck: update sections whitelist
In c20ff1150f
and 5f19c8160a
(PR #16897)
_bt_settings_area was replaced with _settings_handlers_area

Update sanitycheck sections whitelist accordingly

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 13:36:13 +02:00
Peter A. Bigot 98a8e293cb scripts: gen_cfb_font_header: correct command sanitization
The code to sanitize the generated font path by removing bindir
improperly stripped the first character of every argument when --bindir
was not provided, corrupting the command documentation when fonts are
generated manually.  Only sanitize if --bindir was provided with
non-empty content.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Peter A. Bigot 8b4d529706 scripts: gen_cfb_font_header: correct last character
DELETE is not a printable character.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Kumar Gala 31052837ed scripts/dts: cleanup 'generation' usage
* Remove dead code that referenced 'generation' but didn't do anything
  with it
* Replace looking at 'generation' with a simple check for property
  starting with # (for things like #address-cells, etc)  or ending in
  -map (for things like gpio-map) to skip

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-11 06:51:17 -04:00
Peter A. Bigot 79765b8ad7 scripts/dts: fix support for uint8-array property values
uint8-array is the name for what the devicetree specification calls a
bytestring.

The original parsing code treated square brackets as equivalent to
quotes or angle brackets, failing to interpret elements as hex-encoded.
This was a bug, corrected in this patch by processing content as a
sequence of hex-encoded bytes with arbitrary whitespace.

The original generating code emitted the property as individual
elements.  Replace that with generation of a single structure
initializer, which is more useful in cases where the length of a
property value may vary between nodes.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 06:50:27 -04:00
Manivannan Sadhasivam 7ac767b880 scripts: dts: Match alpha numeric property values
For using alpha numeric property values in a devicetree node, we
need to match the values starts with a number. Current scenario will
return the value as a numeric literal if it starts with a number. This
will not work for a compatible like, "96b-ls-con" which is proposed in
issue #15598.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
2019-07-11 06:15:22 -04:00
Michael Scott 4ca54390f4 sanitycheck: handle west-flash dependency for device-testing
sanitycheck --help mentions that west-flash requires device-testing
to be enabled (and indeed it does because DeviceHandler will never
be called where west_flash option is used.  Let's generate an error
if west-flash is used w/o specifying device-testing.

Also cleanup help text which looks odd in both sanitycheck --help
and in the file itself.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Michael Scott 421ce46f4b sanitycheck: add west-runner parameter
Several boards have multiple runners setup.  We need a way to specify
which runner to use with sanitycheck.  Introduce --west-runner option.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Kumar Gala 8083f8a6b0 scripts/dts: deprecate DT_<COMPAT>_<INSTANCE ID>_BUS_<BUS>
We don't have any uses of this form of define so deprecate it for now.
If needed this should be DT_INST_<INSTANCE ID>_<COMPAT>_BUS_<BUS>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 06:24:39 -05:00
Kumar Gala 81072b53a6 dts/scripts: mark all non DT_ prefix deprecated
Now that we've converted LED and SW to use DT_ prefix we can mark the
non-DT_ prefixed versions as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala 5c78b93c6b scripts/dts: Mark 'non-S' forms of cell and controllers as deprecated
Now that we've converted all _GPIO_ to _GPIOS_ we can mark the _GPIO_
form as deprecated (same for _PWM_ / _PWMS_).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala e5d838081c dts: Fix handling of fixed-clock
A number of minor issues with the 'fixed-clock' support:
* Fix the #clock-cells to be 0
* Fix nxp_ke1xf.dtsi to set #clock-cells 0 and the clock reference to
  only be a phandle.
* Fix the generation script to only generate what it should for a
  'fixed-clock'

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:07 -05:00
Marc Herbert c7633de038 sanitycheck: fix priority of --extra-args=CONFIG_ over testcase.yaml
sanitycheck takes any "extra_config" list found in the testcase.yaml
file and generates an "overlay" file from it. This file is placed in
the per-test build directory and passed to cmake/kconfig.cmake through a
-DOVERLAY_CONF= option set in the (also) generated sanity-out/Makefile.

This commit moves this generated config overlay to a subdirectory one
level down from the build directory, otherwise kconfig.cmake picks it
up *twice*: once from the -DOVERLAY_CONF= option already mentioned above
and a second time because kconfig.cmake scans the build directory and
blindly picks up ALL files ending with .conf[*]. The second pickup is
problematic because kconfig.cmake currently gives it the top precedence,
even higher than anything the user espressed with --extra-args=CONFIG_*

Here's a quick and simple demonstration of the issue fixed by this
commit:

  cd $ZEPHYR_BASE/samples/net/sockets/net_mgmt/
  sanitycheck -T. -p qemu_x86 -b -v # --extra-args=CONFIG_USERSPACE=y|n
  grep CONFIG_USERSPACE $(find sanity-out/ -name .config)

  .net_mgmt.kernelmode/zephyr/.config: # CONFIG_USERSPACE is not set
    .net_mgmt.usermode/zephyr/.config: CONFIG_USERSPACE=y

  grep 'Merged configuration' $(find sanity-out/ -name build.log)

Without this commit, attemps to override anything with
--extra-args=CONFIG_ are silently dropped on the floor.

For more background this issue was found while using the recipe in
commit message 4afcc0f8af

[*] picking up all .conf files is debatable but a much bigger debate
with backward compatibility implications. This small fix makes
absolutely zero difference to anyone or anything not using sanitycheck.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-09 19:16:30 -07:00
Andrew Boie 49cf4867c7 sanitycheck: select gcov tool based on target
Use PATH gcov if no SDK in use, or native-posix derived
targets are in use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-09 10:20:31 +02:00
Wentong Wu 859ca42006 scripts: elf_helper: correct kernel object address check range
kernel object should not located in app_smem area.

Fixes: #17338.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-08 13:28:13 -07:00
Marc Herbert e5cedcae43 sanitycheck help: how to spot tests that were not run and just built
There are a few reasons why sanitycheck will only build a test and not
run it: list them in the developer guide. Also lists the --options that
provide that information and update their --help message.

A couple other --help fixes.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:07:57 -04:00
Marc Herbert d7ca903b72 sanitycheck: replace cmake -H "not meant for public use" with -S
https://www.mail-archive.com/cmake-developers@cmake.org/msg16769.html

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:04:39 -04:00
Sebastian Bøe b8d4a77ecd size_report: Don't crash when observing stripped symbols
When stripped symbols are present in the build size_report has crashed
due to it not being aware of this case.

This patch causes stripped symbols to be ignored instead, allowing
analysis of the non-stripped symbols.

It is left as future work to somehow display how much space stripped
symbols are taking up.

Stripped symbols can be present when third-party binaries are linked
into the build.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-07-05 14:18:49 -04:00
Marc Herbert 9e57338efd sanitycheck: fix --jobs help message + other cosmetic fixes
The --jobs default was recently changed in commit 9f4f57eed3, update
its help message.

Add the hopefully last missing verbose("Spawning...") statement.

Fix comment updated in commit 095b82a301.

Replace two tags with whitespace.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-03 23:04:16 -07:00
Andrew Boie 1578ef71b1 sanitycheck: workaround file limits
sanitycheck is opening an insane number of file descriptors
simultaneously as it opens up communication pipes with
every test that supports emulation, on every emulated
board target.

Increase the resource limit on open files until this code
can be properly refactored.

Workaround for: #17239

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-03 16:49:21 -04:00
Andrew Boie 8047a6f4d0 sanitycheck: make coverage reports easier to do
- The --gcov-tool argument now has a reasonable default
  if the Zephyr SDK is in use.
- --coverage-platform, if unspecified, defaults to what
  was passed to --platform
- --coverage implies --enable-slow, so that tests with
  the 'slow' tag are built and run.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:56:14 -04:00
Andrew Boie 095b82a301 sanitycheck: correct timeout extension comment
This wasn't discussing why there is extra time for when
code coverage is enabled.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:54:24 -04:00
Andrew Boie 9f4f57eed3 sanitycheck: don't multiply CPU count
We have a number of timing sensitive tests which run
correctly on a much more frequent basis if the system
is not so heavily loaded. Instead of squeezing a few
more crumbs of performance by doubling the CPU count,
just use the number of CPUs reported by the system.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:51:24 -04:00
Andrew Boie d62e229680 sanitycheck: extend coverage timeout
If GCOV coverage is enabled, the coverage dump happens after
"PROJECT EXECUTION SUCCESSFUL" is printed. In some cases,
the additional time added was not enough to capture all the
GCOV output on a heavily loaded system before the emulator
gets killed.

Ideally, the decision to kill the emulator needs to be smarter
and less race-prone, but that can wait for a future
enhancement.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 20:30:41 -04:00
Alberto Escolar Piedras b91f37494b sanitycheck: Minor optimization in BinaryHandler
Do not search for the valgrind binary (which) if valgrind
was not selected to be used.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Alberto Escolar Piedras 649368c018 sanitycheck: Record time used in BinaryHandler
So the time used to run boards which use the BinaryHandler can be
reported, record the time used from spawning the process until
it finnishes or is killed.
The BinaryHandler is used by the "native" boards, unit tests,
nsim and Renode.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Andrzej Głąbek bb8dddfa99 scripts: dts: Add extraction of clock frequency for fixed clock
This patch adds generation of `*_FIXED_CLOCK_FREQUENCY` macros for
clock consuming nodes that are provided with a fixed rate clock.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 09:45:35 -05:00
Andrzej Głąbek c782ba82ae scripts: dts: Add special handling of '+' in 'str_to_label' function
This patch is needed in order to get compilable macros for compatibles
like "arm,cortex-m0+".

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 07:32:01 -05:00
Marc Herbert eefea9da9c gen_app_partitions: deterministic _and_ decreasing app_smem_*.ld
Commit 212ec9a29a / feature #14121 already ordered partitions by
decreasing size, however it was common in samples/userspace/shared_mem/
/sample.kernel.memory_protection.shared_mem for two partitions to have
the same size and be randomly ordered between them. This adds the
partition name as a second sort key.

Unlike previous attempt in commit 725abdf430 this doesn't use the
partition name as the first (and only) key and doesn't break the
decreasing size order.  Huge thanks to Sigvart Hovland for spotting this
in a post-merge but prompt code review.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert 85bc0d2fe5 Revert "gen_app_partitions.py: make generated/app_smem_*.ld files deterministic"
This reverts commit 725abdf430 which did get rid of randomness in the
order of the partition _names_ as claimed but regressed commit
212ec9a29a / feature #14121 and broke the previous size order which I
missed. Huge thanks to Sigvart Hovland for spotting this in a post-merge
but prompt code review. Proper fix in the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert 725abdf430 gen_app_partitions.py: make generated/app_smem_*.ld files deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
in a row could lead to a different partition order, different
build/zephyr/include/generated/app_smem_*.ld files and different
binaries.

Fix with a minor change to the "for" loop in the output function:
make it iterate on sorted(partitions.items()) instead of the raw and
randomly ordered partitions dictionary.

It is easy to reproduce the issue even without downgrading to an
obsolete Python version; pick a test like samples/userspace/shared_mem/
and simply change the code to this:

--- a/scripts/gen_app_partitions.py
+++ b/scripts/gen_app_partitions.py
@@ -159,10 +159,12 @@ def parse_elf_file(partitions):
                     partitions[partition_name][SZ] += size

+import random
 def generate_final_linker(linker_file, partitions):
     string = linker_start_seq
     size_string = ''
-    for partition, item in sorted(partitions.items()):
+    for partition, item in sorted(partitions.items(),
+                                  key=lambda x: random.random()):
         string += data_template.format(partition)
         if LIB in item:
             for lib in item[LIB]:

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Marc Herbert 6ccd026f7c gen_relocate_app.py: make generated/linker_relocate.ld deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
${ZEPHYR_BASE}/samples/application_development/code_relocation/
in a row could lead to a different sections order, different
build/zephyr/include/generated/linker_relocate.d and code_relocation.c
and different binaries.

Fix with a minor change to three "for" loops in the output functions:
make them iterate on sorted(list of sections) instead of the raw and
randomly ordered dictionaries of sections.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Carles Cufi 41f1f648f6 west: runners: Guess build folder
When using a build folder format with build.dir-fmt that includes any
parameters that need resolving, the west runners cannot find the folder
since the required information (board, source dir or app) is not
available.
Add a very simple heuristic to support the case where a build folder
starts with a hardcoded prefix (for example 'build/') and a single build
is present under that prefix.
The heuristic is gated behind a new configuration option:
build.guess-dir

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 11:50:26 -04:00
Carles Cufi 3b3bbc1ee0 Bluetooth: hci_uart: Use DT for the hci_uart UART
Use Device Tree,and in particular a new 'bt-c2h-uart' to select which
UART is being used to communicate with an external BLE Host when acting
as a Controller.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 12:37:04 +02:00
Marti Bolivar 814bc7ba8e scripts: remove runner related cruft
There's some old cache variables that don't matter anymore in here.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar f08935fa73 scripts: west_commands: improve run_common error handling
Print a friendlier error message on ValueError, but don't throw away
the stack trace.

Move another call to log.die().

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar e6873b8d7e scripts: tweak west build output
Follow along with changes made in west flash/debug/etc to make it
easier to see the output steps visually.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar ec8dbf34dc scripts: tweak runner-related command output
Having common log handlers now lets us improve our logging output so
that info messages are prefixed with the runner they came from, and
doing something similar with the high level steps as we go, like this:

   -- west <command>: using runners
   -- runners.RUNNER_NAME: doing something
   <output from RUNNER_NAME subprocesses go here>
   -- runners.RUNNER_NAME: all done, bye

We can also colorize the west output to make it stand out better from
subprocesses, using the same output formatting style that west
commands like west list do.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar ddce583ca2 scripts: west_commands: decouple runners pkg from west
I've had some requests to be able to use code in the runners package
without having west installed.

It turns out to be pretty easy to make this happen, as west is
currently only used for west.log and some trivial helper methods:

- To replace west log, use the standard logging module
- Add an appropriate handler for each runner's logger in
  run_common.py which delegates to west.log, to keep
  output working as expected.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar d176cc36ad scripts: runners: update ZephyrBinaryRunner docstring
Fix some things that are incorrect and try to make it more clear.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar 6440514bec scripts: west_commands: fix help string
The option name is wrong.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar 725e98e300 scripts: fix pyocd runner --flash-opt help
This doesn't work. Fix the example.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-25 22:33:02 +02:00
Kumar Gala 95f28eb764 scripts/dts: Remove DT_CHOSEN generation
We never used DT_CHOSEN so lets remove this and add it back if/when
there's an actual need for it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:58 -05:00
Kumar Gala 08a5f9f8eb scripts/dts: Add ability to mark cell/ctrl defines as deprecated
Add support so that we can flag any "defines" associated with a call to
either extract_cells or extract_controller as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:35 -05:00
Kumar Gala 79b59e8205 scripts/dts: Remove 'use-prop-name' from clock generation
We never set 'use-prop-name' on clock bindings so lets just always
use CLOCK_CONTROLLER as the define name we generate.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:42:31 -05:00
Kumar Gala 593d628e32 dts/extract: Remove 'use-prop-name' from cell and controller handling
We have 'use-prop-name' flag in the bindings which is specifically used
for GPIO properties to control if we get "GPIO" or "GPIOS" as the
generated define name.

Lets remove the inconsistancy and use "GPIOS" as the preferred name as
this matches the DTS property name.  Towards that we will generate both
forms and remove support for 'use-prop-name'.

This also impacts "PWM" generation.  So we'll have "PWM" and "PWMS"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:40:33 -05:00
Jukka Rissanen c9aaab74db net: ptp: clock: Add usermode support to ptp_clock_get()
It is useful that the ptp_clock_get() function can be called from
the userspace. Create also unit test for calling that function
from userspace.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-06-25 15:22:51 +03:00
Wentong Wu 5611e92347 kernel: add futex support
A k_futex is a lightweight mutual exclusion primitive designed
to minimize kernel involvement. Uncontended operation relies
only on atomic access to shared memory. k_futex structure lives
in application memory. And when using futexes, the majority of
the synchronization operations are performed in user mode. A
user-mode thread employs the futex wait system call only when
it is likely that the program has to block for a longer time
until the condition becomes true. When the condition comes true,
futex wake operation will be used to wake up one or more threads
waiting on that futex.

This patch implements two futex operations: k_futex_wait and
k_futex_wake. For k_futex_wait, the comparison with the expected
value, and starting to sleep are performed atomically to prevent
lost wake-ups. If different context changed futex's value after
the calling use-mode thread decided to block himself based on
the old value, the comparison will help observing the value
change and will not start to sleep. And for k_futex_wake, it
will wake at most num_waiters of the waiters that are sleeping
on that futex. But no guarantees are made on which threads are
woken, that means scheduling priority is not taken into
consideration.

Fixes: #14493.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-24 15:38:21 -07:00
Ulf Magnusson a570b40252 genrest: De-spam docs by skipping direct deps. in more places
Similar deal to commit cc14c40a2d ("kconfiglib: Unclutter symbol
strings, avoid redundant writes, misc.").

Hide the direct dependencies in the defaults, selects, and implies
sections. Do the same in menuconfig/guiconfig as well.

This uses a new Kconfiglib API, so update Kconfiglib to upstream
revision 164ef007a8. This also includes some minor optimizations and
cleanups.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-24 12:14:06 -07:00
Ioannis Glaropoulos 88959e75dd scripts: gen_priv_stacks.py: use the right alignment for priv stacks
The privilege stacks are not sandboxed inside an MPU region,
so they do not have to be aligned with the stack buffer size.
We fix this by using the PRIVILEGE_STACK_ALIGN macro, which
is defined in arch.h and reflects the minimum alignment
requirement for privilege stack buffers.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-06-24 10:16:57 -07:00
Kumar Gala 74818bcb49 scripts/dts: Treat array always as a list
If the type of property is a 'array' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-24 07:16:59 -04:00
Kumar Gala 344737d759 scripts/dts: Treat string-list always as a list
If the type of property is a 'string-list' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-22 10:03:17 -05:00
Marc Herbert 35dc96399c sanitycheck: show handler_time in -v output (qemu 2.032s)
We already have the info so let's show it. This helps spots intermittent
issues[*], gives an indication of the time --build-only saves, can help
spot an overloaded test system, highlights the most time-consuming tests
which may need a longer timeout in their config, shows the effective
timeout value when one occurs... all this for a dirt cheap screen estate
price and two extra lines of code.

Sample -v output:

32/81 board123  tests/testme              PASSED (qemu 2.049s)
33/81 board456  samples/hello             PASSED (build)
34/81 qemu_x3   tests/kernel.stack.usage  FAILED: timeout (qemu 60.029s)
     see: sanity-out/qemu_x3/tests/kernel.stack.usage/handler.log
35/81 board456  tests/testme              PASSED (build)
36/81 qemu_x5   tests/kernel.queue        FAILED: failed (qemu 2.191s)
     see: sanity-out/qemu_x5/tests/kernel.queue/handler.log

[*] running qemu in heavily packed cloud virtual machines comes to mind,
    also see #12553, #14173 etc.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-21 13:47:32 -04:00
Kumar Gala bf0f6d911d dts: Add concept of sub-nodes to YAML and generator
Several bindings have an expectation of sub-nodes that describe the
actual infomation.  The sub-nodes don't have any compatiable so we can't
key on that.

So we can add the concept of a sub-node to the YAML to handle cases like
'gpio-keys', 'gpio-leds', 'pwm-leds', etc..

The sub-node in the YAML is effective the "binding" params that describe
what properties should exist in the sub-node.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 15:34:28 +02:00
Kumar Gala fe2062b38b dts/extract: deprecate old IRQ alias define generation
Make the old IRQ aliases that where missing the _0, _1, etc. part as
deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Kumar Gala 7b638bf3f6 dts/extract: Fix IRQ alias define generation
The alias generation wasn't doing the right thing with regards to
keeping the names consistent.  We would drop the index from the define
name for aliases.

So we'd get
	DT_NXP_KINETIS_GPIO_GPIO_D_IRQ
which should be
	DT_NXP_KINETIS_GPIO_GPIO_D_IRQ_0

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Kumar Gala 6a79bf2571 dts/extract: Add ability to mark aliases as deprecated
Add a deprecate flag to add_prop_aliases so we can make the aliases it
generates as deprecated if needed.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Anas Nashif cbfba62182 ci: increase sleep time between sanitycheck runs
Give the system more time to settle before we try to re-run the tests
again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-20 22:15:27 +02:00
Ulf Magnusson e1f0b61d23 scripts/dts: Clarify extract/flash.py some more
Think I understand it now, and that was the goal.

 - _extract_partition() adds index-based entries. extract_partition()
   adds label-based entries.

   Rename them to _add_partition_index_entries() and
   _add_partition_label_entries(), and call them from a top-level
   extract_partition() function.

   This makes the logic clearer. It took me a long time to spot it.

 - Generate indicies with a simple counter and remove the _flash_area
   logic. This would break if partitions were extracted more than once,
   but they aren't, and now you know that they can't be.

 - Rename _create_legacy_label() to add_legacy_alias() and make it
   global. It doesn't use 'self'.

 - Break the logic for finding the flash controller into a separate
   helper function

 - Add doc-comments for the new functions

 - Misc. other small clean-ups

generated_dts_board.conf and generated_dts_board_unfixed.h were verified
to be identical for disco_l475_iot1 and frdm_kw41z before and after the
cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-20 11:35:05 +02:00
Anas Nashif f2cb20c772 docs: fix misspelling across the tree
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-19 15:34:13 -05:00
Andy Ross 5efdd6a525 sanitycheck harness: Correct ordered regex handling
The way sanitycheck did its ordered regexes is that it would test
every regex against every line, and store the matching lines and their
regexes in an OrderedDict and check that they happened in the right
order.

That's wrong, because it disallows matching against a line that
previously appeared (and should have been ignored) in the input
stream.  The watchdog sample is the best illustration: the first boot
will (by definition) contain all the output already, but the regex has
to match against a line from the SECOND boot and not the same one it
saw earlier.

Do this the simple way: keep a counter of which regex we're trying to
apply next and increment it on a match.  This is faster too as we only
need to check one pattern per line.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-06-19 14:37:20 -04:00
Tyler Hall bb359257bf scripts/sanitycheck: fix ZEPHYR_BASE subdir check edge case
There is a case where using startswith to determine if a path is a
subdirectory of another path can erroneously match. When using a
testcase root outside of ZEPHYR_BASE, an erroneous match will cause the
relative path containing ".." to get prepended to the test output
directory.

Example:

$HOME/zephyr/zephyr # ZEPHYR_BASE
$HOME/zephyr/zephyr-rust/tests # testcase root

The relative path prepended to the testcase name is ../zephyr-rust/tests
and an example test output dir is
./sanity-out/qemu_x86/../zephyr-rust/tests/rust/rust.main

In this case, the build directory escapes the board directory and is no
longer unique. Parallel tests then clobber each other.

Use pathlib instead of string matching to cover this case.

Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
2019-06-19 13:56:26 -04:00
Marc Herbert 6f98db61f1 generated/cfb_font_dice.h: don't leak absolute paths in comment
The start of generated/cfb_font_dice.h looked like this:

/*
 * This file was automatically generated using the following command:
 * /home/john/zephyrproject/zephyr/scripts/gen_cfb_font_header.py
 * --input fonts/dice.png --output
 * /home/john/tmp/build/zephyr/include/generated//cfb_font_dice.h
 * --width 32 --height 32 --first 49 --last 54
 */

For build reproduction and "privacy" reasons, change it to this:

/*
 * This file was automatically generated using the following command:
 * ${ZEPHYR_BASE}/scripts/gen_cfb_font_header.py
 * --input fonts/dice.png --output
 * zephyr/include/generated//cfb_font_dice.h
 * --width 32 --height 32 --first 49 --last 54
 */

Test with:
  sanitycheck  -p reel_board \
  -T $ZEPHYR_BASE/samples/display/cfb_custom_font/

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-19 08:40:59 -04:00
Erwan Gouriou 8a9ca0d864 scripts: Upgrade pyocd for pack support
pyocd 0.21.0 provides pack support 'pack support' functionality,
as opposed to current 'buitlin support'.
This new feature enables the possibility to add pyocd support
for any chip that is documented in Keil database. Then one doesn't
need anymore to wait pyocd is updated with a new target to use
pyocd with his target, as long as it is populated in Keil database.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-06-18 23:03:35 -04:00
Kumar Gala 84cf9dc671 scripts/sanitycheck: Add --cmake-only option
Add an option that only invokes the cmake phase of sanitycheck.  This
can be useful for any testing that only needs to initial generation
phase of cmake, for example device tree.  Also useful if we want to
just generate compile_commands.json files from cmake via:

./sanitycheck -xCMAKE_EXPORT_COMPILE_COMMANDS=1 --cmake-only

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-18 12:02:03 -04:00
Ioannis Glaropoulos a4afe8c0fa scripts: gen_priv_stacks: remove unused variable
Variable priv_stack_decl_size is not used by the script,
so we can remove it.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-06-17 12:33:58 -04:00
Marti Bolivar c24e044c6e scripts: runners: check for required programs
Add self.require() checks before running commands. Increase test
coverage, including for this feature, while we are here.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-17 15:06:21 +02:00
Marti Bolivar c07267a26a scripts: runners: abstract jlink's missing program support
The runners/jlink.py script has a mechanism for erroring out if a host
tool is not installed. Abstract it into runners/core.py and handle it
from run_common.py. This will let it be used in more places.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-17 15:06:21 +02:00
Anas Nashif 49b22d4fc3 sanitycheck: report tests that never build/run
Add new option --report-excluded to list all those tests with bad
filtering that never build or run. This option produces accurate results
with --all but can be used with default sanitycheck options to see what
does not run/build in CI for example. (limited coverage).

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-16 14:13:25 -04:00
Marti Bolivar d371c54f51 scripts: west sign: test for input file existence
Check the CONFIG_BUILD_OUTPUT_HEX and CONFIG_BUILD_OUTPUT_BIN options
are enabled before attempting to build signed versions of these formats.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar 2f839da163 scripts: west sign: improve user friendliness
Add more error handling and warnings. Doing this nicely requires a bit
of re-work to the control flow.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar 33cbba3457 scripts: west sign: move hex and bin default locations
When signing binaries from multiple build directories, it is
inconvenient to have to specify the output file locations by hand each
time. It's also a little weird that they're not next to zephyr.bin and
zephyr.hex.

Move them to the build directory, next to their unsigned variants.

Suggested by Piotr Mienkowski.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar 06c9f8e9e6 scripts: west sign: code style refactor
Let's leave self.args as the actual parsed argument namespace.
Pass the final computed build directory separately.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar 3bd07a244a scripts: west sign: make it work again
This got broken in the patches which added the build.dir-fmt config
option when BUiLD_DIR_DESCRIPTION was renamed.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar fe91e48418 scripts: runners: add __contains__ to BuildConfiguration
Implement the in operator for this class.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Ulf Magnusson 045c809561 scripts/dts: Make extract/flash.py easier to read
Clean up the code a bit:

 - Simplify the loops over the flash 'reg' properties by using range()

 - Build identifier names with a plain .format() where possible. This
   makes them stand out better in the code.

 - Remove redundant variables

 - Move variables close to where they're used

 - Misc. other minor improvements

generated_dts_board.conf and generated_dts_board_unfixed.h were verified
to be identical for disco_l475_iot1 and frdm_kw41z before and after the
cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-16 10:10:06 -04:00
Piotr Mienkowski 9ac4dc6631 scripts/dts: generate DT_ALIAS_<ALIAS>_<PROP> defines
DT_ALIAS_<ALIAS>_<PROP> defines are a convenient and portable way to get
the device instance name despite different naming conventions used by
the device drivers.

Signed-off-by: Piotr Mienkowski <piotr.mienkowski@gmail.com>
2019-06-15 11:15:26 -05:00
Anas Nashif b58aa20e13 ci: skip handling coverage data
We not analysing coverage data at all in CI right now. Disable this
while we figure out a better solution for reporting data.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-14 09:26:40 -04:00
Kumar Gala 01e54a5472 scripts/dts: Change DT_<COMPAT>_<INSTANCE>_<PROP> define
Its possible that the <INSTANCE> number could conflict with the register
number.  This is shown to happen for a device like soc-nv-flash at
address 0.

So change naming convention to DT_INST_<INSTANCE>_<COMPAT>_<PROP> and
make DT_<COMPAT>_<INSTANCE>_<PROP> as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-14 08:02:15 -05:00
Kumar Gala 062bd0656a scripts/dts: Sort instance IDs by reg addr
Because of how generate defines for instances its possible that we have
a name conflict if the instance ID and reg addr space clash.

For example on qemu_x86 there are current two 'soc-nv-flash' nodes and
one is at reg addr 0, but instance id 1, the other is reg addr 0x1000
and instance id 0.  We'd possibly get this conflict:

For the 'soc-nv-flash' at 0x1000 (instance 0):
	(instance define)
	#define DT_SOC_NV_FLASH_0_BASE_ADDRESS          0x1000

For the 'soc-nv-flash' at 0x0 (instance 1):
	(address define)
	#define DT_SOC_NV_FLASH_0_BASE_ADDRESS          0x0

To deal with this we make sure that the lower reg address is instance 0,
than things work out ok.  To handle this case, if we sort the instance
IDs based on reg addr than if we have something at reg addr 0, it will
also than be an instand ID 0.

The longer term solution will be to deprecated the old defines and
remove the conflict between instance ID defines and normal DT defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-13 14:11:13 -05:00
Marti Bolivar 8718d5c84b scripts: west: improve error message on missing cache
In the west flash/debug commands, if the user gives an invalid build
directory, they'll get a stack trace instead of a helpful error
message when the cache can't be built.

Fix that.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-10 17:46:27 -05:00
Carles Cufi 98980c6b7c west: build: Configurable build folder format
Add the possibility of configuring the build folder format in west's
configuration system.
The build.dir-fmt configuration option controls how west will create
build folders when not specified, the following parameters are currently
accepted:

- board: The board name
- source_dir: The relative path from CWD to the source directory
- app: The name of the source directory

If CWD is below source_dir in the directory hierarchy then source_dir is
set to an empty string.

This means that if one sets:

[build]
dir-fmt = build/{board}/{source_dir}

Then when building samples/hello_world from zephyr's root for the
reel_board the build folder will be:

./build/reel_board/samples/hello_world

but when building it from inside the samples/hello_world folder it will
instead be:

./build/reel_board

Fixes https://github.com/zephyrproject-rtos/west/issues/124

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-11 00:00:34 +02:00
Carles Cufi 49c4b1c303 west: Use find_build_dir in run_common
In preparation for upcoming changes to the way the default build folder
is defined, switch to using the common find_build_dir() function in the
runners.
This actually changes the behavior for the west build command slightly,
since the current working directory (cwd) will now be checked after the
default build folder ('build'). This brings it in line with what is
used for the runners.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-11 00:00:34 +02:00
Andrew Boie a42313d488 requirements.txt: add gcovr
This tool is cited in our documentation for producing coverage
reports, add it to the list of packages pulled in by pip during
workstation setup.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-06-07 14:15:16 -07:00
Kumar Gala 1c55882ae3 dts/extract: Remove bogus pinctrl generation
The pinctrl generation has never done anything useful so lets remove it
as its just dead code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-07 10:35:58 -06:00
Ulf Magnusson f46ebc3c97 kconfig.py: Use messages returned by load_config() and write_config()
load_config() and write_config() now return a message to print. This
message also says whether the configuration was loaded (replace=True) or
merged (replace=False), and whether the new .config is different from
the old (for write_config()).

Print the returned messages and remove some old print()s.

Also switch to an improved warning control API (the old one is still
supported, but might as well).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-07 18:20:22 +02:00
Ulf Magnusson cc14c40a2d kconfiglib: Unclutter symbol strings, avoid redundant writes, misc.
Update kconfiglib, menuconfig, and guiconfig to upstream revision
5c904f4549 to get various improvements and fixes in:

 - Marc Herbert found an issue involving symlinks, absolute paths,
   and rsource that could lead to files not being found. The root cause
   was relpath() assuming that symlink/../bar is the same as bar/, which
   isn't guaranteed.

   Fix it by handling paths in a simpler, more textual way.

 - Propagated dependencies from 'depends on' are now stripped from
   properties when symbols are printed (e.g. in information dialogs and
   generated documentation).

   The printed representation now also uses shorthands.

   Before:

     config A
             bool
             prompt "foo" if C && D
             default A if B && C && D
             depends on C && D

   After:

     config A
             bool "foo"
             default A if B
             depends on C && D

 - Before writing a configuration file or header, Kconfiglib now
   compares the previous contents of the file against the new contents,
   and skips the write if there's no change. This avoids updating the
   modification time, and can save work.

   A message like "No change to '.config'" is shown when there's no
   change.

 - .config now has '# end of <menu>' comments to make it easier to see
   where a menu ends. This was taken from a change to the C tools.

 - load_config() and write_(min_)config() now return a message that can
   be printed with print(kconf.load_config()). This allows messages to
   be reused in e.g. the configuration interfaces (nice now that there's
   also a "No change to..." string).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-07 18:20:22 +02:00
Alexey Brodkin bceaed67b3 simulation: ARC EM in nSIM to use nsim_em.props
This matches a naming scheme used for other flavour of ARC EM
SoC flavors.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
2019-06-06 14:20:42 -04:00
Kumar Gala 442df97366 scripts/dts: Remove alias defines for labels
We added generation of aliases for "alt-label" (which was the outer
label of a node) for use with shields and connectors.  However we've
never used these defines and the generation is a bit inconsistent.

This removes generation of defines like for label 'arduino_spi':
	#define ARDUINO_SPI_BASE_ADDRESS ... (already deprecated)
	#define DT_ST_STM32_SPI_FIFO_ARDUINO_SPI_BASE_ADDRESS ...

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-06 09:08:16 -06:00
Nathaniel Graff bd2919dcf1 runners/jlink: Add --tool-opt option for more args
Adds the --tool-opt option to allow additional arguments to be
passed to JLink Commander

Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2019-05-30 16:54:02 -04:00
Ulf Magnusson a1c3cc6660 guiconfig: Add a graphical configuration interface
This is a graphical configuration interface written in Tkinter. Like
menuconfig.py, it supports showing all symbols (with invisible symbols
in red) and jumping directly to symbols. Symbol values can also be
changed directly from the jump-to dialog.

This interface should feel a lot smoother than menuconfig.py on Windows.

When single-menu mode is enabled, a single menu is shown at a time, like
in the terminal menuconfig. Only this mode distinguishes between symbols
defined with 'config' and symbols defined with 'menuconfig'.

Compatible with both Python 2 and Python 3. Has been tested on X11,
Windows, and macOS.

To avoid having to carry around a bunch of GIFs, the image data is
embedded in guiconfig.py. To use separate GIF files instead, change
_USE_EMBEDDED_IMAGES to False. The image files can be found in
https://github.com/ulfalizer/Kconfiglib/tree/screenshots/guiconfig.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-30 09:54:06 -04:00
Ulf Magnusson 6cfc13522c menuconfig: Small Space/Enter improvement + fix for obscure bug
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
7e0b9f7ae1, to get these commits in:

  menuconfig: Improve space/enter behavior slightly

  Space toggles value is possible, and enters menus otherwise. Enter
  works the other way around.

  Make this explicit in the code, which also fixes some corner cases,
  like space doing nothing on a y-selected menuconfig symbol.

  -------------------------------------------------------------------

  menuconfig: Fix display issue for unsatisfied-deps selected symbol
  with children

  A symbol with unsatisfied direct dependencies can end up with visible
  children in an implicit submenu if it is selected (though that
  generates a warning), so the optimization in _shown_nodes() isn't
  safe, and causes the child nodes to not be shown outside show-all
  mode.

  Just remove the optimization. Trying things out some more,
  everything's plenty fast enough anyway.

  Checking the direct dependencies of the parent instead would be safe.

The menu path now says "(Top)" instead of "(top menu)" too, which is a
bit more compact. Make the same change in genrest.py.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-30 09:54:06 -04:00
Luiz Augusto von Dentz a3bea8872b Bluetooth: L2CAP: Move fixed channels to its own section in ROM
This changes the declaration of fixed channels to be statically defined
with use of BT_L2CAP_CHANNEL_DEFINE since fixed channels are never
unregistered.

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2019-05-29 16:31:03 +03:00
Maureen Helm 4de99994f0 scripts: west: Add reset-after-load argument to jlink runner
Adds a new argument to the jlink runner to reset the device after
loading code to flash. This fixes a problem with the lpcxpresso54114
board where it was necessary to manually reset the board to get new code
to start running after the 'ninja flash' command. This new argument is
optional and false by default because there are some cases were we must
not reset after load, such as when we load the application into ITCM on
imx rt devices.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-25 10:01:30 -04:00
Marti Bolivar 8b9b7e7c8a scripts: avoid unnecessary 'west build --force'
Commit 88fb8bacfb ("scripts: improve west build's board handling")
lets us specify the board with a build.board config or BOARD
environment variable.

However, under some conditions, e.g. if the use has
build.pristine=auto and build.board=some_board, the following fails a
check_force call:

west build samples/hello_world
west build samples/philosophers

The problem is that the check_force wasn't made aware of the other
places a board can come from. Fix that.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-24 11:07:18 +02:00
Jan Van Winkel 6b9e160b5a sanitycheck: Execute binary in output directory
Execute the test binary from the output directory instead of directory
where sanitycheck was started.

This will ensure that any artifact created with a relative path by the
test binary will be placed in the output directory instead of creating
the artifact in the directory where sanitycheck was executed and prevent
any possible conflicts.

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-05-23 08:37:09 -04:00
Kumar Gala 0d4290a2cb scripts/dts: Fix issue in label generation of registers
If we have something like:

       #address-cells = <1>;
       #size-cells = <0>;

       intc: ioapic@fec00000  {
	       compatible = "intel,ioapic";
	       reg = <0xfec00000 0x100000>;
       };

We should generate:

DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_0
DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_1

Instead we generated:

DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS
DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_1

This was due to logic deciding if '_0' should be used not taking into
account #address-cells & #size-cells correctly.

Fixes: #16296

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-22 17:27:53 -05:00
Antony Pavlov 8245195306 dts: import vendor-prefixes.txt from linux kernel
import linux-v5.1:Documentation/devicetree/bindings/vendor-prefixes.txt

Use vendor-prefixes.txt to check vendor prefixes
used in compatible strings and property names.

Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
2019-05-22 12:40:20 -04:00
Antony Pavlov f08148e640 scripts/checkpatch.pl: fix root dir detection and enable it
Don't try to find files specific for linux kernel.

Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
2019-05-22 12:40:20 -04:00
Wentong Wu c8f43b9c25 scripts: make code relocation work for non-XIP system
on non-XIP system, loading address shouldn't be located on
FLASH memory region.

Fixes: #16090.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-05-21 15:28:59 -04:00
Wentong Wu 743a184b2d scripts: mpu align for ro section of new memory region on non-XIP system
on non-XIP system, SRAM is the default region, and relocated .data
section and .bss section of SRAM shouldn't be inserted between
_image_rom_start and _image_rom_end, because the memory region between
_image_rom_start and _image_rom_end will construct the mpu ro region.
Also for the newly added memory region on non-XIP system, the
relocated .text secition and .rodata section should also be mpu aligned.

Fixes: #16090.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-05-21 15:28:59 -04:00
Kumar Gala fba8ac988c scripts/dts: Remove alias defines for bus parents with an alias
We generated some alias defines for children of a bus in which we had a
path alias for the bus node.  We never used these defines, we don't
recommend they get used (child of busses should use instance defines)
and they were only generated in small handful of cases (for dts that had
path aliases to the bus node - i2c or spi).

Remove this as effectively dead code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-21 09:20:31 -04:00
Marc Herbert 75276a7d97 sanitycheck: when present, point at handler.log instead of run.log
When Zephyr crashes immediately QEMU reports an error immediately. This
is immediately reported by "make run". Then sanitycheck points the user
at the output of "make run". However the error message(s) are in QEMU's
output which is in a different .log file.

To address this situation point the error message at handler.log
instead of run.log if and only if handler.log is not empty.

To reproduce here's an artificial but very simple crash:

  sanitycheck --extra-args=CONFIG_TEST_USERSPACE=n \
    -p qemu_x86 -T tests/kernel/mem_protect/stackprot/

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-21 08:28:07 -04:00
Kumar Gala 983b493ac1 scripts/dts: Add support for enum's
Add specific enum generation support related to usb 'maximum-speed'
property.  This will generate a define with _ENUM with the integer
value of the enum as its ordered in the YAML.  The assumption right
now is that there's a matching enum in the code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-17 09:53:20 -05:00
Øyvind Rønningstad 79d2e31cff scripts: mergehex: Add --overlap argument
Allows specifying the 'overlap' argument in IntelHex::merge().
This is identical to the --overlap argument in hexmerge.py, which
is bundled with IntelHex.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2019-05-16 07:16:55 -05:00
Andy Doan cbecaddff1 sanitycheck: Add a "test-only" option
This makes it easy to re-run tests from an existing build directory
w/o trying to rebuild artifacts.

Signed-off-by: Andy Doan <andy@foundries.io>
2019-05-16 07:08:01 -05:00
Andy Doan 79c48849fe sanitycheck: Allow custom arguments for west-flash
This allows you to use west to flash the device and pass custom
arguments to the flash command.

Signed-off-by: Andy Doan <andy@foundries.io>
2019-05-16 07:08:01 -05:00
Marti Bolivar b3674f596b scripts: fix west TypeError on unknown arguments
Let's not mess with CommandContextError here, as the APIs have gotten
messed around a bit in various versions. Just use log.die() as that
will work with current and future west versions, and is clearer anyway.

Fixes west 247.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-16 14:00:53 +02:00
Jukka Rissanen 89bf1578d9 net: sockets: Add a way to register a socket family handler
Allow automatic handling of registered socket families.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-05-10 22:43:27 +03:00
Carles Cufi 66fdd48fdd west: Add completion command
Add a completion command that dumps the contents of a shell
completion file present in the zephyr repository.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-10 13:08:27 +02:00
Carles Cufi f43d306527 west: Add bash completion file
This file was previously located in the west repository, under scripts/.
Since it now includes knowledge about specific behavior ef zephyr
extension commands, we move it here after overhauling it completely.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-10 13:08:27 +02:00
Marc Herbert 1c8632cfaa sanitycheck: support symbolic links in ZEPHYR_BASE
Fix issue where sanitycheck wrongly assumed tests inside ZEPHYR_BASE
to be outside ZEPHYR_BASE and dropped the prefix in their name. This
happened when:
- ZEPHYR_BASE contains symbolic link(s), and
- relative --testcase-root argument(s) are passed

To generate unique names, TestCase.get_unique(testcase_root) first
checks whether "testcase_root" starts with ZEPHYR_BASE. Either may or
may not include symbolic links so both must be canonicalized before
comparison. While fixing this method, replace explicit forward slash
"/" and string replace with os.path.relpath() and make a couple other
simplifications and minor pydoc fixes.

Add new canonical_zephyr_base = os.path.realpath(ZEPHYR_BASE) constant
and corresponding comments and guidelines.

The most visible effect of this mismatch was sanitycheck dropping the
--testcase-root prefix from the unique name of tests inside
ZEPHYR_BASE. This means some test names could be not unique anymore
and silently overwrite each other's results, example:

 bash# cd zephyr_dir_with_symlink; export ZEPHYR_BASE=$(pwd)
 ./scripts/sanitycheck -T samples/portability/cmsis_rtos_v1 \
                       -T samples/portability/cmsis_rtos_v2

The more systematic and practical consequence (and how I actually
found this) was test outputs landing in unexpected locations.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-07 23:19:35 -04:00
Henrik Brix Andersen f9dd53624c scripts/dts: allow node to be child on one bus and parent on another
Allow a device tree node to be child on one bus and parent on another
bus (e.g. an I2C slave device with multiple sub-devices).

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2019-05-07 22:33:06 -04:00
Erwan Gouriou 1e314df89a scripts/dts: globals: parse gpio-map properties for connector nodes
gpio-map is a property of "nexus node", defined in dts v0.3.
It allows to describe a pin connector so it can be referenced
through phandles and hence used in expansion device nodes like a
shield header (typically implemented through overlays).

This change implements gpio controller resolution through these maps.
Few assumptions were taken in order to simplify the implementation.
These assumptions bring some limitations to the use of gpio-map
but my understanding is that this should still allow to cover most
use cases.

Assumptions:
-gpio-size is the same for all gpio-controllers referenced in a map
-optional properties gpio-map-mask and gpio-map-pass-thru are
supposed to be omitted
The understanding of this last assumption is that flags provided in
the expansion device node will overwrite the connector flags.

In a latter stage, when need happen, these limitations can be
revisited to unlock fully fledged gpio-map usage.

Fixes #15637

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-05-07 22:16:42 -04:00
Ulf Magnusson 9d3435692d scripts: gen_kobject_list: Remove redundant parentheses
Making a clean slate for a pylint test in CI.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-07 08:46:41 -07:00
Marti Bolivar 0396b6e64c scripts: west build: propagate verbosity to cmake generator
When run as "west -v build", make sure that the underlying build tool
is run in verbose mode as well (if the generator is known to support
it, which is the case for Unix Makefiles and Ninja based generators).

The per-generator hacks here are needed to support CMake 3.13. If we
move to CMake 3.14 or later, we can just run "cmake --build BUILD -v"
and be done with it.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-07 15:49:16 +02:00
Jun Li 2d5fb6da4c west: fix import error on sign.py
Fix broken imports caused by changes
from its dependencies.

Signed-off-by: Jun Li <jun.r.li@intel.com>
2019-05-07 08:36:29 -04:00
Ulf Magnusson 859c4ed2cd west: Fix unused import and variable
Making a clean slate for a pylint test in CI.

'_' is a common name for non-problematic unused variables in Python.
pylint knows not to flag it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-07 08:12:34 -04:00
Marti Bolivar 83a5fd8d4c scripts: make west build -h fit on a screen
The west build --help output no longer fits in a single page.  Move
details and examples into the documentation, so the -h output doesn't
require scrolling around.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 3a486a85bc scripts: add --build-opt option to west build
This allows passing arguments to the underlying build tool (ninja,
make, etc.).

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 8465cf25c8 scripts: add --dry-run flags to west build
Analogously to the Make options with the same names, these print the
commands which would have run without running them.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar a1ef6962ae scripts: fix up west build verbosity levels
Adjust them so "west build -v" prints ZEPHYR_BASE and any CMake
commands, but none of the other more esoteric bits of information.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar bbe890ab30 scripts: add build.generator config option for west build
This can be used to override the default CMake generator
permanently. Its values are the same as those acceptable to cmake's -G
option.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 1f5e6d897f scripts: add --cmake-only to west build
This forces CMake to run, but skips running any build system target
afterwards.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 88fb8bacfb scripts: improve west build's board handling
- Respect the BOARD environment setting.
- Don't require --force if the board can't be figured out: it might be
  set in CMakeLists.txt, for example. Instead, downgrade to a warning
  which can be disabled with "west config build.board_warn false".
- Add a build.board configuration option used as a final BOARD fallback
  after CACHED_BOARD (in the CMake cache), --board (command line), and
  BOARD (environment).
- Keep the config docs up to date.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar d15950353d scripts: add helpers to west build for getting config options
These will be used more in a later patch.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 69099e3d99 scripts: flake8 cleanup for west build
Clean up coding nits to match west's style.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar acda25716a west: build: use CMAKE_HOME_DIRECTORY to locate source
Rather than relying on APPLICATION_SOURCE_DIR to be in the cache, just
use CMAKE_HOME_DIRECTORY instead.

https://cmake.org/cmake/help/latest/variable/CMAKE_HOME_DIRECTORY.html

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar 4dbf5f1535 scripts: fix and tweak west boards
- This script didn't get fixed when cmake.py was renamed zcmake, so it
  won't run; fix that.
- Change the default format string to '{name}' to keep things simple
- Flake8 lint

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Maureen Helm fe4b1ff0c3 boards: rv32m1_vega: Use different openocd config file for each core
Adds a new argument to the openocd runner to optionally specify the
config file. Updates the rv32m1_vega board to use different openocd
config files for the ri5cy and zero-riscy cores.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-06 14:52:17 -05:00
Carles Cufi 851fa12d51 west: boards: Use the new zcmake module name
Switch to the recently renamed zcmake module when importing the
run_cmake function.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-05 12:58:22 +02:00
Marc Herbert af1090c3bd sanitycheck: clarify who spawns processes and threads and where
Notably fix the wrong comment I added in commit 6f011c95c4: when
testing with QEmu sanitycheck does _not_ spawn QEmu; it relies on "make
run" instead.

Searching the code for "Spawning" now cycles directly to all the places
starting processes and threads.

- Sample -v -v verbose output (lines wrapped for commit message check)

 Spawning QEMUHandler Thread for \
     qemu_x86/samples/hello_world/sample.helloworld 'make run'

- native_posix example:

 Spawning process /home/.../sanity-out/native_posix/\
    /samples/hello_world/sample.helloworld/zephyr/zephyr.exe
 Spawning BinaryHandler Thread for native_posix/\
     samples/hello_world/sample.helloworld

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-04 18:29:01 -04:00
Carles Cufi c9f4bb6799 west: build: Construct CalledProcessError with positional args
Since west's main.py relies on the args tuple with the returncode
and the cmd, create the CalledProcessError using the correct
positional args.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-04 20:59:15 +02:00
Jan Van Winkel 4d975dba79 west: Renamed cmake.py to zcmake.py
Renamed cmake.py to zcmake.py to prevent import conflict if cmake is
installed through pip.

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-05-04 16:40:06 +02:00
Carles Cufi d6f033e690 west: Add a boards command
Add a new "boards" command that is able to list all the boards in the
upstream tree. There is currently no support for out-of-tree boards.

The command executes cmake to use the built-in CMake script,
boards.cmake, to list the boards, and then stores the information
retrieved and allows the user to present it in a user-definable format.

Fixes https://github.com/zephyrproject-rtos/west/issues/53

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-03 21:55:13 +02:00
Carles Cufi 31bdad5299 west: Implement CMake helpers in scripts/west_commands
Move the existing CMake and build functionality from the west repository
to zephyr. The rationale behind this move is that it's very tightly
coupled with the Zephyr build system and is only used by the extension
commands implemented in the zephyr tree.
If additional extension commands in third-party repos want to use the
functionality they can add $ZEPHYR_BASE/scripts/west_commands to the
Python system path.

The implmentations in the west repo will be deprecated.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-03 21:55:13 +02:00
Sigvart M. Hovland 06365a5ed1 cmake: zephyr_modules: Add west executable argument
On some systems where you don't have access to `PATH` and you can't
set the `ENV{PATH}` variable. You need to be able to pass the path
to the west executable down to the python script so it is better
for it to be set explicitly than assuming that it exsists as a
part of the PATH/executables in the shell being called.

Signed-off-by: Sigvart M. Hovland <sigvart.hovland@nordicsemi.no>
2019-05-03 14:23:09 -04:00
Maureen Helm d6d1e27a97 scripts: west_commands: Print error message when jlink exe not found
Adds a check in the jlink runner to look for the jlink executables and
print a more useful error message if they are not found.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-02 22:50:55 -04:00
Luiz Augusto von Dentz e0f3ab6bf2 Bluetooth: GATT: Add support for static services
This reintroduces support for static service in the form of a new API,
BT_GATT_SERVICE_DEFINE, and changes the internal services (GAP/GATT)
to be defined as const as they are never register/unregistered.

Internal service needed to be renamed in order to keep the same order
as before since the section elements are sorted by name.

The result is the following (make ram_report):

before:
      gatt.c                                    572     0.66%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gap_attrs				180     0.21%
        gap_svc                                  12     0.01%
        gatt_attrs                              160     0.18%
        gatt_sc                                  80     0.09%
        gatt_svc                                 12     0.01%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

after:
      gatt.c                                    210     0.24%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gatt_sc                                  80     0.09%
        last_static_handle                        2     0.00%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2019-05-02 08:29:23 +03:00
Flavio Ceolin bf878ced12 sanitycheck: Fix an error in scan_path exception
The function error expects only one parameter. The excpetion handler in
scan_path was calling this function with multiple parameters instead of
formatting the string.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2019-04-28 13:38:32 -04:00
Torsten Rasmussen 0dd3b421c4 west: zephyr_module: Improved error messaging during west failures
Fixes: #15664

This commit improve error messaging in case `west` list fails.
Previously any error messages and stack trace reported by `west` will
was thrown away by zephyr_module.py.

Now the error, as well as any stack traces, printed by `west` will be
re-printed to the user.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2019-04-27 12:38:32 -07:00
Marc Herbert 4a10eea0eb gen_kobject_list.py: better comments and --help. Zero code change.
- The script can but does not always generate five files, in fact the
  current build invokes it at least three times requesting different
  outputs every time.

- --kobj-types-output produces a code fragment; not a standalone/usable
  header file. It outputs enum constants for a single enum type and not
  several enum types.

- Some outputs include driver instances and others not: clarify which.

- There's an entire and great section in the documentation that took
  me ages to find because it's not referenced anywhere in the --help
  or code. Fixed.

- Highlight the massive duplication in the CMakeLists.txt to save
  déjà vu confusion and minimize future divergence.

- Other minor tweaks.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-26 03:21:59 -07:00
Carles Cufi 3a88dce9c5 scripts: west: Run pristine.cmake directly instead of the target
When making a build folder pristine until now we were running the
'pristine' build target. The issue with that is that ninja/make or
whatever build tool is being used might decide to re-run CMake itself if
some of the dependencies have changes. This might trigger an error that
is unfriendly and unnecessary, since the user is explicitly asking for
the build folder to be wiped before starting a fresh build.
To avoid this issue restor to running directly the CMake script that the
'pristine' build target itself uses, so as to make sure that the build
folder is wiped unconditionally regardless of changes made to the tree.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-23 13:24:41 -07:00
Anas Nashif d18ec53575 sanitycheck: show handler in verbose mode
When running in verbose mode, show what handler is being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif d6476eee3e sanitycheck: handle pid file not being present
Do not try to open non-existing pid files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif 042d9b7d83 sanitycheck: fix reporting: build vs. run
If build fails, report a build failure instead of handler error.

Fixes #15272

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif 654ec598ac sanitycheck: add timestamps to messages
Using the new option --timestamps, any output from sanitycheck will have
a timestamp to help identify bottle necks and monitor execution time.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif 924987d72f ci: detect changes to sub-board definitions
When detecting changes to boards, make sure we test all board
configurations available in that board directory, not only the main one.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:29 -04:00
Patrik Flykt 83de530d5a scripts/coccinelle: Add script for counting identifier length
Add a simple Coccinelle script that counts identifier lengths and
prints out a warning if it is longer than 31 characters.

The script can be run with:
spatch -D report --very-quiet \
--include-headers --recursive-includes \
--cocci-file $ZEPHYR_BASE/scripts/coccinelle/identifier_length.cocci \
--dir $ZEPHYR_BASE \
kernel/

Where '--include-headers' and '--recursive-includes' can be omitted
if neede.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-18 12:29:07 -04:00
Andy Ross aa2b8a1bc7 scripts/sanitycheck: Precompile test-time regular expressions
The Harness handlers for tests were parsing the realtime stream out of
qemu pipes by recompiling and executing every regex for every line (!)
of output from the simulator.  That's a significant CPU load, and it's
(1) in a separate thread not tracked by the JOBS limit and (2)
happening at the worst possible time and contending with the qemu
process for host CPU cycles that it needs to hit its (real world)
timer targets on time.

Compile them just once, please.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-04-18 12:21:13 -04:00
Ulf Magnusson 86b0c22dc4 kconfig: Avoid potential issue parsing generated_dts_board.conf
Splitting a string like 'foo="bar=baz"' on '=' will give ['foo', '"bar',
'baz"'] instead of the intended ['foo', '"bar=baz"']. split() with
maxsplit=1 to avoid potential issues.

Not seen in practice. Just some future safety.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:49:46 -04:00
Ulf Magnusson aec74f692c kconfiglib/menuconfig: Various behavior/UI improvements
Update Kconfiglib and menuconfig to upstream revision 90c5573c19, to get
these commits in:

    Warn for unquoted argument to 'source', etc.

    Print a warning suggesting to add quotes for things like

      source foo/bar/Kconfig
      menu title
      prompt unquoted

    Example warning:

      Kconfig:32: warning: style: quotes recommended around
      'lib/Kconfig.debug' in 'source lib/Kconfig.debug'

    That quoteless syntax is supported for compatibility with old
    versions of the C tools. It only works for a single word.

    ==================================================================

    menuconfig: Include all parents in menu paths

    Previously, symbols not defined with 'menuconfig' with children
    weren't listed in the children's menu paths. It was deliberate, but
    it's probably an anti-feature in retrospect, because it can make it
    harder to find stuff by following the menu path.

    Don't try to be clever and just list all the parent nodes in the
    menu path.

    ==================================================================

    menuconfig: Fix display issue for optional-prompt menuconfigs

    _shown_nodes() needs to check whether invisible 'menuconfig' symbols
    with optional prompts have visible children, so that they can be
    shown outside show-all mode. Previously, only 'config' symbols were
    checked.

    ==================================================================

    menuconfig: Remember last saved/loaded path and improve
    _conf_changed

    Remember the last path that was manually saved/loaded instead of
    reverting back to standard_config_filename() (e.g. .config).

    Remember the path to the last saved minimal configuration separately
    as well.

    Also improve the _conf_changed behavior when loading a .config
    within the interface. Instead of always treating it as needing to be
    saved, check if it's outdated, like for the .config file loaded on
    startup.

    Also make the exit message ("No changes to save", etc.) always
    include the target .config file, which is helpful. Previously, only
    the save message did.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:46:15 -04:00
Carles Cufi b7c75915e0 west: build: Add new pristine cmd-line and config option
Add a new command-line and build config option, `pristine`, that the
user can pass to `west build` or set in its configuration file(s) in
order to automatically trigger a pristine build on every build or
whenever west considers it required.

The option can take the following values:

- never: Never run the target
- always: Always run the pristine target before building
- auto: Run the pristine target when required

With `auto`, the pristine target will be run when running
west with an existing build folder containing a build system and:

- Selecting a different board from the one currently in the build system
- Selecting a different application from the one currently in the build
  system

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Carles Cufi 3c6584d6d5 west: build: Use officially documented CMake options
The -B option has always existed but was first officially documented in
CMake 3.13.0. In that same release the -S option was introduced,
replacing the old undocumented -H. Switch to using the officially
documented options.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Marc Herbert 0f7255c3b8 sanitycheck: order results.csv and discards.csv deterministically
One of the first things needed when comparing builds of tests across
different environments/systems is to make sure the same (sub)tests were
selected and run in the first place. For that purpose sort the output of
--testcase-report and --discard-report as they were in random order.

Actually make the entire class TestInstance sortable by adding a
standard __lt__() method comparing unique instance names; it could be
useful again.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:28:54 -04:00
Marc Herbert 3061c92aaf file2hex.py: new --gzip-mtime option that defaults to zero + test
This makes the output of file2hex.py deterministic by default while also
letting the user set the mtime in the gzip header manually if desired.

Use the option without any argument to restore the previous behavior
that sets the current (and obviously changing) "now" timestamp.

To test:  ./sanitycheck --tag gen_inc_file

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:27:40 -04:00
Anas Nashif a94f2b0b12 release: update footprint data
Update footprint data for 1.14 release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-15 20:31:52 -04:00
Ulf Magnusson 4f920df29a scripts: Remove checkconfig.py
This is an old script for finding references to undefined Kconfig
symbols that assumes Kconfiglib 1 (an older API).

There's now a different check for references to undefined symbols (see
commit 1d0834b35f ("checks: kconfig: Check for references to undefined
Kconfig symbols") in the ci-tools repo).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-12 08:28:03 -04:00
Marti Bolivar 100a2bba6c scripts: update pyocd in requirements.txt
This includes a bugfix for a pyocd requirement on pyyaml
that could not be satisfied using officially supported
releases.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-10 17:52:15 -04:00
Andrew Boie 9e69c54060 sanitycheck: add more info during processing
After the testcase configs are built, there is a step to
filter all the test case information to determine the set
of tests to run.

As this step takes a nontrivial amount of time, add an
informational message about it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-10 13:09:28 -04:00
Torsten Rasmussen ad026c09b1 cmake: modules: using posix path for Kconfig generated file
Fixes: #15289

Kconfig requires posix style path when sourcing other files.
As abspath in python will use native host style, i.e. backslash '\' in
windows this will cause invalid paths in Kconfig generated file and
thus the file will never be loaded.

This commit uses PurePath to convert the path to posix style, ensuring
Kconfig can load the modules.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-04-09 16:47:45 +02:00
Anas Nashif f6a163fc5c ci: remove unused script merge_junit.py
This script is now part of ci-tools repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 10:26:27 -04:00
Anas Nashif 3ae52624ff license: cleanup: add SPDX Apache-2.0 license identifier
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier.  Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.

By default all files without license information are under the default
license of Zephyr, which is Apache version 2.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Anas Nashif 8287f7c855 scripts: remove old xtools config files
Those files are outdated. We now have them as part of the sdk-ng git
repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Patrik Flykt 7c0a245d32 arch: Rename reserved function names
Rename reserved function names in arch/ subdirectory. The Python
script gen_priv_stacks.py was updated to follow the 'z_' prefix
naming.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-03 17:31:00 -04:00
Andrew Boie f0835674a3 lib: os: add sys_mutex data type
For systems without userspace enabled, these work the same
as a k_mutex.

For systems with userspace, the sys_mutex may exist in user
memory. It is still tracked as a kernel object, but has an
underlying k_mutex that is looked up in the kernel object
table.

Future enhancements will optimize sys_mutex to not require
syscalls for uncontended sys_mutexes, using atomic ops
instead.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie 1dc6612d50 userspace: do not track net_context as a kobject
The socket APIs no longer deal with direct net context
pointers.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie a00eff731c gen_kobject_list: allow object data to be C code
The data value in a kernel object structure is specific
to that type of object. Allow this to be a reference to
another C symbol or other compiled code by populating as
a string.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00