scripts/requirements: Split & document requirement.txt

Split up requirements.txt into several files so that CI tools can
utilize/reference the specific requirements-<FOO>.txt they may need
while keep things in sync with the development.  This is to reduce
both time and amount of work CI actions due to python package install.

Create the following groupings:

1. BASE - needed to build or create zephyr images
2. BUILD-TEST - need to run compile/build tests
3. DOC - need to build the docs
4. RUN-TEST - need for runtime testing
5. EXTRAS - optional or useful for development/developers workflow

Also tried to add a comment about what or why a given package is being
pulled in for.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>

CODEOWNERS

@@ -386,6 +386,7 @@
 /scripts/net/                             @jukkar @pfl
 /scripts/process_gperf.py                 @andrewboie
 /scripts/gen_relocate_app.py              @wentongwu
+/scripts/requirements*.txt                @mbolivar @galak @nashif
 /scripts/tracing/                         @wentongwu
 /scripts/sanity_chk/                      @nashif
 /scripts/sanitycheck                      @nashif

scripts/requirements-base.txt

@@ -0,0 +1,24 @@
+# BASE: required to build or create images with zephyr
+#
+# While technically west isn't required it's considered in base since it's
+# part of the recommended workflow
+
+# used by various build scripts
+pyelftools>=0.24
+
+# used by dts generation to parse binding YAMLs, also used by
+# sanitycheck to parse YAMLs
+PyYAML>=5.1
+
+# used by west_commands
+packaging
+
+# intelhex used by mergehex.py
+intelhex
+
+# it's west
+west>=0.6.2
+
+# used for windows based 'menuconfig'
+# "win32" is used for 64-bit Windows as well
+windows-curses; sys_platform == "win32"

scripts/requirements-build-test.txt

@@ -0,0 +1,19 @@
+# BUILD-TEST: required to do build tests of zephyr
+#
+# things used by sanitycheck or other things like code coverage or python
+# testing
+
+# used by sanitycheck for ansi color
+colorama
+
+# python lex/yex used by sanitycheck
+ply>=3.10
+
+# optional, but used for validation of YAML
+pykwalify
+
+# used for code coverage
+gcovr>=4.2
+
+# used for west-command testing
+pytest

scripts/requirements-doc.txt

@@ -0,0 +1,8 @@
+# DOC: used to generate docs
+
+breathe>=4.9.1
+docutils>=0.14
+sphinx>=1.7.5
+sphinx_rtd_theme
+sphinx-tabs
+sphinxcontrib-svg2pdfconverter

scripts/requirements-extras.txt

@@ -0,0 +1,13 @@
+# EXTRAS: optional or useful for development/developers workflow
+
+# used by sanitycheck for --test-tree option
+anytree
+
+# helper for developers - check git commit messages
+gitlint
+
+# helper for developers
+junit2html
+
+# used by scripts/gen_cfb_font_header.py - helper script for user
+Pillow

scripts/requirements-run-test.txt

@@ -0,0 +1,12 @@
+# RUN-TEST: required to do run time tests of zephyr
+#
+# things used by sanitycheck or related in run time testing
+
+# used to connect to boards for console IO
+pyserial
+
+# used to flash & debug various boards
+pyocd>=0.24.0
+
+# used by sanitycheck for board/hardware map
+tabulate

scripts/requirements.txt

@@ -1,25 +1,5 @@
-Pillow
+-r requirements-base.txt
-PyYAML>=5.1
+-r requirements-build-test.txt
-anytree
+-r requirements-doc.txt
-breathe>=4.9.1
+-r requirements-run-test.txt
-colorama
+-r requirements-extras.txt
-docutils>=0.14
-gcovr>=4.2
-gitlint
-intelhex
-junit2html
-packaging
-ply>=3.10
-pyelftools>=0.24
-pykwalify
-pyocd>=0.24.0
-pyserial
-pytest
-sphinx>=1.7.5
-sphinx_rtd_theme
-sphinx-tabs
-sphinxcontrib-svg2pdfconverter
-tabulate
-west>=0.6.2
-# "win32" is used for 64-bit Windows as well
-windows-curses; sys_platform == "win32"