Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
COPY/NOCOPY flag was added to the end of the string with ':'
as separators. The python script then split the line on ':'
which breaks on Windows build because the string was
[MEM]:[FILE]:[COPY]
In windows you will have 'c:\' in the file path so the
line.split() in script will split on the 'c:'.
Move the COPY/NOCOPY to:
[MEM]:[COPY/NOCOPY]:[FILE]
Note that the comments at top of gen_relocate_app.py also
indicates this format.
Fixes#43950
Signed-off-by: David Leach <david.leach@nxp.com>
* Use case of interest:
Some platforms are shipping in parallel to the internal FLASH some other
storage / external FLASH (usually a QSPI FLASH) that can be used to
execute (XIP) code from.
The content of this external FLASH can usually be written at flash time
using proper tools (see for example the case of the external FLASH on
the nRF5340DK that can be written at flash time using nrfjprog).
The external FLASH is a nice addition that is extremely useful when a
large application code doesn't entirely fit on the internal FLASH so
that we could want to move part of it in the auxiliary FLASH to XIP the
code from there.
* The problem:
Right now Zephyr doesn't have a formal and generic way to move at build
time part of the code to a different memory region.
* The current status:
Zephyr is indeed shipping a code_relocation feature but that doesn't
entirely match our needs.
When XIP is enabled, the code_relocation feature is used in Zephyr to
move the selected code (that is to copy text section, to initialize data
and zero bss) from FLASH to SRAM at run time and execute code from SRAM.
The relocation is done by a generated snippet of code that is
memcpy()-ing the right content to the destination region also using some
build-time generated portions of linker script to retrieve start and
destination addresses and regions.
* This patch:
This patch is leveraging the code_relocation code and adding a NOCOPY
feature. This feature is using the code_relocation feature to
dynamically build the linker script snippets but entirely skipping the
run-time code relocation so that the code can be XIP-ed from the
destination region.
* Example:
Let's say that we have a big file called `huge_file.c` that we want to
XIP from the external FLASH that is mapped in an EXTFLASH region.
In this case we should enable `CONFIG_XIP` and
`CONFIG_CODE_DATA_RELOCATION` and instruct cmake as follows:
zephyr_code_relocate(src/huge_file.c EXTFLASH_TEXT NOCOPY)
zephyr_code_relocate(src/huge_file.c SRAM_DATA)
this means that:
- The .text section of the `huge_file.c` must reside in the EXTFLASH
memory region and we do not need to copy the section there because we
are going to XIP it (and we assume that the file is going to be placed
in the external FLASH at flash time).
- The .data section of the `huge_file.c` must still reside in the SRAM
memory region.
* TODOs:
It's desirable to have the possibility to relocate libraries and
pre-build files instead of source code files.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Zeroing the BSS and copying data to RAM with regular memset/memcpy may
cause problems when those functions are assuming a fully initialized
system for their optimizations to work e.g. some instructions require
an active MMU, but turning the MMU on needs the .bss section to be
cleared first, etc.
Commit c5b898743a ("aarch64: Fix alignment fault on z_bss_zero()")
provides a detailed explanation of such a case.
Replacing z_bss_zero() with an architecture specific one is problematic
as the former may see new sections added to it that would be missed by
the later. The same reasoning goes for z_data_copy().
Let's make maintenance much easier by providing weak versions of
memset/memcpy that can be overridden by architecture-specific safe
versions when needed.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The problem with append is that when doing incremental (non-clean)
build, the content of these files are appended each time to the previous
leftover content.
Example:
# west build -p -b qemu_cortex_m3 \
samples/application_development/code_relocation/
# // disable for example CONFIG_XIP
# west build -b qemu_cortex_m3 \
samples/application_development/code_relocation/
at this point the content of the generated files (i.e.
linker_relocate.ld) is wrong.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Check each line whether contains the correct pattern
${location}:${file} before splitting because it
turns out that script passes also compile
definitions added in project CMakeLists.txt such as:
add_compile_definitions(ABCD="XYZ")
which results in the build failure.
Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
As pylint says "Consider iterating the dictionary directly
instead of calling .keys()" don't use .keys() method.
Signed-off-by: Bartosz Bilas <bartosz.bilas@hotmail.com>
With the addition of #34185, it is not longer gauranteed that for a
memory region `NAME` there exists a symbol `NAME_ADDR`. Use the linker
builtin function `ORIGIN` to instead directly get the start address of
the selected memory region.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Fixes: #28847
This commit fixes two places that was causing gen_relocate_app to fail
in windows.
gen_relocate_app.py now splits only on first `:` in
`<MEM_REGION>:<file>`.
Windows contains `:` in path, thus only first `:` is valid for splitting
mem region and file path.
Second part of the issue is fixed in CMake where `'` was used for
quoting of command arguments.
This causes a file not found on Windows because the final `'` would be
treated as part of the filename. Similar the first `'` would be treated
as path of the mem region name.
This is fixed by using `"` for quoting, which works correctly on all
platforms.
gen_relocate_app.py:403: UserWarning: File: .../kernel/sem.c' Not found
Note the stray `'` ^^^
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
We have a collection of python scripts that are part of our build
system. This PR collects docstring comments added to these scripts into
a summary document. Previous references to just the script name in
other documentation are updated to point to this build tool
documentation.
Some of the scripts needed an update to be processed (via include
directives) consistently.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Promote a handy and often-overlooked sys.exit() feature: Passing it a
string (or any other non-int object) prints it to stderr and exits with
status 1.
See the documentation at
https://docs.python.org/3/library/sys.html#sys.exit.
This indirectly prints some errors to stderr that previously went to
stdout.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Getting slightly subjective, but fixes this pylint warning:
scripts/gen_relocate_app.py:228:38: R1714: Consider merging these
comparisons with "in" to "region in ('data', 'bss')"
(consider-using-in)
Use a set literal instead of a tuple literal, as recent Python 3
versions optimize set literals with constant keys nicely.
Getting rid of pylint warnings for a CI check. I could disable any
controversial ones (it's already a list of warnings to enable anyway).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Dictionaries are not ordered in Python 3.5 and before, so building twice
${ZEPHYR_BASE}/samples/application_development/code_relocation/
in a row could lead to a different sections order, different
build/zephyr/include/generated/linker_relocate.d and code_relocation.c
and different binaries.
Fix with a minor change to three "for" loops in the output functions:
make them iterate on sorted(list of sections) instead of the raw and
randomly ordered dictionaries of sections.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
on non-XIP system, SRAM is the default region, and relocated .data
section and .bss section of SRAM shouldn't be inserted between
_image_rom_start and _image_rom_end, because the memory region between
_image_rom_start and _image_rom_end will construct the mpu ro region.
Also for the newly added memory region on non-XIP system, the
relocated .text secition and .rodata section should also be mpu aligned.
Fixes: #16090.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
Discovered with pylint3.
Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.
Python tip:
for i in range(n):
some_list.append(0)
can be replaced with
some_list += n*[0]
Similarly, 3*'\t' gives '\t\t\t'.
(Relevant here because pylint flagged the loop index as unused.)
To do integer division in Python 3, use // instead of /.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
(OPTIONAL) was a vestiage from the initial import of the Zephyr code
base and we dont utilize it with the GNU linker. Additionally, the way
(OPTIONAL) gets defined to nothing creates a linker script that lld
(from llvm) doesn't like.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This seems to solve the issue with GH-12033. There seems to be
some compiler optimization that was causing this issue. This
occurs only when the build is re-run by invoking cmake without
clearing the previous build.
Fixes:GH-12033
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
This script will relocate .text .data and .bss sections from
required files and places it in the required memory region. This
memory region and file are given to this python script in the form
of a string. Refer to the script for the format of this string and
the procedure to invoke it.
The main goal of this script is to provide a robust way to re-order
the memory contents without actually having to modify the code
(C source code and the linker code).
In simple terms this script will do the job of
__attribute__((section("name"))) for a bunch of files together.
Signed-off-by: Varun Sharma <varun.sharma@intel.com>
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
