For zero sized buffers, instead of pointing to a buffer, net_buf->__buf
is NULL. For this reason, when cloning a buffer, the code needs to check
__buf before dereferencing it.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This functionality is useful on the following scenario:
1) The first buffer in a net_pkt contains user data which is relevant
for the (whole) net_pkt.
2) When inserting a new buffer in front of the net_pkt, the (previously)
first buffer (and its user data) are no longer accessible via
net_pkt->buffer.
3) net_buf_user_data_copy() allows to simply copy the user data from the
old to the new first buffer.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
Previously, there was no way to determine maximum number of bytes
that can be allocated using only net_buf structure. This commit
introduces such field.
Moreover, this commit fixes an issue where allocation of less than
maximum number of bytes from a fixed buffer pool would set buffer's
size to this number instead of the whole buffer size.
Signed-off-by: Konrad Derda <konrad.derda@nordicsemi.no>
This commit adds a new function the net_buf's API that allow an user
to match the net_buf's content with a data without copying it to a
temporary buffer.
Signed-off-by: Konrad Derda <konrad.derda@nordicsemi.no>
There are several subsystems and boards which require a relatively large
system heap (used by k_malloc()) to function properly. This became even
more notable with the recent introduction of the ACPICA library, which
causes ACPI-using boards to require a system heap of up to several
megabytes in size.
Until now, subsystems and boards have tried to solve this by having
Kconfig overlays which modify the default value of HEAP_MEM_POOL_SIZE.
This works ok, except when applications start explicitly setting values
in their prj.conf files:
$ git grep CONFIG_HEAP_MEM_POOL_SIZE= tests samples|wc -l
157
The vast majority of values set by current sample or test applications
is much too small for subsystems like ACPI, which results in the
application not being able to run on such boards.
To solve this situation, we introduce support for subsystems to specify
their own custom system heap size requirement. Subsystems do
this by defining Kconfig options with the prefix HEAP_MEM_POOL_ADD_SIZE_.
The final value of the system heap is the sum of the custom
minimum requirements, or the value existing HEAP_MEM_POOL_SIZE option,
whichever is greater.
We also introduce a new HEAP_MEM_POOL_IGNORE_MIN Kconfig option which
applications can use to force a lower value than what subsystems have
specficied, however this behavior is disabled by default.
Whenever the minimum is greater than the requested value a CMake warning
will be issued in the build output.
This patch ends up modifying several places outside of kernel code,
since the presence of the system heap is no longer detected using a
non-zero CONFIG_HEAP_MEM_POOL_SIZE value, rather it's now detected using
a new K_HEAP_MEM_POOL_SIZE value that's evaluated at build.
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
The use case is to have a netbuf pool that is used exclusively with
net_buf_alloc_with_data() where the destroy callback takes care of
freeing the actual data buffer pointed to by __buf.
Signed-off-by: Tomasz Moń <tomasz.mon@nordicsemi.no>
This patch fixes commit b70f92e570
("net: buf: keep memory alignment provided by k_heap_alloc and k_malloc").
One-line was overlooked in the above patch and may result in a
cloned net_buf using a data block that has already been freed.
Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
Cleanup linker scripts for net_buf_pool section to use the linker
script related iterable section macros.
Also replace _net_buf_pool_list with macro's instead to complete
iterable section usage.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
This moves net_buf_simple related code to separate source file.
Having those in separate file makes unit testing easier as simple
network buffers do not use kernel objects, thus can be used
in unit tests without a need for adding any mocks.
Signed-off-by: Mariusz Skamra <mariusz.skamra@codecoup.pl>
As there is dedicated NET_BUF_SIMPLE_* logging macros set,
NET_BUF_SIMPLE_DBG shall be used in this place instead.
Signed-off-by: Mariusz Skamra <mariusz.skamra@codecoup.pl>
This patch reworks how fragments are handled in the net_buf
infrastructure.
In particular, it removes the union around the node and frags members in
the main net_buf structure. This is done so that both can be used at the
same time, at a cost of 4 bytes per net_buf instance.
This implies that the layout of net_buf instances changes whenever being
inserted into a queue (fifo or lifo) or a linked list (slist).
Until now, this is what happened when enqueueing a net_buf with frags in
a queue or linked list:
1.1 Before enqueueing:
+--------+ +--------+ +--------+
|#1 node|\ |#2 node|\ |#3 node|\
| | \ | | \ | | \
| frags |------| frags |------| frags |------NULL
+--------+ +--------+ +--------+
net_buf #1 has 2 fragments, net_bufs #2 and #3. Both the node and frags
pointers (they are the same, since they are unioned) point to the next
fragment.
1.2 After enqueueing:
+--------+ +--------+ +--------+ +--------+ +--------+
|q/slist |------|#1 node|------|#2 node|------|#3 node|------|q/slist |
|node | | *flag | / | *flag | / | | / |node |
| | | frags |/ | frags |/ | frags |/ | |
+--------+ +--------+ +--------+ +--------+ +--------+
When enqueing a net_buf (in this case #1) that contains fragments, the
current net_buf implementation actually enqueues all the fragments (in
this case #2 and #3) as actual queue/slist items, since node and frags
are one and the same in memory. This makes the enqueuing operation
expensive and it makes it impossible to atomically dequeue. The `*flag`
notation here means that the `flags` member has been set to
`NET_BUF_FRAGS` in order to be able to reconstruct the frags pointers
when dequeuing.
After this patch, the layout changes considerably:
2.1 Before enqueueing:
+--------+ +--------+ +--------+
|#1 node|--NULL |#2 node|--NULL |#3 node|--NULL
| | | | | |
| frags |-------| frags |-------| frags |------NULL
+--------+ +--------+ +--------+
This is very similar to 1.1, except that now node and frags are
different pointers, so node is just set to NULL.
2.2 After enqueueing:
+--------+ +--------+ +--------+
|q/slist |-------|#1 node|-------|q/slist |
|node | | | |node |
| | | frags | | |
+--------+ +--------+ +--------+
| +--------+ +--------+
| |#2 node|--NULL |#3 node|--NULL
| | | | |
+------------| frags |-------| frags |------NULL
+--------+ +--------+
When enqueuing net_buf #1, now we only enqueue that very item, instead
of enqueing the frags as well, since now node and frags are separate
pointers. This simplifies the operation and makes it atomic.
Resolves#52718.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Allocation callback in net_buf_heap_cb and net_buf_var_cb
used for net_bufs with variable size payloads, defined by
NET_BUF_POOL_HEAP_DEFINE or NET_BUF_POOL_VAR_DEFINE,
allocate one more octet for internal variable ref_count,
used in function generic_data_ref(), which in turn is needed
for net_buf_clone()).
The user gets a buffer which is shifted by one octet in memory
block. This breaks alignment provided k_heap_alloc and k_malloc.
Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
In order to bring consistency in-tree, migrate all subsystems code to
the new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Transition the `user_data` field in `struct net_buf` to be a flexible
array member instead of a hardcoded array. Compile-time asserts are
introduced at the location of the intermediate struct usage to ensure
that the assumptions utilised in runtime code hold true.
The primary assumptions are that the two `user_data` fields exist at the
same memory offset, and that the instantiated struct size can be
determined from the generic struct size and the length of the user data.
`net_buf_id` and `pool_get_uninit` must now use manual address
calculations as the `__bufs` type is no longer the actual size of the
instantiated variable.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Store the `user_data` array size on both the pool and net_buf structs.
This will enable length validation once `user_data` fields are not
globally the same size. The new variables fit inside existing padding,
and therefore do not increase the size of either structure.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
net_buf_max_len() provides the maximum number of bytes which can be put
behind the data pointer. This provides a save alternative to using the
size field of the net_buf structure directly, which does not take the
reserved bytes (headroom) into account.
This commit also replaces the usage of the size field in places where
size got used directly. Code has not been adjusted when it is easy to
recognise that the buffer does not have any reserved bytes, which is the
case after allocation or reset. Same goes for the faulty usage by
net_pkt as exposed by the last commit and begin fixed by a separate
commit.
Even though it would be cleaner, I decided to not rename the size field
to e.g. __buf_size in order to keep the amount of code changes low.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This enables to use net_buf_append_bytes without passing an allocator in
which case the code would attempt to use the net_buf_pool of the
original buffer.
Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Adds a new family of `struct net_buf` operations that remove data from
the end of the buffer.
The semantics of `net_buf_remove_mem` have been chosen to match those of
`net_buf_pull_mem`, i.e. the return value is a pointer to the memory
that was removed.
The opposite of this function, `net_buf_remove`, would need to return
the old end of the data buffer to be useful. However this value is
always an invalid target for reading or writing data to (It points to
the middle of unused data).The existance of the function would be
misleading, therefore it is not implemented.
Fixes#31069.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Use the core k_heap API pervasively within our tree instead of the
z_mem_pool wrapper that provided compatibility with the older mempool
implementation.
Almost all of this is straightforward swapping of one alloc/free call
for another. In a few cases where code was holding onto an old-style
"mem_block" a local compatibility struct with a single field has been
swapped in to keep the invasiveness of the changes down.
Note that not all the relevant changes in this patch have in-tree test
coverage, though I validated that it all builds.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Mark all k_mem_pool APIs deprecated for future code. Remaining
internal usage now uses equivalent "z_mem_pool" symbols instead.
Fixes#24358
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
using CONFIG_NET_BUF_POOL_USAGE monitor avail_count,
this variable should be protect.
Protecting it by using atomic variable
Signed-off-by: Ehud Naim <ehudn@marvell.com>
This patch updates the net_buf API to use k_timeout_t in essentially
all places where "s32_t timeout" was previously used. For the most
part the conversion is trivial, except for the places where
intermediate decrements of remaining timeout is needed. For this the
z_timeout_end_calc() API is used.
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
Remove net bufs own assertion mechanism and use the system assert
instead. This changes the assertion messaged printed from printing
expression to printing the line and file name. This provides more
context as the same expression could be asserted upon multiple times and
would then not provide enough clarity in the message.
This removes the option to enable only net buf assertions.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
NET_BUF_ASSERT() tranlates into nothing in presence
of CONFIG_ASSERT=y, which is misleading.
Use __ASSERT() in NET_BUF_ASSERT().
Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
This adds net_bug_simple_init_with_data which can be used to initialize
a net_buf_simple pointer with an external data pointer.
Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Use sys_put_xyz() helpers instead of memcpy() whenever possible. This
brings in straight-line inline code for pushes and adds of known,
small sizes.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Provides a way to clone a net_buf_simple without altering the state of
the original buffer. The primary usage scenario is for manipulating a
previously allocated PDU inside a buffer without altering the length and
offset of the buffer.
Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
move misc/byteorder.h to sys/byteorder.h and
create a shim for backward-compatibility.
No functional changes to the headers.
A warning in the shim can be controlled with CONFIG_COMPAT_INCLUDES.
Related to #16539
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
There are issues using lowercase min and max macros when compiling a C++
application with a third-party toolchain such as GNU ARM Embedded when
using some STL headers i.e. <chrono>.
This is because there are actual C++ functions called min and max
defined in some of the STL headers and these macros interfere with them.
By changing the macros to UPPERCASE, which is consistent with almost all
other pre-processor macros this naming conflict is avoided.
All files that use these macros have been updated.
Signed-off-by: Carlos Stuart <carlosstuart1970@gmail.com>
This is the same as net_buf_pull(), except that instead of returning
the new buf->data it returns the old buf->data. This was recently
discussed in github issue #12562.
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
net_buf_linearize() used to clear the contents of output buffer,
just to fill it with data as the next step. The only effect that
would have is if less data was written to the output buffer. But
it's not reliable for a caller to rely on net_buf_linearize() for
that, instead callers should take care to handle any conditions
like that themselves. For example, a caller which wants to process
the data as zero-terminated string, must reserve a byte for it
in the output buffer explicitly (and set it to zero).
The only in-tree user which relied on clearing output buffer was
wncm14a2a.c. But either had buffer sizes calculated very precisely
to always accommodate extra trailing zero byte (without providing
code comments about this), or arguably could suffer from buffer
overruns (at least if data received from a modem was invalid and
filled up all destination buffer, leaving no space for trailing
zero).
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Don't try to find "errors" in the values of dst_len and len params
passed to net_buf_linearize(). Instead, do what entails with the
common sense from the values passed in, specifically:
1. Never read more than dst_len (or it would lead to buffer
overflow).
2. It's absolutely ok to read than specified by "len" param, that's
why this function returns number of bytes read in the first place.
The motivation for this change is that it's not useful with its
current behavior. For example, a number of Ethernet drivers linearize
a packet to send, but each does it with its own duplicated adhoc
routine, because net_buf_linearize() would just return error for the
natural use of:
net_buf_linearize(buf, sizeof(buf), pkt->frags, 0, sizeof(buf));
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
