I am working on a modification of PKTGEN for sending packets containing sequences of the Fibonacci series. This is my very first time with kernel development, so I am not very familiar with the available functions for memory allocation. I am also not a C guru :)
I store the iterative steps of the algorithm in an array, that I would like to be dynamic if somebody asks for a great Fibonacci n parameter.
Realloc is not available. Do you know a way for dynamically enlarge an array size?
Thank you
See Dave Hansen Flexible Arrays added to the 2.6.31-rc5
https://lwn.net/Articles/345273/
The creation of a flexible array is done with:
#include <linux/flex_array.h>
struct flex_array *flex_array_alloc(int element_size, int total, gfp_t flags);
The individual object size is provided by element_size, while total is the maximum number of objects which can be stored in the array. The flags argument is passed directly to the internal memory allocation calls. With the current code, using flags to ask for high memory is likely to lead to notably unpleasant side effects.
Storing data into a flexible array is accomplished with a call to:
int flex_array_put(struct flex_array *array, int element_nr, void *src, gfp_t flags);
This call will copy the data from src into the array, in the position indicated by element_nr (which must be less than the maximum specified when the array was created). If any memory allocations must be performed, flags will be used. The return value is zero on success, a negative error code otherwise.
There might possibly be a need to store data into a flexible array while running in some sort of atomic context; in this situation, sleeping in the memory allocator would be a bad thing. That can be avoided by using GFP_ATOMIC for the flags value, but, often, there is a better way. The trick is to ensure that any needed memory allocations are done before entering atomic context, using:
int flex_array_prealloc(struct flex_array *array, int start, int end, gfp_t flags);
This function will ensure that memory for the elements indexed in the range defined by start and end has been allocated. Thereafter, a flex_array_put() call on an element in that range is guaranteed not to block.
Getting data back out of the array is done with:
void *flex_array_get(struct flex_array *fa, int element_nr);
The return value is a pointer to the data element, or NULL if that particular element has never been allocated.
Note that it is possible to get back a valid pointer for an element which has never been stored in the array. Memory for array elements is allocated one page at a time; a single allocation could provide memory for several adjacent elements. The flexible array code does not know if a specific element has been written to; it only knows if the associated memory is present. So a flex_array_get() call on an element which was never stored in the array has the potential to return a pointer to random data. If the caller does not have a separate way to know which elements were actually stored, it might be wise, at least, to add GFP_ZERO to the flags argument to ensure that all elements are zeroed.
There is no way to remove a single element from the array. It is possible, though, to remove all elements with a call to:
void flex_array_free_parts(struct flex_array *array);
This call frees all elements, but leaves the array itself in place. Freeing the entire array is done with:
void flex_array_free(struct flex_array *array);
As of this writing, there are no users of flexible arrays in the mainline kernel. The functions described here are also not exported to modules; that will probably be fixed when somebody comes up with a need for it.
This isn't the sort of thing that kernel development is intended to support. It would be far more appropriate to make this a user mode program.
However, the way to do it is to implement your own dynamic length array. Track how big the array is. If it needs to grow, call kmalloc() (usually with the GFP_KERNEL parameter) with the new size, copy the old data to the new, and dispose of the old (kfree()). See the kernel header file
If the array will be larger than about 4K or 8K, consider using __get_free_pages() or vmalloc() instead.
kmalloc() and kfree() are in linux-2.X.XX.XX/include/linux/slob_def.h
__get_free_pages() is in linux-2.X.XX.XX/include/linux/gfp.h
vmalloc() is in linux-2.X.XX.XX/include/linux/vmalloc.h
I tryed hand coded a very simple educational example of a byte dynamic array (std::string-lie) that can grow as needed to support write + fseek semantics, i.e. writing past end increases file size, fseek past end increases size and fills gap with NULs.
I used to use the kzalloc family because it seemed the one that was easiest to work with and with the ability to allocate larger memory chunks, though there would be performance tradeoffs with other options such as kmalloc.
#include <linux/slab.h> /* kvzalloc, kvrealloc, kvfree */
#include <linux/string.h> /* memset */
typedef struct {
char *buf;
size_t used;
size_t _size;
} dyn_arr_t;
int dyn_arr_init(dyn_arr_t *a, size_t size);
int dyn_arr_init(dyn_arr_t *a, size_t size)
{
a->buf = kvzalloc(size, GFP_KERNEL);
if (!a->buf)
return -ENOMEM;
a->used = 0;
a->_size = size;
return 0;
}
/* Reserve the required space for a future data insertion of size len at offset off.
* We don't do the actual insertion here as there are multiple possible insertion methods
* e.g. copy_from_user or strcpy.
*/
int dyn_arr_reserve(dyn_arr_t *a, size_t off, size_t len)
{
size_t new_used, new_size;
new_used = off + len;
if (new_used > a->_size) {
new_size = new_used * 2;
a->buf = kvrealloc(a->buf, a->_size, new_size, GFP_KERNEL);
if (!a->buf)
return -ENOMEM;
a->_size = new_size;
}
if (off > a->used)
memset(a->buf + a->used, '\0', off - a->used);
if (new_used > a->used)
a->used = new_used;
if (log) pr_info("dyn_arr_reserve _size:=%zu used:=%zu\n", a->_size, a->used);
return 0;
}
void dyn_arr_free(dyn_arr_t *a)
{
kvfree(a->buf);
a->buf = NULL;
a->used = 0;
a->_size = 0;
}
And sample usage would be something like:
// Init on module init
dyn_arr_t arr;
ret = dyn_arr_init(&arr, 1);
if (ret)
return ret;
// Sample write
static ssize_t write(struct file *filp, const char __user *buf, size_t len, loff_t *off)
{
ssize_t ret;
down_write(&rwsem);
ret = dyn_arr_reserve(&data, *off, len);
if (ret) {
ret = -ENOSPC;
goto out;
}
if (copy_from_user(data.buf + *off, buf, len)) {
ret = -EFAULT;
goto out;
} else {
ret = len;
*off += ret;
}
out:
up_write(&rwsem);
return ret;
}
// Cleanup
dyn_arr_free(data);
Also worth having a look at the "scull" example in LDD3 and its extensions under chapter 8 "allocating memory". Not the most up-to-date source nowadays, but worthwhile
Tested on Linux kernel v6.8.12.