0

When you allocate memory with Vulkan you do it on a particular type of heap and memory type that the Vulkan API has listed. Some of these memory types have the flags like for example:

  • HOST_COHERENT
  • HOST_CACHED

And among these sometimes there the same heap but different memory types among these heaps, meaning you can choose in some caches whether to have coherent or cached memory or not. I'm wondering how it does this as from user space, for example when you do 'new' or 'malloc' you get memory from the heap, but you don't decide what type it is, or whether you need to flush and invalidate caches manually. How does Vulkan do this? In my experience programming in C++ I've never come across the ability to be able to pick a 'type' of memory? Can I do this myself? Or is it only something Vulkan can do?

Improve this question

3 Answers 3

Reset to default
4

I'm wondering how it does this from user space ...

It doesn't.

Graphics drivers have a kernel driver component, and this is what allows control over how the memory is mapped on the CPU. Normal user-space allocations don't get control over this, especially for anything that ends up device visible as that often needs modification of GPU-side page tables.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

So what type of memory do you get as a programmer when you do new or malloc?
Most probably cached on the CPU, GPU can't access at all. ... but technically "implementation defined" and depends on the system. I can imagine embedded systems where all memory is uncached for some potions of the boot process, for example.
3

To expand on what solidpixel said:

Most OS' configuration of the control registers and page flags are limited to the kernel, but they often still expose some of that functionality to userland.

For instance, on Windows, you have functions like VirtualAlloc, VirtualAlloc2, VirtualProtect where you can set your own access flags. This is how JIT languages like Java, C#, Javascript, etc, allocate executable memory (or more accurately, using VirtualProtect, change memory to executable). There are a whole load of protection constants available in windows: https://learn.microsoft.com/en-us/windows/win32/Memory/memory-protection-constants. Some other fun ones include PAGE_GUARD which is useful for debugging memory overruns. Notable here are the flags PAGE_NOCACHE and PAGE_WRITECOMBINE. In general, you rarely use these, except when communicating with specific devices.

But as solidpixel said, the user mode GPU driver doesn't actually use these. Instead they call D3DKMTCreateAllocation (or pfnAllocateCb when running in D3D10 or greater) and pass some driver specific information. In the kernel, those calls both get passed to DxgkDdiCreateAllocation where the Kernel Mode Display Miniport Driver decodes the private data, and sets members in the allocation info structures, namely pCreateAllocation->pAllocationInfo[i].Flags to indicate the page protections and memory access flags. After DxgkDdiCreateAllocation returns, VidMM will take the information about the allocation and work to secure regions of memory on the CPU and GPU as needed. For CPU memory VidMM is in complete control. For GPU memory, VidMM gets broad control over the protection flags and location (including to some degree, paging structure), but asks the Display Miniport Driver to handle the implementation details on how its paging structure works.

Improve this answer

Comments

1

Your questions seems to arise from this:

do 'new' or 'malloc' you get memory from the heap, but you don't decide what type it is,

Now, here's the thing: Neither new nor malloc are the fundamental operations that allocate the memory. They are merely wrappers defined by the C++ and C programming language standards, that need an actual implementation. This implementation uses operating system functions, so called "syscalls" that will do the actual memory allocation.

These syscalls may either work with generic memory allocation infrastructure offered by the operating system kernel (anonymous mmap on Linux and the *BSDs, VirtualAlloc on Windows) or they may call into a specific device driver to allocate device specific memory and map it into address space (open("/dev/…") → mmap(device_fd,…) on Linux and the *BSDs, CreateFileEx("\\<DEVICE>\…") → CreateFileMapping(…) → MapViewOfFile on Windows).

And that's how these different kinds of memory end up in user space: Operating System low level access.

Improve this answer

2 Comments

Are you saying malloc returns mmapped file/memory? Like when you do mmap on Linux?
@Zebrafish: kind of, yes. Most malloc implementations don't do a mmap(flags|=MAP_ANONYMOUS, fd=-1) for each and every allocation, but instead maintain a pool of memory from which they cut smaller allocations; most allocations are small. You can look up the mappings in /proc/<PID>/maps and you'll find pointers returned malloc to reside within one of those mappings. There's one mapping called [heap] which is the mapping manipulated with sbrk, the underlying mechanism in the kernel is mmap (sbrk/sys_brk is a wrapper elixir.bootlin.com/linux/v6.15/source/mm/mmap.c#L115 )

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.