3

I have been trying to setup shared memory IPC between a process running on the host machine and a program running on the guest machine. I have been making the following attempts at launching QEMU based on information found on the web:

Attempt #1: qemu-system-x86_64 -hda disk.qcow2 -m 4096M -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:34,hostfwd=tcp::10022-:22 -smp 32 -monitor pty -vnc :12 -qmp tcp:localhost:4444,server,wait=off -object memory-backend-file,id=hostmem,size=4096M,mem-path=/dev/shm/data.dat,share=on -machine memory-backend=hostmem

This was documented here: Accessing the RAM of a QEMU Emulated System from another Process – REDS blog, and seemed promising.

Attempt #2: qemu-system-x86_64 -hda disk.qcow2 -m 4096M -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:34,hostfwd=tcp::10022-:22 -smp 32 -monitor pty -vnc :12 -qmp tcp:localhost:4444,server,wait=off -device ivshmem-plain,memdev=hostmem,master=on -object memory-backend-file,size=256M,share=on,mem-path=/dev/shm/data.dat,id=hostmem

This is documented on QEMU website. Not sure ivshmem is supposed to be used for this purpose, but I thought it was worth a try: Inter-VM Shared Memory device — QEMU documentation

Attempt #3: I have read this post: qemu/docs/memory-hotplug.txt at master · qemu/qemu · GitHub, the only difference with what I was already doing was the suggestion of adding the slots=3 option to the -m option like so:

qemu-system-x86_64 -hda disk.qcow2 -m 4096M,slots=3,maxmem=8G -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:34,hostfwd=tcp::10022-:22 -smp 32 -monitor pty -vnc :12 -qmp tcp:localhost:4444,server,wait=off -object memory-backend-file,id=hostmem,size=4096M,mem-path=/dev/shm/data.dat,share=on -machine memory-backend=hostmem

Attempt #4: I have finally tried to follow a different approach by hotplugging the shared memory device via QEMU Monitor as described here: QEMU tutorial:How to use ivshmem-plain - L (liujunming.top)

In all these attempts a backend file is correctly instantiated on the host machine at "/dev/shm/data.dat". However there is no trace of the same file at the same location on the guest. So a program running on the guest that tries to get a file descriptor to that file to be then used by mmap and enable writing to or reading from shared memory, crashes.

Does anyone know why this is happening? Does shared memory need to be explicitly enabled on the guest for this to work? If so, how do you do it?

If I run: lspci on the guest I can see the IV-SHMEM PCI device!! So why is the /dev/shm/data.dat file not created on the guest?

1
  • This seems to answer the question above: Commented Jan 18, 2024 at 13:23

1 Answer 1

2

I found the following explanation:

Shared memory IPC (inter-process communication) is a method of exchanging data between processes or virtual machines using a shared memory region. Qemu is a software that emulates various hardware architectures and allows running different operating systems on the same host machine.

One way to enable shared memory IPC in qemu is to use the ivshmem device, which is a PCI device that provides access to a shared memory region on the host. The ivshmem device can be configured to use either a POSIX shared memory file (such as /dev/shm/data.dat) or a memory backend object (such as memory-backend-file) as the source of the shared memory12.

To use the ivshmem device, you need to specify the following options when launching qemu:

-device ivshmem-plain,memdev=hostmem where hostmem is the name of the memory backend object that defines the size, path, and sharing mode of the shared memory region. -object memory-backend-file,size=...,share=on,mem-path=...,id=hostmem where you specify the size, path, and ID of the memory backend object. The share=on option is important to enable sharing the memory region with other processes or VMs. For example, the following command will create a 256 MB shared memory region backed by the file /dev/shm/data.dat and expose it to the guest VM as an ivshmem device:

qemu-system-x86_64 -device ivshmem-plain,memdev=hostmem -object memory-backend-file,size=256M,share=on,mem-path=/dev/shm/data.dat,id=hostmem ...

Note that the file /dev/shm/data.dat will not be visible on the guest VM, as it is only used by the host to allocate the shared memory region. The guest VM will see the shared memory region as a PCI device with a BAR (base address register) that maps to the shared memory address space. You can use the lspci command on the guest VM to verify that the ivshmem device is present and get its BAR address2.

To access the shared memory region from the guest VM, you need to use the mmap system call to map the BAR address to the guest’s virtual address space. For example, the following C code snippet will map the first 4 KB of the shared memory region to a pointer shm:

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>

#define SHM_SIZE 4096 // size of the shared memory region to map
#define SHM_BAR 0xfebf0000 // BAR address of the ivshmem device (obtained from lspci)

int main() {
    int fd = open("/dev/mem", O_RDWR); // open the /dev/mem device file
    if (fd == -1) {
        perror("open");
        return -1;
    }
    void *shm = mmap(NULL, SHM_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, SHM_BAR); // map the BAR address to shm
    if (shm == MAP_FAILED) {
        perror("mmap");
        return -1;
    }
    // do something with shm, such as reading or writing data
    munmap(shm, SHM_SIZE); // unmap the shared memory region
    close(fd); // close the /dev/mem device file
    return 0;
}
Sign up to request clarification or add additional context in comments.

2 Comments

The BAR address (start address) of the ivshmem on the guest can be retrieved by executing the following command in the guest: lspci -vv -s 00:04.0
What if guest is baremetal and there is no os. How can I get the bar BAR address and could it be used directly with pointer?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.