15

I've watched the presentation and still have one question about working of shared buffers. As the slide 16 shows, when the server handles an incoming request, the postmaster process calls fork() to create a child one for handling the incoming request. Here is a picture from there: enter image description here

So, we have the entire copy of the postmaster process except its pid. Now, if the child process update some data belonging to shared memory (putting in shared buffers, as shown in the slide 17), we need the other threads be awared of the changes. The picture:

enter image description here

The synchronization process is what I don't understand. Any process owns a copy of the shared memory and while copying it doesn't know if another thread will write something to its copy of the shared memory. What if after creating proc1 by calling fork(), another process proc2 is created a little bit later and start writing something into the its copy of the shared memory.

Question: How does proc1 know what to do with the part of the shared memory that are being modified by proc2?

Improve this question
2
  • 7
    I totally disagree with the close votes. This is entirely on topic, it's a question about POSIX APIs, C programming, etc. Please stop close-voting. Commented Oct 4, 2015 at 7:46
  • It probably belongs in both. This is both a fundamental question about postgres design, but also about programming. The issue is it doesn't really show how it's about programming. Commented Oct 4, 2015 at 7:48

2 Answers 2

Reset to default
18

The crucial thing to understand is that there are two different types of memory sharing used.

One is the copy-on-write sharing used by fork() (without exec()), where the child process inherits the parent process's memory and state. In this case when the child or parent modify anything, a new private copy of the modified memory page is allocated. So the child doesn't see changes made by the parent after fork() and the parent doesn't see changes made by the child after fork(). Peer children cannot see each other's changes either. They're all isolated as far as memory is concerned, they just share a common ancestor.

That memory is what's shown in the Program (text), data and stack sections of the diagram.

Because of that isolation, PostgreSQL also uses POSIX shared memory - or, in older versions, system V shared memory. These are explicitly shared memory segments that are mapped to a range of addresses. Each process sees the same memory, and it is not copy-on-write. It's fully read/write shared.

This is what is shown in the purple "shared memory" section of the diagram.

POSIX shared memory is used for inter-process communication for locking, for shared_buffers, etc etc. Not the memory inherited from fork()ing.

While memory from fork is often shared copy-on-write, that's really an operating system implementation detail. The operating system could choose not to share it at all, and make an immediate copy of the parent's whole address space for the child at fork time. The only way the copy-on-write sharing is really relevant is when looking at top etc.

When PostgreSQL refers to "shared memory" it's always talking about the POSIX or System V shared memory block(s) that are mapped into each process's address space. Not copy-on-write sharing from fork().

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

You explained these concepts really clear, thanks much. But what do the forked processes do with the copy-on-write memory? If postgresql always means POSIX shared memory, it's not quite clear why they show the copy-on-write shared memory in the first picture I cited? PostgreSQL doesn't concern about COW-shared memory at all, does it?
@St.Antario PostgreSQL does care about the memory copying from fork(), but it doesn't care when it's copied at all. It could be CoW, or immediately copied. It wouldn't care. They use the memory inherited from the postmaster during fork() to look up server settings like the database encoding, to avoid re-doing all sorts of startup processing, to know where to look for the shared memory segment, and tons more. They just don't use it to communicate with other backends, or send data back "up" to the postmaster.
@St.Antario Mostly the fork()ed memory is just an optimisation to avoid lots of startup overhread. On windows there's no fork() so PostgreSQL has to do full startup for each backend, and it passes the options backends need from the postmaster on the command line and/or via temp files, rather than in inherited memory. It's slower, but it works. That's controlled by the EXEC_BACKEND compile option; taking a look at that might help clarify how it works the rest of the time.
1

I don't know about this special case but generally in linux and most other operating systems in order to speedup creating a new process, when a process asks operating system to create a new process then OS creates the new one with minimum requirements (specifically in DB applications) and share most of parent memory space with child. Now when a child want to modify some part of shared memory, OS uses COW (copy on write) concept and create a new copy of that part of the memory for child process usage. So this part becomes specific for child process and is no longer shared with parent process.

Improve this answer

1 Comment

All right, but how does another child process know about modification made by another child?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.