Opening Boost Interprocess segment in Constructor of object

Question

I create a map<int, vector<int>> in a Boost Interprocess managed_shared_memory in one process and then want to open it in another process to write to it continually. The other process tries to open the shared memory in the constructor of an object Communication and then presents a method addData(int _data).

Example:

// Typedefs for shared data structures

using boost::interprocess;
using std::pair;
using std::scoped_allocator_adaptor;

typedef managed_shared_memory::segment_manager man;

typedef allocator <int , man> Int_Allocator;
typedef vector <int, Int_Allocator> Int_Vector;

typedef pair <const int, Int_Vector> Int_Vector_Map_Type;
typedef scoped_allocator_adaptor <allocator<Int_Vector_Map_Type, man>> Int_Vector_Map_Type_Allocator;
typedef map <int, Int_Vector, std::less<int>, Int_Vector_Map_Type_Allocator> Int_Vector_Map;

// Let's call this process 'parent'

managed_shared_memory segment(create_only, "Shared");
Int_Vector_Map_Type_Allocator alloc = segment.get_segment_manager();
segment.construct<Int_Vector_Map>("Map")(std::less<int>(), alloc);

// 'child' process

class Communciation{
    Int_Vector_Map* map;

public:
    Communication{
        managed_shared_memory segment(open_only, "Shared");
        map = segment.find<Int_Vector_Map>("Map").first;
    }
    void addData(int _key, int _value){
        if(map->size() == 0 || map->find(_key) == map->end(){
            managed_shared_memory segment(open_only, "Shared");
            Int_Allocator alloc = segment.get_segment_manager();
            map->insert(Int_Vector_Map_Type(_key, Int_Vector(alloc)));
        } // create the vector and its allocator, otherwise runtime error

        map->at(_key).push_back(_value);
    }
}

This is a broken down example of what I am looking at, but I do not see the immediate error. When trying to run this code I get the runtime error "Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address (...)". Changing the addData(...) method to:

void addData(int _key, int _value){
    managed_shared_memory segment(open_only, "Shared");
    map = segment.find<Int_Vector_Map>("Map").first;

    if(map->size() == 0 || map->find(_key) == map->end(){
        Int_Allocator alloc = segment.get_segment_manager();
        map->insert(Int_Vector_Map_Type(_key, Int_Vector(alloc)));
    } // create the vector and its allocator, otherwise runtime error

    map->at(_key).push_back(_value);
}

i.e. opening the map every call fixes this issue but is not applicable in my case as I want to be able to call this method many times per frame without impacting fps too much.

What is the cause of this issue and is it even possible to use boost's interprocess as described?

Edit: For added context, the 'child' process is running in another process in which I inject a dll into.

While multi-level allocation on shared memory makes a good question(e.g vector<string>) did you try multimap( or unordered_multimap) instead of map<int, vector>? cppreference example on scoped_allocator_adaptor use adaptive_pool instead of allocator. I guess you need more tags; scoped_allocator_adaptor and boost.interprocess included. — Red.Wave
– Red.Wave, Commented Nov 11, 2023 at 18:46

sehe · Accepted Answer · 2023-11-11 22:11:56Z

1

The answer has many components:

use simplified type aliases
the segment MUST remain mapped, so you MUST have it as a member in the class (or with longer lifetime)
allocator-construct only happens when scoped allocator is consistently used and insertion uses allocator-construction (e.g. emplace method)

I have good experience using Boost Container's implementations, so let's show that.

Simplified type aliases

// Typedefs for shared data structures
namespace bip = boost::interprocess;

namespace Shared {
    namespace bc  = boost::container;
    using Segment = bip::managed_shared_memory;
    using Mgr     = Segment::segment_manager;

    template <typename T> using Alloc  = bc::scoped_allocator_adaptor<bip::allocator<T, Mgr>>;
    template <typename T> using Vector = bc::vector<T, Alloc<T>>;
    template <typename K, typename V, typename Cmp = std::less<K>, typename P = std::pair<K const, V>>
    using Map = bc::map<K, V, Cmp, Alloc<P>>;
} // namespace Shared

Now you can simply state:

using IVMap = Shared::Map<int, Shared::Vector<int>>; // Int_Vector_Map

And it will expand to the right set of comparators, allocators etc.

Lifetime

To construct the segment from the constructor, you will need to use the initializer list. E.g.:

class Communication {
  public:
    Communication() //
        : segment_{bip::open_only, "Shared"}
        , map_(*segment_.find<IVMap>("Map").first) {}

    void addData(int key, int value);

  private:
    Shared::Segment segment_;
    IVMap&          map_;
};

Emplace

To insert new elements, implement addData like:

void addData(int key, int value) {
    auto it = map_.find(key);

    if (it == map_.end())
        it = map_.emplace(key).first;

    it->second.push_back(value);
}

Note that the segment simply exists. Note that the allocator is propagated due to the scoped allocator adaptor in combination with the emplace construction.

Edit Sadly I cannot make the emplace compile. I'm sure I'm missing something trivial, but I have worked around it like:

if (it == map_.end()) {
    IVMap::mapped_type v(map_.get_allocator());
    it = map_.emplace(key, std::move(v)).first;
}

Live Demo

Live On Coliru

#include <boost/container/scoped_allocator.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/map.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <set>

// Typedefs for shared data structures
namespace bip = boost::interprocess;

namespace Shared {
    namespace bc  = boost::container;
#ifdef COLIRU // online compiler
    using Segment = bip::managed_mapped_file;
#else
    using Segment = bip::managed_shared_memory;
#endif
    using Mgr     = Segment::segment_manager;

    template <typename T> using Alloc  = bc::scoped_allocator_adaptor<bip::allocator<T, Mgr>>;
    template <typename T> using Vector = bc::vector<T, Alloc<T>>;
    template <typename K, typename V, typename Cmp = std::less<K>, typename P = std::pair<K const, V>>
    using Map = bc::map<K, V, Cmp, Alloc<P>>;
} // namespace Shared

using IVMap = Shared::Map<int, Shared::Vector<int>>; // Int_Vector_Map

class Communication {
  public:
    Communication() //
        : segment_{bip::open_only, "Shared"}
        , map_(*segment_.find<IVMap>("Map").first) {}

    void addData(int key, int value) {
        auto it = map_.find(key);

        if (it == map_.end()) {
            IVMap::mapped_type v(map_.get_allocator());
            it = map_.emplace(key, std::move(v)).first;
        }

        it->second.push_back(value);
    }

    auto const& get() const { return map_; }

  private:
    Shared::Segment segment_;
    IVMap&          map_;
};

#include <fmt/ranges.h>
#include <functional>
#include <random>
static auto vals = bind(std::uniform_int_distribution(100, 999), std::mt19937{std::random_device{}()});

int main(int argc, char** argv) {
    auto const args = std::set<std::string_view>(argv + 1, argv + argc);

    if (args.contains("parent")) {
        Shared::Segment seg{bip::open_or_create, "Shared", 10 << 10};
        seg.find_or_construct<IVMap>("Map")(seg.get_segment_manager());
    }

    if (args.contains("child")) {
        Communication comm;

        for (unsigned n = 10; n--;) {
            auto v = vals();
            comm.addData(v / 100, v);
        }

        fmt::print("After insertion:\n - {}\n", fmt::join(comm.get(), "\n - "));
    }
}

Printing

+ ./a.out parent
+ ./a.out child
After insertion:
 - (1, [155, 170])
 - (2, [248])
 - (4, [418])
 - (5, [542, 562])
 - (6, [642, 674, 659])
 - (7, [783])
+ ./a.out child
After insertion:
 - (1, [155, 170, 143, 130])
 - (2, [248, 222])
 - (3, [325])
 - (4, [418, 428])
 - (5, [542, 562, 556])
 - (6, [642, 674, 659, 671])
 - (7, [783, 793, 733, 745])
+ ./a.out child
After insertion:
 - (1, [155, 170, 143, 130])
 - (2, [248, 222])
 - (3, [325, 320, 362])
 - (4, [418, 428, 486, 437])
 - (5, [542, 562, 556])
 - (6, [642, 674, 659, 671, 695, 609])
 - (7, [783, 793, 733, 745, 786, 777, 793])
 - (9, [995])

Live Demo locally:

answered Nov 11, 2023 at 22:11

sehe

400k49 gold badges475 silver badges673 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

sehe Over a year ago

Using a careful mix of std::scoped_allocator_adaptor and boost::container::map I was able to get marginally closer to the goal of painless insertion: coliru.stacked-crooked.com/a/70e948f8285d180d This is the first time I've seen anything related to (scoped) allocators work better in the libstdc++ implementation than in the boost implementation, though. I'd still focus on the boost types around here.

Moritz Perschke Over a year ago

Thank you for the beatiful implementation of the DS. But the issue remains that making the pointer to the DS in shared memory causes a runtime error "Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address (...)" Should this work theoretically and my issue is elsewhere? What confuses me most is that is a reading address error

sehe Over a year ago

It doesn't, though? I supplied a working example that you can see working for yourself. Did you try it? Maybe you can base a small, self-contained example to illustrate your issue on it. (In other words, your problem is probably "elsewhere", in that "elsewhere" might be exactly addressed in the answer code, namely bullet #2 ("the segment MUST remain mapped, so you MUST have it as a member in the class (or with longer lifetime)"). That's just my best guess though, because I can't see how you adjusted your code)

Moritz Perschke Over a year ago

Yes I just assumed the issue was with my implementation of the DS as I was a bit out of my depth with interprocess, thank you for the clarification, the issue has to be elsewhere.

sehe Over a year ago

It's unclear what you mean. If you mean std::vector then, yes, you can, but you have to adjust other code accordingly (because std::allocator<> is NOT convertible from bip::allocator<>). CAUTION None of this seems useful? It would not work in shared memory every because dynamic allocations would be outside of the shared memory segment (i.e. std::vector allocates from the heap, as always). Pro tip: You can always just show the cryptic compiler messages: coliru.stacked-crooked.com/a/3093530e54d94039

Collectives™ on Stack Overflow

Opening Boost Interprocess segment in Constructor of object

1 Answer 1

Simplified type aliases

Lifetime

Emplace

Live Demo

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Simplified type aliases

Lifetime

Emplace

Live Demo

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related