Memory allocator with custom pointer type

Question

I tried to create a custom memory allocator which uses a smart pointer. I do not post the code because it's too big and doesn't add much of information. Then I tested it with a std::vector. It works perfectly well on Xcode. But when I tried to build the same code in Visual Studio 12 (2013), the build failed with the following error:

...vector(873): error C2660: 'std::_Wrap_alloc< my_allocator< int > >::construct' : function does not take 2 arguments

the problem is in push_back method:

void push_back(value_type&& _Val)
    {
    ....
        this->_Getal().construct(this->_Mylast,
            _STD forward<value_type>(this->_Myfirst[_Idx]));
    ....
    }

The error message is a bit confusing. Real problem is that this->_Mylast is of type my_allocator< int >::pointer, which is a smart pointer, and construct method expects int*.

So, the question is simple: what are the requirements to pointer types used in a custom memory allocator? Should X::pointer be convertible to a raw pointer? If yes, it makes them pretty useless.

Actually I would expect that line of code to look like:

this->_Getal().construct(addressof(*(this->_Mylast)),
            _STD forward<value_type>(this->_Myfirst[_Idx]));

Let's try to find an answer in C++ standard, which says:

[17.6.3.5-5] An allocator type X shall satisfy the requirements of CopyConstructible (17.6.3.1). The X::pointer, X::const_pointer, X::void_pointer, and X::const_void_pointer types shall satisfy the requirements of NullablePointer (17.6.3.3). No constructor, comparison operator, copy operation, move operation, or swap operation on these types shall exit via an exception. X::pointer and X::const_pointer shall also satisfy the requirements for a random access iterator (24.2)

If we take a look at NullablePointer reqs, they add few other requirements:

[17.6.3.3] A NullablePointer type is a pointer-like type that supports null values. A type P meets the requirements of NullablePointer if:
(1.1) — P satisfies the requirements of EqualityComparable, DefaultConstructible, CopyConstructible, CopyAssignable, and Destructible...

If I check random access iterator requirements, I also don't find any explicit mentioning of its casting to a raw pointer. But in few places the approach with addressof is used (e. g. 24.2.1-5).

Also, it's not the only place in Microsoft's std::vector implementation where X::pointer and raw pointer are assumed to be equal. I'm wondering, what do I miss?

EDIT: I'll add a piece of my_allocator deffinition here:

class my_allocator
{
public:

typedef std::size_t          size_type;
typedef std::ptrdiff_t       difference_type;
typedef my_ptr<T>            pointer;
typedef my_ptr<const T>      const_pointer;
typedef T&                   reference;
typedef const T&             const_reference;
typedef T                    value_type;
typedef my_ptr<void>         void_pointer;
typedef my_ptr<const void>   const_void_pointer;

<constructors>

pointer allocate(size_type n, const_void_pointer = nullptr);
void deallocate(const pointer& ptr, size_type elements_num);
};

Can you provide us with all typedefs in your allocator? E.g. const_pointer and so forth? (It would absolutely not surprise me if VC++' standard library is not fully compliant) — Columbo
– Columbo, Commented Dec 25, 2014 at 15:09
Most likely VC++ standard library is buggy. I'd thought that construction of std contained objects should be done via allocator_traits::construct rather than directly allocator::construct. — Walter
– Walter, Commented Dec 25, 2014 at 16:31
@Walter, actually it does use allocator_traits. _Getal() returns a wrapper std::_Wrap_alloc< my_allocator< int > >. — MaksymB
– MaksymB, Commented Dec 25, 2014 at 16:45
Hello Maxym, were you able to run your allocator with custom pointer on VS STL finally? I try to build similar code on VS2015 and stuck on the same issue. I see boost/interprocess has dedicated set of containers to deal with such problem (it persists data in mem mapped file for instance), but I'd like to stick to minimal set of third party libs as possible... — Aleksander Stankiewicz
– Aleksander Stankiewicz, Commented Aug 25, 2016 at 18:15

Community · Accepted Answer · 2017-05-23 10:30:40Z

14

To solve this problem I created a to_raw_pointer function which happens to work on any "fancy pointer" which implements operator->(). You can find it in the libc++ implementation.

Here it is:

template <class _Tp>
inline _LIBCPP_INLINE_VISIBILITY
_Tp*
__to_raw_pointer(_Tp* __p) _NOEXCEPT
{
    return __p;
}

template <class _Pointer>
inline _LIBCPP_INLINE_VISIBILITY
typename pointer_traits<_Pointer>::element_type*
__to_raw_pointer(_Pointer __p) _NOEXCEPT
{
    return _VSTD::__to_raw_pointer(__p.operator->());
}

It works by calling the operator->() in an unconventional way. This operator must either call another operator->(), or return a real pointer. The overload for real pointers breaks the recursion with an identity function. So this would be used like:

this->_Getal().construct(__to_raw_pointer(this->_Mylast),
            _STD forward<value_type>(this->_Myfirst[_Idx]));

construct is specified to take a real pointer, not a fancy pointer. And there is no implicit conversion specified from fancy pointers to real pointers. The container must use something such as to_raw_pointer, or addressof.

The container is also required to call construct via allocator_traits, instead of calling it directly on the stored allocator as shown. This is to allow construct to be "defaulted" by allocator_traits, as opposed to requiring the allocator to implement construct.

Currently both operator*() and operator->() generally require the fancy pointer to be non-null prior to calling that operator on it. However I am anticipating that this requirement will be relaxed for operator->() in the future.

Update

I was in a bit of a hurry when I wrote the above. Now that I have the time, I was going to include the complete requirements on the allocator::pointer types. However on re-reading the question I see that Maxym has already done a good job of that in the question, so I won't repeat them here.

The one thing that is in the std, but is not completely obvious, is the implicit and explicit conversions among the four pointer types: pointer, const_pointer, void_pointer, and const_void_pointer:

implicit allocator pointer conversions:
+--------------------------------------+
| pointer      -->  const_pointer      |
|    |    \               |            |
|    |      ---------     |            |
|   \|/             _\|  \|/           |
| void_pointer -->  const_void_pointer |
+--------------------------------------+


explicit allocator pointer conversions:
+--------------------------------------+
| pointer           const_pointer      |
|   /|\                  /|\           |
|    |                    |            |
|    |                    |            |
| void_pointer      const_void_pointer |
+--------------------------------------+

That is, you can implicitly convert from non-const to const, and from non-void to void, and you can explicitly convert from void to non-void. But there is no way for a container to const_cast (cast away const-ness) from an allocator::const_pointer or allocator::const_void_pointer. Once the container goes const, it can never get back.

edited May 23, 2017 at 10:30

CommunityBot

11 silver badge

answered Dec 25, 2014 at 17:28

Howard Hinnant

225k55 gold badges493 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

MaksymB Over a year ago

Cool, but it's actually the same as addressof(*(this->_Mylast)) because pointer indirection is anyway required by the standard. What I try to figure out is whether I do something wrong (because I misinterpret C++ standard) or it's a bug in VS STL implementation.

MaksymB Over a year ago

So, am I right to conclude that it's a bug in VC++ STL implementation? (Since X::pointer to raw pointer and vice versa is not allowed).

Howard Hinnant Over a year ago

Yes: construct is specified to take a real pointer, not a fancy pointer. And there is no implicit conversion specified from fancy pointers to real pointers. The container must use something such as to_raw_pointer, or addressof.

Potatoswatter Over a year ago

Is this preferable to addressof( * ptr )? Neither operator* nor operator-> are required, but * intuitively seems like a safer bet.

Howard Hinnant Over a year ago

A subtlety is the question of whether you can dereference ptr or not. If you can't, then obviously addressof( * ptr ) won't work. Currently the standard says that to use operator-> with a ptr also requires that you be able to dereference ptr. I believe the standard is too strict in this requirement. Only if you select a member with operator-> do you need to dereference. If you call ptr.operator->(), ptr doesn't actually get dereferenced. The standard should be clarified to say so.

|

Collectives™ on Stack Overflow

Memory allocator with custom pointer type

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related