Assign std::vector<std::byte> to std::vector<char> WITHOUT copying memory

Question

I have a function that returns a std::vector<std::byte>

I am aware that std::byte is not a character type nor an integral type, and that converting it to char is only possible through a typecast. So far so good.

So I would like (in cases where I know that the vector only contains character data) to transfer ownership of the underlying buffer from the std::vector<std::byte> to a std::vector<char> using std::move, so as to avoid copying the entire underlying buffer.

When I try doing this, I get this error:

no suitable user-defined conversion from "std::vector<std::byte, std::allocatorstd::byte>" to "std::vector<char,std::allocator>" exists

Is this at all possible using C++? I think there are real use cases where one would want to do this

I think you can't because when you've defined them you've made them std::bytes, hence you can't move them as new types. moving means transferring the ownership i.e the elements themselves will be moved, so If you can change the type of some object (without copying), you can do what you want. — asmmo
– asmmo, Commented Jun 21, 2020 at 22:31
I don't think this is possible. Casting may work in practice (or it may not) but it will be undefined technically. I think it may be worth investing in developing a std::vector like type that has the ability to adopt external memory. Or consider std::span. — Galik
– Galik, Commented Jun 21, 2020 at 22:36
@matt I doubt the STL will ever have such features as it is a very general purpose library. If you want to do low-level (advanced/dangerous) stuff you're on your own (or find a library that supports that). — Galik
– Galik, Commented Jun 21, 2020 at 22:58
@matt: "It would make sense to have a std::vector class that can adopt external memory." But vector<T> doesn't store memory; it stores an array of Ts. It is reasonable to allow a vector<T> instance to adopt the storage from another vector<T> instance. It makes far less sense for it to be able to adopt the "memory" of some unrelated type vector<U>, since that is an array of Us, which is not an array of Ts. — Nicol Bolas
– Nicol Bolas, Commented Jun 22, 2020 at 0:19

Ted Lyngmo · Accepted Answer · 2020-08-15 06:33:02Z

8

I would probably leave the data in the original vector<byte> and make a small class that keeps a reference to the original vector<byte> and does the necessary casting when you need it.

Example:

#include <cstddef>
#include <iostream>
#include <vector>

template<typename T>
struct char_view {
    explicit char_view(std::vector<T>& bytes) : bv(bytes) {}

    char_view(const char_view&) = default;
    char_view(char_view&&) = delete;
    char_view& operator=(const char_view&) = delete;
    char_view& operator=(char_view&&) = delete;

    // capacity
    size_t element_count() const { return bv.size(); }
    size_t size() const { return element_count() * sizeof(T); }

    // direct access
    auto data() const { return reinterpret_cast<const char*>(bv.data()); }
    auto data() { return reinterpret_cast<char*>(bv.data()); }

    // element access
    char operator[](size_t idx) const { return data()[idx]; }
    char& operator[](size_t idx) { return data()[idx]; }

    // iterators - with possibility to iterate over individual T elements
    using iterator = char*;
    using const_iterator = const char*;

    const_iterator cbegin(size_t elem = 0) const { return data() + elem * sizeof(T); }
    const_iterator cend(size_t elem) const { return data() + (elem + 1) * sizeof(T); }
    const_iterator cend() const { return data() + size(); }

    const_iterator begin(size_t elem = 0) const { return cbegin(elem); }
    const_iterator end(size_t elem) const { return cend(elem); }
    const_iterator end() const { return cend(); }
    
    iterator begin(size_t elem = 0) { return data() + elem * sizeof(T); }
    iterator end(size_t elem) { return data() + (elem + 1) * sizeof(T); }
    iterator end() { return data() + size(); }

private:
    std::vector<T>& bv;
};

int main() {
    using std::byte;

    std::vector<byte> byte_vector{byte{'a'}, byte{'b'}, byte{'c'}};

    char_view cv(byte_vector);

    for(char& ch : cv) {
        std::cout << ch << '\n';
    }
}

Output:

a
b
c

A simpler option if you only need const access could be to create a string_view:

template<typename T>
std::string_view to_string_view(const std::vector<T>& v) {
    return {reinterpret_cast<const char*>(v.data()), v.size() * sizeof(T)};
}
//...
auto strv = to_string_view(byte_vector);

edited Aug 15, 2020 at 6:33

answered Jun 21, 2020 at 23:35

Ted Lyngmo

124k7 gold badges93 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Aykhan Hagverdili Over a year ago

Off-topic: user-defined byte literals would probably be neater than C style casts

Aykhan Hagverdili Over a year ago

While we're at it, let's generalize this view thing to help spread UB gcc.godbolt.org/z/TR6rsy

Ted Lyngmo Over a year ago

@＿Static＿assert "user-defined byte literals" I totally agree. I tried to keep it short and I couldn't find any pre-defined user-defined byte literals. "let's generalize this view thing to help spread UB" :-) I think that by staying at char we should be fine?

Aykhan Hagverdili Over a year ago

Sure, reinterpreting as char array should be fine, but where's the fun in that? :)

Andrew Over a year ago

Is there some good reason why some move/conversion isn't supported in the language? I run into this frequently and it's a PITA.

|

M.M · Accepted Answer · 2020-06-22 02:59:38Z

2

std::vector does not allow attaching or detaching to memory allocations , other than moves from a vector of exactly the same type. This has been proposed but people raised (valid) objections about the allocator for attaching and so on.

The function returning vector<byte> constrains you to work with a vector<byte> as your data container unless you want to copy the data out.

Of course, you can alias the bytes as char in-place for doing character operations.

answered Jun 22, 2020 at 2:59

M.M

143k26 gold badges223 silver badges400 bronze badges

Comments

catnip · Accepted Answer · 2020-06-23 20:48:37Z

-6

You can achieve this with a cast, as shown below. This is legal because the cast is to a char reference (if casting to any other type it would be UB) but, with gcc at least, you still have to compile it with -fno-strict-aliasing to silence the compiler warning. Anyway, here's the cast:

std::vector <char> char_vector = reinterpret_cast <std::vector <char> &&> (byte_vector);

And here's a live demo

edited Jun 23, 2020 at 20:48

answered Jun 21, 2020 at 23:02

catnip

26.3k4 gold badges33 silver badges58 bronze badges

2 Comments

Galik Over a year ago

flak flak flak.

Aykhan Hagverdili Over a year ago

Since you're bringing the big guns, why do you even need std::move anymore. Just auto char_vector = reinterpret_cast<std::vector<char> &&>(byte_vector);. To UB and beyond!

Collectives™ on Stack Overflow

Assign std::vector<std::byte> to std::vector<char> WITHOUT copying memory

3 Answers 3

8 Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related