6

I'm trying to build a simple convertor for any object to byte array in Python. I took a look on pickle but it only works creating a file and this is not what I need. I also checked json.dump but some objects need a serializer to be dumped.

I need a convertor that keeps my object in memory and can convert any object to byte array.

Improve this question
10
  • Have you looked at the bytearray function? Commented Jun 25, 2020 at 0:19
  • Yes, it seems like bytearray only convert strings to bytearray. Do you have an example in which I can convert any type of object to byte array @Abion47? Commented Jun 25, 2020 at 0:20
  • Any arbitrary type, no questions asked? No, and I don't believe it is possible without some serious reflection work, and even then it's debatably more trouble than its worth. If you want to serialize a type (especially a custom type), you typically have to define how it can be serialized (e.g. by conforming to the buffer interface so you can serialize it with bytearray). Commented Jun 25, 2020 at 0:22
  • 1
    To talk about pickle, though, what makes you think it only works by writing it to files? Have you tried the dumps function? Commented Jun 25, 2020 at 0:23
  • 1
    @R.Karlus you can use pickle.dumps, but why do you need to convert it into bytes? What is the purpose? Commented Jun 25, 2020 at 0:34

2 Answers 2

Reset to default
20

I'm adding this as an actual answer since it seemed to address your needs. Your use of pickle.dump is the problem since it is designed to take the serialized object and write it to a file (though as pointed out in other answers, it doesn't necessarily have to, but that's beside the point).

The function you want to be using is pickle.dumps, which returns the serialized object directly as a byte array:

someobject = 123
someobject_bytes = pickle.dumps(someobject)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Awesome! Fantastic! Voted up.
pickle.loads(someobject_bytes) to deserialized the bytes
1

The file parameter to Pickler just needs to have a write(b) method that receives the bytes. In principle you can write your own class with your method write(self,b) that does whatever you want with those bites and pass an object of that class to Pickler

Example:

import pickle

class File:
  def write(self,b):
    print(b)

f = File()
p = pickle.Pickler(f)

object = 3.14
p.dump(object)

Output: b'\x80\x03G@\t\x1e\xb8Q\xeb\x85\x1f.'

Instead of printing the bytes, like in this example, you could append them in a list, or anything else you want.

Improve this answer

10 Comments

This is a lot of work to make dump work when you can just use dumps.
I solved the problem using pickle dumps method. But here is my upvote for the help.
@Abion47 This is not 'to make dump work', but to show that it doesn't have to be a file when it is used. The two forms of delivering values, as a return value or as a callback have their cases in which one is more preferable than the other.
Sure, you can use dump with something that isn't a file object. But you have to define a custom class that adheres to a specific interface that defines a function which receives the bytes as a parameter. Or... you can skip all that and get the bytes directly using dumps. Your answer is an interesting factoid, but I am unable to think of a situation where it would actually be a useful alternative solution for OP.
@Abion47 There should be many posts online where you can learn when using callbacks rather than return values is more convenient. This one is one. I can think of some: The serialization doesn't fit in memory and you want to do something with it, like compare it to another, for example.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.