Django import export - Unable to import model with BinaryField

Question

I'm trying to import data from a model which has nullable BinaryField. The data doesn't contain the field and I want it to be imported with a null value in the field. If the field already exists in the database for a given id, it should keep the value as it is.

I removed the field from the fields whitelist in the corresponding Resource object and added it to the exclude blacklist. However, I'm getting this error while importing - can't pickle memoryview objects.

Traceback:

Traceback (most recent call last):
File "/lib/python3.5/site-packages/import_export/resources.py", line 451, in import_row
original = deepcopy(instance)
File "/lib/python3.5/copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "/lib/python3.5/copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "/lib/python3.5/copy.py", line 155, in deepcopy
y = copier(x, memo)
File "/lib/python3.5/copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/lib/python3.5/copy.py", line 174, in deepcopy
rv = reductor(4)
TypeError: can't pickle memoryview objects

Package versions - django==1.11, django-import-export==0.6

EDIT:

class ABC(models.Model):
    name = models.CharField('Name', max_length=128, blank=False, null=False)
    binary_field = models.BinaryField('Some name', null=True, blank=True)

class ABCResource(resources.ModelResource):
    class Meta:
        model = ABC
        fields = (
            'id',
            'name',
        )
        import_id_fields = ('id',)
        exclude = ('binary_field',)

class ABCAdmin(ImportExportModelAdmin):
    form = ABCModelForm
    list_display = (
        'id',
        'name',
    )
    exclude = ('binary_field',)
    resource_class = ABCResource

class ABCModelForm(forms.ModelForm):
    class Meta:
        model = ABC
        exclude = ['binary_field']

brain-fart, sorry. @Roshan does this occur with the latest 0.7 release? — Jason
– Jason, Commented Jan 31, 2018 at 13:44
It's really a huge waste of resources, time and effort to use django import export or to even attempt to dump a database at application level at all. Use the tools that are custom made for this - that is your databases dump tool. — e4c5
– e4c5, Commented Feb 19, 2018 at 5:29

John Moutafis · Accepted Answer · 2018-02-14 11:14:03Z

Sadly this seems to be a problem with django-import-export module.

In Python 2.7+ and 3.5.2+, pickling memoryview objects, is permitted. Try the following code just to confirm that:

import pickle

test = memoryview(b'abc')
test.__reduce__()

The above will raise the following error:

Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle memoryview objects

Apparently, in line 451 of django-import-export (version 0.6), the BinaryField is passed as a memoryview object to

original = deepcopy(instance)

which causes the error (instance contains the BinaryField as a memoryview object).
In the current version (1.0.0) this happens in line 446.

The instance retrieved/generated by the instance_loader of the module doesn't take into consideration the BinaryField.

You should probably open a ticket at django-import-export about this.

shashank · Accepted Answer · 2018-02-07 08:19:35Z

0

Maybe passing the actual array instead of the memory view can solve your problem. If you want to execute a function in parallel, all of it parameters have to be picklable if i recall correctly. At least that is the case with python multiprocessing. So you could pass the array to the function and create the memoryview inside your function.

reference: moreinfo

answered Feb 7, 2018 at 8:19

shashank

1,1612 gold badges10 silver badges24 bronze badges

2 Comments

Roshan Over a year ago

Um, I'm not passing a memoryview anywhere. It's probably done implicitly by django-import-export

Matt Hardcastle Over a year ago

The memoryview is created in BinaryFields.to_python method: <github.com/django/django/blob/…> The link is from the current head, but the behavior has been consistent for awhile.

shashank · Accepted Answer · 2018-02-07 08:24:28Z

0

have you added binary_field in ABCModelForm ? If yes, then it is creating issue because It is not possible to include a BinaryField in a ModelForm. For reference : binaryfield

edited Feb 7, 2018 at 8:24

shashank

1,1612 gold badges10 silver badges24 bronze badges

answered Feb 3, 2018 at 19:49

Pushplata

6025 silver badges10 bronze badges

1 Comment

Roshan Over a year ago

No, I have exclude = ['binary_field'] in the ModelForm's Meta class

Collectives™ on Stack Overflow

Django import export - Unable to import model with BinaryField

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related