1

Is there any way to create many objects at once in Python?

Let say, that I want to populate a list with 1.000.000 object instances. Instead of initializing objects one-by-one in loop, is there any way to create them in one call?

Iterating one-by-one takes some time and I am looking for a way other than running many threads to reduce this time.

My adventure with Python started a month ago, so I am quite unaware of many Python specific tricks. Any hints how to deal with this problem?

Improve this question
5
  • 1
    What about lazily creating objects as you use them instead of creating from the start? check defaultDict for example. Commented Aug 16, 2014 at 18:38
  • some packages such as numpy let you create a large array quickly, but generally, you have to create them one at a time. List comprehensions like [MyClass(x) for x in range(1000000)] tend to be faster than for loops because there are fewer name lookups. Commented Aug 16, 2014 at 18:40
  • 1
    Is there enough memory for 1,000,000 objects? What kind of post processing? Can they be dealt with 1 by 1? Is random access to any of the objects needed? What kind of objects? Are they muteable or immutable? Do they depend on some value of X when created? Do you need to search them for a value? Do you need to be able to order them or save them? Do you need to insert values at X location? Are they to be kept unique (like a set)? Commented Aug 16, 2014 at 20:27
  • There is an existing list of N objects that contain some data, that by creating an BLL object for each existing objects, is parsed to JSON and returned. Random access is not needed, I am looking for a solution to boost parsing process that is being carried out by creating N BLL's. Thought that something like mass object allocation could work. Commented Aug 17, 2014 at 13:10
  • VTR. This question and the linked duplicate are both talking about a list of objects, but this one is asking about the objects while the other one is asking about the list. Commented Mar 22, 2021 at 0:46

3 Answers 3

Reset to default
4

You must iterate N times to make N objects, either in a for loop or list comp.

objectList = [object() for _ in range(1000000)]

For example to instatiate a list of int

intList = [int() for _ in range(1000000)]

For a small list of 10 int:

>>> [int() for _ in range(10)]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1 
objectList = [None] * 1000  # Make a list of 1000 None's

ADDED

If you are wanting to preallocate a list to avoid the overhead of reallocation the list as it continues to grow, this construct is appropriate as is will allocate the entire memory for the list and initialize each member of the list using the same object.

If you need the initialize each member to a different object, you have to initialize each list element separately via a loop, list comprehension, etc.

Memory reallocation under Windows tends to be a good bit slower than under Unix based systems, so this approach can be faster than simply initialing the loop as this construction runs at native speed.

Improve this answer

2 Comments

What if all the objects aren't None or aren't all the same?
See explanation that I was apparently typing when you asked this.
0

I would use a generator:

>>> def obj(x):
...    print 'obj called with', x
...    return x
... 
>>> gen=(obj(x) for x in xrange(10))

Notice at this point, obj has not been called. Once you actually convert to a list (or much better still, iterate over it) the objects will be created in turn:

>>> list(gen)
obj called with 0
obj called with 1
obj called with 2
obj called with 3
obj called with 4
obj called with 5
obj called with 6
obj called with 7
obj called with 8
obj called with 9
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Improve this answer

2 Comments

How would this be faster?
@martineau: That is the right question, but the OP did not really specify information that would allow one to determine. See my questions on the post...

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.