Creating a Persistent Data Object In Django

Question

I have a Python-based maximum entropy classifier. It's large, stored as a Pickle, and takes about a minute to unserialize. It's also not thread safe. However, it runs fast and can classify a sample (a simple Python dictionary) in a few milliseconds.

I'd like to create a basic Django web app, so users can submit samples to classify in realtime. How would I load the classifier into persistent memory once, and then regulate it so that each request can access the object without conflicting with other requests?

bmaeser · Accepted Answer · 2010-02-27 20:27:33Z

4

you could use djangos cache-framework and set the timeout to a extreme value

answered Feb 27, 2010 at 20:27

bmaeser

1,0008 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Cerin Over a year ago

Clever and dead-simple to implement. Great suggestion.

SumNeuron Over a year ago

@Cerin could you elaborate a bit more on the solution

Brian Luft · Accepted Answer · 2010-02-26 22:41:20Z

2

Consider running it in another process. You could have your Django application submit samples via a socket that the classifier process listens on, or you could run a queue and have Django submit requests to the queue.

answered Feb 26, 2010 at 22:41

Brian Luft

1,1738 silver badges13 bronze badges

1 Comment

Cerin Over a year ago

Yes, running the classifier in it's own server would work, but that seems overkill. I was looking for something that utilizes Django's framework.

Collectives™ on Stack Overflow

Creating a Persistent Data Object In Django

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related