sqlite performance reading blob using sqlite4java

Question

I have a table with about 100 records. One record contains a String (identifier ) and a Blob. The Blob is a serialized Path ( Android ) The Blobs have a size of between 20kb and 100kb. ( I want the data be stored in the db and not on the disk as file ). The Database is readonly.

I like to load the blobs into path-objects and display as fast as possible at startup. After loading from the db ( dbcache ), I store them in a LRUCache. After referencing the pathes from the LRUCache its very fast. First load from DB is slow. ( Loading about 100 pathes ( 20kb - 100kb ) dures 5 seconds, which I like to boost ). Any hints ?

I found the free sqlite4java ( JNI ) libraries, which where mentoined regarding performance and sqlite.

Could this be a solution for me to enhance the performance of reading ? Any experiences with this ?

duffymo · Accepted Answer · 2013-09-16 12:07:02Z

1

I don't think so.

You don't say if these are read-only reference data or writable transactions. That makes a difference.

Unless you're going to be reading these data into an array repeatedly, I'd say that the performance is unlikely to be an issue for CPU or memory. 1000 documents of 0.2MB each will consume 200MB of RAM. If they're static Strings in a Map, I'd bet they'll end up in perm gen. You should size that accordingly.

Choose the data structure well. Make it a WeakHashMap with SoftReference value types.

Are you sure you'll need all of them, all the time?

You're assuming that you're going to have a performance issue before you have one. I'd do the simplest thing that can work, measure the performance, and only complicate matters when necessary.

answered Sep 16, 2013 at 12:07

duffymo

310k46 gold badges376 silver badges571 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

mcfly soft Over a year ago

Thanks for your help. I precised my question. I am only reading. ( READONLY ). I have to load them all at start, which is not a memory problem for me.

duffymo Over a year ago

I don't believe you know where the time is being spent. I'd measure it before making this more complicated. If it's a one-time read-only situation, maybe you could just read them in a background thread or during down times.

mcfly soft Over a year ago

I already have a background thread and the shapes popup each after the other which is working fine. No GUI Issue. I just searching for a faster way.

Collectives™ on Stack Overflow

sqlite performance reading blob using sqlite4java

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related