SQLite: ON disk Vs Memory Database

Question

We are trying to Integrate SQLite in our Application and are trying to populate as a Cache. We are planning to use it as a In Memory Database. Using it for the first time. Our Application is C++ based.

Our Application interacts with the Master Database to fetch data and performs numerous operations. These Operations are generally concerned with one Table which is quite huge in size. We replicated this Table in SQLite and following are the observations:

Number of Fields: 60 Number of Records: 1,00,000

As the data population starts, the memory of the Application, shoots up drastically to ~1.4 GB from 120MB. At this time our application is in idle state and not doing any major operations. But normally, once the Operations start, the Memory Utilization shoots up. Now with SQLite as in Memory DB and this high memory usage, we don’t think we will be able to support these many records.

When I create the DB on Disk, the DB size sums to ~40MB. But still the Memory Usage of the Application remains very high.

Q. Is there a reason for this high usage. All buffers have been cleared and as said before the DB is not in memory?

Any help would be deeply appreciated.

Thanks and Regards Sachin

60 fields sounds like a spreadsheet. Look at your data to see if normalising could save you a buttload of space. — Emyr
– Emyr, Commented Nov 8, 2010 at 11:35
Where do you suspect the leak is? in SQLite or our Application? In my Application, the structures and buffers are getting cleaned up properly. — chingupt
– chingupt, Commented Nov 10, 2010 at 4:26

AnonGeek · Accepted Answer · 2012-06-16 06:31:24Z

2

You can use the vacuum command to free up memory by reducing the size of sqlite database.

If you are doing a lot of insert update operations then the db size may increase. You can use vaccum command to free up space.

answered Jun 16, 2012 at 6:31

AnonGeek

8,05811 gold badges42 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

netcoder · Accepted Answer · 2010-11-08 20:38:09Z

1

SQLite uses memory for things other than the data itself. It holds not only the data, but also the connections, prepared statements, query cache, query results, etc. You can read more on SQLite Memory Allocation and tweak it. Make sure you are properly destroying your objects too (sqlite3_finalize(), etc.).

answered Nov 8, 2010 at 20:38

netcoder

68k19 gold badges129 silver badges142 bronze badges

Collectives™ on Stack Overflow

SQLite: ON disk Vs Memory Database

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related