Delete struct from binary file in C++

Question

I'm currently trying to implement a function that deletes a specific struct which is already saved in a binary file. I'm a little confused on how to do it. I have a solution that will work but isn't particularly elegant: I could read the entire file of structs to a vector of structs, delete the old file, delete the struct I want to remove from the vector, then save the vector of structs as a file again. I'm pretty confident this would work but if the file is large it seems like a lengthy and perhaps unnecessary solution. I know how to find the struct I want to delete, and I know how to change the values in it (by overwriting it) but how exactly can I delete it?

This is how I save my struct:

std::fstream binary_file(fileLocation.c_str(), std::ios::out | std::ios::binary | std::ios::app);
binary_file.write(reinterpret_cast<char *>(&myStruct), sizeof(myStruct));
binary_file.close();

each struct has an integer with its unique ID, I use that to find the appropriate struct like so:

myStream.open(fileLocation.c_str(), std::ios::in | std::ios::out | std::ios::binary);
    while (!myStream.eof())
    {
        myStream.read(reinterpret_cast<char*>(&myStruct), sizeof(myStruct));
        if (myStruct.ID == given_ID)
        {
            temp_fstream.seekg(-(sizeof(myStruct), std::ios::cur);
            //delete struct
            return;
        }
    }

I'm at a loss at what to do here, is it even possible? I've toyed around with the idea of just flagging the struct as invalid so even though my program will read it it won't use it for anything, but again, seems like a poor idea.

Any suggestions?

The usual solution is to copy, skipping what you don't want to copy. But your read loop is broken; you don't verify that your read succeeded before using the results. (And of course, if you write the file this way, you're almost guaranteed that some time in the future, you'll be unable to read it.) — James Kanze
– James Kanze, Commented Oct 1, 2014 at 17:20

Jonathan Wood · Accepted Answer · 2014-10-01 17:16:26Z

2

You can recreate the file without the deleted item, but yes that is horribly inefficient.

What's needed here is something a little more involved. For example, you could come up with a system of marking a struct as deleted. Your code would need to write some special value at that location so it would know it was deleted, and then reuse that slot when saving another struct.

This would increase the complexity of your code. You wouldn't be able to simply read and write a collection of structs in a single statement.

There are many variations on this type of approach. The exact code needed kind of depends on your application.

answered Oct 1, 2014 at 17:16

Jonathan Wood

68.1k86 gold badges309 silver badges542 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user3932479 Over a year ago

I guess I'll go with this solution out of the myriad of answers here, simply because I guess it fits my needs the best. I will constantly be removing elements and writing new ones to the files, it's a shame I'll no longer be able to just append when writing new elements, but hopefully it'll work out

Jonathan Wood Over a year ago

If correctly implemented, this route should give you the best performance.

Alnitak Over a year ago

@user3932479 if your structs are all fixed size and the order doesn't matter then it would be easy to refill unused spaces. FWIW, mmap would equally well work in this case too, and in my experience the performance improvement over stdio or streams is staggering.

Martin James Over a year ago

I fthey're fixed-size, you could link all the deleted items into a free list and store the first list index as part of the filename.

Alnitak · Accepted Answer · 2014-10-01 17:17:15Z

2

A potentially very fast method if the file isn't larger than available RAM is to use mmap:

mmap the file
find the memory location of the struct in question
memmove everything above that location down by the size of the structure
munmap the file.
ftruncate the file to its new length

answered Oct 1, 2014 at 17:17

Alnitak

341k72 gold badges420 silver badges503 bronze badges

Comments

tomdzon · Accepted Answer · 2014-10-01 17:18:17Z

0

I'm not sure if I understand you correctly... But, can you just get to that structure (find it in binary file), copy all bytes until that position, skip next sizeof(that struct) bytes, and copy the rest of the binary file to that new file?!

answered Oct 1, 2014 at 17:18

tomdzon

911 silver badge3 bronze badges

Comments

anthonyvd · Accepted Answer · 2014-10-01 17:22:03Z

0

Assuming all the structs in the file are the same size and type and that order doesn't matter, here's what the algorithm would look like:

Locate the struct to be deleted, remember its location
Find the last struct in your file
Copy the last struct's content into the space occupied by the one to be deleted
Either pad the end of your file or use OS-specific functions (truncate() on Unix, SetEndOfFile() on Windows)

AFAIK there's no platform agnostic way to shrink the size of a file so you'll have to use platform specific stuff for that part.

answered Oct 1, 2014 at 17:22

anthonyvd

7,6604 gold badges32 silver badges51 bronze badges

Collectives™ on Stack Overflow

Delete struct from binary file in C++

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related