Is it possible to delete N bytes from begining of a file in C?
N is 30 bytes only.
I can't copy a file because they are very large (sometimes even 100 of GBs)
3
-
1The proper question is not if C will do this, but if the unspecified file system and operating system support such a (mostly) in-place modification or change. If they do, C is one of the most transparent of the many langauges which can take advantage of that fact. If they do not, then no programming language can.Chris Stratton– Chris Stratton2012-12-02 19:33:00 +00:00Commented Dec 2, 2012 at 19:33
-
And the answer is almost surely no. I know of no OS that can do this.R.. GitHub STOP HELPING ICE– R.. GitHub STOP HELPING ICE2012-12-02 19:37:27 +00:00Commented Dec 2, 2012 at 19:37
-
You can move the filecontent after byte 30 to the begin and than truncate the file with truncate on POSIX systems and _chsize on Windows systems.user411313– user4113132012-12-02 21:05:08 +00:00Commented Dec 2, 2012 at 21:05
Add a comment
|
2 Answers
On Linux you can create a loopback device pointing to an offset within another file. Here is such an example from unix.stackexchange:
#!/bin/bash
for ((i=0;i<10000;i++)); do
printf "%7d\n" $i
done >/var/tmp/file
losetup -v -f -o 512 --sizelimit 512 /var/tmp/file
losetup -a
head -2 /var/tmp/file
echo ...
tail -2 /var/tmp/file
echo ===
head -2 /dev/loop0
echo ...
tail -2 /dev/loop0
printf "I was here" > /dev/loop0
grep here /var/tmp/file
losetup -d /dev/loop0
output:
loop device: /dev/loop0
/dev/loop0: [0808]:136392 (/var/tmp/file), offset 512, size 512
0
1
...
9998
9999
===
64
65
...
126
127
I was here 65
5 Comments
R.. GitHub STOP HELPING ICE
Sparse files will not help.
Mahmoud Al-Qudsi
@R I've updated my solution, but I'm almost certain it can be pulled off with sparse files. If you split the file to two blocks, then you should be able to bypass the first somehow. I'm just not sure what that way would be off the top of my head.
R.. GitHub STOP HELPING ICE
There's not a way, and the amount OP wants to delete is not a multiple of the blocksize anyway.
R.. GitHub STOP HELPING ICE
Huh? What does the loopback device have to do with anything? Actually the important size is the page size anyway, not the hardware block size. You cannot change the alignment of data modulo the page size because it would make cache/mappings of the file go haywire.
R.. GitHub STOP HELPING ICE
Ah, I see your updated answer for a trick that'll work, It's a rather expensive trick though -- creating a new device for each "shifted" file you need to work with. But it may solve OP's problem
You can't delete 30 bytes from the beginning of the file, but nothing stops you from overwriting the file (at least, on most Unix filesystems). This amounts to a copy in terms of the amount of time it takes, but it doesn't use up your free disk space.
Here's a little function which might help, although I didn't test it thoroughly.
One warning: it used to be the case that off_t was a 32-bit signed integer, and therefore pread, pwrite and ftruncate could not be used on files larger than 2GB. On Linux, this is no longer the case. However, it may be necessary to use pread64, pwrite64 and ftruncate64. Don't try this on a really large file you care about until you've verified that it actually works on your OS.
int fshift(int fd, off_t to, off_t from) {
if (from <= to) {
fprintf(stderr, "%s\n", "Unimplemented: fshift can only shift left");
errno = ERANGE;
return -1;
}
char buffer[BUFFER_SIZE];
for (;;) {
ssize_t n_read = pread(fd, buffer, BUFFER_SIZE, from);
if (n_read < 0) {
int tmp = errno;
perror("fshift: could not read file");
errno = tmp;
return -1;
}
from += n_read;
if (n_read == 0) {
int rc = ftruncate(fd, to);
if (rc < 0) {
int tmp = errno;
perror("fshift: could not truncate file");
errno = tmp;
}
return rc;
}
for (char* p = buffer, *lim = &buffer[n_read]; p < lim;) {
ssize_t n_write = pwrite(fd, p, lim - p, to);
if (n_write < 0) {
int tmp = errno;
perror("fshift: could not write file");
errno = tmp;
return -1;
}
p += n_write;
to += n_write;
}
}
}
