About PHP parallel file read/write

Question

Have a file in a website. A PHP script modifies it like this:

$contents   = file_get_contents("MyFile");

// ** Modify $contents **

// Now rewrite:
$file       = fopen("MyFile","w+");
fwrite($file, $contents);
fclose($file);

The modification is pretty simple. It grabs the file's contents and adds a few lines. Then it overwrites the file.

I am aware that PHP has a function for appending contents to a file rather than overwriting it all over again. However, I want to keep using this method since I'll probably change the modification algorithm in the future (so appending may not be enough).

Anyway, I was testing this out, making like 100 requests. Each time I call the script, I add a new line to the file:

First call:

First!

Second call:

First!

Second!

Third call:

First!

Second!

Third!

Pretty cool. But then:

Fourth call:

Fourth!

Fifth call:

Fourth!

Fifth!

As you can see, the first, second and third lines simply disappeared.

I've determined that the problem isn't the contents string modification algorithm (I've tested it separately). Something is messed up either when reading or writing the file.

I think it is very likely that the issue is when the file's contents are read: if $contents, for some odd reason, is empty, then the behavior shown above makes sense.

I'm no expert with PHP, but perhaps the fact that I performed 100 calls almost simultaneously caused this issue. What if there are two processes, and one is writing the file while the other is reading it?

What is the recommended approach for this issue? How should I manage file modifications when several processes could be writing/reading the same file?

If those requests are not sequential, file_get_contents() may be being called before data from the previous call is effectively flushed to the file. You may want to use fflush() and flock() to make sure the file is only processed by one script at a time. — Havenard
– Havenard, Commented Aug 21, 2014 at 1:10
And always verify that file_get_contents() returned a string instead of FALSE. — Havenard
– Havenard, Commented Aug 21, 2014 at 1:11
file_get_contents may fail sometimes, try checking it with empty(). I also suggest that you try curl. — user3079266
– user3079266, Commented Aug 21, 2014 at 1:13
empty() doesn't mean it failed, the file may be indeed empty. Check if it is_string() or !== FALSE. — Havenard
– Havenard, Commented Aug 21, 2014 at 1:19

Nicholas Summers · Accepted Answer · 2014-08-21 01:17:38Z

3

What you need to do is use flock() (file lock)

What I think is happening is your script is grabbing the file while the previous script is still writing to it. Since the file is still being written to, it doesn't exist at the moment when PHP grabs it, so php gets an empty string, and once the later processes is done it overwrites the previous file.

The solution is to have the script usleep() for a few milliseconds when the file is locked and then try again. Just be sure to put a limit on how many times your script can try.

NOTICE:

If another PHP script or application accesses the file, it may not necessarily use/check for file locks. This is because file locks are often seen as an optional extra, since in most cases they aren't needed.

edited Aug 21, 2014 at 1:17

answered Aug 21, 2014 at 1:11

Nicholas Summers

4,8014 gold badges24 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jay · Accepted Answer · 2017-10-19 19:09:45Z

3

So the issue is parallel accesses to the same file, while one is writing to the file another instance is reading before the file has been updated. PHP luckily has a mechanisms for locking the file so no one can read from it until the lock is released and the file has been updated.

flock()

can be used and the documentation is here

edited Oct 19, 2017 at 19:09

answered Aug 21, 2014 at 1:10

Jay

2,6861 gold badge18 silver badges24 bronze badges

Comments

Sverri M. Olsen · Accepted Answer · 2014-08-21 01:40:59Z

2

You need to create a lock, so that any concurrent requests will have to wait their turn. This can be done using the flock() function. You will have to use fopen(), as opposed to file_get_contents(), but it should not be a problem:

$file = 'file.txt';
$fh = fopen($file, 'r+');
if (flock($fh, LOCK_EX)) { // Get an exclusive lock
    $data = fread($fh, filesize($file)); // Get the contents of file
    // Do something with data here...
    ftruncate($fh, 0); // Empty the file
    fwrite($fh, $newData); // Write new data to file
    fclose($fh); // Close handle and release lock
} else {
    die('Unable to get a lock on file: '.$file);
}

answered Aug 21, 2014 at 1:40

Sverri M. Olsen

13.2k3 gold badges38 silver badges54 bronze badges

1 Comment

Klox Over a year ago

Good example. Note that as of 5.3.2 you need to call flock($fh, LOCK_UN); before closing the file.

Collectives™ on Stack Overflow

About PHP parallel file read/write

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related