MongoDB and PHP Library Cursor Timeout

Question

While using official MongoDB's PHP Library (https://docs.mongodb.com/php-library/master/tutorial/install-php-library/), how to set the cursor timeout as infinite? I read mixed documentation and it's often hard to understand if it refers to the old PHP Driver or the new one (which I am talking about).

For instance:

$cursor = $col->find();
foreach ($cursor as $document) {
   // slow code..
}

How to prevent the cursor from timing out (see error below) and making sure the cursor is closed afterwards without any memory leaks?

Fatal error: Uncaught MongoDB\Driver\Exception\RuntimeException: 
cursor id 123456789 not found in /var/www/html/code.php:1

There are some similar questions here (like this) but it seems we lack a definite reference.

Have you tried setting noCursorTimeout option (although not a good idea because setting cursor timeout as infinite as will keep the cursor open for forever) for find operation or specify batchSize for query ? — s7vr
– s7vr, Commented Aug 13, 2018 at 17:45
re noCursorTimeout .. but then how to close the cursor manually? just setting this option to true seems to be just half of the solution? — JoeSlav
– JoeSlav, Commented Aug 13, 2018 at 18:15
I agree that is not even a recommend option. I'm afraid I don't know a way to close the cursor from client side ( may be cursor.close when error is intercepted ) but one can set timeout on the server. Something like mongod --setParameter cursorTimeoutMillis=time in millis kills the cursor after a timeout. What about the other solution ? using batchsize ? Set to lower number so your slow code have enough time to run and request the results frequently on the server. — s7vr
– s7vr, Commented Aug 13, 2018 at 19:26
Which PHPLIB version are you using ? Also, did you try to set batchSize() as mentioned on the other post ? — Wan B.
– Wan B., Commented Aug 14, 2018 at 7:19
I use: MongoDB server version: 4.0.1 / pecl mongodb 1.4.3 stable / Library is probably 1.3.2. I have not fiddle yet with batchSize as I am debating as to whether that will cause races in my code. But your point is understood. Thanks to both! — JoeSlav
– JoeSlav, Commented Aug 14, 2018 at 8:02

Wan B. · Accepted Answer · 2018-08-15 04:49:44Z

cursor id 123456789 not found in /var/www/html/code.php:1

Typically this is because the application is spending too long between getMore commands. In other words, the cursor returns a number of records on the first iteration and the loop takes a long time before it requests more records.

$cursor = $collection->find( [ 'field' => 'foobar'] );
foreach ($cursor as $document) {
    // long running processes (slow)
}

Cursors have a timeout on the server, after ~10 minutes if the client has not sent any commands to the server it will be closed due to inactivity. In the case above, when it's requesting for the next batch, the cursor has been killed resulting in the error message cursor id not found.

Some tried to disable the cursor timeouts by setting noCursorTimeout:true on the cursor. Although this is not recommended, because you may end up with a cursor that lives forever (zombie) when there's an issue with getMore results returning from the server. i.e. cursors with noCursorTimeout may remain alive on the server long after the client disconnected.

There are a couple of possible solutions:

Decrease cursor.batchSize(). This is because reducing the number of records per batch reduces the cursor inactivity. i.e. Previously 100 records processed for 15 minutes, now it's only 50 records for 7.5 minutes.
Manually create a session i.e. PHPLIB MongoDB\Client::startSession(). Then pass this as a session option on the queries. At regular intervals during the long-running iteration, perform some other database interaction using the same session i.e. ping the server. This will keep the session active (30 minutes timeout); however, it will do nothing to keep the cursor alive, so you may combine this with the use of noCursorTimeout.

Collectives™ on Stack Overflow

MongoDB and PHP Library Cursor Timeout

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related