Issue upgrading from Neo4j 2 to 3 using embedded Java API: Exception in hasNextIO

Question

Thanks in advance for the attention and any suggestions. I hope my explanation of the issue is clear enough.

As noted in the subject line, I'm upgrading my java based project to Version 3 of the embedded API. Unfortunately, I'm getting the following error:

SEVERE: exception in hasNext: The statement has been closed.

Code is below, but here's the explanation. In order to provide a layer of isolation in between my application code and the database code (in this case Neo4j), I'm using the DAO pattern. I'm assigning the ResourceIterator I'm getting back from the findNodes call to a variable in the user level iterator I'm defining. According to this page (https://neo4j.com/docs/java-reference/current/javadocs/org/neo4j/graphdb/Transaction.html)

All ResourceIterables that where returned from operations executed inside a transaction will be automatically closed when the transaction is committed or rolled back. Note however, that the ResourceIterator should be closed as soon as possible if you don't intend to exhaust the iterator

That's clear enough and seems to explain why I am getting the exception. But there has to be some way to return an iterator for later use. At least I hope so. The other choice that comes to mind would be to store all of the nodes in user memory so my user level interface can iterate through them, and that would be a bit problematic. I'll note that this code (which used to use the GlobalGraphOperations class) worked in Neo4j 2. but maybe that was a bug in Neo4j 2? Note that I have tried not closing the transaction (or even using a transaction at all).

Any ideas?

Here's the actual source of the problem: It's in my definition of an iterator for the user visible structure representing nodes. The line throwing the exception is the call to hasNext.

  public boolean hasNext() {
       // Wrap the neo4j iterator
       boolean hasN = false;
       GraphDatabaseService theDB = Neo4jDAOFactory.getTheNetworkDB();
       Transaction tx = theDB.beginTx();
       try {
          hasN = nodeIterator.hasNext();
       } catch (Exception e) {
           // should send this back using the message logs eventually
           this.logger.log (Level.SEVERE, "exception in hasNext: " + e.getMessage(), e);
       } finally {
           tx.close();
       }
       return hasN;
   }

But it doesn't make sense without also looking at:

public Iterator<NetworkNodeTransferObject>
    getNetworkNodes(String nameSpace, String key, Object value){

    Neo4jNetworkNodeDAOIterator theIterator = null;
    GraphDatabaseService theDB = Neo4jDAOFactory.getTheNetworkDB();
    Transaction tx = theDB.beginTx();
    try {

        Label newLabel = Label.label(nameSpace);
        Iterator<Node> neo4jNodeList = theDB.findNodes(newLabel, key, value);
        theIterator = new Neo4jNetworkNodeDAOIterator();
        theIterator.nodeIterator = neo4jNodeList;
    } catch (Exception e) {
        // should send this back using the message logs eventually
        this.logger.log (Level.SEVERE, "exception in getNetworkNodes: " + e.getMessage(), e);
    } finally {
        tx.close();
    }

    return theIterator;
}

Thanks Howard

Frank Pavageau · Accepted Answer · 2016-08-22 10:33:42Z

0

I'll address your hasNext() implementation first: you can't wrap nodeIterator.hasNext() in some new transaction at that point, it has to be used in the context of the transaction in which it was created.

So basically it doesn't need to be more than

public boolean hasNext() {
    return nodeIterator.hasNext();
}

Now, regarding your transaction problem: to return an iterator, you need to keep the transaction open at least as long as the iterator itself. There are multiple ways of doing that:

you can manage transactions at a higher layer, i.e. in the caller of getNetworkNodes or above, so getNetworkNodes would expect a transaction to exist and not concern itself with its creation

you can keep the transaction in your user-level iterator, alongside the nodeIterator, but it can be tricky to close it reliably. If you always consume the iterator until the end, closing the transaction either when you reach the end or when there's an error (since it will probably interrupt the iteration anyway) should do the trick:

public boolean hasNext() {
    boolean hasN = nodeIterator.hasNext();
    if (!hasN) {
        tx.close();
    }
    return hasN;
}

public NetworkNodeTransferObject next() {
    try {
         Node node = nodeIterator.next();
         // Create the NetworkNodeTransferObject
         return networkNodeTo;
    } catch (RuntimeException e) {
        tx.close();
        throw e;
    }
}

answered Aug 22, 2016 at 10:33

Frank Pavageau

11.7k1 gold badge45 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

HowardLander Over a year ago

Thanks much for the info. One further question: let's say I store the transaction in the iterator and the iterator was not consumed to the end. When the iterator goes out of scope, will the transaction be closed? Is it reasonable to depend on that?

Frank Pavageau Over a year ago

No, it will absolutely not be closed: that would require the Transaction implementation to also implement the finalize() method to maybe get a chance to close the transaction during garbage collection, and that's really brittle (and adds GC overhead). If you don't want to consume the whole iterator, you need to be able to close the transaction manually, e.g. by creating a sub-interface ResourceIterator extends Iterator, AutoCloseable, like Neo4j's ResourceIterable, and closing it explicitly or via a try-with-resources block.

Collectives™ on Stack Overflow

Issue upgrading from Neo4j 2 to 3 using embedded Java API: Exception in hasNextIO

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related