save and find json string in mongodb

Question

I'm building a logging application that does the following:

gets JSON strings from many loggers continuously and saves them to a db
serves the collected data as a per logger bulk

my intention is to use a document based NoSQL storage to have the bulk structure right away. After some research I decided to go for MongoDB because of the following features: - comprehensive functions to insert data into existing structures ($push, (capped) collection) - automatic sharding with a key I choose (so I can shard on a per logger basis and therefore serve bulk data in no time - all data already being on the same db server)

The JSON I get from the loggers looks like this:

[
  {"bdy":{
    "cat":{"id":"36494h89h","toc":55,"boc":99},
    "dataT":"2013-08-12T13:44:03Z","had":0,
    "rng":23,"Iss":[{"id":10,"par":"dim, 10, dak"}]
  },"hdr":{
    "v":"0.2.7","N":2,"Id":"KBZD348940"}
  }
]

The logger can send more than one element in the same array. I this example it is just one.

I started coding in Java with the mongo driver and the first problem I discovered was: I have to parse my with no doubt valid JSON to be able to save it in mongoDB. I learned that this is due to BSON being the native format of MongoDB. I would have liked to forward the JSON string to the db directly to save that extra execution time.

so what I do in a first Java test to save just this JSON string is:

String loggerMessage = "...the above JSON string...";
DBCollection coll = db.getCollection("logData");
DBObject message = (DBObject) JSON.parse(loggerMessage);
coll.insert(message);

the last line of this code causes the following exception:

Exception in thread "main" java.lang.IllegalArgumentException: BasicBSONList can only work with numeric keys, not: [_id]
at org.bson.types.BasicBSONList._getInt(BasicBSONList.java:161)
at org.bson.types.BasicBSONList._getInt(BasicBSONList.java:152)
at org.bson.types.BasicBSONList.get(BasicBSONList.java:104)
at com.mongodb.DBCollection.apply(DBCollection.java:767)
at com.mongodb.DBCollection.apply(DBCollection.java:756)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:220)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:204)
at com.mongodb.DBCollection.insert(DBCollection.java:76)
at com.mongodb.DBCollection.insert(DBCollection.java:60)
at com.mongodb.DBCollection.insert(DBCollection.java:105)
at mongomockup.MongoMockup.main(MongoMockup.java:65)

I tried to save this JSON via the mongo shell and it works perfectly.

How can I get this done in Java? How could I maybe save the extra parsing? What structure would you choose to save the data? Array of messages in the same document, collection of messages in single documents, ....

Have you tried removing the [] around your JSON string? If you remove them it shuold work fine. — assylias
– assylias, Commented Aug 12, 2013 at 23:10

Mike Dynamite · Accepted Answer · 2013-08-15 09:41:45Z

1

It didn't work because of the array. You need a BasicDBList to be able to save multiple messages. Here is my new solution that works perfectly:

    BasicDBList data = (BasicDBList) JSON.parse(loggerMessage);
    for(int i=0; i < data.size(); i++){
        coll.insert((DBObject) data.get(i));
    }

answered Aug 15, 2013 at 9:41

Mike Dynamite

652 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

save and find json string in mongodb

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related