json structure for bulk insert not valid

Question

Elasticsearch version : 7.1
Postman version : 7.8.0

Elastic Search Url: http://localhost:9200/menu/_bulk

mapping

 "mappings": {
            "properties": {
                "input": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "output": {
                    "properties": {
                        "category": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "item": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "items": {
                            "properties": {
                                "category": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                },
                                "item": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                },
                                "modifiers": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                }
                            }
                        },
                        "modifiers": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "quantity": {
                            "type": "long"
                        }
                    }
                }
            }
        }

Error I am receving:

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Malformed action/metadata line [3], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Malformed action/metadata line [3], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"
    },
    "status": 400

Expected Result: Successfully adding new documents to index menu

Procedure

I am trying to a bulk insert with elastic search. I have referred to the documentation and this is an example they provided below.

{ "index" : { "_index" : "testindex", "_type" : "somerandomtype", "_id" : "1" } }
{ "somefield" : "value1" }
{ "index" : { "_index" : "testindex", "_type" : "somerandomtype", "_id" : "2" } }
{ "somefield" : "hello hello hello" }

I have based my formatting in the same manner but I keep getting the error. This is what my body looks like thats going into postman.

{"index": { "_index": "menu", "_type":"_doc" } }
{"input": "angus-burger", "output": {
"category": "Sides", "item": "Angus-Deluxe Burger", "modifiers": [], "quantity": 1} }

What am I doing wrong here?

Atur · Accepted Answer · 2019-10-18 15:46:26Z

Your Json format is indeed incorrect. Postman body section will show error with given Json. Additionally Bulk request body is not meant to be in single valid Json.

Use the same data with curl, and result will be success.

Moreover, when using the command data with POSTMAN, each 'section' should be within a single line (i.e. each line represents a single valid json). Moreover, there should be no blank lines. (there are some similarities here to 'bcp' command)

So, this would work

 {"index": { "_index": "menu", "_type":"_doc" } }
 {"input": "angus-burger", "output": {"category": "Sides", "item": "Angus-Deluxe Burger", "modifiers": [], "quantity": 2} }

But this won't work in postman for bulk insert

{
    "index": {
        "_index": "menu",
        "_type": "_doc"
    }
}
{
    "input": "angus-burger",
    "output": {
        "category": "Sides",
        "item": "Angus-Deluxe Burger",
        "modifiers": [],
        "quantity": 2
    }
}

Lakhwinder Singh · Accepted Answer · 2019-10-18 10:46:27Z

0

"your json format not correct .you can copy the code and check " Visit http://json.parser.online.fr/ regularly!

answered Oct 18, 2019 at 10:46

Lakhwinder Singh

291 gold badge1 silver badge6 bronze badges

2 Comments

Mohamed Ali Over a year ago

The format to insert into elasticsearch is it's own version of json. It will not be valid if you compare it to standard json.

Mohamed Ali Over a year ago

I was able to solve on my own and just to prove to you the formatting is different, I am still failing in the JSON validator you posted but it posts successfully in elastic search

Mohamed Ali · Accepted Answer · 2019-10-18 11:03:31Z

0

It seems that the format of the body was incorrect. After posting the below, I was able to successfully post to elasticsearch. One thing to note is that in postman, you must have it so each line is as close to the other. By that I mean you must have no spacing between your body at the end of a new line.

{"index": { "_index": "menu", "_type":"_doc" } }
{"input": "angus-burger", "output": {"category": "Sides", "item": "Angus-Deluxe Burger", "modifiers": [], "quantity": 1} }

answered Oct 18, 2019 at 11:03

Mohamed Ali

8122 gold badges12 silver badges36 bronze badges

Collectives™ on Stack Overflow

json structure for bulk insert not valid

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related