Elastic Search: Transform CSV to array on index

Question

I have a field containing a csv string coming from the JDBC river plugin which I want to be an array of integers.

I have specified the following mapping containing a transform:

{
    "mytype" : {
        "transform": {
            "script":"ctx._source['AllowedAccountIds'] = ctx._source['AllowedAccountIds']?.tokenize(',')",
            "lang":"groovy"
        },
         "properties": {
           "AllowedAccountIds": { "type": "integer"},          
           "Name": { "type": "string" },
           "Description":{"type":"string"}
        }
    }
}

I can see that this sort of works if I request a document with the ?_source_transform parameter:

{
    "_index": "myindex",
    "_type": "mytype",
    "_id": "1",
    "_version": 1,
    "found": true,
    "_source": {
        "Name": "Lorem Ipsum",
        "AllowedAccountIds": [
            "1",
            "3",
            "19",
            "14",
            "26"
        ],
        "Description": "Vivamus ex libero, pellentesque ultricies sodales vitae, mollis a magna. Vestibulum est purus, placerat eu enim a, ullamcorper lacinia ligula. Nullam pellentesque tristique mollis. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; In feugiat augue sem, eu scelerisque augue gravida nec. Vivamus fermentum finibus arcu, at commodo sapien imperdiet ac. Ut posuere cursus neque, eu ornare lorem ornare sit amet. Vivamus pulvinar ultricies eros. Aliquam euismod ex non dapibus consequat. Donec tristique tortor id feugiat porttitor. Morbi turpis purus, vestibulum sit amet accumsan sed, ultricies quis justo."
    }
}

But if I don't specify the parameter I see the following:

{
    "_index": "myindex",
    "_type": "mytype",
    "_id": "1",
    "_version": 1,
    "found": true,
    "_source": {
        "Name": "Lorem Ipsum",
        "AllowedAccountIds": "1,3,19,14,26",
        "Description": "Vivamus ex libero, pellentesque ultricies sodales vitae, mollis a magna. Vestibulum est purus, placerat eu enim a, ullamcorper lacinia ligula. Nullam pellentesque tristique mollis. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; In feugiat augue sem, eu scelerisque augue gravida nec. Vivamus fermentum finibus arcu, at commodo sapien imperdiet ac. Ut posuere cursus neque, eu ornare lorem ornare sit amet. Vivamus pulvinar ultricies eros. Aliquam euismod ex non dapibus consequat. Donec tristique tortor id feugiat porttitor. Morbi turpis purus, vestibulum sit amet accumsan sed, ultricies quis justo."
    }
}

I was expecting to see the first result all the time. Have I misunderstood what Transforms are for?

Can anyone shed any light on this? The documentation isn't much help.

Edit: This does seem to have fixed my problem when querying from ElasticSearch.NET though - but I'd still like to understand what is going on.

Vineeth Mohan · Accepted Answer · 2015-01-07 15:37:58Z

1

As per the document - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-transform.html#mapping-transform

but the original source is stored in the _source

This means that the actual JSON wont be transformed but then the data that goes to the store ( If enabled ) and reverse index are transformed.

Hence your search only work based on your transformed content , but then the data that you will retrieve would not be transformed by default.

answered Jan 7, 2015 at 15:37

Vineeth Mohan

19.4k9 gold badges70 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elastic Search: Transform CSV to array on index

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related