How to Insert unique column data into elasticsearch?

Question

I have a webcrawler that scrapes Business Names and Business Contact numbers and inserts into ElasticSearch

The problem is i dot NOT want duplicate phone numbers inserted into the column

Is there anyway i can achieve this internally in elasticsearch?

If not, is there a better solution than the below steps

Fetch Record -> Check for Duplicate phone number -> Insert

Andrei Stefan · Accepted Answer · 2015-07-14 06:15:20Z

1

If the phone numbers are put in the same field of one document, then create a tokenizer that should output as tokens each phone number.

And for this tokenizer define a filter of type unique that will keep the unique tokens. Something around these lines:

  "analyzer": {
    "numbers": {
      "tokenizer": "[whatever]",
      "filter": [
        ...,
        "unique"
      ]
    }
  }

answered Jul 14, 2015 at 6:15

Andrei Stefan

52.5k6 gold badges102 silver badges92 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment