MongoDB query to remove duplicate documents from a collection

Question

I take data from a search box and then insert into MongoDB as a document using the regular insert query. The data is stored in a collection for the word "cancer" in the following format with unique "_id".

{
  "_id": {
    "$oid": "553862fa49aa20a608ee2b7b"
  },
  "0": "c",
  "1": "a",
  "2": "n",
  "3": "c",
  "4": "e",
  "5": "r"
}

Each document has a single word stored in the same format as above. I have many documents as such. Now, I want to remove the duplicate documents from the collection. I am unable to figure out a way to do that. Help me.

Does stackoverflow.com/questions/14184099/… help ? or stackoverflow.com/questions/13190370/… ? — Zee
– Zee, Commented Apr 23, 2015 at 9:08
No Sourabh. Here, I am confused why the alphabets of a word are being assigned a value. — Vamshi
– Vamshi, Commented Apr 23, 2015 at 9:12
Normally you would do this by making the word the key since that is unique — Sammaye
– Sammaye, Commented Apr 23, 2015 at 9:15
Now, I have many number of duplicate documents with same word. How can I remove them? — Vamshi
– Vamshi, Commented Apr 23, 2015 at 9:21

nickmilon · Accepted Answer · 2015-04-23 14:43:20Z

2

an easy solution in mongo shell: `

use your_db
db.your_collection.createIndex({'1': 1, '2': 1, '3': 1, etc until you reach maximum expected letter count}, {unique: true, dropDups: true, sparse:true, name: 'dropdups'})
db.your_collection.dropIndex('dropdups')

notes:

if you have many documents expect this procedure to take very long time
be careful this will remove documents in place, better clone your collection first and try it there.

answered Apr 23, 2015 at 14:43

nickmilon

1,3721 gold badge10 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

MongoDB query to remove duplicate documents from a collection

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related