Custom search using Mongodb

Question

I have a MongoDB as my database and the backend is written in node.js. I am trying to implement a search for a table which returns me all results with the string entered AND string matching.

For example searching "foo" will return (In that order)

foo maker moo
doo foo doo //The order of word search does not matter as long as it puts the word search first
foobar
fooboo

Currently I have this but I am convinced there is a better way to do it without searching the db twice:


    async function(req, res) {
        var customerName = req.params.customerName;
   //word match
        var customers1 = await Models.DummyContactTable.find({
            customerName: {
                $regex: "^" + customerName,
                $options: 'i'
            },
            IsActive: true
        });
    //String match
   
     var customers2 = await Models.DummyContactTable.find({
          $and: [
              {
                customerName: {
                  $regex: customerName, $options: 'i'
                }
              },
              {
                customerName: {
                  $not: {
                    $regex: "^" + customerName,
                  }
                },
                IsActive: true
              }
          ]
        });
    //Since sometimes we get duplicates, doing a filter and find to de-dup
      var customers = customers1.concat(customers2.filter((customer) => !customers1.find(f => f.uuid === customer.uuid)));

Using user input as regular expressions is a security vulnerability. — D. SM
– D. SM, Commented Aug 29, 2020 at 3:30
I will eventually add the checks to the user input before sending it off to the db but for this demo code I just want to get the search right. — user1152262
– user1152262, Commented Aug 29, 2020 at 4:30
Problems like this are often solved with ngrams. Have you looked at Mongodb Atlas Search? You could use the autocomplete operator to return all the results quickly in one pass. docs.atlas.mongodb.com/atlas-search — Doug
– Doug, Commented Aug 30, 2020 at 21:07
Just spent sometime researching that. It would be good if you can provide a working sample I can quickly try to see if that would work if possible otherwise I would have to do a lot of trial and error to see what would work. — user1152262
– user1152262, Commented Aug 31, 2020 at 0:14

Doug · Accepted Answer · 2020-08-31 19:07:11Z

If you were using Atlas Search, you could write a query like this:

{ 
  $search: {
    autocomplete: { 
      path: "customerName",
      query: "foo"
}}}

// atlas search index definition
{
  "mappings": {
    "fields": {
      "customerName" : { 
        "type" : "autocomplete"
}}}

If you needed to control the result scores, you could use compound

{
  $search: {
    compound: {
      should: [ 
        {autocomplete: {path: "customerName", query: "foo" }},
        {text: {path: "customerName", query: "foo" , score: { boost: { "value" : 3" }}}}
    ]}}}

In this case, we're using the text operator to split on word boundaries using the lucene.standard analyzer, and boosting those results above. Results from Atlas Search are automatically sorted by score with top results first. Queries are optimized for performance and this query would be done in one pass.

There are a lot of other knobs in the docs to turn depending on your sorting and querying needs (such as using different analyzers, prefix searches, phrase searches, regex, etc).

D. SM · Accepted Answer · 2020-08-29 04:39:37Z

0

If you want those kinds of ordering rules I would load up all of your customer names into an application that does the search and perform search & sort entirely in the application. I don't expect even Atlas search to provide this kind of flexibility.

(I don't think the queries you provided achieve the ordering you want either.)

answered Aug 29, 2020 at 4:39

D. SM

14.6k3 gold badges15 silver badges22 bronze badges

2 Comments

user1152262 Over a year ago

The order of the first two dont matter as they can be interchanged, what matters is that the "words" come before the sentences,

D. SM Over a year ago

If that is the case update your question to clearly state the requirements.

Collectives™ on Stack Overflow

Custom search using Mongodb

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related