Rails Elasticsearch - multiple filter terms

Question

I have a model Campaign which can be created by a user of types: "NonProfit" and "Individual".

Now I want to search campaigns who are created by either NonProfit or Individual only, or by any of them.

Here is my model:

class Campaign < ActiveRecord::Base
after_commit lambda { __elasticsearch__.index_document  },  on: :create
after_commit lambda { __elasticsearch__.update_document },  on: :update

belongs_to :user

enum status: { waiting: 0, approved: 1, disapproved: 2, expired: 3, canceled: 4, draft: 5, published: 6}


def user_type
   self.user.rolable_type
end

def as_json(options={})
  super(:only => [:id, :status, :title, :description],
        :methods => [:user_type])
end

settings index: {
number_of_shards: 1,
analysis: {
  filter: {
    trigrams_filter: {
      type: 'ngram',
      min_gram: 2,
      max_gram: 10
    },
    content_filter: {
      type: 'ngram',
      min_gram: 4,
      max_gram: 20
    }
  },
  analyzer: {
    index_trigrams_analyzer: {
      type: 'custom',
      tokenizer: 'standard',
      filter: ['lowercase', 'trigrams_filter', 'asciifolding']
    },
    search_trigrams_analyzer: {
      type: 'custom',
      tokenizer: 'whitespace',
      filter: ['lowercase']
    },
    english: {
      tokenizer: 'standard',
      filter: ['standard', 'lowercase', 'content_filter']
    },
    czech: {
      tokenizer: 'standard',
      filter: ['standard','lowercase','content_filter', 'asciifolding']
    }
  }
}
} do
mappings dynamic: 'false' do
  indexes :status, type: 'string'
  indexes :user_type, type: 'string'
  indexes :en_title, index_analyzer: 'english', search_analyzer: 'english'
  indexes :ma_title, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :cs_title, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :en_description, index_analyzer: 'english', search_analyzer: 'english'
  indexes :ma_description, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :cs_description, index_analyzer: 'czech', search_analyzer: 'czech'
end
end

  def as_indexed_json(options={})
{ id: id,
  status: status,
  user_type: user_type,
  ma_title: ma_title,
  cs_title: cs_title,
  en_title: en_title,
  ma_description: ma_description,
  cs_description: cs_description,
  en_description: en_description  
}
  end

   def self.search(query, user_type)
__elasticsearch__.search(
  {
    query: {
      filtered: {
        query: {
          multi_match: {
            query: query,
            fields: ['ma_title^10', 'cs_title^10', 'en_title^10', 'ma_description', 'cs_description', 'en_description']
          }
        },
        filter: {
          term: {
            status: "published"               
          },
          term: {
            user_type: user_type
          }
        }
      }
    }       
  }
)
 end

end

My Controller would look something like this:

Campaign.search(params[:q], 'NonProfit') # to search only NonProfit campaigns
Campaign.search(params[:q], 'Individual') # to search only Individual campaigns
Campaign.search(params[:q], ['NonProfit','Individual']) # to search any of the campaigns

However I always get 0 results. I works just fine when I remove the filter term 'user_type'. I am not succesffull to get the user_type filter term working as expected.

Any idea, how to make it working? Thanks, Miroslav

UPDATE 1 (the filter user_type still does not work - 0 results):

def as_json(options={})
  super(:only => [:id, :status, :title, :description],
        :methods => [:user_type]
     #   :include => {
     #     :employers => {:only => [:title]},
     #     :roles => {:only => [:name]}
     #   }
  )
end

settings index: {
number_of_shards: 1,
analysis: {
  filter: {
    trigrams_filter: {
      type: 'ngram',
      min_gram: 2,
      max_gram: 10
    },
    content_filter: {
      type: 'ngram',
      min_gram: 4,
      max_gram: 20
    }
  },
  analyzer: {
    index_trigrams_analyzer: {
      type: 'custom',
      tokenizer: 'standard',
      filter: ['lowercase', 'trigrams_filter', 'asciifolding']
    },
    search_trigrams_analyzer: {
      type: 'custom',
      tokenizer: 'whitespace',
      filter: ['lowercase']
    },
    english: {
      tokenizer: 'standard',
      filter: ['standard', 'lowercase', 'content_filter']
    },
    czech: {
      tokenizer: 'standard',
      filter: ['standard','lowercase','content_filter', 'asciifolding']
    }
  }
}
} do
mappings dynamic: 'false' do
  indexes :status, type: 'string', index: 'not_analyzed'
  indexes :user_type, type: 'string', index: 'not_analyzed'
  indexes :en_title, index_analyzer: 'english', search_analyzer: 'english'
  indexes :ma_title, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :cs_title, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :en_description, index_analyzer: 'english', search_analyzer: 'english'
  indexes :ma_description, index_analyzer: 'czech', search_analyzer: 'czech'
  indexes :cs_description, index_analyzer: 'czech', search_analyzer: 'czech'
end
end

def as_indexed_json(options={})
{ id: id,
  status: status,
  ma_title: ma_title,
  cs_title: cs_title,
  en_title: en_title,
  ma_description: ma_description,
  cs_description: cs_description,
  en_description: en_description,
  :methods => [:user_type]
}
end




def self.search(query, user_type)
__elasticsearch__.search(
  {
    query: {
      filtered: {
        query: {
          multi_match: {
            query: query,
            fields: ['ma_title^10', 'cs_title^10', 'en_title^10', 'ma_description', 'cs_description', 'en_description']
          }
        },
        filter: {
          bool: {
            must: [
              {
                term: {
                  status: "published"
                }
              },
              {
                terms: {
                  user_type: user_type
                }
              }
            ]
          }
        }
      }
    }       
  }
)
end

This hasn't to do with your question directly, but take a look into the Searchkick Gem. It's pretty popular in the Rails-Searchkick world. It provides a great interface for communicating with Elasticsearch and is very well tested. — Tobias
– Tobias, Commented Nov 27, 2015 at 18:38
Thanks Tobias. This is in my plan in future to look at that in more detail and reimplement the current app parts, but at the moment I need to finish the first version of the product as soon as possible and push it live. — Miroslav
– Miroslav, Commented Nov 27, 2015 at 18:53

ChintanShah25 · Accepted Answer · 2015-11-27 19:06:19Z

1

You need to add "index" : "not_analyzed" to your mapping for the fields status and user_type. As you did not specify any analyzer ES was using standard analyzer for those fields.

mappings dynamic: 'false' do
  indexes :status, type: 'string', index : 'not_analyzed' <--- here
  indexes :user_type, type: 'string', index : 'not_analyzed' <--- here

If you need to use term filter, you need to make sure it is exact match as term queries do not perform any analysis.

Also for Campaign.search(params[:q], ['NonProfit','Individual']) # to search any of the campaigns you need to use terms filter as you are searching for more than one value.

EDIT Terms Query

Terms Query expect array of values

Campaign.search(params[:q], ['NonProfit'])

Try this

{
   "terms": {
      "user_type": ['NonProfit']
   }
 }

I hope this helps.

edited Nov 27, 2015 at 19:06

answered Nov 27, 2015 at 16:54

ChintanShah25

12.7k3 gold badges45 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Miroslav Over a year ago

Thanks @ChintanShah25. I made the changes, but for some reason the "user_type" filter still does not work :/. I edit the content above - UPDATE 1.

ChintanShah25 Over a year ago

I have updated my answer, can you try again? It should work now.

Miroslav Over a year ago

Actually I am passing an array like this, but 0 results: Campaign.search(params[:q], ['NonProfit']) Also this is the value stored in ES DB: {"id":7,"status":"published","user_type":"NonProfit",....}

ChintanShah25 Over a year ago

could you try querying only for user_type and see if it returns expected results? like curl -XGET 'localhost:9200/your_index/_search?pretty=true' -d '{"query":{"terms":{"user_type":["NonProfit"]}}}'

Miroslav Over a year ago

When I remove the user_type filter and keep the status one, it works just fine. If vice-versa (remove status and keep user_type filter) it does not work. I think there is something wrong with the user_type method as this is the difference between status and user_type. Are the definitions for all user_type occurrences in the model correct?

|

Collectives™ on Stack Overflow

Rails Elasticsearch - multiple filter terms

UPDATE 1 (the filter user_type still does not work - 0 results):

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

UPDATE 1 (the filter user_type still does not work - 0 results):

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related