9

I'm trying to set up an existing project using elasticsearch and ruby-on-rails. I can get the project up and running however elasticsearch queries return various error message for me and none of my colleagues:

[400] {"error":{"root_cause":[{"type":"parsing_exception","reason":"[and] query malformed, no start_object after query name","line":1,"col":896}],"type":"parsing_exception","reason":"[and] query malformed, no start_object after query name","line":1,"col":896},"status":400}

This obviously leads me to believe it's something to do with my setup. I've installed the same version of them (2.4), however even the setup command that was supplied with the project:

rake environment elasticsearch:import:all DIR=app/models FORCE=y

gives me a timeout error. I've checked and the service is running. Any help as to what could be wrong with my set up would be greatly appreciated as I'm at a dead end for 2 days on this.

Thanks in advance.

EDIT: I don't think the issue is with the query itself as this works for my colleagues however it is as follows:

{: date_range => {: from => Mon,
    01 Aug 2016 00: 00: 00 UTC + 00: 00,
    : to => Mon,
    31 Jul 2017 00: 00: 00 UTC + 00: 00
  }, : activities_only => nil
}, @pupils = [# < Pupil id: 1042, first_name: "Mark", last_name: "Mekhaiel", date_of_birth: "2017-03-19", level: 0, group_id: 95, created_at: "2017-07-31 08:59:59", updated_at: "2017-07-31 09:48:27", username: "m.mekhaiel1", picture: 0, color: 0, archived: false, sub_group_id: nil, score: 20, deleted_at: nil > ], @search = # < PupilCreditSearch: 0x007fc961801308 @interval = : month, @page = 1, @per = 25, @pupil_id = [1042], @date_range = {: from => Mon,
  01 Aug 2016 00: 00: 00 UTC + 00: 00,
  : to => Mon,
  31 Jul 2017 00: 00: 00 UTC + 00: 00
}, @queries = [# < Search::Query: 0x007fc964791bb8 @query = nil, @fields = [], @type = : multi_match, @options = {: fields => [],
  : type => : cross_fields
} > ], @filters = [# < Search::Filter: 0x007fc964756db0 @name = : pupil_id, @value = [1042], @type = : terms, @options = {} > ], @aggregations = {: years_created => # < Search::Aggregation: 0x007fc9647a32f0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : year
    }
  } > ,
  : months_created => # < Search::Aggregation: 0x007fc9647a30c0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : month
    }
  } > ,
  : date_ranges => # < Search::Aggregation: 0x007fc9647a2800 @type = : date_range,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_range,
    : params => {: keyed => true,
      : ranges => [{: from => "now/w",
        : to => "now+1w/w",
        : key => : this_week
      }, {: from => "now/M",
        : to => "now+1M/M",
        : key => : this_month
      }, {: from => "now-1w/w",
        : to => "now-2w/w",
        : key => : last_week
      }, {: from => "now-1M/M",
        : to => "now-2M/M",
        : key => : last_month
      }, {: from => "2016-09-01",
        : to => "2017-08-31",
        : key => : academic_year
      }]
    }
  } > ,
  : groups => # < Search::Aggregation: 0x007fc9647a2260 @type = : terms,
  @field = : pupil_id,
  @options = {: aggs => {: all_dates => # < Search::Aggregation: 0x007fc9647a22d8 @type = : date_histogram,
      @field = : created_at,
      @options = {: params => {: interval => : month
        },
        : aggs => {: total_points => # < Search::Aggregation: 0x007fc9647a2440 @type = : sum,
          @field = : points,
          @options = {} > ,
          : running_total => # < Search::Aggregation: 0x007fc9647a2350 @type = : cumulative_sum,
          @field = nil,
          @options = {: params => {: buckets_path => "total_points"
            }
          } >
        }
      } >
    }
  } > ,
  : options => # < Search::Aggregation: 0x007fc9647a1518 @type = : global,
  @field = nil,
  @options = {: aggs => {: filtered => # < Search::Aggregation: 0x007fc9647a1590 @type = : filter,
      @field = nil,
      @options = {: params => {: and => [{: range => {: created_at => {: gte => Mon,
                01 Aug 2016 00: 00: 00 UTC + 00: 00,
                : lt => Mon,
                31 Jul 2017 00: 00: 00 UTC + 00: 00
              }
            }
          }, {: terms => {: pupil_id => [1042]
            }
          }]
        },
        : aggs => {: categories => # < Search::Aggregation: 0x007fc9647a1b80 @type = : terms,
          @field = "category",
          @options = {: ignore => true
          } > ,
          : topic_ids => # < Search::Aggregation: 0x007fc9647a19a0 @type = : terms,
          @field = "topic_id",
          @options = {: ignore => true
          } > ,
          : learning_objective_ids => # < Search::Aggregation: 0x007fc9647a17c0 @type = : terms,
          @field = "learning_objective_id",
          @options = {: ignore => true
          } > ,
          : completed_activities => # < Search::Aggregation: 0x007fc9647a1608 @type = : terms,
          @field = : pupil_id,
          @options = {: aggs => {: activities => # < Search::Aggregation: 0x007fc9647a1680 @type = : filter,
              @field = nil,
              @options = {: params => {: and => [{: terms => {: category => [0, 1, 2, 3]
                    }
                  }]
                }
              } >
            }
          } >
        }
      } >
    }
  } >
}, @request = # < Search::Request: 0x007fc964753548 @params = {: query => {: bool => {: must => [{: match_all => {}
      }],
      : filter => [{: terms => {: pupil_id => [1042]
        }
      }]
    }
  },
  : aggs => {: years_created => {: date_histogram => {: field => : created_at,
        : interval => : year
      }
    },
    : months_created => {: date_histogram => {: field => : created_at,
        : interval => : month
      }
    },
    : date_ranges => {: date_range => {: field => : created_at,
        : keyed => true,
        : ranges => [{: from => "now/w",
          : to => "now+1w/w",
          : key => : this_week
        }, {: from => "now/M",
          : to => "now+1M/M",
          : key => : this_month
        }, {: from => "now-1w/w",
          : to => "now-2w/w",
          : key => : last_week
        }, {: from => "now-1M/M",
          : to => "now-2M/M",
          : key => : last_month
        }, {: from => "2016-09-01",
          : to => "2017-08-31",
          : key => : academic_year
        }]
      }
    },
    : groups => {: terms => {: field => : pupil_id
      },
      : aggs => {: all_dates => {: date_histogram => {: field => : created_at,
            : interval => : month
          },
          : aggs => {: total_points => {: sum => {: field => : points
              }
            },
            : running_total => {: cumulative_sum => {: buckets_path => "total_points"
              }
            }
          }
        }
      }
    },
    : options => {: global => {},
      : aggs => {: filtered => {: filter => {: and => [{: range => {: created_at => {: gte => Mon,
                  01 Aug 2016 00: 00: 00 UTC + 00: 00,
                  : lt => Mon,
                  31 Jul 2017 00: 00: 00 UTC + 00: 00
                }
              }
            }, {: terms => {: pupil_id => [1042]
              }
            }]
          },
          : aggs => {: categories => {: terms => {: field => "category"
              }
            },
            : topic_ids => {: terms => {: field => "topic_id"
              }
            },
            : learning_objective_ids => {: terms => {: field => "learning_objective_id"
              }
            },
            : completed_activities => {: terms => {: field => : pupil_id
              },
              : aggs => {: activities => {: filter => {: and => [{: terms => {: category => [0, 1, 2, 3]
                      }
                    }]
                  }
                }
              }
            }
          }
        }
      }
    }
  }
} >> , @options = # < ReportOptions: 0x007fc96b663258 @search = # < PupilCreditSearch: 0x007fc961801308 @interval = : month, @page = 1, @per = 25, @pupil_id = [1042], @date_range = {: from => Mon,
  01 Aug 2016 00: 00: 00 UTC + 00: 00,
  : to => Mon,
  31 Jul 2017 00: 00: 00 UTC + 00: 00
}, @queries = [# < Search::Query: 0x007fc964791bb8 @query = nil, @fields = [], @type = : multi_match, @options = {: fields => [],
  : type => : cross_fields
} > ], @filters = [# < Search::Filter: 0x007fc964756db0 @name = : pupil_id, @value = [1042], @type = : terms, @options = {} > ], @aggregations = {: years_created => # < Search::Aggregation: 0x007fc9647a32f0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : year
    }
  } > ,
  : months_created => # < Search::Aggregation: 0x007fc9647a30c0 @type = : date_histogram,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_histogram,
    : params => {: interval => : month
    }
  } > ,
  : date_ranges => # < Search::Aggregation: 0x007fc9647a2800 @type = : date_range,
  @field = : created_at,
  @options = {: field => : created_at,
    : type => : date_range,
    : params => {: keyed => true,
      : ranges => [{: from => "now/w",
        : to => "now+1w/w",
        : key => : this_week
      }, {: from => "now/M",
        : to => "now+1M/M",
        : key => : this_month
      }, {: from => "now-1w/w",
        : to => "now-2w/w",
        : key => : last_week
      }, {: from => "now-1M/M",
        : to => "now-2M/M",
        : key => : last_month
      }, {: from => "2016-09-01",
        : to => "2017-08-31",
        : key => : academic_year
      }]
    }
  } > ,
  : groups => # < Search::Aggregation: 0x007fc9647a2260 @type = : terms,
  @field = : pupil_id,
  @options = {: aggs => {: all_dates => # < Search::Aggregation: 0x007fc9647a22d8 @type = : date_histogram,
      @field = : created_at,
      @options = {: params => {: interval => : month
        },
        : aggs => {: total_points => # < Search::Aggregation: 0x007fc9647a2440 @type = : sum,
          @field = : points,
          @options = {} > ,
          : running_total => # < Search::Aggregation: 0x007fc9647a2350 @type = : cumulative_sum,
          @field = nil,
          @options = {: params => {: buckets_path => "total_points"
            }
          } >
        }
      } >
    }
  } > ,
  : options => # < Search::Aggregation: 0x007fc9647a1518 @type = : global,
  @field = nil,
  @options = {: aggs => {: filtered => # < Search::Aggregation: 0x007fc9647a1590 @type = : filter,
      @field = nil,
      @options = {: params => {: and => [{: range => {: created_at => {: gte => Mon,
                01 Aug 2016 00: 00: 00 UTC + 00: 00,
                : lt => Mon,
                31 Jul 2017 00: 00: 00 UTC + 00: 00
              }
            }
          }, {: terms => {: pupil_id => [1042]
            }
          }]
        },
        : aggs => {: categories => # < Search::Aggregation: 0x007fc9647a1b80 @type = : terms,
          @field = "category",
          @options = {: ignore => true
          } > ,
          : topic_ids => # < Search::Aggregation: 0x007fc9647a19a0 @type = : terms,
          @field = "topic_id",
          @options = {: ignore => true
          } > ,
          : learning_objective_ids => # < Search::Aggregation: 0x007fc9647a17c0 @type = : terms,
          @field = "learning_objective_id",
          @options = {: ignore => true
          } > ,
          : completed_activities => # < Search::Aggregation: 0x007fc9647a1608 @type = : terms,
          @field = : pupil_id,
          @options = {: aggs => {: activities => # < Search::Aggregation: 0x007fc9647a1680 @type = : filter,
              @field = nil,
              @options = {: params => {: and => [{: terms => {: category => [0, 1, 2, 3]
                    }
                  }]
                }
              } >
            }
          } >
        }
      } >
    }
  } >
}, @request = # < Search::Request: 0x007fc964753548 @params = {: query => {: bool => {: must => [{: match_all => {}
      }],
      : filter => [{: terms => {: pupil_id => [1042]
        }
      }]
    }
  },
  : aggs => {: years_created => {: date_histogram => {: field => : created_at,
        : interval => : year
      }
    },
    : months_created => {: date_histogram => {: field => : created_at,
        : interval => : month
      }
    },
    : date_ranges => {: date_range => {: field => : created_at,
        : keyed => true,
        : ranges => [{: from => "now/w",
          : to => "now+1w/w",
          : key => : this_week
        }, {: from => "now/M",
          : to => "now+1M/M",
          : key => : this_month
        }, {: from => "now-1w/w",
          : to => "now-2w/w",
          : key => : last_week
        }, {: from => "now-1M/M",
          : to => "now-2M/M",
          : key => : last_month
        }, {: from => "2016-09-01",
          : to => "2017-08-31",
          : key => : academic_year
        }]
      }
    },
    : groups => {: terms => {: field => : pupil_id
      },
      : aggs => {: all_dates => {: date_histogram => {: field => : created_at,
            : interval => : month
          },
          : aggs => {: total_points => {: sum => {: field => : points
              }
            },
            : running_total => {: cumulative_sum => {: buckets_path => "total_points"
              }
            }
          }
        }
      }
    },
    : options => {: global => {},
      : aggs => {: filtered => {: filter => {: and => [{: range => {: created_at => {: gte => Mon,
                  01 Aug 2016 00: 00: 00 UTC + 00: 00,
                  : lt => Mon,
                  31 Jul 2017 00: 00: 00 UTC + 00: 00
                }
              }
            }, {: terms => {: pupil_id => [1042]
              }
            }]
          },
          : aggs => {: categories => {: terms => {: field => "category"
              }
            },
            : topic_ids => {: terms => {: field => "topic_id"
              }
            },
            : learning_objective_ids => {: terms => {: field => "learning_objective_id"
              }
            },
            : completed_activities => {: terms => {: field => : pupil_id
              },
              : aggs => {: activities => {: filter => {: and => [{: terms => {: category => [0, 1, 2, 3]
                      }
                    }]
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
9
  • do you use the [elasticsearch-ruby ](github.com/elastic/elasticsearch-ruby) gem ? Commented Aug 4, 2017 at 17:48
  • 2
    Can you post the query here you are sending ? Commented Aug 6, 2017 at 16:06
  • posting your query and sample document you are trying to query and output you are trying to achieve will help Commented Aug 7, 2017 at 2:03
  • 1
    elasticsearch : parsing_exception: no [query] registered for [filtered] :github.com/laravel/scout/issues/122,Elastic Search is returning a 400 parsing_exception error github.com/nuagenetworks/visualization-framework/issues/324 Commented Aug 10, 2017 at 18:08
  • 1
    It has to be a malformed query. Looks like what you have pasted for query is Ruby code that uses instance variables and custom Ruby objects. Any one of these might be outputting invalid components that break the query. Can you please paste the eventual JSON object that is passed to Elasticsearch? Commented Aug 11, 2017 at 0:05

1 Answer 1

2

The problem is with one of the aggregations, namely:

:filter => { :and => [...] }

...which leads me to conclusion that you actually may be running elasticsearch 5 and not 2.4, on your machine, because this syntax got dropped in version 5 in favour of:

:filter => { :bool => { :must: [...] } }

Verify the elasticsearch version with curl localhost:9200 (or the actual host/port your app is connecting to).

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.