Elasticsearch and spring data with empty index

Question

I have a question about elasticsearch with spring data.

@Data
@NoArgsConstructor
@AllArgsConstructor
@Document(indexName = "my_es_index")
public class MyEsIndex {

    private String id;

    private Long counter;

    private Long timestamp;
}

and repository

public interface MyEsIndexRepository extends ElasticsearchRepository<MyEsIndex, String> {

    Optional<MyEsIndex> findFirstByIdOrderByTimestampDesc(String id);
}

so I have a service where I have to search first for previous one saved record to retrieve previous value, always doing search ordered by timestamp.

@Service
@RequiredArgsConstructor
public class MyEsService {

private final MyEsIndexRepository repository;

  public MyEsIndex insert(String previousId) {
    Long previousCounter = 
repository.findFirstByIdOrderByTimestampDesc(previousId).map(MyEsIndex::getCounter).orElse(0L);
    var index = new MyEsIndex(UUID.randomUUID().toString(), ++previousCounter, 
    Instant.now().toEpochMilli());
    return repository.save(index);
  }
}

and when trying to do the operation receiving {"error":{"root_cause":[{"type":"query_shard_exception","reason":"No mapping found for [timestamp] in order to sort on","index":"my_es_index"}

is it possible to do initialization for fields in elasticsearch on empty index? because the solution of init config is not that clear because it will be used only once when starting working with empty index where never saved a record

@Configuration
public class InitElasticsearchConfig {

private final MyEsIndexRepository repository;

  @EventListener(ApplicationReadyEvent.class)
  public void initIndex() {
  if (repository.findAll(PageRequest.of(0, 1)).isEmpty()) {
    var initIndex = new MyEsIndex("initId", 0L, 0L);
    repository.save(initIndex);
    repository.delete(initIndex);
  }
}

is it possible to delegate this solution to spring? I didn't find any

Abacus · Accepted Answer · 2022-12-15 17:33:57Z

When using Spring Data Elasticsearch repositories - as you do - the normal behaviour is that the mapping is written to Elasticsearch after index creation on application startup when the index does not yet exist.

The problem in your code is that you do not define to what types the properties of your entity should be mapped; you need to add @Field annotations to do that:

@Document(indexName = "my_es_index")
public class MyEsIndex {

    private String id;

    @Field(type = FieldType.Long)
    private Long counter;

    @Field(type = FieldType.Long)
    private Long timestamp;
}

Properties that are not annotated with a @Field annotation are not written to the mapping but left for automatic mapping by Elasticsearch, that's the cause for the sort not working. As there is no document written to the index, Elasticsearch does not know what type it is and how to sort on that.

In your code there is another thing that might probably not match your desired application logic. In Spring Data Elasticsearch an entity needs to have an id property, that's the property that will be used as the document's id in Elasticsearch. This is normally defined by annotating the property with @Id, if that is missing - as in your case - a property with the name of "id" or "document" is used. So in your case the property id is used.

A document's id is unique in Elasticsearch, if you store a new document under an existing id, the previous content will be overwritten. If that's what you want, the you should add the @Id annotation to your property to make it clear that this is the unique id. But in this case then your code findFirstByIdOrderByTimestamp does not make sense, as a find by id will always return at most one document, so the order by is irrelevant, you could just use a findById() then. I assume that the id should be unique as you initialize it with a UUID.

If your id is not unique and you have multiple documents with the same id and different timestamps, the you'll need to add a new unique property to your entity and annotate that with @Id to prevent id to be used as a unique identifier.

Collectives™ on Stack Overflow

Elasticsearch and spring data with empty index

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related