2

Suppose there are some POJOs to be written to an Elasticsearch index. From my limited experience with ES, each field gets a type (e.g. text, keyword, ...). Is it possible to configure the Jackson ObjectMapper with the annotations from spring-data-elasticsearch to customize the type of the serialized Json? As an example:

@Document(indexName = "books", type = "book")
public class Book {
    @Id
    @Field(type = FieldType.Keyword)
    String isbn;
    @Field(type = FieldType.Text)
    String title;
}

Then the mapping should include something like

{
  "mappings":
    {
        "book": {
            "properties": {
                "id": {
                    "type": "keyword"
                },
                "title": {
                    "type": "text"
                }
            }
        }
    }
}

Please let me know if I think about this wrong!

1
  • Ftm I work around this issue by posting the mappings, settings etc. directly to the index before pushing any documents. Commented Jun 28, 2019 at 13:55

2 Answers 2

4

With Elasticsearch you can either let it define mappings automatically using Dynamic Mapping (you persist your docs and ES builds mapping according to it input), or you can define your schema prior to indexing by sending ES with relevant mappings/templates (.json files).

Using dynamic mapping option is less recommended and might cause issues with mappings (wrong date format, int/float issues, etc..)

I usually set my mappings without the dynamic feature - setting it to 'strict', this will raise Exception automatically by ES (and will propagate with any library you use)

"dynamic": "strict",

strict: If new fields are detected, an exception is thrown and the document is rejected. New fields must be explicitly added to the mapping.

With Spring-Data-Elasticsearch, you can define your Object just the way you have posted - annotating your fields with wanted types - this will cause ElasticsearchTemplate.java to parse your Object (take a look at public <T> boolean putMapping(Class<T> clazz) and public ElasticsearchPersistentEntity getPersistentEntityFor(Class clazz)) and feed it to ES.

You can verify spring-es index/mapping creation according to your annotations with ElasticsearchTemplate: (Test taken from Spring-Data-ES project)

@Test
public void shouldReturnMappingForGivenEntityClass() {

    // given

    // when
    boolean created = elasticsearchTemplate.createIndex(SampleEntity.class);
    elasticsearchTemplate.putMapping(SampleEntity.class);
    Map<String, Object> mapping = elasticsearchTemplate.getMapping(SampleEntity.class);

    // then
    assertThat(created).isTrue();
    assertThat(mapping).isNotNull();
    assertThat(((Map<String, Object>) ((Map<String, Object>) mapping.get("properties")).get("message")).get("type"))
            .isEqualTo("text");
}

And the SampleEntity:

@Document(indexName = "test-index-sample-mapping", type = "mapping")
static class SampleMappingEntity {

    @Id private String id;

    @Field(type = Text, index = false, store = true, analyzer = "standard")
    private String message;
}

I would recommend separating index construction from the application and not to use the implicit POJO annotation option - this will be easier to control later on.

Sign up to request clarification or add additional context in comments.

1 Comment

Thanks, will try that out when I get the chance!
1

I don't know which version of Spring Data Elasticsearch you are using; the support of using the Jackson Object Mapper together with the @Field annotation is limited.

If you are using 3.2.0RC1, then change to use the ElasticsearchEntityMapper (see reference docs how to configure it). Although the Jackson Mapper still is the default one if no other is configured, current work is done on ElasticsearchEntityMapper. I think after release of 3.2 we probably will change the default mapper to ElasticsearchEntityMapper.

I know the reference documentation still lacks information, I'm just in the process of reworking it.

When creating an index with the Repository classes from 3.2, the annotations like @Field are taken into account and the correct mapping is written to the index, no need to write the mapping by yourself.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.