How to set Elasticsearch document property to null using the Java API Client and Spring Data Elasticsearch

Question

I have a Schedule document which contains a map of Todos. It looks something like this:

{
  "_index": "schedule-index",
  "_id": "sched-id",
  "_source": {
    "id": "sched-id",
    "todos": {

      "todo-id-1": {
        "id": "todo-id-1"
        "name": "laundry"
      },
    
      "todo-id-2": {
        "id": "todo-id-2"
        "name": "dishes"
      }
    }
  }
}

Occasionally, I need to set one of the Todos to null, while retaining the key. e.g.

{
  "_index": "schedule-index",
  "_id": "sched-id",
  "_source": {
    "id": "sched-id",
    "todos": {

      "todo-id-1": null, <-- SET VALUE TO NULL

      "todo-id-2": {
        "id": "todo-id-2"
        "name": "dishes"
      }
    }
  }
}

I was able to achieve this functionality using the RestHighLevelClient, as shown below:

// using HighLevelRestClient
public BulkResponse upsert(List<Schedule> schedules) {
  // 'schedules' has 1 schedule with id=sched-id
  // that has 1 Todo with id=todo-id-1 and value=null

  BulkRequest bulkRequest = new BulkRequest();

  schedules.forEach(schedule-> {
    UpdateRequest updateRequest = new UpdateRequest("schedule-index", schedule.getId())
      .docAsUpsert(true)
      .doc(asJson(schedule), XContentType.JSON);
    bulkRequest.add(updateRequst);
  })

  BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
}

Which produces the query:

{
  "update": {
    "_index": "schedule-index",
    "_id": "sched-id"
  }
}
{
  "doc_as_upsert": true,
  "doc": {
    "id": "sched-id"
    "todos": {
      "a": null    <----- SET 'a' TO NULL
    }
  }
}

However, after converting to the new Java API Client (Spring Data Elasticsearch fragment), the client seems to refuse to put a null value in the generated query.

// using Java API Client
public BulkResponse upsert(List<Schedule> schedules) {
  // 'schedules' has 1 schedule with id=sched-id
  // that has 1 Todo with id=todo-id-1 and value=null
  BulkRequest.Builder br = new BulkRequest.Builder();

  schedules.forEach(schedule-> {
    br.operations(op -> op 
      .update(ur -> ur
        .index("schedule-index")
        .id(todo.getId())
        .action(a -> a
          .docAsUpsert(true)
          .doc(schedule)
        )))
  })

  cleint.bulk(br.build());
}

Which produces the query:

{
  "update": {
    "_index": "schedule-index",
    "_id": "sched-id"
  }
}
{
  "doc_as_upsert": true,
  "doc": {
    "id": "sched-id"
    "todos": {      <----- TODO MAP IS EMPTY
    }
  }
}

How can I tell the Java API Client to upsert null values for map values?

Edit

This seems similar to ElasticsearchRepository skip null values, but in that case the root properties were being ignored.

This is not related to Spring Data Elasticsearch, as you are building your query only with the Elasticsearch client library. But Spring Data Elasticsearch wouldn't write null values in maps contained in a document either. — Abacus
– Abacus, Commented Feb 8, 2024 at 17:57

Seanimus · Accepted Answer · 2024-02-09 23:13:11Z

1

Fixed the issue.

Looks like the Spring Boot autoconfiguration class (ElasticsearchClientConfigurations.java) is setting the Elasticsearch object mapper as new JacksonJsonpMapper().

Looking at the constructor, the issue seems clear:


public JacksonJsonpMapper() {
  this((new ObjectMapper())
    .configure(SerializationFeature.INDENT_OUTPUT, false)
    .setSerializationInclusion(Include.NON_NULL));      // <----- CULPRIT
}

Fixing this was straightforward, as Spring will use any JsonpMapper I provide.

Adding the following bean in a @Configuration class fixed the issue:


@Bean
public JacksonJsonMapper jacksonJsonMapper() {
  return new JacksonJsonpMapper(new ObjectMapper()
    .configure(SerilaizationFeature.INDENT_OUTPUT, false)
    .setSerializationInclusion(JsonInclude.Include.ALWAYS));
}

answered Feb 9, 2024 at 23:13

Seanimus

5676 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

King Midas Over a year ago

The method returns a JacksonJsonpMapper. But solves the problem

Collectives™ on Stack Overflow

How to set Elasticsearch document property to null using the Java API Client and Spring Data Elasticsearch

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related