AWS Step Functions: Filter an array using JsonPath

Question

I need to filter an array in my AWS Step Functions state. This seems like something I should easily be able to achieve with JsonPath but I am struggling for some reason.

The state I want to process looks like this:

{
  "items": [
    {
      "id": "A"
    },
    {
      "id": "B"
    },
    {
      "id": "C"
    }
  ]
}

I want to filter this array by removing entries for which id is not in a specified whitelist.

To do this, I define a Pass state in the following way:

"ApplyFilter": {
  "Type": "Pass",
  "ResultPath": "$.items",
  "InputPath": "$.items.[?(@.id in ['A'])]",
  "Next": "MapDeployments"
}

This makes use of the JsonPath in operator.

Unfortunately when I execute the state machine I receive an error:

{
  "error": "States.Runtime",
  "cause": "An error occurred while executing the state 'ApplyFilter' (entered at the event id #8). Invalid path '$.items.[?(@.id in ['A'])]' : com.jayway.jsonpath.InvalidPathException: com.jayway.jsonpath.InvalidPathException: Space not allowed in path"
}

However, I don't understand what is incorrect with the syntax. When I test here everything works correctly.

What is wrong with what I have done? Is there another way of achieving this sort of filter using JsonPath?

AWS might use an older release of the Jayway JasonPath dependency that does not support the in operator, but that's actually unlikely since this feature is around for quite some time. Maybe it's the Pass filter. Could you try to reverse the logic and use the nin operator instead? — wp78de
– wp78de, Commented Feb 9, 2021 at 22:48
@wp78de Yes nin doesn't work either. However == does. I think you are right, and that AWS are using a very old version of JsonPath for some reason. From what I can see in and nin have been available since version 2.1.0 released in 2015 — timxyz
– timxyz, Commented Feb 10, 2021 at 10:10
I combed the AWS GitHub repo and I think the current implementation is using Jayway JasonPath 2.0(pre)? — wp78de
– wp78de, Commented Feb 11, 2021 at 18:20

pancake0314 · Accepted Answer · 2024-05-19 10:49:04Z

1

You can use regular expression evaluation as alternatives. Note that you don't need to put a dot between "items" and the following square brackets.

"ApplyFilter": {
  "Type": "Pass",
  "ResultPath": "$.items",
  "InputPath": "$.items[?(@.id =~ /A|C/)]",
  "Next": "MapDeployments"
}

You can also use expressions like /.*A.*|.*C.*/, if you want partial match. Explore this link for other useful expressions.

On the other hand, based on the testing I did today, in expressions seem to be unsupported as of now.

answered May 19, 2024 at 10:49

pancake0314

261 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

yoodan · Accepted Answer · 2021-02-16 05:52:48Z

0

According to the official AWS docs for Step Functions,

The following in paths are not supported @ .. , : ? *

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-paths.html

answered Feb 16, 2021 at 5:52

yoodan

852 bronze badges

1 Comment

timxyz Over a year ago

This only applies to "reference paths" and InputPath is not a reference path. For instance $.items.[?(@.id == 'A')] works fine.

Collectives™ on Stack Overflow

AWS Step Functions: Filter an array using JsonPath

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related