Single value extraction from ResultSelector in AWS Step Function?

Question

I'm struggling with something that seems like it should be simple, but the 'obvious' approach isn't working :-|. I have an AWS Step Function Lambda step. The input to that step (and function) looks like so:

{ "foo":"aaa" , "bar":"bbb" }

The Lambda's output looks like so:

{
  // a bunch of Lambda execution details here
  "Payload" : { "baz":"ccc" , "qux":"ddd" }
}

I'd like for the step's output to be:

{ "foo":"aaa" , "bar":"bbb" , "fred":"ccc" }

... i.e. I want to overlay the step's input with a new value at key "fred" referencing the Lambda output's value at "baz".

My first attempt (which still seems the most logical) is to 'extract' only "ccc" from the Lambda output and inject it into the step's output, like so:

"ResultSelector": "$.Payload.baz",
"ResultPath": "$.fred"

This puts the literal string "$.Payload.baz" into the output, not the dereferenced value.

It seems ResultSelector wants to build an object, so then I tried:

"ResultSelector": { "fred.$":"$.Payload.baz" },
"ResultPath": "$"

This replaces the entire step's input object, yielding just:

{ "fred":"ccc" }

And finally, the hybrid of those two:

"ResultSelector": { "fred.$":"$.Payload.baz" },
"ResultPath": "$.fred"

... yields:

{ "foo":"aaa" , "bar":"bbb", "fred":{"fred":"ccc"} }

Close (but no cigar)!

I'm a bit stumped for how to craft ResultSelector and ResultPath to achieve this simple overlay, and I feel like there's just some simple syntax trick that somehow I haven't found yet in the docs ... any ideas?

I'd be curious to see if someone can find a better solution, but the only thing I have managed in these cases is to have a Pass state afterward where I change the layout of the whole state. — Maurice
– Maurice, Commented Aug 31, 2023 at 14:16
Yeah I thought about that, but it seems inconceivable(!) to me that there's no way to do this ... especially given that the Lambda function output will always be nested in the "Payload" key (and Lambdas are clearly advertised as the canonical step in Step Functions). ¯_(ツ)_/¯ — mmuurr
– mmuurr, Commented Aug 31, 2023 at 14:35
I'm following this question in the hopes that some SFN-Grandmaster is able to solve this, but I'm not too optimistic. If you check out the data flow simulator it seems like the expectation is to provide a JSON-Like structure for the ResultSelector... — Maurice
– Maurice, Commented Aug 31, 2023 at 15:13

fedonev · Accepted Answer · 2024-05-15 14:52:21Z

4

Note: As @Maurice points out, this solution works in cases where the task input is equal to (or is a subset of) the execution input. The execution input is exposed as $$.Execution.Input, which we can reference in ResultSelector to get the desired output.

To output the union of the task (execution) input and the task output, use JsonMerge in ResultSelector. That intrinsic function shallowly merges two objects. Reference the execution input from the context object and the output from the task payload. ResultSelector always expects an object, so we need a temporary key. I used merged, which I get rid of with OutputPath.

"ResultSelector": {
    "merged.$": "States.JsonMerge($$.Execution.Input, $.Payload, false)",
},
"OutputPath": "$.merged",

🤔 Output:

{
  "bar": "bbb",
  "baz": "ccc",
  "qux": "ddd",
  "foo": "aaa"
}

That's close, but not exactly what you wanted. You want a *partial* union of the task input and output, dropping qux and renaming baz. To do that, we do the JsonMerge thing, but with a subset of the task output constructed with the Format and StringToJson intrinsic functions:

"ResultSelector": {
  "merged.$": "States.JsonMerge($$.Execution.Input, States.StringToJson(States.Format('\\{\"fred\":\"{}\"\\}', $.Payload.baz)), false)"
},
"OutputPath": "$.merged",

✅ Output:

{
  "bar": "bbb",
  "foo": "aaa",
  "fred": "ccc",
}

edited May 15, 2024 at 14:52

answered Aug 31, 2023 at 17:00

fedonev

26.1k2 gold badges43 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Maurice Over a year ago

My understanding was that $$.Execution.Input refers to the original Input of the stepfunction and not the task input. If that's true this is still cool but solves a different problem, correct?

fedonev Over a year ago

@Maurice OP says it's the "input to that step (and function)", which I read as: task input = step function input.

lvthillo Over a year ago

Is it possible to access the original input in that ResultSelector step?

fedonev Over a year ago

@lvthillo If by "original input" you mean the execution input, yes you can. If by "original input" you mean something in the task input that's not in the execution input or task output, then I think no.

Collectives™ on Stack Overflow

Single value extraction from ResultSelector in AWS Step Function?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related