How do I modify parameters in a Container Overrides section of a Step Functions machine?

Question

I have a Step Functions Machine the definition file of which looks as such:

                          "ContainerOverrides": [
                            {
                              "Name": "Foo",
                              "Environment": [
                                {
                                  "Name": "Foo"
                                  "Value": "Bar"
                                },
                              ],
                              "Command.$": "States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)"
                            }
                          ]
                        },

that I am trying to rewrite into Typescript (CDK). I've gotten the following few lines.

      containerOverrides: [{
        containerDefinition: Foo,
        environment: [
          { name: 'Foo', value: 'Bar'},
        ],
        command: ['States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)'],
      }],

I'm a bit confused about how to go about this.

When I deploy the above CDK code, I get as output:

                            "Command": [
                     "States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)"
                            ],

My confusion is in regards to the following: The ContainerOverrides method doesn't accept parameters, but I need to modify a parameter (Command.$), so how can I possibly do that? I came across this post where somebody seems to have a similar issue, but when I try to apply the proposed solution, of simply writing

command: JsonPath.arrayAt('States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)'

I get told that ''Cannot use JsonPath fields in an array, they must be used in objects''

Which CDK task construct are you using?

fedonev
– fedonev

2023-02-01 18:20:55 +00:00
Commented Feb 1, 2023 at 18:20 — fedonev
– fedonev, Commented Feb 1, 2023 at 18:20
@fedonev I'm using an EcsRunTask

DevBob
– DevBob

2023-02-02 09:33:15 +00:00
Commented Feb 2, 2023 at 9:33 — DevBob
– DevBob, Commented Feb 2, 2023 at 9:33

fedonev · Accepted Answer · 2023-02-03 08:18:04Z

2

TL;DR The current implementation of EcsRunTask doesn't permit this. The general-purpose CallAwsService construct does.

The EcsRunTask construct is the CDK's implementation of the ECS optimised integration. The construct only accepts an array of strings as override commands. It cannot produce substitutable output like "Command.$": "$.commands" that's needed to read the override command from the execution input. This is a limitation of the CDK implementation, not of the ECS optimized integration itself.

The cleanest solution is to use the CallAwsService construct, which implements the SDK service integration. It requires manual configuration. The API-specific config goes in the parameters prop. The prop is loosely typed as { [string]: any }. It's flexible, but it's your job to provide the expected syntax for the ecs:RunTask SDK call. Here is the relevant bit for your question:

parameters {
    Overrides: {
        ContainerOverrides: [
            { Command: sfn.JsonPath.array("sh", "-c", sfn.JsonPath.stringAt("$.cmd")), },
        ],
    },
}

It produces the expected command override in the Step Functions task definition:

"Command.$": "States.Array('sh', '-c', $.cmd)"

answered Feb 3, 2023 at 8:18

fedonev

26.1k2 gold badges43 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

DevBob Over a year ago

This is incredibly useful, thanks for sharing. I've tried the CallAwsService construct, and it works perfect. Is there anyway to still add the .sync integrationpattern that was available with the EcsRunTask? My issue is that I need both command.$ parameter to be overridden and the state to have a .sync integration pattern.

fedonev Over a year ago

@DevBob Glad to help. Unfortunately, the SDK integrations do not support the .sync pattern.

fedonev Over a year ago

@DevBob If you really need the .sync pattern, I think your best option is a CustomState construct to pass arbitrary State Machine JSON. It's for cases such as yours where the CDK constructs are overly restrictive.

DevBob Over a year ago

Thank you, I'll get back to experimenting with this later. For now, it seems to implement the .sync pattern, but struggles with follow up states in the Catch field akin to what's described here, as I got the same issue of Invalid State Machine Definition: 'MISSING_TRANSITION_TARGET: Missing 'Next' target: $.Foo despite the follow up state being present in the catch field, and it needing to exit otherwise.

Graeme Wilson · Accepted Answer · 2024-02-02 07:58:32Z

If you can tolerate an extra step then you can use step_function_tasks EvaluateExpression to build your command array and pass that directly to EcsRunTask:

create_command_builder_task = tasks.EvaluateExpression(scope, 'Build CLI command',
        expression="""[
            'sh',
            '-c',
            $.Foo,
            $.Foo,
            $.Bar,
            $.Bar,
            $.Bar
        ]""",
        result_path="$.Command"
)

Then use JsonPath.list_at to retrieve the Command:

tasks.ContainerOverride(
  command=step_functions.JsonPath.list_at('$.Command')
)

Then:

StateMachine(scope,
        'StateMachine',
        definition_body=step_functions.DefinitionBody.from_chainable(create_command_builder_task.next(ecs_task))
        )

Collectives™ on Stack Overflow

How do I modify parameters in a Container Overrides section of a Step Functions machine?

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related