Getting 400 Bad Request Error while creating S3 Batch Job from Java Code

Question

As per the doc, I am trying to create a batch job from Java Code. I am able to create a job from console with same role and lambda arn, but from code, I am getting 400 Bad Request. Also, I don't see any error message as per this doc

Here is my code snippet -

JobOperation jobOperation = new JobOperation().withLambdaInvoke(new LambdaInvokeOperation()
                    .withFunctionArn("arn:aws:lambda:eu-west-1:<account_id>:function:s3BatchOperarationsPOCLambda"));

            JobManifest manifest = new JobManifest()
                    .withSpec(new JobManifestSpec().withFormat(JobManifestFormat.S3InventoryReport_CSV_20161130)
                            .withFields(new String[] { "Bucket", "Key" }))
                    .withLocation(
                            new JobManifestLocation().withObjectArn("arn:aws:s3:::<bucket_name>/manifest.csv")
                                    .withETag("e55392fa1ad40a08e40b13b3c000a0aa"));

            JobReport jobReport = new JobReport().withBucket(reportBucketName).withPrefix("testreport")
                    .withFormat(JobReportFormat.Report_CSV_20180820).withEnabled(true).withReportScope("AllTasks");

            AWSS3Control s3ControlClient = AWSS3ControlClientBuilder.standard().withRegion(Regions.US_WEST_1).build();

            String roleArn = "arn:aws:iam::<account_id>:role/S3-Batch-Role";
            String accountId = <account_id>;
            s3ControlClient.createJob(new CreateJobRequest().withAccountId(accountId).withOperation(jobOperation)
                    .withManifest(manifest).withPriority(12).withRoleArn(roleArn).withReport(jobReport)
                    .withClientRequestToken(uuid).withDescription("S3 job").withConfirmationRequired(false));


        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it and returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            System.out.println("test2" + e.getMessage());

            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }

Role has full IAM and s3 batch operation permissions, also lambda has access permission for s3. Trust policy is also defined for batch operations.

Here is my error log -

 (Service: AWSS3Control; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; Proxy: null)
com.amazonaws.services.s3control.model.AWSS3ControlException: null (Service: AWSS3Control; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; Proxy: null)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
    at com.amazonaws.services.s3control.AWSS3ControlClient.doInvoke(AWSS3ControlClient.java:1532)
    at com.amazonaws.services.s3control.AWSS3ControlClient.invoke(AWSS3ControlClient.java:1499)
    at com.amazonaws.services.s3control.AWSS3ControlClient.invoke(AWSS3ControlClient.java:1488)
    at com.amazonaws.services.s3control.AWSS3ControlClient.executeCreateJob(AWSS3ControlClient.java:265)
    at com.amazonaws.services.s3control.AWSS3ControlClient.createJob(AWSS3ControlClient.java:236)
    at com.code.platformintegrationsscheduler.handlers.test.createS3Job(test.java:68)
    at com.code.platformintegrationsscheduler.handlers.test.main(test.java:27)

I'm assuming this is not the block of code you are actually running. Am I right? — Nghia Do
– Nghia Do, Commented Aug 14, 2020 at 13:05

zer0 · Accepted Answer · 2020-08-16 19:57:35Z

1

I was stuck with the same issue today and after some debugging and trying out the same operation on CLI, I found that new JobReport().withBucket(reportBucketName) takes a bucketArn instead of a bucket name.

The actual issue might be different in your case. I suggest you serialize your request from code and try out the same operation in CLI and match both the requests.

AWS Error messages are often not very helpful when we actually need them.

edited Aug 16, 2020 at 19:57

answered Aug 16, 2020 at 19:49

zer0

112 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

GAGAN BHATIA Over a year ago

Updated name with Arn for the bucket, doesn't seem to be working. To add to this point, I cant create batch from CLI also. Will add cli command as next comment

GAGAN BHATIA · Accepted Answer · 2020-08-19 10:46:25Z

0

I got the issue, issue was related to the gradle versions, we need to make sure we have all aws services gradle versions to be same.

In my case -

    compile group: 'com.amazonaws', name: 'aws-java-sdk-dynamodb', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-iam', version: '1.11.844' 
    compile group: 'com.amazonaws', name: 'aws-java-sdk-events', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-s3', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-batch', version: '1.11.844'
    compile group: 'com.amazonaws', name: 'aws-java-sdk-s3control', version:'1.11.844'

answered Aug 19, 2020 at 10:46

GAGAN BHATIA

6016 silver badges17 bronze badges

Collectives™ on Stack Overflow

Getting 400 Bad Request Error while creating S3 Batch Job from Java Code

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related