4

I created a cluster according to AWS' tutorials and it supposed to be working just fine but when I try to connect to it, I get the following error

java.io.IOException: failed to configure cluster endpoints from hosts: [mydaxcluster.ximfba.clustercfg.dax.use1.cache.amazonaws.com:8111] at com.amazon.dax.client.cluster.Source$AutoconfSource.pull(Source.java:128) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Source$AutoconfSource.update(Source.java:60) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Source$AutoconfSource.refresh(Source.java:51) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Cluster.refresh(Cluster.java:311) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Cluster.refresh(Cluster.java:294) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Cluster.startup(Cluster.java:191) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Cluster.startup(Cluster.java:143) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.dynamodbv2.ClusterDaxClient.(ClusterDaxClient.java:159) [DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.dynamodbv2.ClusterDaxClient.(ClusterDaxClient.java:130) [DaxJavaClient-latest.jar:1.0.792.0] at com.poc.dynamo.controller.DataController.testDax(DataController.java:190) [classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:108) [spring-boot-actuator-1.5.1.RELEASE.jar:1.5.1.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) [spring-boot-actuator-1.5.1.RELEASE.jar:1.5.1.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.6.RELEASE.jar:4.3.6.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:474) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434) [tomcat-embed-core-8.5.11.jar:8.5.11] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.11.jar:8.5.11] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.11.jar:8.5.11] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] Caused by: java.net.ConnectException: timeout waiting for an available tube at com.amazon.dax.client.SocketTubePool.alloc(SocketTubePool.java:77) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.generated.DaxClientStubs.endpoints_455855874_1(DaxClientStubs.java:449) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.dynamodbv2.DaxClient.endpoints(DaxClient.java:1825) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Source$AutoconfSource.pullFrom(Source.java:138) ~[DaxJavaClient-latest.jar:1.0.792.0] at com.amazon.dax.client.cluster.Source$AutoconfSource.pull(Source.java:106) ~[DaxJavaClient-latest.jar:1.0.792.0] ... 75 common frames omitted Suppressed: java.net.ConnectException: timeout waiting for an available tube ... 80 common frames omitted Suppressed: java.net.ConnectException: timeout waiting for an available tube ... 80 common frames omitted

Unable to read item Unable to call cluster endpoint: No endpoints available

Improve this question

3 Answers 3

Reset to default
5

We will need to know more info about how you setup your DAX cluster and client instances to be able to help. Some general ideas to try:

  1. Are you able to connect to the DAX cluster from the EC2 client instance? Try running: nc -z mydaxcluster.ximfba.clustercfg.dax.use1.cache.amazonaws.com 8111
  2. Have you used the default security group for your VPC to create DAX cluster? You will need to authorize inbound traffic on TCP port 8111 to the security group used to create the DAX cluster from the security group of the client instance. Please follow the steps for configuring security group rules for your VPC
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Thanks for the reply, I did all the steps on AWS' website, including configuring the security group rules. I wasn't able to connect to my cluster.
Can you try the nc -z command above from your EC2 client instance and paste the output. If the result is not Connection to mydaxcluster.ximfba.clustercfg.dax.use1.cache.amazonaws.com 8111 port [tcp/*] succeeded!. Then we need to troubleshoot your VPC setup.
I'm new to AWS so what do you mean when you say "your EC2 client instance"? Thanks.
To be able to use DAX, your application (DAX client software) needs to run on an EC2 instance that is in the same VPC used to create DAX cluster. Please refer to the steps in the documentation on how to create an instance on EC2 and how to run DAX client software on that instance.
For some reason I have never seen these particular steps before, the ones I was following are different. I'll get started on these and update you, thanks a lot!
|
0

I had faced the same issue, when I tried to connect to a Dax cluster from ECS containers.

Here is the thing that worked for me, as suggested by Abdelrahman Elhaddad

  1. Make sure the subnets and the VPC of the application server and the dax cluster is same.

  2. On your Dax Cluster's security group, ensure inbound TCP traffic is enabled on port 8111 (if your DAX cluster is unencrypted) and 9111 (for encrypted).

  3. And on your server's security group, make sure that TCP outbound requests on 8111/9111 are allowed.

PS: If you are on ECS, modifying docker images can be very cumbersome, use any EC2 instances and try to alter the variables if needed. Also, you can make use of command line network tool to check your security group rules

Improve this answer

Comments

0

Faced the same issue, but resolved it via IAM role. My NC to the Dax Endpoint from an EC2 instance did not return anything like a success message either, but modifying the IAM role fixed it. For ECS, you have to modify the Task Role. Adding the AmazonDynamoDBFullAccess policy to the role worked for me. The IAM policy should have these two actions: "dynamodb:*", "dax:*", Your ECS or EC2 does NOT have to be in the same subnet as the DAX endpoint for it to work. It only needs to be in the same VPC.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.