Hello again.
Currently, I am installing the ActiveMQ component from MOSIP External components in the cluster using the install.sh script.
The installation process seems to be smooth without any errors. However, after the deployment, the statefulset.apps/activemq-activemq-artemis-slave remains in a Not Ready state.
NAME READY STATUS RESTARTS AGE
pod/activemq-activemq-artemis-master-0 2/2 Running 0 56m
pod/activemq-activemq-artemis-slave-0 1/2 Running 0 56m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/activemq-activemq-artemis ClusterIP 10.43.207.210 <none> 8161/TCP,61616/TCP,5672/TCP 56m
service/activemq-activemq-artemis-master ClusterIP 10.43.169.194 <none> 8161/TCP,61616/TCP,5672/TCP,9494/TCP 56m
service/activemq-activemq-artemis-slave ClusterIP None <none> 8161/TCP,61616/TCP,5672/TCP,9494/TCP 56m
NAME READY AGE
statefulset.apps/activemq-activemq-artemis-master 1/1 56m
statefulset.apps/activemq-activemq-artemis-slave 0/1 56m
I checked the logs of both the master pod and the slave pod, and found the following error: “AMQ214033: Cannot resolve host: java.net.UnknownHostException: activemq-activemq-artemis-slave-0.activemq-activemq-artemis-slave.activemq”.
Although the master pod shows as “Ready,” the error messages in the pod logs are the same as the slave pod.
Merging input with '/var/lib/artemis/etc-override/broker-10.xml'
Merging input with '/var/lib/artemis/etc-override/broker-11.xml'
Merging input with '/opt/assets/enable-jmx.xml'
Calculating performance journal ...
256000
_ _ _
/ \ ____| |_ ___ __ __(_) _____
/ _ \| _ \ __|/ _ \ \/ | |/ __/
/ ___ \ | \/ |_/ __/ |\/| | |\___ \
/_/ \_\| \__\____|_| |_|_|/___ /
Apache ActiveMQ Artemis 2.16.0
2023-07-28 05:07:51,597 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server
2023-07-28 05:07:51,774 INFO [org.apache.activemq.artemis.core.server] AMQ221000: backup Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=data/journal,bindingsDirectory=data/bindings,largeMessagesDirectory=data/large-messages,pagingDirectory=data/paging)
2023-07-28 05:07:51,798 INFO [org.apache.activemq.artemis.core.server] AMQ222162: Moving data directory /var/lib/artemis/data/journal to /var/lib/artemis/data/journal/oldreplica.1
2023-07-28 05:07:52,004 INFO [org.apache.activemq.artemis.core.server] AMQ221012: Using AIO Journal
2023-07-28 05:07:52,214 INFO [org.apache.activemq.artemis.core.server] AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size (-Xmx). being defined as 2,097,152,000
2023-07-28 05:07:52,383 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
2023-07-28 05:07:52,384 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
2023-07-28 05:07:52,384 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2023-07-28 05:07:52,385 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT
2023-07-28 05:07:52,385 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE
2023-07-28 05:07:52,386 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
2023-07-28 05:07:52,795 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized activemq-branding plugin
2023-07-28 05:07:53,024 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized artemis-plugin plugin
2023-07-28 05:07:53,278 ERROR [org.apache.activemq.artemis.core.client] AMQ214016: Failed to create netty connection: java.net.UnknownHostException: activemq-activemq-artemis-master-0.activemq-activemq-artemis-master.activemq
at java.net.InetAddress.getAllByName0(InetAddress.java:1281) [rt.jar:1.8.0_232]
at java.net.InetAddress.getAllByName(InetAddress.java:1193) [rt.jar:1.8.0_232]
at java.net.InetAddress.getAllByName(InetAddress.java:1127) [rt.jar:1.8.0_232]
at java.net.InetAddress.getByName(InetAddress.java:1077) [rt.jar:1.8.0_232]
at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:156) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:153) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_232]
at io.netty.util.internal.SocketUtils.addressByName(SocketUtils.java:153) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.DefaultNameResolver.doResolve(DefaultNameResolver.java:41) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:61) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:53) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:55) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:31) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.resolver.AbstractAddressResolver.resolve(AbstractAddressResolver.java:106) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:206) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.bootstrap.Bootstrap.access$000(Bootstrap.java:46) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:180) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:166) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetSuccess(AbstractChannel.java:984) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:504) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:417) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:474) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:387) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.51.Final.jar:4.1.51.Final]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.16.0.jar:2.16.0]
I tried using the ping command in the pod, but couldn’t ping this address. However, I can ping activemq-activemq-artemis-slave-0.activemq-activemq-artemis-slave.activemq.svc.cluster.local successfully.
I also attempted to add a rule in the cluster’s coreDNS to map “activemq-activemq-artemis-slave-0.activemq-activemq-artemis-slave.activemq” to “activemq-activemq-artemis-slave-0.activemq-activemq-artemis-slave.activemq.svc.cluster.local”, but it didn’t work. Have you encountered similar issues, or do you have any troubleshooting suggestions? Thank you very much.
Also, I am using tag v1.2.0.1-B3 version.
update:
sorry I didn’t notice the note in the README before:
- Master/slave: Note that after successful installation you will see Artemis master pod in
2/2state while slave pod will continue to be in1/2state. This is the high-availability master-slave configuration. If master fails, the slave will become2/2receiving traffic from Artemis service and vice-a-versa. You will see “readiness probe failed” message for the slave pod - this is normal and expected.
i saw the readiness probe failed message，does this mean the error logs mentioned above are also normal?
Thank you so much.