MOSIP services v3 1.2.0.1 installation issues - kernel (notifier) , keymanager

Support
1

Hello! I would like to ask for help in solving the issues encountered during installation of MOSIP services v3 1.2.0.1, particularly: kernel (notifier) , keymanager

image
image1022×503 81 KB

Events:
Type Reason Age From Message

Warning Unhealthy 10m (x217 over 164m) kubelet Startup probe failed: HTTP probe failed with statuscode: 500
Normal Pulling 5m14s (x30 over 165m) kubelet Pulling image “docker.io/mosipqa/kernel-notification-service:1.2.0.1”
Warning Unhealthy 24s (x669 over 163m) kubelet Startup probe failed: Get “http://10.42.3.69:15020/app-health/notifier/startupz”: context deadline exceeded (Client.Timeout exceeded while awaiting headers)

image
image1022×564 80.6 KB

{“@timestamp”:“2022-09-07T06:06:15.676Z”,“@version”:“1”,“message”:“Could not locate PropertySource: I/O error on GET request for "http://config-server.config-server/config/kernel/default/develop3-v3\”: Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)",“logger_name”:“org.springframework.cloud.config.client.ConfigServicePropertySourceLocator”,“thread_name”:“main”,“level”:“WARN”,“level_value”:30000,“appName”:“keys-generator”}

Sorry. I don’t know how to attach log files that may be useful without making a very long post.

Thank you.

2

@rcsampang Yes, we see this problem in our environment as well. This seems to be a little specific to the V3 setup, We are investigating this problem, and will soon come back to you. For a workaround we moved the timeout to 300

3

@gsasikumar Thank you. It looks like changing the timeout to 300 did not fix it for me. Here is an excerpt of the log file:

2022-09-14 03:32:39,999 [http-nio-8083-exec-7] WARN [o.s.b.a.h.AbstractHealthIndicator].health.89 : Mail health check failed

javax.mail.MessagingException: Could not connect to SMTP host: smtp.gmail.com, port: 587, response: 421
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2197)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:740)

at javax.mail.Service.connect(Service.java:366)
at java.base/java.lang.Thread.run(Thread.java:829)

Further reading led me to changing the application-default.properties, is this the correct version to use?

mosip-config/application-default.properties at 1.2.0.1-v3 · mosip/mosip-config · GitHub.

Also in activemq, what is the password?
I read that i should " Password: Run get_pwd.sh", but I can’t find the script/file? Could you point/send me the script/file?

Best regards!

5

Looks like the SMTP port is not accessible. Can you check if you are able to connect to the SMTP port? Gmail also has limitations in the number of emails that can be sent.

6

I tested using telnet and ncat . Telnet results to SMTP service while ncat connects to UDP. The same results for Yahoo SMTP

rke@dcsmosip:~$ telnet smtp.gmail.com 587
Trying 74.125.23.109…
Connected to smtp.gmail.com.
Escape character is ‘^]’.
421 No SMTP service here
Connection closed by foreign host.

rke@dcsmosip:~$ nc -v -u smtp.gmail.com 587
Connection to smtp.gmail.com 587 port [udp/*] succeeded!
^C

rke@dcsmosip:~$ telnet smtp.mail.yahoo.com 587
Trying 66.218.88.167…
Connected to smtp.mail.us.am0.yahoodns.net.
Escape character is ‘^]’.
421 No SMTP service here
Connection closed by foreign host.

rke@dcsmosip:~$ nc -v -u smtp.mail.yahoo.com 587
Connection to smtp.mail.yahoo.com 587 port [udp/*] succeeded!
^C

How do I proceed from here? Can i setup a local SMTP server using Posix and use that instead?

I also did kubectl describe pod notifier-8459f88bc8-4krdx -n kernel
…
Events:
Type Reason Age From Message

Warning Unhealthy 54m (x9158 over 5d2h) kubelet Startup probe failed: HTTP probe failed with statuscode: 500
Normal Pulling 49m (x1343 over 5d2h) kubelet Pulling image “docker.io/mosipqa/kernel-notification-service:1.2.0.1”
Warning BackOff 14m (x2214 over 5d2h) kubelet Back-off restarting failed container
Warning Unhealthy 4m29s (x28381 over 5d2h) kubelet Startup probe failed: Get “http://10.42.3.110:15020/app-health/notifier/startupz”: context deadline exceeded (Client.Timeout exceeded while awaiting headers)

AND kubectl logs notifier-8459f88bc8-4krdx -n kernel
…
{“@timestamp”:“2022-09-21T05:34:49.422Z”,“level”:“ACCESS”,“level_value”:70000,“traceId”:“ed0d462536d3677e”,“statusCode”:503,“req.requestURI”:“/v1/notifier/actuator/health”,“bytesSent”:353,“timeTaken”:10.010,“appName”:“kernel-notification-service”,“req.userAgent”:“kube-probe/1.23”,“req.xForwardedFor”:“-”,“req.referer”:“-”,“req.method”:“GET”,“req.remoteHost”:“127.0.0.6”}
{“@timestamp”:“2022-09-21T05:34:57.317Z”,“level”:“ACCESS”,“level_value”:70000,“traceId”:“b09c1978b9983691016575de6c188cc3”,“statusCode”:200,“req.requestURI”:“/v1/notifier/actuator/prometheus”,“bytesSent”:10952,“timeTaken”:0.008,“appName”:“kernel-notification-service”,“req.userAgent”:“Prometheus/2.28.1”,“req.xForwardedFor”:“-”,“req.referer”:“-”,“req.method”:“GET”,“req.remoteHost”:“127.0.0.6”}

7

This is a little strange.

Because the first error that you pointed out is with the SMTP response code 421. The 421 in Gmail means a server-side error or something about your IP address is unusual. Refer to the error code here Gmail SMTP errors and codes - Google Workspace Admin Help

Have you tried with any other SMTP server? We just recently completed the development of a mock SMTP server to avoid these limitations. Please refer here. GitHub - mosip/mock-smtp

We are in the process of testing this better. But you can run the docker or node version of this on a VM and point your SMTP to that SMTP server & port (8025). This could help overcome this problem.

8

I also tried using Yahoo. No improvement.
I will try it using the mock SMTP server you have just set up and report back the results.
Is there a default username and password ? How do I add/configure users and passwords?
Thanks!

9

Thank you. Using the mock SMTP worked. I deleted and then re-installed the MOSIP modules. I used the new credentials in the msg-gateway installation
Please enter the SMTP host x.x.x.x (IP of VM)
Please enter the SMTP user sasi@yazhi.io (user that worked in the test mail)
Please enter the SMTP secret key (blank, no pasword)
*** I don’t know yet how to add users and passwords in the mock SMTP server

Then edited kernel-default.properties (1.2.0.1-v3) before re-instaling MOSIP modules
mosip.kernel.notification.email.from=sasi@yazhi.io
…
spring.mail.port=8025

Successful! There is no longer error in kernel notifier installation.

While for keymanager I increased the idle timeout from 0 to 60s at deployment/v3/mosip/keymanager/idle_timeout_envoyfilter.yaml
…
name: envoy.filters.network.tcp_proxy
typed_config:
‘@type’: type.googleapis.com/envoy.config.filter.network.tcp_proxy.v2.TcpProxy
idle_timeout: 60s

Thus kernel and keymanager successfully installed.
I have also managed to get the ActiveMQ password after deleting and reinstalling the module.

Unfortunately, now I am encountering issues in installing Registration Processor - I will create a new topic for that since the issues in kernel notifier and keymanager are resolved.

1 Like
10

Hello,
We are installing the platform mosip behind a proxy, when installing the SMTP we put it the host value gmail.com
we get this error:
WARN [o.s.b.a.h.AbstractHealthIndicator].health.89 : Mail health check failed
javax.mail.MessagingException: Could not connect to SMTP host: gmail.com, port: 8025, response: -1

Any recommendation to resolve the problem or do we have to deploy a local SMTP server to avoid connecting to outside

thanks

11

What version of MOSIP are you using? I have just upgraded to v1.2.0.1-B2 because my previous installation crashed due to ungraceful shutdown of the cluster caused by a power failure.

In this version mock-smtp is fully integrated, although you have to include it in the install-all.sh script at mosip-infra/deployment/v3/mosip/all. In my case I placed it after artifactory.

In v1.2.0.1-B2, the new mock-stmp is deployed as an application in the MOSIP K8s cluster, so it is easier to manage and monitor. Also in this version, while installing msg-gateway you will be prompted with a question if you will use mock-smtp? If you answer Yes, everything will be configured for you. You just have to install/deploy mock-smtp by going to the mock-smtp directory and run the install.sh script or include it in the install-all.sh as I have stated above.

In version 1.2.0.1, I was not able to make it run on Gmail, so I installed the MOSIP mock-smtp and run it in a separate VM, in my case the Nginx VM. That worked but it means I have to monitor it separately from the cluster.

I hope that helps.

12

@slimab I think Gmail has blocked all SMTP access until it comes through a secure channel. I would recommend using mockSMTP for development or QA needs.

1 Like