MOSIP cluster - NGINX VMs, DNS Requirements, Istio ...using RKE/Rancher

Good day!
I am able to install Rancher on RKE cluster and now in the process of installing the MOSIP cluster.
I want to clarify, the VM for NGINX LB for MOSIP cluster is different from the VM for Rancher RKE cluster? Right?
Thus the internal IP for this VM is the one needed in the DNS Requirements’ internal IP?
One more thing, in installing NGINX in this VM, the install.sh script is configured to get IP from dummy-ips-nginx.service. Will this work on prem?
I tried it but the installation hangs at getting IP. What did I miss?
Also, any step by step procedure in installing Istio? Where to put the values for $INTERNAL and $Public or what to edit and what information to put.
Thank you for your kind assistance.

Good Day

Yes the VM’s for Rancher cluster nginx and Mosip cluster nginx are different.

Yes the internal IP is the one needed for the internal communication and external ip is the one needed for the external interaction with publically accessible modules.

Actually our install.sh script prompts there to input the internal and external ips seperately along with other parameters and then sets the same and install the nginx there.

Can you again brief where are we facing issues. also we have now stopped using dummy-ips-nginx.service.

Right away I saw where my mistake is -

I have been following this MOSIP-infra github - mosip-infra/deployment/v3 at master · mosip/mosip-infra · GitHub

Thank you!

I have to read through this and try again. GitHub - mosip/k8s-infra: Kubernetes infrastructure to deploy MOSIP modules.

After going through this briefly, it only takes care of setting up the Kubernetes cluster.
I guess it means going back to MOSIP-infra github v3 and then follow the instructions in installing the MOSIP external components and core modules:

Is that right?

Yes… this is the correct sequence…
Basically start from mosip-infra. It will point you to cluster related scripts and docs in k8s-infra, once cluster is ready start with external and then MOSIP modules from mosip-infra.

@ckm007 Thank you.

Everything seems to be installing fine, now that I am working on 1.2.0.1 branch.

I have additional questions though. Do I install Monitoring (Prometheus) before installing Istio?
I encountered this error when installing Istio, although afterwards everything looks to be fine.

:~/k8s-infra/mosip/on-prem/istio$ ./install.sh
Operator init
Installing operator controller in namespace: istio-operator using image: docker.io/istio/operator:1.14.3
Operator controller will watch namespaces: istio-system
:heavy_check_mark: Istio operator installed
:heavy_check_mark: Installation complete
Create ingress gateways, load balancers and istio monitoring
istiooperator.install.istio.io/istio-operators-mosip created
resource mapping not found for name: “envoy-stats-monitor” namespace: “istio-system” from “istio-monitoring/PodMonitor.yaml”: no matches for kind “PodMonitor” in version “monitoring.coreos.com/v1
ensure CRDs are installed first
resource mapping not found for name: “istio-component-monitor” namespace: “istio-system” from “istio-monitoring/ServiceMonitor.yaml”: no matches for kind “ServiceMonitor” in version “monitoring.coreos.com/v1
ensure CRDs are installed first
Wait for all resources to come up
Error from server (NotFound): deployments.apps “istiod” not found
Error from server (NotFound): deployments.apps “istio-ingressgateway” not found
Error from server (NotFound): deployments.apps “istio-ingressgateway-internal” not found
Installing gateways, proxy protocol, authpolicies
Public domain: api.dcsmosip.science.upd.edu.ph
Internal dome: api-internal.dcsmosip.science.upd.edu.ph
NAME: istio-addons
LAST DEPLOYED: Thu Aug 11 18:52:45 2022
NAMESPACE: istio-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
------ IMPORTANT ---------
If you already have pods running with envoy sidecars, restart all of them NOW. Check if all of them appear with command istioctl proxy-status
--------------------------

:~/k8s-infra/mosip/on-prem/istio$ istioctl proxy-status
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
istio-ingressgateway-55cc548579-ljvfs.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-5947f988df-zd7nd 1.14.3
istio-ingressgateway-internal-78bd7888f-mt7h7.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-5947f988df-zd7nd 1.14.3

:~/k8s-infra/mosip/on-prem/istio$ kubectl get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway NodePort 10.43.214.92 15021:30521/TCP,80:30080/TCP 3m17s
istio-ingressgateway-internal NodePort 10.43.69.173 15021:31521/TCP,80:31080/TCP,61616:31616/TCP,5432:31432/TCP 3m17s
istiod ClusterIP 10.43.65.178 15010/TCP,15012/TCP,443/TCP,15014/TCP 3m27s

:~/k8s-infra/mosip/on-prem/istio$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
cattle-fleet-system fleet-agent-96f6f455c-b68c9 1/1 Running 0 75m
cattle-system cattle-cluster-agent-6569ffb898-stmms 1/1 Running 3 77m
cattle-system cattle-cluster-agent-6569ffb898-t6bkf 1/1 Running 0 76m
cattle-system helm-operation-hv42d 0/2 Completed 0 54m
istio-operator istio-operator-84d9f8d555-fqbhs 1/1 Running 0 5m58s
istio-system istio-ingressgateway-55cc548579-ljvfs 1/1 Running 0 5m23s
istio-system istio-ingressgateway-internal-78bd7888f-mt7h7 1/1 Running 0 5m23s
istio-system istiod-5947f988df-zd7nd 1/1 Running 0 5m33s
kube-system calico-kube-controllers-85f985646c-2xr9c 1/1 Running 2 108m
kube-system canal-5hngh 2/2 Running 0 108m
kube-system canal-c22pl 2/2 Running 0 108m
kube-system canal-dlhkt 2/2 Running 0 108m
kube-system canal-jbzrn 2/2 Running 0 108m
kube-system canal-tn5qr 2/2 Running 0 108m
kube-system coredns-685d6d555d-56dh5 1/1 Running 0 108m
kube-system coredns-685d6d555d-96dtt 1/1 Running 0 107m
kube-system coredns-autoscaler-96789f8f7-gs4v2 1/1 Running 0 108m
kube-system metrics-server-d67876c44-j2pjg 0/1 Running 0 108m
kube-system rke-coredns-addon-deploy-job-zph4f 0/1 Completed 0 108m
kube-system rke-metrics-addon-deploy-job-46w49 0/1 Completed 0 108m
kube-system rke-network-plugin-deploy-job-cjmtl 0/1 Completed 0 108m
longhorn-system csi-attacher-5ddf9c48cf-4sqv5 1/1 Running 0 54m
longhorn-system csi-attacher-5ddf9c48cf-74tt9 1/1 Running 0 54m
longhorn-system csi-attacher-5ddf9c48cf-lq5kg 1/1 Running 0 54m
longhorn-system csi-provisioner-59b7b8b7b8-bbsc2 1/1 Running 0 54m
longhorn-system csi-provisioner-59b7b8b7b8-fcnt6 1/1 Running 0 54m
longhorn-system csi-provisioner-59b7b8b7b8-mj98b 1/1 Running 0 54m
longhorn-system csi-resizer-68ccff94-75kbs 1/1 Running 0 54m
longhorn-system csi-resizer-68ccff94-vjqfz 1/1 Running 0 54m
longhorn-system csi-resizer-68ccff94-zc2pc 1/1 Running 0 54m
longhorn-system csi-snapshotter-6d7d679c98-8gprg 1/1 Running 0 54m
longhorn-system csi-snapshotter-6d7d679c98-n2fzr 1/1 Running 0 54m
longhorn-system csi-snapshotter-6d7d679c98-q24vk 1/1 Running 0 54m
longhorn-system engine-image-ei-dae99989-24l44 1/1 Running 0 54m
longhorn-system engine-image-ei-dae99989-bvnpm 1/1 Running 0 54m
longhorn-system engine-image-ei-dae99989-n4rql 1/1 Running 0 54m
longhorn-system engine-image-ei-dae99989-tnwfr 1/1 Running 0 54m
longhorn-system engine-image-ei-dae99989-z5zr7 1/1 Running 0 54m
longhorn-system instance-manager-e-21c0775e 1/1 Running 0 31m
longhorn-system instance-manager-e-2b38f706 1/1 Running 0 31m
longhorn-system instance-manager-e-52b8c8a9 1/1 Running 0 31m
longhorn-system instance-manager-e-819e3e0e 1/1 Running 0 31m
longhorn-system instance-manager-e-864a1812 1/1 Running 0 31m
longhorn-system instance-manager-r-1bfeb938 1/1 Running 0 31m
longhorn-system instance-manager-r-25790442 1/1 Running 0 31m
longhorn-system instance-manager-r-3356dc3a 1/1 Running 0 31m
longhorn-system instance-manager-r-8a0e0d51 1/1 Running 0 31m
longhorn-system instance-manager-r-bfdb03da 1/1 Running 0 31m
longhorn-system longhorn-admission-webhook-57ddddf8f8-8xtnn 1/1 Running 0 54m
longhorn-system longhorn-admission-webhook-57ddddf8f8-r4m55 1/1 Running 0 54m
longhorn-system longhorn-conversion-webhook-7d7cf6f877-qgnwh 1/1 Running 0 54m
longhorn-system longhorn-conversion-webhook-7d7cf6f877-wcjc7 1/1 Running 0 54m
longhorn-system longhorn-csi-plugin-5ztkr 2/2 Running 0 54m
longhorn-system longhorn-csi-plugin-dsdjv 2/2 Running 0 54m
longhorn-system longhorn-csi-plugin-g9ppp 2/2 Running 0 54m
longhorn-system longhorn-csi-plugin-h874p 2/2 Running 0 54m
longhorn-system longhorn-csi-plugin-r9ngt 2/2 Running 0 54m
longhorn-system longhorn-driver-deployer-69fcc75f85-j9xb2 1/1 Running 0 54m
longhorn-system longhorn-iscsi-installation-6lfpz 1/1 Running 0 65m
longhorn-system longhorn-iscsi-installation-bvrwd 1/1 Running 0 65m
longhorn-system longhorn-iscsi-installation-l78c7 1/1 Running 0 65m
longhorn-system longhorn-iscsi-installation-pcg7z 1/1 Running 0 65m
longhorn-system longhorn-iscsi-installation-xwjq9 1/1 Running 0 65m
longhorn-system longhorn-manager-cn5n7 1/1 Running 1 54m
longhorn-system longhorn-manager-ftjfw 1/1 Running 0 54m
longhorn-system longhorn-manager-lldfj 1/1 Running 0 54m
longhorn-system longhorn-manager-lrg59 1/1 Running 1 54m
longhorn-system longhorn-manager-pr5dm 1/1 Running 0 54m
longhorn-system longhorn-nfs-installation-cnsjf 1/1 Running 0 65m
longhorn-system longhorn-nfs-installation-jh49h 1/1 Running 0 65m
longhorn-system longhorn-nfs-installation-lm49n 1/1 Running 0 65m
longhorn-system longhorn-nfs-installation-rc4l4 1/1 Running 0 65m
longhorn-system longhorn-nfs-installation-vxwh6 1/1 Running 0 65m
longhorn-system longhorn-ui-6f45bd5c65-zrnxt 1/1 Running 0 54m

Also, what needs to be done in this part?

Istio injection

To enable Istio injection in a namespace:

*kubectl label ns <namespace> istio-injection=enabled --overwrite*

What namespaces do I need to configure with istio-injection ?