# MOSIP cluster - NGINX VMs, DNS Requirements, Istio ...using RKE/Rancher **URL:** \ **Category:** Platform\ **Created:** [July 29, 2022, 6:13am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106 "2022-07-29T06:13:56Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![rcsampang](https://avatars.discourse-cdn.com/v4/letter/r/ecccb3/32.png) [@rcsampang](https://community.mosip.io/u/rcsampang)\ **Post date:** [July 29, 2022, 6:13am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/1 "2022-07-29T06:13:56Z")
Good day! I am able to install Rancher on RKE cluster and now in the process of installing the MOSIP cluster. I want to clarify, the VM for NGINX LB for MOSIP cluster is different from the VM for Rancher RKE cluster? Right? Thus the internal IP for this VM is the one needed in the DNS Requirements’ internal IP? One more thing, in installing NGINX in this VM, the install.sh script is configured to get IP from dummy-ips-nginx.service. Will this work on prem? I tried it but the installation hangs at getting IP. What did I miss? Also, any step by step procedure in installing Istio? Where to put the values for $INTERNAL and $Public or what to edit and what information to put. Thank you for your kind assistance. ---
**Author:** ![ckm007](https://dub1.discourse-cdn.com/flex017/user_avatar/community.mosip.io/ckm007/32/264_2.png) [@ckm007](https://community.mosip.io/u/ckm007)\ **Post date:** [August 1, 2022, 8:35am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/2 "2022-08-01T08:35:59Z")
Good Day Yes the VM’s for Rancher cluster nginx and Mosip cluster nginx are different. Yes the internal IP is the one needed for the internal communication and external ip is the one needed for the external interaction with publically accessible modules. Actually our install.sh script prompts there to input the internal and external ips seperately along with other parameters and then sets the same and install the nginx there. > Can you again brief where are we facing issues. also we have now stopped using dummy-ips-nginx.service. > ---
**Author:** ![rcsampang](https://avatars.discourse-cdn.com/v4/letter/r/ecccb3/32.png) [@rcsampang](https://community.mosip.io/u/rcsampang)\ **Post date:** [August 1, 2022, 3:58pm UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/3 "2022-08-01T15:58:14Z")
Right away I saw where my mistake is - I have been following this MOSIP-infra github - [mosip-infra/deployment/v3 at master · mosip/mosip-infra · GitHub](https://github.com/mosip/mosip-infra/tree/master/deployment/v3) Thank you! I have to read through this and try again. [GitHub - mosip/k8s-infra: Kubernetes infrastructure to deploy MOSIP modules.](https://github.com/mosip/k8s-infra) After going through this briefly, it only takes care of setting up the Kubernetes cluster. I guess it means going back to MOSIP-infra github v3 and then follow the instructions in installing the MOSIP external components and core modules: > > Is that right? ---
**Author:** ![ckm007](https://dub1.discourse-cdn.com/flex017/user_avatar/community.mosip.io/ckm007/32/264_2.png) [@ckm007](https://community.mosip.io/u/ckm007)\ **Post date:** [August 9, 2022, 10:01am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/4 "2022-08-09T10:01:25Z")
Yes… this is the correct sequence… Basically start from mosip-infra. It will point you to cluster related scripts and docs in k8s-infra, once cluster is ready start with external and then MOSIP modules from mosip-infra. ---
**Author:** ![rcsampang](https://avatars.discourse-cdn.com/v4/letter/r/ecccb3/32.png) [@rcsampang](https://community.mosip.io/u/rcsampang)\ **Post date:** [August 11, 2022, 7:45pm UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/5 "2022-08-11T19:45:45Z")
@ckm007 Thank you. Everything seems to be installing fine, now that I am working on 1.2.0.1 branch. I have additional questions though. Do I install Monitoring (Prometheus) before installing Istio? I encountered this error when installing Istio, although afterwards everything looks to be fine. _:~/k8s-infra/mosip/on-prem/istio$ ./install.sh_ _Operator init_ _Installing operator controller in namespace: istio-operator using image: [docker.io/istio/operator:1.14.3](http://docker.io/istio/operator:1.14.3)_ _Operator controller will watch namespaces: istio-system_ _✔ Istio operator installed_ _✔ Installation complete_ _Create ingress gateways, load balancers and istio monitoring_ _[istiooperator.install.istio.io/istio-operators-mosip](http://istiooperator.install.istio.io/istio-operators-mosip) created_ _**resource mapping not found for name: “envoy-stats-monitor” namespace: “istio-system” from “istio-monitoring/PodMonitor.yaml”: no matches for kind “PodMonitor” in version “[monitoring.coreos.com/v1](http://monitoring.coreos.com/v1)”**_ _ **ensure CRDs are installed first** _ _**resource mapping not found for name: “istio-component-monitor” namespace: “istio-system” from “istio-monitoring/ServiceMonitor.yaml”: no matches for kind “ServiceMonitor” in version “[monitoring.coreos.com/v1](http://monitoring.coreos.com/v1)”**_ _ **ensure CRDs are installed first** _ _ **Wait for all resources to come up** _ _**Error from server (NotFound): deployments.apps “istiod” not found**_ _**Error from server (NotFound): deployments.apps “istio-ingressgateway” not found**_ _**Error from server (NotFound): deployments.apps “istio-ingressgateway-internal” not found**_ _Installing gateways, proxy protocol, authpolicies_ _Public domain: api.dcsmosip.science.upd.edu.ph_ _Internal dome: api-internal.dcsmosip.science.upd.edu.ph_ _NAME: istio-addons_ _LAST DEPLOYED: Thu Aug 11 18:52:45 2022_ _NAMESPACE: istio-system_ _STATUS: deployed_ _REVISION: 1_ _TEST SUITE: None_ _------ IMPORTANT ---------_ _If you already have pods running with envoy sidecars, restart all of them NOW. Check if all of them appear with command istioctl proxy-status_ _--------------------------_ _ **:~/k8s-infra/mosip/on-prem/istio$ istioctl proxy-status** _ _NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION_ _istio-ingressgateway-55cc548579-ljvfs.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-5947f988df-zd7nd 1.14.3_ _istio-ingressgateway-internal-78bd7888f-mt7h7.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-5947f988df-zd7nd 1.14.3_ _ **:~/k8s-infra/mosip/on-prem/istio$ kubectl get svc -n istio-system** _ _NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE_ _istio-ingressgateway NodePort 10.43.214.92 15021:30521/TCP,80:30080/TCP 3m17s_ _istio-ingressgateway-internal NodePort 10.43.69.173 15021:31521/TCP,80:31080/TCP,61616:31616/TCP,5432:31432/TCP 3m17s_ _istiod ClusterIP 10.43.65.178 15010/TCP,15012/TCP,443/TCP,15014/TCP 3m27s_ _ **:~/k8s-infra/mosip/on-prem/istio$ kubectl get pods -A** _ _NAMESPACE NAME READY STATUS RESTARTS AGE_ _cattle-fleet-system fleet-agent-96f6f455c-b68c9 1/1 Running 0 75m_ _cattle-system cattle-cluster-agent-6569ffb898-stmms 1/1 Running 3 77m_ _cattle-system cattle-cluster-agent-6569ffb898-t6bkf 1/1 Running 0 76m_ _cattle-system helm-operation-hv42d 0/2 Completed 0 54m_ _istio-operator istio-operator-84d9f8d555-fqbhs 1/1 Running 0 5m58s_ _istio-system istio-ingressgateway-55cc548579-ljvfs 1/1 Running 0 5m23s_ _istio-system istio-ingressgateway-internal-78bd7888f-mt7h7 1/1 Running 0 5m23s_ _istio-system istiod-5947f988df-zd7nd 1/1 Running 0 5m33s_ _kube-system calico-kube-controllers-85f985646c-2xr9c 1/1 Running 2 108m_ _kube-system canal-5hngh 2/2 Running 0 108m_ _kube-system canal-c22pl 2/2 Running 0 108m_ _kube-system canal-dlhkt 2/2 Running 0 108m_ _kube-system canal-jbzrn 2/2 Running 0 108m_ _kube-system canal-tn5qr 2/2 Running 0 108m_ _kube-system coredns-685d6d555d-56dh5 1/1 Running 0 108m_ _kube-system coredns-685d6d555d-96dtt 1/1 Running 0 107m_ _kube-system coredns-autoscaler-96789f8f7-gs4v2 1/1 Running 0 108m_ _kube-system metrics-server-d67876c44-j2pjg 0/1 Running 0 108m_ _kube-system rke-coredns-addon-deploy-job-zph4f 0/1 Completed 0 108m_ _kube-system rke-metrics-addon-deploy-job-46w49 0/1 Completed 0 108m_ _kube-system rke-network-plugin-deploy-job-cjmtl 0/1 Completed 0 108m_ _longhorn-system csi-attacher-5ddf9c48cf-4sqv5 1/1 Running 0 54m_ _longhorn-system csi-attacher-5ddf9c48cf-74tt9 1/1 Running 0 54m_ _longhorn-system csi-attacher-5ddf9c48cf-lq5kg 1/1 Running 0 54m_ _longhorn-system csi-provisioner-59b7b8b7b8-bbsc2 1/1 Running 0 54m_ _longhorn-system csi-provisioner-59b7b8b7b8-fcnt6 1/1 Running 0 54m_ _longhorn-system csi-provisioner-59b7b8b7b8-mj98b 1/1 Running 0 54m_ _longhorn-system csi-resizer-68ccff94-75kbs 1/1 Running 0 54m_ _longhorn-system csi-resizer-68ccff94-vjqfz 1/1 Running 0 54m_ _longhorn-system csi-resizer-68ccff94-zc2pc 1/1 Running 0 54m_ _longhorn-system csi-snapshotter-6d7d679c98-8gprg 1/1 Running 0 54m_ _longhorn-system csi-snapshotter-6d7d679c98-n2fzr 1/1 Running 0 54m_ _longhorn-system csi-snapshotter-6d7d679c98-q24vk 1/1 Running 0 54m_ _longhorn-system engine-image-ei-dae99989-24l44 1/1 Running 0 54m_ _longhorn-system engine-image-ei-dae99989-bvnpm 1/1 Running 0 54m_ _longhorn-system engine-image-ei-dae99989-n4rql 1/1 Running 0 54m_ _longhorn-system engine-image-ei-dae99989-tnwfr 1/1 Running 0 54m_ _longhorn-system engine-image-ei-dae99989-z5zr7 1/1 Running 0 54m_ _longhorn-system instance-manager-e-21c0775e 1/1 Running 0 31m_ _longhorn-system instance-manager-e-2b38f706 1/1 Running 0 31m_ _longhorn-system instance-manager-e-52b8c8a9 1/1 Running 0 31m_ _longhorn-system instance-manager-e-819e3e0e 1/1 Running 0 31m_ _longhorn-system instance-manager-e-864a1812 1/1 Running 0 31m_ _longhorn-system instance-manager-r-1bfeb938 1/1 Running 0 31m_ _longhorn-system instance-manager-r-25790442 1/1 Running 0 31m_ _longhorn-system instance-manager-r-3356dc3a 1/1 Running 0 31m_ _longhorn-system instance-manager-r-8a0e0d51 1/1 Running 0 31m_ _longhorn-system instance-manager-r-bfdb03da 1/1 Running 0 31m_ _longhorn-system longhorn-admission-webhook-57ddddf8f8-8xtnn 1/1 Running 0 54m_ _longhorn-system longhorn-admission-webhook-57ddddf8f8-r4m55 1/1 Running 0 54m_ _longhorn-system longhorn-conversion-webhook-7d7cf6f877-qgnwh 1/1 Running 0 54m_ _longhorn-system longhorn-conversion-webhook-7d7cf6f877-wcjc7 1/1 Running 0 54m_ _longhorn-system longhorn-csi-plugin-5ztkr 2/2 Running 0 54m_ _longhorn-system longhorn-csi-plugin-dsdjv 2/2 Running 0 54m_ _longhorn-system longhorn-csi-plugin-g9ppp 2/2 Running 0 54m_ _longhorn-system longhorn-csi-plugin-h874p 2/2 Running 0 54m_ _longhorn-system longhorn-csi-plugin-r9ngt 2/2 Running 0 54m_ _longhorn-system longhorn-driver-deployer-69fcc75f85-j9xb2 1/1 Running 0 54m_ _longhorn-system longhorn-iscsi-installation-6lfpz 1/1 Running 0 65m_ _longhorn-system longhorn-iscsi-installation-bvrwd 1/1 Running 0 65m_ _longhorn-system longhorn-iscsi-installation-l78c7 1/1 Running 0 65m_ _longhorn-system longhorn-iscsi-installation-pcg7z 1/1 Running 0 65m_ _longhorn-system longhorn-iscsi-installation-xwjq9 1/1 Running 0 65m_ _longhorn-system longhorn-manager-cn5n7 1/1 Running 1 54m_ _longhorn-system longhorn-manager-ftjfw 1/1 Running 0 54m_ _longhorn-system longhorn-manager-lldfj 1/1 Running 0 54m_ _longhorn-system longhorn-manager-lrg59 1/1 Running 1 54m_ _longhorn-system longhorn-manager-pr5dm 1/1 Running 0 54m_ _longhorn-system longhorn-nfs-installation-cnsjf 1/1 Running 0 65m_ _longhorn-system longhorn-nfs-installation-jh49h 1/1 Running 0 65m_ _longhorn-system longhorn-nfs-installation-lm49n 1/1 Running 0 65m_ _longhorn-system longhorn-nfs-installation-rc4l4 1/1 Running 0 65m_ _longhorn-system longhorn-nfs-installation-vxwh6 1/1 Running 0 65m_ _longhorn-system longhorn-ui-6f45bd5c65-zrnxt 1/1 Running 0 54m_ **Also, what needs to be done in this part?** ## Istio injection To enable Istio injection in a namespace: ```auto *kubectl label ns istio-injection=enabled --overwrite* ``` **What namespaces do I need to configure with istio-injection ?** ---
**Author:** ![ckm007](https://dub1.discourse-cdn.com/flex017/user_avatar/community.mosip.io/ckm007/32/264_2.png) [@ckm007](https://community.mosip.io/u/ckm007)\ **Post date:** [August 16, 2022, 5:30am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/6 "2022-08-16T05:30:15Z")
Noted @rcsampang. Basically as per documents we should install istion first as part of cluster setup. Since the same was not followed it is throwing error for already deployed monitoring services. So you can delete the monitoring services there and re install the same. It will fix the existing issue. Also note the error: **ensure CRDs are installed first** \* resource mapping not found for name: “istio-component-monitor” namespace: “istio-system” from “istio-monitoring/ServiceMonitor.yaml”: no matches for kind Also for Istio injectio part: we have already added this as part of all the install.sh scripts to enable or disable istio injection as and when required. So no need to make any seperate changes for this. example: [mosip-infra/install.sh at 1.2.0.1 · mosip/mosip-infra · GitHub](https://github.com/mosip/mosip-infra/blob/1.2.0.1/deployment/v3/mosip/idrepo/install.sh#L16) ---
**Author:** ![rcsampang](https://avatars.discourse-cdn.com/v4/letter/r/ecccb3/32.png) [@rcsampang](https://community.mosip.io/u/rcsampang)\ **Post date:** [August 17, 2022, 4:45am UTC](https://community.mosip.io/t/mosip-cluster-nginx-vms-dns-requirements-istio-using-rke-rancher/106/7 "2022-08-17T04:45:42Z")
@ckm007 Thank you for the clarifications. I deleted and recreated my MOSIP clusters and followed the instructions as per 1.2.0.1 branch. Happy to say that installation up to external modules went fine without any errors. Did not encounter any more errors in Istio and was able to install Prometheus successfully. The only issue I am having right now is that I have run out of resources halfway through the installation of MOSIP services. The culprit is that the resources I requested was based on the Master branch which has lower specifications than what is needed as per 1.2.0.1 branch. Thus, it wasn’t enough to run MOSIP services. The good news is that it looks like the services are installing fine although it has run out of resources to continue the installation of other services. I have already requested for upgrades and once approved I will do the installation again. Once it is running I would gladly report it. Thank you for all the kind understanding, assistance, and continuous support.