Config Module Deployment

reddi3 · February 21, 2024, 7:47am

Hi Team,

I need a small clarification in config-module deployment for MOSIP. Currently, this is the config url(MOSIP git url) which we deployed for our setup. Now, we need to customize it and so we need to map our repo and redeploy the module. If we need to redeploy the config-server module, other modules will be affected(go down) I believe. So, once this module is redeployed, restarting other services should be enough. Or need to redeploy all the modules again? kindly confirm this and let me know if any queries.

reddi3 · February 22, 2024, 9:55am

I have tried deploying the config-server module with the custom repo url hosted in bitbucket and provided the details in the values.yaml. I have entered the repo url,version details in the file and made private field as true. Then I have entered the username and token details. For token, I tried with both bitbucket app password and Repository Access Token, but still getting unauthorized error. Kindly check and let me know if I am missing anything.

sanchi-singh24 · February 23, 2024, 12:55pm

Hi @reddi3

I understand you’re encountering an issue with deploying config-server. I’ve raised your request to our deployment team, who will be happy to assist you further. However, due to their current workload on high-priority tasks, their response may be slightly delayed. We apologize for any inconvenience this may cause and appreciate your patience.

Best Regards,
Team MOSIP

reddi3 · February 26, 2024, 6:57am

Hi @sanchi-singh24 ,

Thanks for the response. Waiting for the assistance from your team.
Current status is, we have deleted the deployed config server module and made the custom repo details changes in values.yaml and tried to redeploy. There is no error in the config-pod logs but it is stuck after this and finally it is stuck in the same stage and not coming up. Attaching the logs for your reference.Kindly check and let me know if the repo details need to be changed anywhere else.

syed.salman · February 26, 2024, 10:17am

Hi @reddi3

Could you kindly furnish the complete logs from the config-server, along with the repository URL, and the specific details provided in the values.yaml file for the config-server?

reddi3 · February 26, 2024, 11:51am

Hi @syed.salman ,

Kindly find the screenshots of the logs in the attachment. Since I could only attach upto max 5 files, I am sending you the screenshots of first 5 pages and then send it in the next reply.

reddi3 · February 26, 2024, 11:53am

Hi Syed,

Please find the last two Screenshots and screenshot of the repo configuration in the values.yaml file.

syed.salman · February 26, 2024, 1:42pm

@reddi3

As per the logs shared, there are no errors in logs, and also noticed that the application has not started. Sometimes pod takes more time to start the application.
To fix this, increase the startup probe failure threshold to 60:

kubectl -n config-server edit deployment config-server

        startupProbe:
          failureThreshold: 60
          httpGet:
            path: /config/*/*
            port: 51000
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5

reddi3 · February 27, 2024, 8:22am

Hi Syed,

Tried above setting like you suggested, but still stuck in same state.

Thanks
Naveen

sanchi-singh24 · March 12, 2024, 10:29am

Hi @reddi3

Let @syed.salman look into this and try helping you with a different approach.

Best Regards,
Team MOSIP

syed.salman · March 13, 2024, 2:25pm

@reddi3

could you provide me the full logs of the config-server

kubectl -n config-server logs <pod-name> --previous

Describe the pod

kubectl -n config-server describe pod <pod-name>

reddi3 · March 14, 2024, 6:03am

Hi Team,

Thanks for the response. Please find the full logs of config-server in the attachment in form of PNG files.

FYI…Attaching the describe pod screenshots in next reply as the attachment limit is reached.

Thanks
Naveen

reddi3 · March 14, 2024, 6:12am

Hi Team,

I have described the config-module pod and attaching the output below:

root@impiger-OptiPlex-7020:~# kubectl -n config-server describe pod config-server-c8d9465c-kt4c8
Name: config-server-c8d9465c-kt4c8
Namespace: config-server
Priority: 0
Service Account: config-server
Node: impiger-optiplex-7020/10.100.148.74
Start Time: Fri, 08 Mar 2024 16:13:40 +0530
Labels: app.kubernetes.io/instance=config-server
app.kubernetes.io/name=config-server
pod-template-hash=c8d9465c
security.istio.io/tlsMode=istio
service.istio.io/canonical-name=config-server
service.istio.io/canonical-revision=latest
Annotations: cni.projectcalico.org/containerID: e7d6bb420e1b5ad25cd0d72181b5dd9d06e22ddf803826675a1241b434aa53ca
cni.projectcalico.org/podIP: 10.42.0.77/32
cni.projectcalico.org/podIPs: 10.42.0.77/32
istio.io/rev: default
kubectl.kubernetes.io/default-container: config-server
kubectl.kubernetes.io/default-logs-container: config-server
prometheus.io/path: /stats/prometheus
prometheus.io/port: 15020
prometheus.io/scrape: true
sidecar.istio.io/status:
{“initContainers”:[“istio-init”],“containers”:[“istio-proxy”],“volumes”:[“workload-socket”,“credential-socket”,“workload-certs”,"istio-env…
Status: Running
IP: 10.42.0.77
IPs:
IP: 10.42.0.77
Controlled By: ReplicaSet/config-server-c8d9465c
Init Containers:
istio-init:
Container ID: docker://5c5c30dbe84f7e459bb7a968103027ff7b72eb16d593f7ac210c780ab4bb9ae0
Image: docker.io/istio/proxyv2:1.20.3
Image ID: docker-pullable://istio/proxyv2@sha256:18163bd4fdb641bdff1489e124a0b9f1059bb2cec9c8229161b73517db97c05a
Port:
Host Port:
Args:
istio-iptables
-p
15001
-z
15006
-u
1337
-m
REDIRECT
-i
*
-x

  -b
  *
  -d
  15090,15021,15020
  --log_output_level=default:info
State:          Terminated
  Reason:       Completed
  Exit Code:    0
  Started:      Fri, 08 Mar 2024 16:13:43 +0530
  Finished:     Fri, 08 Mar 2024 16:13:44 +0530
Ready:          True
Restart Count:  0
Limits:
  cpu:     2
  memory:  1Gi
Requests:
  cpu:     100m
  memory:  128Mi
Environment:
  ISTIO_META_IDLE_TIMEOUT:  0s
Mounts:
  /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bl8rj (ro)

Containers:
istio-proxy:
Container ID: docker://ca815c668e56dc9867207117010dd84d880ca0bcb52ab1cb26f143a7013bae7f
Image: docker.io/istio/proxyv2:1.20.3
Image ID: docker-pullable://istio/proxyv2@sha256:18163bd4fdb641bdff1489e124a0b9f1059bb2cec9c8229161b73517db97c05a
Port: 15090/TCP
Host Port: 0/TCP
Args:
proxy
sidecar
–domain
$(POD_NAMESPACE).svc.cluster.local
–proxyLogLevel=warning
–proxyComponentLogLevel=misc:error
–log_output_level=default:info
State: Running
Started: Fri, 08 Mar 2024 16:13:46 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 100m
memory: 128Mi
Readiness: http-get http://:15021/healthz/ready delay=0s timeout=3s period=15s #success=1 #failure=4
Startup: http-get http://:15021/healthz/ready delay=0s timeout=3s period=1s #success=1 #failure=600
Environment:
JWT_POLICY: third-party-jwt
PILOT_CERT_PROVIDER: istiod
CA_ADDR: istiod.istio-system.svc:15012
POD_NAME: config-server-c8d9465c-kt4c8 (v1:metadata.name)
POD_NAMESPACE: config-server (v1:metadata.namespace)
INSTANCE_IP: (v1:status.podIP)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
HOST_IP: (v1:status.hostIP)
ISTIO_CPU_LIMIT: 2 (limits.cpu)
PROXY_CONFIG: {“proxyMetadata”:{“ISTIO_META_IDLE_TIMEOUT”:“0s”},“gatewayTopology”:{“numTrustedProxies”:2},“holdApplicationUntilProxyStarts”:true}

  ISTIO_META_POD_PORTS:          [
                                     {"name":"http","containerPort":51000,"protocol":"TCP"}
                                 ]
  ISTIO_META_APP_CONTAINERS:     config-server
  GOMEMLIMIT:                    1073741824 (limits.memory)
  GOMAXPROCS:                    2 (limits.cpu)
  ISTIO_META_CLUSTER_ID:         Kubernetes
  ISTIO_META_NODE_NAME:           (v1:spec.nodeName)
  ISTIO_META_INTERCEPTION_MODE:  REDIRECT
  ISTIO_META_WORKLOAD_NAME:      config-server
  ISTIO_META_OWNER:              kubernetes://apis/apps/v1/namespaces/config-server/deployments/config-server
  ISTIO_META_MESH_ID:            cluster.local
  TRUST_DOMAIN:                  cluster.local
  ISTIO_META_IDLE_TIMEOUT:       0s
  ISTIO_KUBE_APP_PROBERS:        {"/app-health/config-server/livez":{"httpGet":{"path":"/config/*/*","port":51000,"scheme":"HTTP"},"timeoutSeconds":5},"/app-health/config-server/readyz":{"httpGet":{"path":"/config/*/*","port":51000,"scheme":"HTTP"},"timeoutSeconds":5},"/app-health/config-server/startupz":{"httpGet":{"path":"/config/*/*","port":51000,"scheme":"HTTP"},"timeoutSeconds":5}}
Mounts:
  /etc/istio/pod from istio-podinfo (rw)
  /etc/istio/proxy from istio-envoy (rw)
  /var/lib/istio/data from istio-data (rw)
  /var/run/secrets/credential-uds from credential-socket (rw)
  /var/run/secrets/istio from istiod-ca-cert (rw)
  /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bl8rj (ro)
  /var/run/secrets/tokens from istio-token (rw)
  /var/run/secrets/workload-spiffe-credentials from workload-certs (rw)
  /var/run/secrets/workload-spiffe-uds from workload-socket (rw)

Warning Unhealthy 35m (x17208 over 5d19h) kubelet Startup probe failed: HTTP probe failed with statuscode: 500
Normal Pulled 30m (x1514 over 5d19h) kubelet Container image “mosipid/config-server:1.1.2” already present on machine
Warning Unhealthy 10s (x27045 over 5d19h) kubelet Startup probe failed: HTTP probe failed with statuscode: 404

Thanks
Naveen

syed.salman · March 26, 2024, 7:06am

Hello @reddi3,

Could you kindly furnish the complete logs of the configuration server and compress them using 7z to alleviate the size of the log file?

Please utilize the following kubectl command to retrieve the previous pod logs of the config-server:

kubectl -n config-server logs <pod-name> --previous

Furthermore, it seems that I encountered difficulties accessing your mosip-config repository. If it is configured as private, could you please supply the username and token required for accessing the private repository? Additionally, if the repository has not been forked, kindly proceed to fork it from the mosip GitHub account.

Thank you.

reddi3 · March 28, 2024, 2:33pm

Hi @syed.salman ,

Thanks for the response. These are the only allowed formats for the attachments.Kindly advise.

Thanks
Naveen

syed.salman · April 1, 2024, 5:13am

Hi @sanchi-singh24

Please consult the MOSIP Admin team regarding the authorization for uploading files with .zip, .7z, and .txt extensions.

dhanendra06 · April 2, 2024, 4:34am

Hi @syed.salman,

Authorization for uploading files in admin are:

bulkupload/masterdata: Master data upload supported format are csv, xls and xlsx files.
bulkupload/packet: packet upload supported format is only zip file.

reddi3 · April 3, 2024, 8:03am

Hi @syed.salman ,

I have already shared you the username and token along with my git repo details in the private chat. Hope you received it and let me know if you didn’t. FYI, the service is not coming up even with the public repo. So, basically from what I could see it is only working with the MOSIP git repo. Kindly check on this and revert back as this seems to be blocker for us since many days. Waiting for your response.

Thanks
Naveen

reddi3 · April 16, 2024, 2:24pm

Hi @syed.salman ,

Any update regarding the reported issue. We are waiting for your response.

Thanks
Naveen

sanchi-singh24 · April 23, 2024, 5:21am

Hi @reddi3

Apologies for the delayed response let me check with @syed.salman internally if this was resolved and what are the next steps to be taken.

Best Regards,
Team MOSIP

Topic		Replies	Views
How to create a private repo and use it on config-server	7	268	October 19, 2023
Config Module Installation Support	24	295	July 22, 2024
After Changing Private Repository of Config Server, Deployment Failed Pod Stuck at 1/2 Stage Support	12	144	July 8, 2024
How to properly change the repo on config-server Support	8	275	November 7, 2023
Prereg-BatchJob Deployment Support	4	30	August 19, 2024

Config Module Deployment

Related topics