Credential event store & identity_cache empty enrolment

Hi Mosip team
I hope you are doing well.
After doing an enrolment that reach the last step of print service, the tables credential event store & identity_cache in IDA schema are empty, so we cannot authenticate an identity, each time we got uin not found.
Fo example: create a VID from UIN.

BR

@zeddari what version of Mosip are you using? is IDA service up and running? Also do you see no id at all in IDA db?

Whats the status of the credential request generator? do you see any credentials in the request generator?

Hi
below our answers:

  • Mosip version is: 1.2.0.1 release
  • IDA servies are up and running
  • no Id at all in IDA db (IDENTITY_CACHE table is empty)
  • credential request generator is up and running
  • we have credentials in the request generator

Hi @zeddari

You are currently in a place where the issue can be in credential_transaction, ida or websub. Please answer the following questions to help us analyse the issue better,

  • A set of records are created in the credential_transaction table when an ID is issued (top 3 records). What is the status of these records?

    • NEW - then you need to wait for the record to process or check the batch job in credential-request-generator as it might not be running.
    • ERROR or FAILED - there is some issue in credential processing. Please share the logs for credential-service
    • ISSUED - the record is sent successfully sent to WebSub, hence, we need to further check ida.credential_event_store
    • STORED - the records are processed successfully
  • Do you see records in ida.credential_event_store table?

    • No - the issue is in subscription to websub or in websub it self
    • Yes - we should check the status_code in ida.credential_event_store table
      - FAILED or ERROR - Share the Failure status and logs of ida-internal-service
      - STORED - The record is successfully stored in IDA, please check the identity_cache

Hi @nayakrounak
In the credential_transaction, the record status is Failed.
The credential_service has this error: No policy available for given partner: movp & auth

{“log”:“{"@timestamp":"2023-04-10T13:18:55.231Z","@version":"1","message":"service-account-mosip-crereq-client - REQUEST_ID - bf8cd47c-6cb9-4d2e-a106-9d82866cd18c - No policy available for given partner","logger_name":"io.mosip.credentialstore.util.DataShareUtil","thread_name":"http-nio-8095-exec-15","level":"ERROR","level_value":40000,"appName":"credential-service,id-repository,application","traceId":"97b2eb7b69c281ae","spanExportable":"false","req.requestURI":"/v1/credentialservice/issue","X-Span-Export":"false","req.method":"POST","req.userAgent":"Apache-HttpClient/4.5.6 (Java/11.0.16)","spanId":"97b2eb7b69c281ae","X-B3-SpanId":"97b2eb7b69c281ae","X-B3-TraceId":"97b2eb7b69c281ae","req.remoteHost":"192.168.64.1","req.requestURL":"http://10.87.105.140:8075/v1/credentialservice/issue\“}\n”,“stream”:“stdout”,“time”:“2023-04-10T13:18:55.2335597Z”}
{“log”:”{"@timestamp":"2023-04-10T13:18:55.232Z","@version":"1","message":"service-account-mosip-crereq-client - REQUEST_ID - bf8cd47c-6cb9-4d2e-a106-9d82866cd18c - io.mosip.credentialstore.exception.DataShareException: IDR-CRS-011 –\u003e Datashare response is null\n\tat io.mosip.credentialstore.util.DataShareUtil.getDataShare(DataShareUtil.java:121)\n\tat io.mosip.credentialstore.util.DataShareUtil$$FastClassBySpringCGLIB$$77084ded.invoke(\u003cgenerated\u003e)\n\tat org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)\n\tat org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)\n\tat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)\n\tat org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)\n\tat org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)\n\tat org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164)\n\tat org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)\n\tat org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:152)\n\tat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)\n\tat org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)\n\tat io.mosip.credentialstore.util.DataShareUtil$$EnhancerBySpringCGLIB$$4158db56.getDataShare(\u003cgenerated\u003e)\n\tat io.mosip.credentialstore.service.impl.CredentialStoreServiceImpl.createCredentialIssuance(CredentialStoreServiceImpl.java:195)\n\tat io.mosip.credentialstore.controller.CredentialStoreController.credentialIssue(CredentialStoreController.java:57)\n\tat io.mosip.credentialstore.controller.CredentialStoreController$$FastClassBySpringCGLIB$$b491f896.invoke(\u003cgenerated\u003e)\n\tat org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)\n\tat org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)\n\tat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)\n\tat org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)\n\tat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)\n\tat org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)\n\tat io.mosip.credentialstore.controller.CredentialStoreController$$EnhancerBySpringCGLIB$$a0c2a105.credentialIssue(\u003cgenerated\u003e)\n\tat jdk.internal.reflect.GeneratedMethodAccessor205.invoke(Unknown Source)\n\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.base/java.lang.reflect.Method.invoke(Method.java:566)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)\n\tat org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Reque

Hi Zeddari,

If you would have used our default installation, our default IDA auth partner is - “mpartner-default-auth” and the policy associated with is “mpolicy-default-auth”. You can find the partner and policy details in the pms-partner.csv and pms-auth_policy.csv files respectively.

The partner and policy is mapping is kept in the partner-policy table in mosip_pms db, the details for the same are available in the pms-partner_policy.csv file.

As you are using a new partner called “movp”, please make sure these details are available in the respective tables and that your partner ID is also configured in the id-authentication-default.properties.

Hi
we are using the default config, the MOVP exist in:
id-repository-default.properties:212:idrepo-dummy-online-verification-partner-id=MOVP

Hi @zeddari ,
Please ignore the failures related to MOVP partner. We need to check why that entry is getting added to credential_transaction but it is not related to credential issuance to IDA. For IDA, we should have another credential request entry for “mpartner-default-auth” parther in credential_transaction table. First thing is we need to check if we have “mpartner-default-auth” as Online_Verification_Partner in PMS. Please check in PMS with below API for the same.

${mosip.pms.partnermanager.url}/v1/partnermanager/partners?partnerType=Online_Verification_Partner

If it is there, then please check for the mpartner-default-auth related entry in credential_transaction table. You can check it by decrypting the “request” column and see the issuer attribute in it. To decrypt it use keymanager’s decrypt endpoint and use App ID: ID_REPO and Reference ID: credential_request.
Also if any failure is observed for that please try re-issuing it by changing its status to NEW in credential_transaction table and observe if any error occurs in the credential-service logs for that. If it is not having any issue you would need to check in ida-internal-service logs.

Hi Again
now we created the requested key alias and we pout the config in the files.
in ida.credential_event_store table, we have the record but with status: FAILED_NON_RECOVERABLE

we have 2 errors:
1 -
io.mosip.kernel.core.crypto.exception.InvalidDataException: KER-FSE-005 → Data must not be longer than 256 bytes;
nested exception is javax.crypto.IllegalBlockSizeException: Data must not be longer than 256 bytes

2 -
data usage has been expired

@zeddari thanks.
First error is the one to check. Second one is due to retry after the first error occurred.

On the decryption error, I doubt if data-share data thumbprint validation is disabled by flag. Normally the encrypted data will have thumbprint + encrypted sessionId + keysplitter + encrypted data.

SessionId only expected to be 256 bytes. If thumbprint is not trimmed it would be treated as part of sessionId and then only the error would occur.

Either the encrypted data from data share should not have thumbprint or in IDA datashare data thumbprint validation should be enabled.

Please check the property once: mosip.ida.datashare.thumbprint-validation-required

Hi
Thanks for your replay
I have only this:
mosip.ida.internal.thumbprint-validation-required=true
it was fakse and I set it to truen but same error
used by: io.mosip.kernel.core.crypto.exception.InvalidDataException: KER-FSE-005 → Data must not be longer than 256 bytes;
nested exception is javax.crypto.IllegalBlockSizeException: Data must not be longer than 256 bytes
at io.mosip.kernel.crypto.jce.core.CryptoCore.doFinal(CryptoCore.java:489)
at io.mosip.kernel.crypto.jce.core.CryptoCore.asymmetricDecrypt(CryptoCore.java:345)
at io.mosip.kernel.crypto.jce.core.CryptoCore.asymmetricDecrypt(CryptoCore.java:307)
at io.mosip.kernel.crypto.jce.core.CryptoCore.asymmetricDecrypt(CryptoCore.java:75)
at io.mosip.kernel.keymanagerservice.helper.SessionKeyDecrytorHelper.decryptWithKeyAlias(SessionKeyDecrytorHelper.java:537)
at io.mosip.kernel.keymanagerservice.helper.SessionKeyDecrytorHelper.decryptSessionKeyNoKeyIdentifier(SessionKeyDecrytorHelper.java:478)
at io.mosip.kernel.keymanagerservice.helper.SessionKeyDecrytorHelper.decryptSymmetricKeyNoKeyIdentifier(SessionKeyDecrytorHelper.java:415)
at io.mosip.kernel.keymanagerservice.helper.SessionKeyDecrytorHelper.decryptSessionKey(SessionKeyDecrytorHelper.java:117)
at io.mosip.kernel.keymanagerservice.service.impl.KeymanagerServiceImpl.decryptSymmetricKey(KeymanagerServiceImpl.java:375)

Is ida-internal-service restarted after the config change?

Yes , sure
same issue
The last records was Failed

@zeddari Is it possible to share the complete log with this exception.

Thanks @zeddari . Can you please check if below property set somewhere either in application-xyz.properties or in id-authentication-xyz.properties. Either it should should be false or that property should not present. It should not be set to true.

mosip.kernel.keymanager.113nothumbprint.support=false

Hi
Thank you for your replay.
I found it only in sync-data:

Hi @zeddari , Please try adding below property in id-authentication-default.properties file and restart ida-internal-service, and then re-issue the credentials, and see if that works.

mosip.kernel.keymanager.113nothumbprint.support=false

Hi
Same issue, data must not be longer than 256 bit
The exception is in this line:

The message is: Error occurred because of mismatch with keys. Try with other current key for decryption.

Hi Zeddari,

For the IDA version 1.2.0.1, the execution should not enter into this block of code.

Can you please share this decompiled code and the logs after you are setting the above flag to false.

Thanks