Need Clarification on NULL/Empty profile Entries in idrepo.anonymous_profile

We observed records in the idrepo.anonymous_profile table where the profile column contains NULL values.

Could anyone clarify the scenarios or conditions under which the profile field is expected to be NULL? We are analyzing this behavior as part of our project and would like to understand whether these NULL values are expected or indicate a specific processing state.

Sample record as below from the table idrepo.anonymous_profile

“8685846a-1d95-536b-935b-95e7cc4f6220” “{”“processName”“:”“New”“,”“date”“:”“2025-04-30"”,““oldProfile””:null,““newProfile””:null}" “service-account-mosip-regproc-client” “2025-04-30 19:24:56.450633” false

Hi Sowmya_Goudar

Thank you for reaching out.

Our team will look into this and get back to you with an update shortly.

Regards,
Mrudula
on behalf of Team MOSIP

Hi,

A couple of points to clarify first, as “NULL profile” can refer to two different scenarios:

  1. The profile column itself is NULL in the idrepo.anonymous_profile table.

  2. The profile column contains valid JSON, but the oldProfile and/or newProfile fields inside the JSON are null.

Based on the sample you shared:

{
  "processName": "New",
  "date": "2025-04-30",
  "oldProfile": null,
  "newProfile": null
}

Expected Behavior

  • oldProfile: null is expected for a New registration, since there is no existing identity to compare against.

  • However, newProfile: null for a New registration is not expected. Under normal processing, the new identity data should be available and a profile should be generated.

The fact that processName is populated as "New" indicates that the anonymous profile generation flow was triggered, but the identity data required to build the profile was not available when the profile was constructed.

Possible Cause

We are aware of a couple of issues in this area that can result in newProfile being stored as null under certain conditions, particularly during concurrent processing and asynchronous profile generation.

Relevant tickets:

These issues are already identified in the MOSIP codebase and fixes are planned for an upcoming release.

Information Required for Further Analysis

To determine whether your case is related to the known issues, could you please share the following:

  • ID Repository Identity Service logs around 2025-04-30 19:24:56 (the record creation timestamp).

  • Any WARN or ERROR messages from:

    • AnonymousProfileHelper

    • IdentityIssuanceProfileBuilder

  • The MOSIP version currently deployed.

  • Whether the environment was processing a high volume of registrations/UIN generations at that time.

  • Whether the affected records are associated with retries or packet reprocessing.

  • The corresponding record from the idrepo.uin table for the same registration, specifically whether valid uin_data exists.

This information will help confirm whether the behavior is caused by one of the known issues or by an environment-specific condition.

Regards,
Kamesh
on behalf of Team MOSIP

Please find below points for the clarification,

  1. ID Repository Identity Service logs around 2025-04-30 19:24:56 (the record creation timestamp). -We cant get same dated logs
  2. Any WARN or ERROR messages from:There are no error or warn message seen for the reg id with this issues.
  3. The MOSIP version currently deployed for Idrepo modules -1.2.1.0 and for other modules is 1.2.0.1
  4. Whether the environment was processing a high volume of registrations/UIN generations at that time. - Environment was processing high volume of registration like 3lakh processed packets per day.Now packet processing count is less around 25k per day also observing the issue.
  5. Whether the affected records are associated with retries or packet reprocessing. -There is no scenario observed like this.
  6. The corresponding record from the idrepo.uin table for the same registration, specifically whether valid uin_data exists. -Valid data exists and packets processed without any issues Overall we didnt observe any pattern for registration ids for this null values.

Hi Sowmya,

Thank you for the detailed responses — they actually help confirm the diagnosis.

We have now root-caused this under MOSIP-45189, and your records match the known issue exactly. This is a confirmed bug.

Root cause — race condition on a shared singleton

AnonymousProfileHelper is a Spring singleton (@Component) that holds per-request state (newUinData, oldUinData, regId, cbeff references, etc.) as mutable instance fields. Its buildAndsaveProfile() method is @Async, so it runs on a separate thread after the main request thread has returned.

When two registrations are processed close together, this sequence occurs:

  1. Request A sets newUinData on the singleton and submits the async profile build.
  2. Request B arrives, detects a different regId, and calls resetData() — which wipes A’s newUinData to null.
  3. A’s async thread then runs, reads newUinData = null, and builds newProfile = null (while processName stays "New", since it is derived from oldUinData being null).
  4. The record is saved as {"processName":"New","oldProfile":null,"newProfile":null}.

This is not an exception in the builder — it is silent state corruption from concurrent requests sharing one instance. It has been reproduced with a unit test that simulates this exact thread interleaving.

Why this is consistent with everything you observed

  • No WARN/ERROR for the reg IDs — expected, since nothing throws; the profile is simply built from null state.
  • Valid uin_data exists and packets processed cleanly — also expected. The main flow persists the identity correctly; only the asynchronous anonymous-profile snapshot reads the stale/null state.
  • No retry or reprocessing correlation — correct, this is a timing collision between concurrent requests, not a reprocessing artifact.
  • Still seen at ~25k/day — the race can trigger whenever any two requests overlap on the singleton, so it is not limited to peak load. Higher volume simply makes it more frequent; lower volume makes it rarer, not impossible.
  • No discernible pattern across reg IDs — expected, since occurrence depends purely on request timing, not on the data itself.

Impact

Please note these records are confined to the idrepo.anonymous_profile table, which by design holds anonymized snapshots for reporting/analytics purposes only (per the table’s own schema definition). It is written by ID Repository but never read back for identity resolution, authentication, or any functional flow. The affected individuals’ actual identity data in idrepo.uin is fully intact and unaffected — only the anonymized reporting snapshot for these registrations is incomplete.

Suggested workaround (until the fix is released)

Since the source identity data in idrepo.uin is intact, the affected reporting rows are recoverable. We suggest the following:

  1. Identify the affected records:

    SELECT id, cr_dtimes
    FROM idrepo.anonymous_profile
    WHERE profile IS NULL
       OR (profile::jsonb ->> 'newProfile') IS NULL;
    
    
  2. Backfill from idrepo.uin: the anonymous_profile.id is deterministically derived from the registration ID, and idrepo.uin retains the corresponding uin_data. A one-off remediation job can regenerate the anonymized profile for these rows from the existing uin_data, restoring the reporting snapshot without affecting the live flow.

As an interim measure to reduce the frequency of new occurrences, the anonymous-profile async executor pool can be tuned so the async build completes sooner relative to incoming requests. Please note this only narrows the timing window — it reduces, but does not eliminate, the occurrence. The complete resolution is the code fix tracked under MOSIP-45189.

Fix

The fix is being tracked under MOSIP-45189 and is targeted for an upcoming ID Repository release. We will update this thread once the release details are finalized.

Regards, Kamesh on behalf of Team MOSIP