Just In Time Migration

📘

Terminology

local IDP - Next Identity IDP
home IDP - external IDP (ex. third party IDP such as salesforce or a homegrown IDP)
externalSystemsMapping - object in NI record that stores home user ID and IDP and marks user as migrated

Introduction

Just In Time (JIT) migrations take an incremental approach to migrating data. Rather than aim to complete the whole event in a short time window, a Just In Time migration (also sometimes referred to as trickle migration or lazy migration) involves running the old and new systems in parallel and migrating the data as the consumer authenticates. This approach can be completed either by allowing a home IDP to interact with the local IDP via API, or by providing connection to the home IDP through a managed interface.

JIT migration also creates a flag in the user record to indicate where the home IDP of the consumer is - amongst other use cases. One of the benefits of JIT migration is that the home IDP password hash doesn’t have to be supported since the user’s password can be validated with the home IDP and stored in the local IDP using the native hash. There are various scenarios that need to be accounted for with JIT migration including a user who has already been migrated and exists in a second home IDP or if a user already exists in the local IDP (merge flow).

JIT migration will be initiated for clients that have it enabled and have a home IDP configured, only one (1) home IDP is supported at a time.

Migration Preference

If an application needs to do a user migration, the following list is the order that we recommend. Note, this is a process, it is not automated or configured below.

  1. Bulk migration*

  2. Password reset

  3. Just In Time Migration (JIT)

  4. Just In Time Migration (JIT) API

*Contact us for more information on how we can migrate your data from legacy systems.

JIT Migration Basic Overview

From an extremely high level, JIT migration will look like this:

  1. The end user visits the website or app and is prompted to sign in with the home IDP credentials.
  2. End user credentials are used to check if the record exists and matches in the home IDP and/or local IDP.
  3. Depending on the results of those two checks, either a new user record will be created in the local IDP, or an existing record in the local IDP will be updated.

When a record exists in both the local IDP and the home IDP, a check will happen to see if the key user attributes are the same, or different, and the user may be prompted to choose a “primary” set of data to be written into the local IDP (also known as a merge workflow). Details of both the Journeys and API flows are below.

External ID Providers Storage

The external systems mapping object is stored and is a place to determine whether the user has been migrated or not, marks where the user was migrated from, and saves the user’s external UUID. This is an object since there can be more than one external provider.

Example

   "external_systems_mapping": {
        "app1_cognito": { //unique id of source
            "name": "App 1", //human friendly name ID source? 
            "user_id": "b12889c5-1080-641f-db98-6e6304a704e1", //user's external ID
            "created": "[TIMESTAMP]"
        }
    },

The key ID field is a unique identifier of the external IDP. It is a human readable unique ID - uniqueness is important since there can be more than 1 external IDP that are the same platform. Suggested format:

appname_idpname

Example:

loyaltybrand_cognito

someapp_salesforce

newssource_cognito (this is a different Cognito than the first example)

Migration Directions

The migration can occur in either direction. An external provider (system) can utilize the JIT API to facilitate the migration from home to local IDP. Conversely, Next Identity supports integrations into external providers in order to seamlessly migration consumers with only a few simple configurations.

Just In Time Migration API

The JIT API allows an API integration to migrate users from an external system into a local system. The API creates a user in the local system and accepts all parameters of the user record along with the password. The endpoint that facilitates the migration is show below:

Auth: bearer token with the jitm_merge scope

/user/v1/jit-migration
HEADER Bearer 11edf3ne9aankdppoe

Request body
{
  "email": "[email protected]",
  "phone_number": "+1234567890",
  "email_verified": true,
  "phone_verified": false,
  "given_name": "FirstName",
  "family_name": "LastName",
  "password": "password123", //plain text password, taken from user input
  "user_metadata": {
     "external_system_id": "123-456-789-101112",
     "home_idp_id": "readable_unique_name_id", //example app1_cognito
     "home_idp_name": "Cognito" // human-friendly name of source
  },
  "code": "abcd123" //optional based on merge flow
  "overwrite": true | false //optional, if present requires code param
}

Response body
{
  "uuid": "50ea23eb-30d2-4c0b-b76a-ed72b7eb34e0",
    "message": "User has been migrated"
}

📘

Note on values validation

Note that all values above are validated based on regex for input fields.

If the code (auth code) is not present and an account already exists in local system, merge criteria will begin as described below. Note that some scenarios result in the code being required for this call.

JIT Logical Flows

Scenario 1: Migration from external system to local IDP

Entrance criteria: JIT migration enabled for the property. User is logging into external identity provider screen and external system is checking whether the user exists in the home IDP to act accordingly.

Start: External system calls /user/v1/jit-migration endpoint

  • if user exists in local system
    • if user info and password match*, externalSystemMapping is set [end]
    • if user info matches and password does not match
      • if client configuration is set to automated merge the API will respond with user account already exists and user can use their existing account to login. externalSystemsMapping is set [end]
      • if client configuration is set to user driven merge, the API will respond that an account already exists. Once the /user/v1/jit-migration is called again with overwrite = true, and an auth code the externalSystemsMapping is set [end]
    • if email matches but first name or last name do not match (both must match)
      • if client configuration is set to automated merge, API responds that user can use their existing account to login. externalSystemsMapping is set [end]
      • if client configuration is set to user driven merge, external system (user) chooses whether local IDP or home IDP will be primary.
        • If local IDP, /user/v1/jit-migration is called with code and overwrite = false. The externalSystemsMapping is set. [end]
        • If home IDP, /user/v1/jit-migration is called with code and overwrite = true. The user record is overwritten with details from the API call and the externalSystemsMapping is set. [end]
  • if user does not exist in local system, user record is created with details from the API call and the externalSystemsMapping is set. [end]

📘

Note on matched user record

When the user is compared, the first name and last name are compared and potentially "matched". To match, the name must match exactly including case and special characters.

Scenario 2: Migration from external system to local IDP where the external system uses Next Identity Journeys.

Entrance criteria: Client has home IDP realm configured and JIT enabled. User is logging into hosted login screen.

Start: User lookup in local IDP

  • if user exists in local IDP, externalSystemsMapping is checked
    • if externalSystemsMapping exists and indicates user merge has occurred, credentials are verified only in local IDP [end]
    • if externalSystemsMapping doesn't exist, home IDP is checked
      • if user exists in home IDP, credentials are verified against home IDP, credentials and user details are compared with local IDP
        • if password and user details match (email, firstname, and lastname all must match) in local IDP, externalSystemsMapping is set, end-user experience is not impacted in this use case and sign in appears seamless [end]
        • if user details match (email, firstname, and lastname all must match), but password does not match
          • if client configuration is set to automated merge, the user is shown a screen that they can use their local IDP to login, externalSystemsMapping is set [end]
          • if client configuration is set to user driven merge, user gets a message: "You have an existing account with . Please enter those credentials to make it your primary account." Then the user is prompted to enter their local IDP credentials. Upon validation, externalSystemsMapping is set [end]
        • if email matches but first or last name do not match (both must match), user is prompted to choose an account as primary
          • if user chooses local IDP as default
            • If user credentials did not match, user is prompted to enter local IDP credentials, credentials are verified, credentials from home IDP are not migrated and user details are not migrated to local IDP and externalSystemsMapping is set [end]
            • If user credentials did match, credentials from home IDP are not migrated and user details are not migrated to local IDP and externalSystemsMapping is set. [end]
          • if user chooses home IDP as default, user credentials and user info is migrated to local IDP and externalSystemsMapping is set [end]
        • if user credentials fail against home IDP (either password is incorrect or they do not have an account - local IDP will not know the difference), user is credentials are checked against local IDP [end]
      • if user doesn't exist in local IDP, home IDP is checked
        • if user credentials are checked with home IDP, and is successful, the user is migrated and externalSystemsMapping is set [end]
        • if user credentials are checked against home IDP, and fails, the user is shown usr/pw error (since they don't exist anywhere and need to register) [end]

Exit criteria assumptions:

  • If user exists in a second home IDP and has already been migrated, the system will not check or migrate the user from the second home IDP.
  • If the user is being migrated and does not have a verified account, they will be required to verify it before they can authenticate.
  • If a user who is being migrated already has an existing entry for the home IDP, the migration will not continue and the failed event will get clearly logged.

Supported External Identity Providers

External Systems API

Users who are logging into a new application and have not granted access will be prompted to accept. Once the consumer is authenticated, the application needs to be able to map the user to an external system Id. This may happen automatically based on whether the client has JIT configured. If the client doesn’t have JIT configured, the app needs a way to set this value manually. Therefore they need an endpoint to update the externalSystemsMapping array in the NI user record.

  • The API will accept bearer token (from token delegation)
  • The API will have a GET, POST, and DELETE function to get the list of external ids, create a new one, and delete one.
  • The API will only be able to access the external system that they have access to so this will be handled by checking the client setting to get the external system id and then checking the user record and only allowing access to that entry.
  • If the POST is being called and the system id already exists, the API will respond with failed to update a duplicate entry response. This should be clearly logged to be able to access later if needed.

See the External System API reference HERE