Migrating TargetX Data to Custom Fields in WebAdMIT

Overview

Use the TargetX to WebAdMIT data migration process to transfer data from TargetX into custom fields in WebAdMIT. 

This automated process works as follows:

1. TargetX exports a CSV file via Informatica to a designated folder on the Informatica server.
2. A Python script running on the Informatica server picks up the CSV file and uses the WebAdMIT API to load the data into the appropriate custom fields in WebAdMIT.
3. After processing, the script archives the CSV file for data retention and auditing purposes.

Prerequisites

To set up this migration process, make sure the following requirements are in place:

Creating Custom Fields in WebAdMIT

Identify the fields you want to migrate from TargetX to WebAdMIT. Then, create custom fields in WebAdMIT that will be the destination for this data.

$ curl -n https://api.webadmit.org/api/v1/user_identities \
  -H "x-api-key: <ENTER YOUR API KEY>"

$ curl -n https://prelaunch.webadmit.org/api/v1/user_identities \
-H "x-api-key: <ENTER YOUR API KEY>"

{
"href": "/api/v1/user_identities",
"user_identities": [
{
"id": 42,
"type": "Admissions User",
"association": "SOPHAS",
"institution": "Demo University",
"organization": "Demo University School of Public Health",
"cycle": "2014 - 2015"
}
]
}

Retrieving the IDs of Custom Fields in WebAdMIT

$ curl -n https://api.webadmit.org/api/v1/user_identities/:<ENTER USER_IDENTITY_ID>/programs/
-H "x-api-key: <ENTER YOUR API KEY>"

$ curl -n https://api.webadmit.org/api/v1/user_identities/:<ENTER USER_IDENTITY_ID>/programs/:<ENTER PROGRAM_ID>/custom_fields \
-H "x-api-key: <ENTER YOUR API KEY>"

{

"href": "/api/v1/user_identities/1/programs/42023191739237/custom_fields",

  "custom_fields": [
    {
      "id": 4,
      "label": "Preferred house",
      "field_type": "select",
      "options": [
        "Gryffindor",
        "Hufflepuff",
        "Ravenclaw",
        "Slytherin"
      ]
    }
  ]
}

Configuring the TargetX Export

To get data from TargetX to WebAdMIT via Informatica, you need to set up Informatica export tasks.

Here’s a quick walkthrough:

1. Import the attached zip [TargetX-WebAdMIT-CustomFields-Stencils.zip] file into your Informatica.
2. During import, make sure to choose your runtime environment.
3. Set up connections for your flat file dropbox folder and TargetX Salesforce (sandbox or production).

4. Test the import and then click Import.
5. You’ll see 6 jobs created (3 mappings and 3 mapping tasks). 
   
   
   
   
6. Edit the ContactExport mapping:
   
   • Set the source connection to your TargetX sandbox or production. 
     
   • Make sure you bring in only the fields Id and TargetX_Base__CAS_Applicant_ID__c. 
     
   • Add a filter to exclude rows where TargetX_Base__CAS_Applicant_ID__c is null.  
     
   • Set the target connection to your Informatica dropbox folder. 

• • SFContactExport.csv file is a newly created file at runtime.
  • Save the Contact Export Mapping.
  • View the Contact Export Mapping Task and make sure source and target connections look correct.
  • Edit the Application Export.
    • Point the source connection to TargetX(Sandbox or Production). 

• Update the query options so they return only application data for the specific CAS and application cycle you’re working with, matching the filters shown in the image.
• Under Fields, limit the selection to TargetX_SRMb__Contact__c plus any additional fields you need to filter applications using the query options described above.

• Set the unconnected lookup to point to the SFContactExport.csv in your dropbox. 

• Set the target to your flat file dropbox.

• Save your mapping to apply the changes.
• Next, open the Application Export Mapping Task and confirm that the source and target connections are configured correctly.
• Then edit the TargetX–WebAdMIT mapping, and verify that the source connection is set to the appropriate TargetX sandbox or production environment.
• Use the query options to filter Application object data so it is specific to your CAS and application cycle.

• Under Fields, include only TargetX_SRMb__Contact__c, the field that stores the Program ID (for the WebAdMIT API), any fields you need to migrate to WebAdMIT, and any fields required to filter application data (for example, Last Data Input Source, Start Term, Start Year).
• For the unconnected lookup unc_CASID_Lookup, ensure its connection uses your flat‑file connection that points to your Dropbox location.
  
  

• For the expression, add one new field for each WebAdMIT custom field you plan to populate. Map each new field to the corresponding incoming field from your source data; for example, you might create a field named “Answer” and map it to the incoming “Name” field. In this step, you can also transform the incoming TargetX data to match the values expected in WebAdMIT custom fields by using DECODE or IIF statements.

• Change the value of the Cycle field to the WebAdMIT cycle you are integrating with, using the required format "2025 - 2026", including the spaces around the dash.
  
  
  

• Next, click the Target and confirm its connection uses your flat‑file Dropbox connection.

• Under Target fields, rearrange the fields so they appear in this order: CASID, Program Unique Identifier String, and Cycle. You may need to create additional fields, depending on how many WebAdMIT custom fields you plan to populate with TargetX data.
  
  
  
• Under Field Mapping, match each incoming field to its corresponding output field.

• Click Save.
• The Python script expects a CSV file whose headers begin with CAS ID, Program ID, Cycle, and the WebAdMIT custom field IDs.
• Because Informatica column names cannot be numeric, use a header file as a workaround.
• Create a CSV file named header.csv.
• Add the following columns to header.csv: CAS ID, Program ID, Cycle, and the custom field IDs you retrieved in Retrieving the IDs of Custom Fields in WebAdMIT.
• Ensure the custom field IDs appear in the same order as the target fields in the Informatica TargetX–WebAdMIT mapping.
• Save header.csv to the Dropbox location used by your flat‑file connection.
• Edit the TargetX‑WebAdMIT mapping task.
• In the mapping task, go to Run Time Options > Advanced Options and update the post‑processing command so it deletes prior export files, concatenates header.csv with data.csv into a new file (for example, FinalFileName.csv), deletes data.csv, and moves the final file into the Python script directory.
• Use a static file name such as CAS_Export_2025_2026.csv for FinalFileName.csv, without any timestamp, and record this exact name for use in later steps.
• Click Save.
• When testing is complete, and you are ready for production, add the following mapping tasks to your existing linear flow, in this order, and schedule them to run after your current tasks:

1. Contact Export Mapping Task
2. Application Export Mapping Task
3. TargetX‑WebAdMIT Mapping Task

Python Installation

You'll need to install Python on the server that will be hosting the script. To do so:

1. Download the software from https://www.python.org/downloads/.
2. Download the latest release for Windows (based on whether the server is a32 bit or 64 bit).
3. Once downloaded, run the Python install file as an administrator.
4. This opens an installation window:

5. Click Customize Installation to display Optional Features:

6. Ensure that pip is selected.
7. The next page displays advanced installation options:

8. Select the path to install Python and click Install.
9. Once Python is successfully installed, restart the server.
10. Once restarted, navigate to your system environment variables to ensure Python is appended to the end of the PATH variable. If it is not, you will need to add it. The path should be the path to the Python executable, such as C:\Users\username\AppData\Local\Programs\Python\Python314
11. To test that your Python installation was successful and added successfully to your PATH variables, run Command Prompt as an administrator, type python, and click enter:

1. If the response is similar to “Python is not recognized as an internal or external command,” it could mean that the PATH variable is not correctly set up or that your Python was not correctly installed.

Creating the Python Executable

Next, you'll need to create a Python executable that will run the script. To do so:

1. Open Command Prompt as an administrator.
2. Run the following command: pip install pyinstaller
   • If you get an error message that pip is not recognized as an internal or external command, this means your Python installation did not include the pip package.
   • The article How to Install PIP on Windows provides guidance on installing pip.
   • Once installed, type pip –version in Command Prompt to verify if the installation was successful.

   • If pip installation was successful, rerun the pyinstaller command: pip install pyinstaller

3. Run the following command to ensure the requests module gets installed correctly: pip install requests
4. If pyinstaller and requests installation was successful, in Command Prompt, navigate to the TargetXToWebAdMIT directory.
5. Type the following command: pyinstaller --onefile decisions.py and press enter.
6. This command converts the script into an executable.
7. The executable is found in the \TargetXToWebAdMIT\dist\ folder.
8. Copy the config.json file from the TargetXToWebAdMIT folder to \TargetXToWebAdMIT\dist\ folder.

Testing the Executable

The following should be completed for prelaunch Outcomes to prelaunch WebAdMIT:

1. Navigate to the \TargetXToWebAdMIT\dist\ folder in Windows Explorer and double-click decisions.exe to open a running Command Prompt window.
2. If the Command Prompt window closes immediately after opening, then there is an issue with your Python code or the config.json file. In that case, navigate to the TargetXToWebAdMIT\dist folder and check the decisions.log file. If this file shows HTTP connection errors, this could mean that your API key, URL, or Cycle is incorrect.
3. Edit the config.json file to make sure the file name, directory paths, API keys, base URL, and cycle are correct.
4. Once verified, run the executable again.
5. If the executable continues to run, this means the JSON file is correct.
6. Run the Informatica tasks configured for this process to drop a csv file into the TargetXToWebAdMIT folder. The executable will pick up, process, and archive the file.
7. Navigate to the Archive folder under TargetXToWebAdMIT\Archive folder. You will see a timestamped_yourfilename.csv file.
8. Open the file, and you will see HTTP status codes printed in each row.
9. Successful loads would have a status code of 200, while unsuccessful loads would have a status code of 404 or 422.
10. The 404 status code could indicate several issues, including the Decision ID not existing in WebAdMIT, incorrect authentication methods (api_key), or CASID not found in WebAdMIT.
11. If error 422 is thrown, the applicant has already had a local status associated with their record.
12. Note that Decision Codes in WebAdMIT can only be updated via API for applicants that do not already have a Local Status associated with them.
13. If all rows show a 200 for each field, then the next step is to check prelaunch WebAdMIT to ensure the decision data was correctly applied in WebAdMIT.If all looks good, then close the running executable command prompt window and delete the archived file.

Cycle Over Cycle Changes

1. Cycle over cycle, review your custom fields in WebAdMIT and make note of any new custom fields that need to be added.
2. If new custom fields are added, make sure you follow the steps in Retrieving the Ids of Custom Fields in WebAdMIT and configure your Informatica export to match accordingly.
3. Edit the filter conditions for the Application mapping as well as the TargetX-WebAdMIT mapping to match filters required for the new cycle. These would be stored in the query options section for each of the sources for the mentioned mappings.
4. Change the cycle value in the TargetX to WebAdMIT mapping expression:

5. Stop the TargetXToWebAdMIT service.
6. Edit the config.json file in your TargetXToWebAdMIT and \TargetXToWebAdMIT\dist folders.
7. Update the filename if you plan to use a new filename for the new cycle in the config.json file. 
8. If you plan to use a completely new WebAdMIT user for the new cycle, change the API Key in the config.json file.
9. Save the config.json file in the TargetXToWebAdMIT and \TargetXToWebAdMIT\dist folders with the changes.
10. Edit the header.csv file in the dropbox folder and change the Custom Field IDs, to the new cycle custom field IDs and save the file.
11. Start the TargetXToWebAdMIT service.

Special Considerations

1. If you have multiple CASs that require custom fields updated in WebAdMIT, then you will need one version of the script, config.json file and Informatica mapping set per CAS.
2. If your CAS cycles overlap, you will need two versions of this process per CAS cycle. Once the previous cycle ends, make sure to stop, delete the service and stop the linear flow containing the old mapping tasks.