Migrating Outcomes Data to Custom Fields in WebAdMIT

Overview

If you'd like to transfer data from Liaison Outcomes to WebAdMIT, you can do so using the Outcomes to WebAdMIT data migration process. This process uses a Python script to move data from Outcomes to custom fields in WebAdMIT.

This works by following these steps:

1. Outcomes exports a CSV file to an SFTP location.
2. The Python script picks up the CSV file and uses the WebAdMIT API to load the data into custom fields in WebAdMIT.
3. The script archives the file for data retention and auditing purposes.

Prerequisites

To set up this migration process, you'll need to have the following in place:

• A Windows server to host the Python script.  Since Outcomes exports data to an SFTP location, this Windows server can be collocated with the SFTP server. To set up the server, you'll need to:
  1. Establish an account with admin rights on the server.
  2. Install Notepad++ on the server to edit the Python script.
  3. Install Python on the server that hosts the script.
  4. Optional: If the SFTP server and the Python script are on two different servers, you'll need a script or batch file to move the data from the SFTP to the script server.
• The fields needed for the export in Outcomes should be configured in prelaunch and production.
• WebAdMIT's custom fields should also be built in prelaunch and production.

Creating custom fields in WebAdMIT

Identify the fields you want to migrate from Outcomes to WebAdMIT. Then, create custom fields in WebAdMIT that will be the destination for this data.

WebAdMIT User Identities

To access the WebAdMIT API, you need an Institution Manager in WebAdMIT that can access data for all departments and programs at the university. You should have an Institution Manager account in the prelaunch and production environments. Once that is in place, complete the following steps in both environments:

1. Log in to WebAdMIT with the Institution Manager account.
2. Click Account on the top right to view Account Details.
3. Click Edit My Account on the top right.
4. Verify that your account has the correct organization assignments and program assignments.
5. In the API Key section, click Generate New Key to generate a unique API key for your account.
6. Click Show Key to view your API Key.
7. Make note of the API Key, which will be required for the script and retrieving your identity ID.

Retrieving Your User Identity ID

1. Make the following API call to retrieve your User Identity ID.
2. Use CURL or POSTMAN to make the below API calls:

• PRODUCTION

$ curl -n https://api.webadmit.org/api/v1/user_identities \
  -H "x-api-key: <ENTER YOUR API KEY>"

• PRELAUNCH

$ curl -n https://prelaunch.webadmit.org/api/v1/user_identities \
-H "x-api-key: <ENTER YOUR API KEY>"

3. Make sure to use the correct API Keys for PRELAUNCH and PRODUCTION.
4. A successful API call should generate a status code of “200” with the following example response body:           

{
"href": "/api/v1/user_identities",
"user_identities": [
{
"id": 42,
"type": "Admissions User",
"association": "SOPHAS",
"institution": "Demo University",
"organization": "Demo University School of Public Health",
"cycle": "2014 - 2015"
}
]
}

5. Make note of the number after "id":. Corresponding to the cycle for which you want your applicant custom fields updated, this is the identity ID and is needed for prelaunch and production. This “id” is needed to retrieve the order of the custom fields in the next section.
6. If the API call is unsuccessful, then it will return a status code of 401. This means the user is not authorized to gain access to the API. Verify the user’s permissions in WebAdMIT and ensure the user has the correct organization and program assignments.

Retrieving the Order of custom fields in WebAdMIT

1. Make the following API calls to retrieve the order of the custom fields in WebAdMIT.  The examples below are for production WebAdMIT. To use the below code for prelaunch, please use the following base URL: https://prelaunch.webadmit.org
2. Retrieve the Program ID associated with the User Identity ID by making the following call:

$ curl -n https://api.webadmit.org/api/v1/user_identities/:<ENTER USER_IDENTITY_ID>/programs/
 -H "x-api-key: <ENTER YOUR API KEY>" 

3. Then, make the following API call to retrieve the Custom Field list:

$ curl -n https://api.webadmit.org/api/v1/user_identities/:<ENTER USER_IDENTITY_ID>/programs/:<ENTER PROGRAM_ID>/custom_fields \
-H "x-api-key: <ENTER YOUR API KEY>"  

4. A successful API call should generate a status code of “200” with the following example response body:           

{
"href": "/api/v1/user_identities/1/programs/42023191739237/custom_fields",
  "custom_fields": [
    {
      "id": 4,
      "label": "Preferred house",
      "field_type": "select",
      "options": [
        "Gryffindor",
        "Hufflepuff",
        "Ravenclaw",
        "Slytherin"
      ]
    }
  ]
}

5. Note all the labels as they are listed one after the other. You will need the list of labels in this exact order to build the export in Outcomes.

Configuring the Outcomes Export

To move any data points from Outcomes to WebAdMIT via SFTP, you must configure an export in Outcomes. Before setting up the export in Outcomes, it is important to have the list of labels for the custom fields in WebAdMIT in the configured order, as shown previously. Then:

1. Create an export in CSV format as outlined in the article Working with Application Exports. The script to move data from Outcomes to WebAdMIT expects a CSV file as an input.
2. Create the name of the CSV file without a date parameter, as that is something the script will handle as part of the archiving process.
3. The CSV file should contain column headers.
4. WebAdMIT custom fields are data type sensitive, so it is important to ensure:
   • Numeric fields in Outcomes are mapped to numeric custom fields in WebAdMIT.
   • Boolean fields in Outcomes are mapped to Boolean fields in WebAdMIT.
   • Multi-select fields in Ouctomes can be mapped to string fields or a select list custom field in WebAdMIT. If using select field in WebAdMIT, ensure the drop-down values match all possibilities that are derived from Outcomes.
   • String fields in Outcomes are mapped to string custom fields in WebAdMIT.
5. The script  expects the export fields to be in this order:
   • CASID
   • Program Unique Identifier String
   • Data fields (in the same order as WebAdMIT API returns for custom fields created in WebAdMIT).
6. Reference the article Working with Export Destinations to configure the export to send the data to an SFTP server.

Python Installation

You'll need to install Python on the server that will be hosting the script. To do so:

1. Download the software from https://www.python.org/downloads/.
2. Download the latest release for Windows (based on whether the server is a32 bit or 64 bit).
3. Once downloaded, run the Python install file as an administrator.
4. This opens an installation window:

6. Click Customize Installation to display  Optional Features:

7. Ensure that pip is selected.
8. The next page displays advanced installation options:

9.  Select the path to install Python and click Install.
10. Once Python is successfully installed, restart the server.
11. Once restarted, navigate to your system environment variables to ensure Python is appended to the end of the PATH variable. If it is not, you will need to add it. The path should be the path to the Python executable, such as C:\Users\username\AppData\Local\Programs\Python\Python310

12. To test that your Python installation was successful and added successfully to your PATH variables, run Command Prompt as an administrator, type python, and hit enter:

1. If the response is similar to “Python is not recognized as an internal or external command”, then it could mean that the PATH variable is not correctly set up or your Python was not correctly installed.

Cycle Over Cycle Changes

1. Cycle over cycle, review your custom fields in WebAdMIT and make note of any new custom fields that need to be added.
2. If new custom fields are added, make sure you follow the steps in Retrieving the Order Custom Fields in WebAdMIT and configure your Outcomes export to match accordingly.
3. If a new export is created in Outcomes each cycle, then make sure the new export is configured to deliver files to SFTP and is named the same as the previous one. If the file name needs to change, make note of the new filename.
4. Stop the OutcomesToWebAdMIT service.
5. Edit the config.json file in your OutcomesToWebAdMIT and \OutcomesToWebAdMIT\dist folders.
6. Update the filename if you plan to use a new filename for the new cycle in the config.json file.
7. Update the cycle to correspond with the new cycle in the config.json file.
8. If you plan to use a completely new WebAdMIT user for the new cycle, change the API Key and cycle in the config.json file.
9. Save the config.json file in the OutcomesToWebAdMIT and \OutcomesToWebAdMIT\dist folders with the changes.
10. Start the OutcomesToWebAdMIT service.

Special Considerations

1. If you have multiple CASs that require custom fields updated in WebAdMIT, then you will need one version of the script and config.json file per CAS.
2. If you have overlapping cycles for your CAS, then you will need two versions of this process per CAS cycles that are overlapping. Make sure to stop and delete the service for the previous cycle once it ends.