Skip to main content
Liaison

How to Create Summed Scores for Application Reviews

Overview

Some programs sum the scores given by reviewers on evaluation forms. This can be managed within the Program Portal using the Phase Calculated Fields and Application Properties features.

Workflow

The steps in this workflow use the following example scenario: 

Suppose each reviewer rates an application based on 3 criteria using a scale of 1–5. An additional bonus point of 1 is also added for applicants who are in-state applicants. These items are then summed to calculate a final score. 

Step 1: Create a Scoring Model Table

We strongly recommend outlining your model in a table that includes the criteria and score range. As you move through the process, you'll add key information to this table, allowing you to quickly reference items as needed.

Criteria

Score Range

Academic Credentials

0–5

Research Experience

0–5

Clinical Skills

0–5

In-state Applicant (Bonus Point)

1

Step 2: Create Your Evaluation Form

Create an evaluation form that includes:

  1. Scale Group with 3 questions, each corresponding to a value in the scoring model. 

    ResidencyCAS Program Portal – Summed Scores 1.png
     
  2. A Number question for the bonus point. 

    ResidencyCAS Program Portal – Summed Scores 2.png

Step 3: Link the Evaluation Form to the Review Phase

  1. Once the evaluation form is created, link it to the phase during which the review will be conducted. 

    ResidencyCAS Program Portal – Summed Scores 3.png
    • While editing the phase, make note of the Key – you'll need this later. In this example, the key is "primaryApplicationReview" – we'll add this to the bottom of our table for easy access later. 
       

      Criteria

      Score Range

      Academic Credentials

      0–5

      Research Experience

      0–5

      Clinical Skills

      0–5

      In-state Applicant (Bonus Point)

      1

      Phase Key: primaryApplicationReview

       
    • Also while editing the phase, create Phase Calculated Fields for the ratings and bonus points. 

      ResidencyCAS Program Portal – Summed Scores 4.1.png

      In this example, note that we created two calculated fields, yet we have three ratings and one bonus point on our form. This is because we used a Scale Group question to capture ratings for the first three items (academic credentials, research experience, clinical skills). When you use a scale group in your form, the system recognizes that a table of ratings exists. As a result, in addition to recognizing each question on the form, it also provides two options for handling all data within that grouping: sum and average. Therefore, we can simply select the sum option instead of creating individual calculated fields for academic credentials, research experience, and clinical skills.

      ResidencyCAS Program Portal – Summed Scores 5.png

      When creating  phase calculated fields, it's also important to decide how you will handle multiple reviews for the same applicant – whether to average the scores for each question, find the median, or sum them. In other words, you're helping the system understand what to do with the scores for each question. In our example, we'll average the scores. This ensures that when there are multiple forms completed by multiple reviewers, we'll see the average results. 
       
  2. Update your table to include the phase calculated field keys for the criteria (you may need to manually generate the keys). 
     

    Criteria

    Score Range

    Phase Calculated Field Key

    Academic Credentials

    0–5

    avgRatings 

    (This includes Academic Credentials + Research Experience + Clinical Skills.)

    Research Experience

    0–5

     

    Clinical Skills

    0–5

     

    In-state Applicant (Bonus Point)

    1

    avgBonus
    Phase Key: primaryApplicationReview

     
  3. After linking the evaluation form to the phase, move one of your applicants into that phase and assign them to yourself. Then, complete a review of that applicant and submit it. This will allow you to test the form later.

Step 4: Create an Application Property for the Final Score

Next, you'll create one application property for the final score for each applicant. Application properties control various aspects of the system. Here, we're creating a property that allow the system to sum the calculated fields we created above.  

  1. Start by navigating to Settings, then Application Setup, then Application Properties. Click the pencil icon in the upper-right corner to edit.
    1. Click Calculated Property in the left menu. 

      ResidencyCAS Program Portal – Summed Scores 6.png
    2. Enter a label for the calculated field. This field represents the score for the criteria, so we'll use “Overall Application Review Score.” Then, set the data type as Number.

      In the Expression section, enter the formula that calculates the weight for the criteria. That formula is:

      phases.primaryApplicationReview.avgRatings + phases.primaryApplicationReview.avgBonus

      Here's how each part of the formula is composed:
       
      •  phases. = the type of data you're using for the calculation, which in this case is phase calculated fields (be sure to include the ending period). 
      •  primaryApplicationReview= the specific phase with which your phase calculated field is associated. You had identified this key in step 3 and added it to the bottom of your table (be sure to include the ending period). 
      • avgRatings and avgBonus = the specific phase calculated field keys for the criteria (do not add anything to the end).

        ResidencyCAS Program Portal – Summed Scores 7.png
        Tips: 
        – If you can't remember your key, you can use the Data Dictionary to locate it. Use the Find function in your browser to quickly search for it in the list of fields.
        – If you put a test applicant through the review process (step 3) , you can use the Try It function to see if your formula works. Search for and select the applicant to view the output. 

        ResidencyCAS Program Portal – Summed Scores 8.png
        ResidencyCAS Program Portal – Summed Scores 9.png
    3. Click Add to add the field to the system.​​​​​​
    4. Click Save in the upper right corner to save your new field.

Now you have a total review score composed of the sum of all evaluation form questions (remember, each question's score is averaged when multiple reviewers complete the form). You can use this score just like any other piece of application data by adding it to the application grid, adding it to the application summary view, creating a segment with it, or exporting it.

 

  • Was this article helpful?