How to Create Weighted and Summed Scores for Application Reviews

Step 1: Create a Scoring Model Table

We strongly recommend outlining your model in a table that includes the criteria, score range, and weight. As you move through the process, you'll add key information to this table, allowing you to quickly reference items as needed.

Step 2: Create Your Evaluation Form

Create an evaluation form that includes a Scale Group with 6 questions, each corresponding to a value in the scoring model. 

Step 3: Link the Evaluation Form to the Review Phase

1. Once the evaluation form is created, link it to the phase during which the review will be conducted. 
   
   • While editing the phase, make note of the Key – you'll need this later. In this example, the key is "weightedEvaluationReview" – we'll add this to the bottom of our table for easy access later. 
      

     Criteria	Score Range	Weight
     Academic Credentials	0–10	2
     Research Experience	0–10	1
     Clinical Skills	0–10	2
     Communication Skills	0–10	1
     Other Work Experience	0–10	1
     Distance Travelled	0–10	3
     Phase Key: weightedEvaluationReview

     
      
   • Also while editing the phase, create Phase Calculated Fields for each of the 6 criteria and take note of the Phase Calculated Field Keys. When creating these phase calculated fields, it's important to decide how you will handle multiple reviews for the same applicant—whether to average the scores for each question, find the median, or sum them. In other words, you're helping the system understand what to do with the scores for each question. In our example, we'll average the scores for each question. 
     
2. Update your table to include the phase calculated field keys for the criteria (you may need to manually generate the keys). 
    

   Criteria	Score Range	Weight	Phase Calculated Field Key
   Academic Credentials	0–10	2	avgAcademicCredentials
   Research Experience	0–10	1	avgResearchExperience
   Clinical Skills	0–10	2	avgClinicalSkills
   Communication Skills	0–10	1	avgCommunicationSkills
   Other Work Experience	0–10	1	avgOtherWorkExperience
   Distance Travelled	0–10	3	avgDistanceTravelled
   Phase Key: weightedEvaluationReview

   
    
3. After linking the evaluation form to the phase, move one of your applicants into that phase and assign them to yourself. Then, complete a review of that applicant and submit it. This will allow you to test the form and use the data later when testing your weighting.

Step 4: Create Application Properties for the Criteria

Next you'll create application properties for your 6 criteria. Application properties control various aspects of the system. Here, we're creating properties that allow the system to calculate the averages for each question.

1. Start by navigating to Settings, then Application Setup, then Application Properties. Click the pencil icon in the upper-right corner to edit, then repeat this process for all 6 criteria: 
   1. Click Calculated Property in the left menu. 
      
   2. Enter a label for the calculated field. This field represents the score for the criteria, so we'll use “Weighted Academic Credentials” for this first entry. Then, set the data type as Number.
      
      
       
   3. Add the application property key to your table – you'll need this for the next step. 
       

      Criteria	Score Range	Weight	Phase Calculated Field Key	Application Property Key for Weighted Criteria
      Academic Credentials	0–10	2	avgAcademicCredentials	WeightedAcademicCredentials
      Research Experience	0–10	1	avgResearchExperience	WeightedResearchExperience
      Clinical Skills	0–10	2	avgClinicalSkills	WeightedClinicalSkills
      Communication Skills	0–10	1	avgCommunicationSkills	WeightedCommunicationSkills
      Other Work Experience	0–10	1	avgOtherWorkExperience	WeightedOtherWorkExperience
      Distance Travelled	0–10	3	avgDistanceTravelled	WeightedDistanceTravelled
      Phase Key: weightedEvaluationReview

      
       
   4. In the Expression section, enter the formula that calculates the weight for the criteria. For our first criteria, academic credentials, that formula is:
      
      phases.weightedEvaluationReview.avgAcademicCredentials*2
      
      Here's how that formula is composed:
       
      •  phases. = the type of data you're using for the calculation, which in this case is phase calculated fields (be sure to include the ending period). Note that this part of the expression will remain the same for each of the next 5 application properties we'll build. 
      •  weightedEvaluationReview. = the specific phase with which your phase calculated field is associated. You had identified this key in step 3 and added it to the bottom of your table (be sure to include the ending period). Note that this part of the expression will remain the same for each of the next 5 application properties we'll build.  
      • avgAcademicCredentials = the specific phase calculated field key for the criteria (do not add anything to the end). 
      • *2 = the weight for the criteria. In this example, the weight for academic credentials is 2. 
        
        
        
        Tips: 
        – If you can't remember your key, you can use the Data Dictionary to locate it. Use the Find function in your browser to quickly search for it in the list of fields.
        – If you put a test applicant through the review process (step 3) , you can use the Try It function to see if your formula works.
         
   5. Click Add to add the field to the system.​​​​​​
   6. Click Save in the upper right corner to save your new field.
   7. Repeat this step for each remaining criteria.

Step 5: Create a Final Application Property to Sum All Values

Next, you'll create one more application property that sums the values of each question score into a total score.

1. Follow the same steps as in step 4 to create the application property. 
2. Give it a name to reflect that it is a total review score. 
3. Set the data type to Number.
4. In the Expression section, enter the formula that sums all scores. In our example, that formula is:
   
   weightedAcademicCredentials+WeightedResearchExperience+WeightedClinicalSkills+WeightedCommunicationSkills+WeightedOtherWorkExperience+WeightedDistanceTravelled
   
   This formula is composed of each application property key for the weighted scores (be sure to include a + between each key).
   
   Tip: if you put a test applicant through the review process (step 3) , you can use the Try It function to see if your formula works.
    
5. Click Add to add the field to the system.
6. Click Save in the upper right corner to save your new field.

Now you have a total review score composed of the sum of all evaluation form questions (remember, each question's score is averaged when multiple reviewers complete the form). You can use this score just like any other piece of application data by adding it to the application grid, adding it to the application summary view, creating a segment with it, or exporting it.

Advanced Tip: if you'd like, you can create application properties that look at data from the ResidencyCAS application. You can then use them in your weighted scoring model by referencing the keys for those data points (you can find those keys in the data dictionary).