Scoring Applications
When evaluating applications, you may wish to implement a system that automatically scores applicants based on a rubric or set of criteria used by your program. This allows you to compare applications using a pre-established set of components and to pinpoint your strongest applicants. Outcomes offers several options to do this:
- Phase Calculated Fields: this feature is configured in the Phase setup and allows you to calculate the sum, mean, or median of fields included in your Evaluation Forms. You can add the resulting score to the Application Summary and use it in filters and exports. Use Phase Calculated Fields when you want to make a simple calculation using the results of your reviews, like averaging the responses submitted by different reviewers.
- Evaluation Calculated Fields: this feature is configured in the Evaluation Form setup and allows you to use JavaScript expressions to build a score, incorporating any data point in Outcomes. By default, the resulting score appears in the Completed Reviews area of the application but can also be added to the Application Summary and used in filters and exports. Use Evaluation Calculated Fields when you want to build an application score and display the results of the calculation alongside the review results.
- Calculated Properties: this feature is configured in the Custom Properties area and allows you to use JavaScript expressions to build a score, incorporating any data point in Outcomes. By default, the resulting score appears in the Application Properties area of the application but can also be added to the Application Summary and used in filters and exports. Use this option when you when you want to build an application score and display the results separate from the review results.
- Translation Tables: when configuring Exports or Calculated Properties, you can also apply a Translation Table to desired fields. This allows you, for example, to build a scoring system that converts the number found in a field to another number, based on the translations you have configured. Translation Tables also support ranges, allowing you to award a predetermined amount of points for test scores, GPAs, or other items that fall within set ranges of numbers.
Using JavaScript Expressions to Build a Score
JavaScript is a programming language commonly used to add interactive elements to a webpage. In Outcomes, JavaScript expressions allow you to dynamically incorporate other data points from the software in a Custom Property. This functionality can be used to design an application scoring system.
To score applicants, JavaScript expressions can be added in two areas:
In both situations, you'll enter JavaScript code into an Expression field. For example, entering the expression below produces a sum of the fields called KEYFIELD1, KEYFIELD2, and KEYFIELD3.
To determine which keys to use in your JavaScript expressions, consult:
- the Field Dictionary, which lists all fields available in the site, or
- the applicable Evaluation Forms, which display keys under each question.
Sample Score Using JavaScript Expressions
In the example below, a score is being calculated on five categories: GPA, degree earned, work experience, and two letters of recommendation. These items were each included as a part of an evaluation form that reviewers complete. In the JavaScript expression below, three main things are happening:
- Default values are set for each category.
- A scale is established in each category to determine how many points should be awarded based on the option the reviewer selects.
- A total score is calculated by adding the five categories together.
Review the sample JavaScript expression along with the legend below it to understand how each area works.
The following legend describes what each section of the JavaScript expression (numbered 1 through 7 above) accomplishes:
- The let function establishes each variable (scored item) and sets the default value for each of them to 0.
- The switch function sets each category's score by establishing what amount to award based on the item the reviewer selects. The keys used here: rubric_gpa_scale, rubric_Degree_scale, rubric_work_exp_scale, rubric_letter_1_scale, rubric_letter_2_scale, are the keys created in each evaluation question (see Key in the image below). In this example, the reviewer manually selects which range the applicant's GPA falls into. The options available to the reviewer (represented here as case 1, case 2, case 3, case 4 and case 5) convert to a value of either 0, 2, 4, 6, or 8 points. This method of manually selecting a GPA range is used instead of capturing the GPA from the application, which may be a useful option when GPAs either aren't included on the application by default or aren't verified. The image below shows how the rubric_gpa_scale question is built on the Evaluation Form.
- The switch function is again used here, this time for the degree information. Similar to the previous category, this is designed for the reviewer to make a selection on a degree-related question. The reviewer's answer is here converted to a value of 0, 4, 6, or 8 points. The image below shows how the rubric_Degree_scale question is configured on the Evaluation Form.
- The switch function is again used to convert a reviewer's selection into a numeric score. This time, the function is converting the reviewer's response to a question about the applicant's work experience (rubric_work_exp_scale).
- Similar to the previous sections, the switch function is used here to award the application either 0 or 2 points based on how the reviewer responded to a question regarding the first letter of recommendation.
- The switch function here works the same as the one above, this time using the second letter of recommendation score.
- Lastly, the return function is used to calculate a final score and display its value. In this example, the score is calculated by adding the scores of each category (score_gpa, score_degree, score_work, score_letter1, and score_letter2). Other equations could be also be used here (e.g., multiplying the scores, subtracting values from each score, or dividing the scores by a value).
In this example, each of the values included in the score calculation are pulled directly from the Evaluation Form results. With JavaScript expressions, you could also choose to pull in other values to make calculations, such as test scores or GPAs.
Working with Evaluation Calculated Fields
One way of building a score that incorporates several components of application data is to add an Evaluation Calculated Field. These are configured using JavaScript, so they are flexible and dynamic but require you to use some basic JavaScript expressions.
To configure an Evaluation Calculated Field:
- Navigate to Evaluation Forms in the Application Review area of the Settings menu.
- Build a new Evaluation Form or click on an existing one to edit it.
- After adding the fields you want reviewers to respond to, click Add Calculated Field at the bottom of the Evaluation Form.
- In the expression area, begin adding JavaScript expressions to establish how your score will be calculated.
Finding Evaluation Calculated Field Results
Once you've created your score using Calculated Fields, you can view the results in several ways.
To find the results of an Evaluation Calculated Field:
- Navigate to an application and click Completed Reviews from the Information Pane.
- Find your desired review and look for the results in the Calculated Scores area. You can also click Show Review Details to see the rest of the review.
- You can also find the results of an Evaluation Calculated Field in the Evaluations section of a PDF export.
- Alternatively, scores can be added to the Application Summary to feature them more prominently on each application record.
Calculating Scale Group Scores
When building an Evaluation Form, one of the Question Type options is Scale Group. This option allows reviewers to select from a scale of options to respond to your review questions. If you've implemented this option, you can use Phase Calculated Fields to calculate a sum, average, or median of the subscores from the Scale Group and display them. To do this:
- Build your Evaluation Form using the Question Type of Scale Group.
- Assign your Evaluation Form to a Phase.
- Add a Phase Calculated Field and select which subscores you'd like to calculate. Note that each Scale Group allows you to select one of the individual scoring components, the sum of each scoring component, or the average of each scoring component. For instance, as seen in the example below, you may have three components in your Scale Group, and want to add those components together. In this case, choose fieldName - Sum when building your Calculated Field.
- Once you select the field you're working with, you can then select how you want these fields to be calculated across different reviewers. For example, if you want to take the sum of the Scale Group scores calculated for each application, but then average them across all reviewers, select Average (Mean) in the second dropdown.
- In the example shown above, your Calculated Field will add up each reviewer's response to all three of the Scale Group questions (academicReview - Sum). It will then take that sum and average it with the sum calculated in other reviews of the same application. Alternatively, you could select each Scale Group question individually and have the software calculate the Sum, Average, or Median of each of those components across all reviews of the application. This results in three separate Calculated Fields.
- Complete the Calculated Fields area, adding a Label and Key for each field.
The results of Phase Calculated Fields are available to use in Exports and Filters.
Working with Calculated Properties
Another way of creating scores in Outcomes is through Calculated Properties. Calculated Properties are custom fields that you can configure with JavaScript expressions, allowing you to calculate a score using the criteria of your choice. These are a type of Custom Properties available to you in Outcomes. Calculated Properties are similar to Evaluation Calculated Fields, except Calculated Properties work outside of the review process. Both methods allow you to build a score that incorporates any data points in the software.
To configure a Calculated Property:
- Navigate to the Application Properties area under Application Setup of the Settings menu.
- Click Manage.
- The following page displays your existing Custom Properties and allows you to add more of them. Click Calculated Property from the menu on the left and drag it into your desired order on the right. This adds a new custom field, allowing you to configure it as desired.
- In the expression area, begin adding JavaScript expressions to establish how your score will be calculated.
- Once you've added a Calculated Property, the results for each applicant can be found by going to Application Properties in the More Options menu of an application record. You can also add scores to the Application Summary to feature them more prominently on each application record.
Using the Field Dictionary to Find Key Names
The Field Dictionary provides a database of every field available to you. Using this, you can determine what keys to use in your expressions. Keys that come from forms can also be found on the applicable form near the associated question.
Fields that aren't sourced from forms can be located in the Field Dictionary. To use this dictionary:
- Click Field Dictionary.
- From the Field Dictionary, click one of the Quick Links or use the Search bar to locate your desired field.
- The Field Dictionary provides information regarding the key to use (e.g., forms.standardized_tests.toefl[].toefl_total_score), the data contained within the field, and the format the data appears in (e.g., String, Number, or Array). Review the information for the fields you want to use.
- Copy the keys you want to use and place them in your JavaScript expression.
- Continue building on your expression or adding new Calculated Fields with different expressions.
- When ready, click Save Changes.