Skip to main content
Liaison

Managing Reviews

In Outcomes, the standard way of assigning applications involves three steps:

  1. Creating Evaluation Forms to provide a place for reviewers to enter their results.
  2. Configuring Phases to establish which reviewers will be assigned and which Evaluation Forms they will use.
  3. Changing applications' Phases to assign them.

Once that is done, Reviewers can complete their reviews.

Assigning applications using this process has several benefits, including the ability for assignees to enter their responses in the Evaluation tab, the ability to create PDF files with evaluation results, and the ability for assignees to be notified of their assignments. For a simpler process, though, you may wish to assign applications using Tags or Tasks instead.

Managing Evaluation Forms

The Application Review section of the Settings menu allows you to create and modify Evaluation Forms. These are forms that Reviewers will complete specific to each application's current Phase.

Creating Evaluation Forms

To create a new Evaluation Form:

  1. Go to the Application Review section of the Settings Menu and click Evaluation Forms.
  2. From here, you can review your existing Forms. Click the plus sign to build a new Form.
  3. Name the form and enter a key to associate with the form.

    New Evaluation Form
  4. Evaluation Forms are divided into different sections. Enter a name in the Section Title.

    Creating an Evaluation Form
  5. Enter an introduction with instructions or information for references.
  6. To begin adding questions, click Add Question.
  7. On the Edit Question page, enter your question.

    Adding an Evaluation question
  8. Enter additional guidance in the Help Text field.
  9. Choose your desired Question Type.

Question Types are as follows:

  • Text: Allows evaluators to type in a text response. Selecting this option also requires setting the Max Length for a response. Use this option for questions requiring a short response.
  • Paragraph Text: Allows evaluators to type a paragraph-length reply.
  • Multiple Choice: Presents evaluators with multiple options to select from. When selecting this Question Type, enter each option individually in the Answer Option area, or add several answer options at once by clicking Paste Options and pasting the desired options in the text box. Each option must be separated by line breaks. Check the Allow Write In option to allow recommender to write in their own response.
  • Drop Down: Works the same as the Multiple Choice option, except answer options are presented to evaluators in a drop-down menu.
  • Checkboxes: Works the same as the Multiple Choice and Drop Down options, except multiple answer options can be selected.
  • File Attachment: Requests that evaluators upload a file in response to the question.
  • Date: Allows evaluators to enter a date in responses to the question.
  • CEEB Code Lookup: Allows evaluators to respond with a school name.
  • Email Address: Allows evaluators to respond with an email address.
  • Phone Number: Allows evaluators to respond with a phone number.
  • Web URL: Allows evaluators to respond with a URL.
  • Number: Allows evaluators to enter a numeric response.
  • Scale: Allows evaluators to rate the applicants on a scale of your choice. Scales can be up to 10. After selecting this option, choose your Scale range and how label will display.
  • Table: Allows evaluators to respond by completing a table. For example, entering their contact information.
  • Address: Allows evaluators to respond with an address.
  • Name: Allows evaluators to respond with a name.
  • Concealed: Allows for the evaluator’s response to be encrypted. Only users with the appropriate permissions can decrypt and view the responses.
  1. Check the Required checkbox to make this a required question.

    Adding a required question
  2. If desired, enter a Javascript condition to modify the question. For example, hide the question or limit its options if certain conditions are true.
  3. After completing your question, click Add.
  4. Continue adding questions as desired.
  5. Click Add Section to add a new section of questions. Including several sections is useful for organizing longer Forms.

    Adding a section to an Evaluation Form
  6. Click Add Calculated Field to add a field to be used in calculating applicant data.

    Adding a Calculated Field to an Evaluation Form
    This section allows your Evaluation Form to combine, display, and adjust data from any field on the application, including fields from the Evaluation Form itself. For example, you can add multiple data points together and perform a calculation to them to develop an applicant score. When working in this area, remember:
    • This is an advanced feature, as the Calculated Fields here can only be added with JavaScript expressions.
    • These Calculated Fields are separate from Phase Calculated Fields, which are used to make calculations solely based on items within the Evaluation Forms.
    • The Field Dictionary is a comprehensive list of application-related fields. Using this, you can determine which field names you need to use in your expressions and what type of data you can expect in each field.
    • The available Data Types for these Calculated Fields are:
      • Boolean: These are fields that can be set to True or False (Yes or No).
      • Number: These are fields to be populated with a numeric value.
      • String: These are fields to be populated with text values.
      • Date: These are fields to be populated with a date.

Working with a JavaScript Calculated Field
If you've added a Calculated Field to an Evaluation Form, once the corresponding reviews are complete, you can see the results of your Calculated Fields by clicking Completed Reviews on the application's Information Pane.

completed-reviews-panel.png

  1. To test the Form and view it as a reviewer would, click Try it.
  2. When you’re done configuring the Evaluation Form, click Save Changes.

    Saving changes to an Evaluation Form
Linking Evaluation Forms to Phases

Once you have created your Evaluation Form, you can associate it with one or more Phases. Once an applicant is assigned to that Phase, the Evaluation Form is linked to them, and the assigned reviewers can complete their review using your Evaluation Form.

To link an Evaluation Form to a Phase:

  1. Navigate to Phases in the Application Review section of the Settings menu.
  2. Click Manage.

    Working with Phases from the Settings menu
  3. From here, you can create a new Phase, or select your desired existing Phase.
  4. In the Review Form area, click Choose Form to select a form. If a form is already associated with this Phase, you can click Edit to select a new form.

    Choosing a Form to connect to a Phase
  5. Select your desired Evaluation Form from the list.
  6. If desired, use the Assignments area to assign the Evaluation Form to specific users or Review Teams. Once an applicant is placed in this Phase, the associated reviewers are assigned to the applicant. Applications you assign are visible to all the users and teams you indicate, regardless of which applications their Role gives them access to.

    Assigning by Segment
  7. When completing the assignment area, select what will be done with applicants in this Phase who don't fall into any of the Segments you've selected. If you decide to leave them unassigned, you can assign them directly from the application.

    Leaving remaining applicants unassigned
Assigning Evaluation Forms to Reviewers Directly

Using Phases, you can link Evaluation Forms to applicants and reviewers. Applicants in a Phase with an associated Evaluation Form, but left unassigned through the configuration of their Phase can be assigned directly to a Reviewer.

To do this:

  1. Navigate to the application you want to assign.
  2. Open the More Options menu and select Assign.

    Assigning an applicant directly
  3. Select your desired reviewers and click Assign In...

For this option to be available, the application needs to be in a Phase that is associated with an Evaluation Form.

Disabling Assignment Notification Emails

By default, any reviewer who is assigned an application through the Evaluation Forms process will also receive a daily email notification. The Assignment Summary Notification emails help reviewers stay up-to-date on their progress in completing their reviews.

Administrators can disable these notifications by going to the Users section and adjusting the Opt Out Assignment Notifications toggle.

Opting out of assignment notifications

Working with Phase Calculated Fields

When building an Evaluation Form to use in reviewing applications, you may include fields that require reviewers to provide a numeric response (e.g., number or scale.) If you wish to make calculations based on these fields, you can create Phase Calculated Fields. For example, you can create a Phase Calculated Field to take an average of how reviewers rated an applicant's Personal Statement.

Phase Calculated Fields can also be configured to count the number of reviewers who responded in a specific way. For example, if your Evaluation Form has a Multiple Choice question with options including Yes, No, and Unsure, you can create a Calculated Field that tallies how many reviewers selected Yes. For more advanced calculations, you may consider working with Evaluation Calculated Fields or Custom Properties, both of which are designed using JavaScript expressions.

Creating Phase Calculated Fields

To add a Phase Calculated Field, you'll first need to have created an Evaluation Form that contains the fields you wish to use in your calculation. Then:

  1. Create a new Phase or edit the Phase where the Evaluation Form should be used.
  2. Add your Evaluation Form in the Review Form area, then click Add under Calculated Fields.

    Adding a Phase Calculated Field
  3. Click Add Calculated Field on the window that appears.

    Adding a Calculated Field on an Evaluation Form
  4. Use the Select Field drop-down to choose a field to use in your calculation. The fields shown in the drop-down are those that are in your selected Evaluation Form.
  5. Use the Label field to write a label for the Calculated Field. This is how it appears in other areas of the software (e.g., when adding the field to the Application Summary or when using it as a filter). Use the Key field to write a key. This is how the data is stored in the background of the software.

    Using an average on a Calculated Field
  6. Use the last drop-down to select what type of calculation you wish to use. If you're working with a numeric field, you can calculate its Sum, Average, or Median. If you're working with multiple choice fields, you can calculate the total number of times any option was selected by a reviewer.

    Adding a Calculated Field for recommended admits
  7. If desired, click Add Calculated Field to create another Calculated Field.
  8. When you're done adding fields, click Update.
  9. Click Update again to save your changes to the Phase.
Using Phase Calculated Fields

Once you've created Calculated Fields, you can use them in several areas. For example, you can add add a filter on the Application View to find all applications that have above or below a specific value for one of your Calculated Fields.

To add a filter using a calculated field:

  1. Navigate to the Applications View and click Add Filter.
  2. Click Phase Calculated Field.

    Using a filter on a Phase Calculated Field
  3. Select the Phase, Calculated Field, Operator, and Value.

    Applying a Phase Calculated Field filter
  4. Click APPLY to update the Application Grid with applications that match your query. In the example above, only applicants with an average Interview Score of greater than 10 will appear on the grid.

Finding Review Results

Once reviews have been completed, you can find their results in several areas.

  1. Navigate to an application and click Completed Reviews from the Information Pane.

    Completed Reviews on the Information Pane
  2. Find your desired review and click Show Review Details to see its results.

    Evaluation form results
  3. You can also find the results of an Evaluation Calculated Field in the Evaluations section of a PDF export.

    PDF of Evaluation results

Managing Review Teams

Outcomes allows you to establish teams of users who will evaluate applicants in the software. By creating teams, you can assign applicants to an entire team at once.

To create a new Review Team:

  1. Go to the Organization section of the Settings menu and click Teams.
  2. From here, you can review the existing teams and click Create to work on a new team.

    Creating a Review Team
  3. Name your new team and, if desired, determine who the team’s Manager and Members are.

    Creating a Review Team
  4. If desired, choose a category. Team categories allow you to organize different teams as you choose.
  5. You can also add an image to associate with the team.
  6. When ready, click Create Team.

You can edit existing teams from the Review Teams page by clicking the name of the team. You can delete or categorize teams by clicking the checkbox to the left of them, and selecting the desired option at the top of the grid.

  • Was this article helpful?