Before fellowship applications begin to roll in, you need to be ready with an evaluation process that’s fair and efficient. But that’s just the start. Let’s talk about a few ways you can be sure your next fellowship awardees are the perfect fit for your program.
By minimizing confusion about your criteria and scoring system, you can help your fellowship selection committee make better decisions. Here are a few strategies to tighten up your review.
If you have a list of evaluation criteria, is there room for interpretation on how they should be applied? Provide your reviewers with a detailed definition of each criterion, so they’re reading each application through the same lens.
Keep the following in mind:
For example, let’s say the criterion is Scientific Merit of the Research Proposal. The definition might be as follows:
This criterion measures how well the research proposal follows the scientific method and its level of potential benefit to participants, society, and/or the field. We want to see that:
Depending on the type of criteria, you may be able to score with a simple Yes or No. But in most cases, you need something more nuanced. The easiest way to do this is by adding a numerical scoring system. However, like before, being clear about definitions is crucial. The number itself doesn’t matter, it’s the meaning behind it that does.
So, if you decide that 1 equals high scientific merit, 2 equals medium, and 3 equals low, you need to be clear about what that means. Since this definition has three elements, you could say:
This makes it so much easier for the reviewer to evaluate. And, when you use a fellowship management system, they can see the application and scorecard side-by-side, rather than toggling between multiple tabs or flipping through physical documents.
As you can see, the numerical scoring system applies a consistent method for evaluation. Plus, when you use the same scale (lower numbers = better), you can simply tally up the applicant’s total score to see the top applicants.
However, this approach doesn’t factor in the relative importance of each criterion. To introduce more precision into your evaluation process, you can assign percentages.
For example, let’s say that your fellowship is known for its scientific rigor and outstanding research outcomes. You might use a breakdown like this:
While you could certainly set up a decision matrix in Excel, a faster way to calculate the final scores is by using fellowship application software. You can configure the scorecard once, and then let the system provide real-time scoring at the touch of a button.
Now that we’ve discussed how to make your evaluation process more objective, let’s focus on the other half of the equation - your review team. After all, the selection of fellowship awardees does come down to a subjective decision. You want to ensure your team makes the right call.
As a starting point, here’s a list of questions you should keep in mind as you’re considering potential reviewers.
As your team is coming together, take a step back, and ask:
We’ve already talked about the importance of defining your evaluation criteria. This is the best way to help your reviewers make well-informed decisions. However, don’t take for granted that everyone is on the same page.
It’s important to host a group orientation before they begin their review assignments. Having discussions about the criteria will reinforce what they’re looking for and clear up any gray areas.
You may want to summarize any clarifications in writing after you meet. If you’re using fellowship management software, you could update the scorecard with customized instructions or reminders.
Since committees are usually working independently on their own assignments, this can inhibit collaboration. Having a broader perspective benefits the decision-making process. Look for ways to let reviewers bounce ideas off each other, ask questions, or raise issues.
Using a fellowship application system makes this easier by allowing for comments directly in the application itself. Reviewers can make a note, and then send a message to fellow committee members or the fellowship coordinator. They can even communicate directly with the candidate if they need more information or clarification.
--
This all adds up to a more robust review process with better outcomes. By clarifying your fellowship evaluation criteria and putting a strong, well-prepared selection committee in place, you can guarantee an excellent decision.
