AIGM home  |  Our Community  |  About Us  | 
AIGM logo Australian Institute of Grants Management An OurCommunity.com.au enterprise


Forum    Login   Join

The assessment process - the key to effective grantmaking

Nothing is more fundamental to effective grantmaking than a sound assessment process. Having the maximum impact means that you have to choose the right projects to fund.

This becomes even more important once you adopt the best practice of committing a substantial portion of available funds to multi-year projects or operational support, since you will be funding fewer projects.

Investing time and effort into your assessment process will go a long way towards establishing your program as well-targeted, transparent and objective.

The Assessment Process

Your assessment process will have multiple steps which will likely include:

  • Conducting an eligibility screening to ensure that only eligible applications are assessed
  • Ranking eligible applications by merit to form a recommendation report
  • Approval by the authorised body of the recommended projects as funding allows

Screening should be pretty cut and dry. Screening criteria should be created to avoid ambiguity as to whether or not a project is eligible. Some electronic systems, allow you to create an "eligibility quiz" for applicants to fill out before they're able to access the application form, thereby preventing ineligible projects from applying in the first place.

If your system doesn't allow this, place your eligibility questions at the beginning of your application.

Eligibility questions should be clear (most can be formatted into a yes or no question), should be stated in your Program Guidelines and should be asked right up front. Applications that do not meet the eligibility requirements should not proceed in the assessment process.

Once you have narrowed the field to include only eligible applications, you need to rate and rank those applications. The people, tools, weighting and criteria used will determine the heart of your assessment process.

The Assessment Panel

Your assessment process should be proportional to the size of grants you are awarding. This principle should apply to:

  • The size of your assessment panel
  • The amount of time allocated for your assessment process
  • The complexity of your assessment tool

If you are assessing grants for a small program, it's okay to have a small panel. In fact, data from SmartyGrants indicates that over 40% of applications are reviewed by single assessors.

However, when you're dealing with grants valued at thousand, or hundreds of thousands of dollars, a panel (three to five members works well) will ensure that differing perspectives are given due consideration.

Who Should Be on Your Assessment Panel?

Your assessment panel may include trustees, members of your staff, representatives from other organisations that share your mission or volunteers from the community.

Strive for a diverse panel that includes the necessary expertise. Consider including members of the target population (low-income people, people with disabilities, etc.), as they will have a unique perspective that is crucial to the success of the project.

In recruiting members of your assessment panel, it's most important to ensure that no one has a real or perceived conflict of interest. You can supply perspective panel members with a list of the applicants and ask them to declare any conflicts (and recuse themselves) as soon as your application round closes.

When Should Assessment Begin?

As soon as possible! Use the time when your grant round is open to prepare for your assessment process. A significant number of grantseekers (37% in the Australian Institute of Grants Management's 2015 Grants In Australia Survey (available via this link)) continue to feel that the delay between the time they submit an application and the time they hear about the result is unacceptable.

Setting clear eligibility criteria, recruiting a the assessment panel ahead of time and preparing the assessment tool before hand will help to keep this step of the process compact.

How Should It Work?

You have several options in how you run your assessment panel. If you have the capacity, you may want to conduct your process electronically- easily done in SmartyGrants for example, allowing assessors to individually complete an assessment form for each application.

Or you may feel that it's important for your assessors to meet face to face and discuss their opinions on the various applications. Both ways have their advantages and you may find it best to use a combination. SmartyGrants allows you to set assessment forms as "shared" or "private", meaning you can determine whether or not assessors are able to view each other's responses.

Either way, you'll need to ensure that your assessors are adequately trained. This means that they have an understanding of the programs goals, know how to access/use the system (SmartyGrants provides a Help Guide for Assessors), and, most importantly, that they understand how to use the assessment tool you're providing them.

The Assessment Tool

Your assessment tool should measure against specific criteria that were laid out in your Program Guidelines and developed to align with the outcomes you will look for in evaluating the success of your program.

Consider what success would look like, research best practices that are proven to lead to those outcomes and integrate those practices into your assessment criteria.

Criteria may consider:

  • The level of need for the project in the community
  • The activities that the project will carry out
  • The expected outcome of those activities and whether or not they align with the desired outcomes of the program
  • The value - the number of people who will be served compared to the amount of money requested
  • The budget, budget justification and its feasibility
  • The level of support for the project from the community (possibly indicated by matching funds)
  • The level of risk associated with the project
  • The applicant's ability to successfully carry out the project
  • The applicant's history of executing previous grants

In creating your assessment form, follow the rules of survey writing. Use neutral language, avoiding loaded or leading words. Place your questions in a logical order, starting broad and then becoming more specific.

Make sure each question asks only one thing and that your questions are clearly worded. Finally, it is recommended that you use a numeric scale for rating each criterion.

Numeric scales are useful because they are precise. It's clear that 7.6 is larger than 7.2 which is larger than 7.1. However, even with a numeric scales there is a risk that not everyone on your panel will use them same way.

Consider a scale that goes from 1 to 10, with 10 being the best possible score. Does that mean that 10 is the answer you're looking for, 1 is unacceptable and 5 is half-way in between? Can you truly be sure that the interval between each number represents the same thing to each assessor?

For numeric scales to truly be more accurate than ordinal scales (Strongly Agree, Agree, Disagree, Strongly Disagree, etc.) the numbers in the scale need to have a fixed relation to one another such that an application that scored a 10 on a specific criterion can be assumed to be 10 times better on this criterion than an application that scored 1.

(Refer to page 61 of the Australia National Audit Office publication Implementing Better Practice Grants Administration for an excellent explanation of this principle).

This is something that will need to be clearly explained to assessors.

Finally, you will need to assign appropriate weight, and perhaps a minimum required score to each of your criteria. There is little case for funding a project that scores high on addressing the program goals and being needed in the community if none of the assessors believe the applicant has the capacity to carry out the project.

Thus, some questions will be more heavily weighted than others in determining the applications overall score, and some questions may be deal breakers.

In a perfect world, creating an accurate assessment tool will make it easier to choose between an array of great applications. But what if the applications that come in are not what the funding body was hoping for and score unacceptably low on key criteria?

The fact that you can objectively rank your applications does not oblige you to fund them. Baring any legal obligations, it may be better to reserve some funding for an upcoming round and focus your efforts on soliciting better applicants in the future.

Approval

The final step should be fairly straightforward. The authorised body, having received a detailed report rating each application and noting any comments or concerns, can simply go down the list and approve the projects until the funding runs out.

There may be cases when the authorised body chooses not to follow the recommendations, but a rigorous rating and ranking process, combined with thorough documentation will make this less likely.

The more defined your strategic direction, the clearer the approval process will be, leaving less opportunity for ad hoc decision making.


Kate Caldecott is the former Executive Director of the AIGM and a founding member of the SmartyGrants leadership team. She now works as a consultant, focussing on reforming and streamlining grants programs, program evaluation and making the most of SmartyGrants. Kate can be contacted on: Kate@katecaldecott.com.au or on 0447 227 598.
News
Great grantmaker team's future focus

Australia's best grantmakers aren't short of ideas for providing funding more effectively, with the NSW Heritage Grants Team proposing several to make things easier for grantseekers. Hear from the 2019 Grantmaker of the Year winners.

News
Top grantmakers shine a light on our funding future

A team of NSW Government grantmakers overseeing a heritage grants budget of around $3 million per annum has been crowned the 2019 Grantmaker of the Year.

News
All the news grantmakers need

Here's a taste of the 'target your funds' edition, with insights from around the world on core funding, deadline data, 'buying' impact and measuring it.

News
SmartyStats reveal how applicants are lodging forms

Analysis of data on applicants using the SmartyGrants system reveals that more than half of all grantseekers who lodge applications complete them in under 48 hours.