Scoring or Weighted Criteria Technique

  • This method represents a more sophisticated version of the checklist technique.

  • The procedure essentially consists of developing a list of several criteria for setting research priorities, and then collecting information on these criteria for a set of commodities or research areas.

  • Part of the information is collected from published sources and rest of it is usually obtained through interaction with the people concerned.

  • Relative weights are attached to the criteria to arrive at the set of priorities.

  • In the simplest form of the scoring technique, weights are determined subjectively.

  • A somewhat more complicated method derives weights from the sensitivity test conducted on social benefit / cost analysis for a number of research programmes.

  • The scoring matrix is nothing but a checklist with the answers to questions assigned numerical values and weights.

  • Criteria weights are multiplied by the values which a particular research programme merits under each criterion to produce a final score.

  • According to their scores, the programmes can then be ranked in order of priority.

  • The scoring technique has been tried more often than any other formal method for ranking research priorities in agriculture.

  • As compared to checklist method, the scoring technique not only forces the research planners to consider all the significant factors which may have a bearing upon research prioritization, but also couples them to try to assess their relative importance.

  • When the final scores do not accord with common sense, the decision makers may wish to adjust the values, and sometimes even the weights.

  • Though the individual decision maker can adjust the scores, it should preferably be done by the group interaction.

  • In either case, the process of adjustment should not be carried too far : results should not be manipulated until they merely reflect existing prejudices.

Merits

  • It is simple to use.

  • It forces researchers to be explicit.

  • It is useful at all levels of planning – national as well as institute    levels.

  • It is applicable for commodity and other types of research, and

  • It may give spurious impression of precision.

    		•

    • While it is possible to express directly the benefits from a commodity research programme, those from research on a factor such as soil can only be derived from assumptions regarding increased productivity over a range of commodities.

    • In order to overcome this problem, some have chosen to treat commodity research programmes differently from factor research programmes.

    • But the results will not then inform the decision maker as to where priorities between, as opposed to within, the two categories of research should lie.

    • Since direct comparison of quantitative results will be extremely difficult in this case, informal discussion between all concerned is the only way to reach a consensus.

    • This technique is straight forward and can be applied without any special training.

    • It can incorporate both qualitative and quantitative information and can be applied to a long list of commodities or research areas in a relatively short period of time.
    Demerits

    • Although the procedure is relatively easy for the decision makers to understand, it does require their time in obtaining the explicit weights for the criteria.

    • Since these weights are inevitably subjective, their elicitation must be carefully structured.

    • The experience and knowledge required for assigning weights are particularly demanding.

    • It should be kept in mind that there is always a risk that weights and scores determined by personal judgements will result in misleading conclusions.

    • The technique also requires researcher’s time to collect information on qualitative criteria.

    • This approach is better suited for periodic or major priority setting efforts than for situations where frequent marginal changes are anticipated.
    • Major shortcomings of the scoring technique are the case of incorporating illogical and / or overlapping criteria and improperly weighting them.
    • This method is often criticized because of the subjective weighing of criteria.
    • The lack of discounting of future benefits and costs, inaccurate accounting of research spill overs and the ignoring of the effects of domestic and trade policies are the other disadvantages.

    • Thus, this method only provides crude estimate of efficiency or distributional effects of research.