Dr. Tariq Rahman

Evaluating Scientists in Pakistan

            The Pakistan Council for Science and Technology (PCST) has been publishing an annual report on the productive scientists of Pakistan since 1999. The first report, called Leading Scientists of Pakistan (1999) laid down the basis of the evaluation of scientists. They looked at the number of articles published in refereed and indexed journals and the citation reports. The idea is that if one’s work is considered valuable by experts in the subject it will be published in a prestigious journal. Further, if many people cite it, as reflected in the citation index, it will have an impact on the scientific field. The Impact Factor, then, is ‘a ratio between citations and the recently citeable papers published’.

            The 1999 report lists scientists who have an impact factor of about 20 or above. There are, of course, problems with this criterion as with all others. It is possible, for instance, that a really good article in an under-researched field is not cited at all. It is also possible that an article is cited but only because it refers to pioneering work not necessarily outstanding work. It is even possible that everybody cites the article as a warning as to how not to do things---but this is something of a joke which occurs once in a blue moon! Nevertheless, whatever the problems, the impact factor is a good way of evaluating scientists.

            In 2000 the PCST published Scientific Research in Pakistan listing scientists with an impact factor of greater than fifteen. This time the average impact factor (1992-2000) was worked out and it was found that 945 scientists (50 per cent) have an impact factor of zero or less than one and 117 people have become full professors with such little impact. It was also found that 13 scientists had an impact factor of 200 or above. The Department of Physics, Quaid-i-Azam University, led physics while the H. E. J Research Institute of Chemistry at Karachi University led chemistry. It was also found that people had received awards and worked as vice chancellors with hardly any impact factor. Something which ought to warn us that al is not well with our system of conferring awards on people and making them vice chancellors.

            In 2002 a voluminous report called Productive Scientists of Pakistan was published. This study added points for journal articles, editing of books, patents, proceedings of conferences, research grants won and supervision of theses. The total number of points was calculated according to a formula which has been given in Annexure-5 of this report. It is possible to get marks for the supervision of M. Sc theses even with little scientific impact. Thus, the overall points do not measure scientific impact, they measure a number of things one of which may be supervision or grants won. They are, on the whole, a useful indicator of how much research work, academic supervision of research and academic resourcefulness (grants) an academic has.

The highest points in the country are those of Dr. Atta ur Rahman (Chemistry) being 5859 (with impact factor of 456.686). Dr. Riazuddin, the Quaid-i-Azam University Physicist, has 2,368 points but an impact factor of 478.47. In Physics the highest points are of Dr. Hameed Ahmad Khan (PAEC) being 2,722 but his impact factor is 312.758. In mathematics the highest points are of Dr. Asghar Qadir being 1125 with an impact factor of 97.435. In the biological sciences Dr. Ajmal Khan, Karachi University, has 1420 points with an impact factor of 81.879. However, Dr. Sheikh Riazuddin (Ministry of Science & Technology) has 1166 points but an impact factor of 286.068 Dr. Rabia Hussain (Aga Khan University) has an impact factor of 281.93 but with only 717 points.

In short, since points can be won from activities other than having impact in one’s field, there are discrepancies between total points and impact factor. It should also be pointed out that some scientists may not have sent their biodata for evaluation because of disagreement with the criteria of evaluation or absence from Pakistan. Thus, like most lists, this list is also incomplete. However, it is an excellent beginning because some kind of quantified criterion is available now to rank scientists.

Another purpose of the list is to rank universities in term of their scientific output.  An extract from this is instructive. The universities are placed in this order:

 

Productive Scientists

Total Points

Quaid-i-Azam University

68

27,984

University of Agriculture (Faisalabad)

39

11,084

University of Karachi

41

10,508

University of Punjab

14

6,521

University of Peshawar

29

5,456

 

After this the productive scientists and points decrease till one comes to zeros. Out of 73 universities, both public and private, 40 have zeros under both heads. Ironically enough, some of the expensive universities which students regard as being very trendy, are in the list of zeros (see Annexure-1 of the report). Apparently fashion is one thing and scientific output another.

These reports constitute a very positive and very welcome step towards quality control in the universities. I wish somebody would do this kind of work for the social sciences, humanities and arts. I have given the outline of such a system of evaluation several times before. I would like to take this opportunity to sum up what I have written earlier about evaluating and ranking social scientists.

Essentially, only articles published in refereed, indexed and abstracted journals should be given points. If the journal is on one index only one point should be given but if it is on five indexes then five points should be given (no extra points beyond that). Points should be more for articles published by one author and less for articles published by more than one author. Books published by prestigious publishers should be given five points provided they are reviewed in indexed journals and the review is positive. If the book is published but not reviewed then only one point should be given to it. However, the publishers should be known as academic and serious publishers and the author should not have paid to get the book printed. Conferences attended may be given points provided the proceedings are published but conferences in Pakistan should carry one point whereas conferences in academically prestigious countries can carry two points while kenote addresses can carry three points. Supervision of theses (M Phil and Ph. D) can carry one and two points provided the theses are sent for external opinions and examined by experts. However, in order to become a professor, the highest rank in a university, one most have won at least half the total points required (say 300 points) through one’s own research publications. For distinguished professors points from research must be 250 while total points may be at least 400.

I advocate such a system especially at this juncture of our history when the HEC has put forward proposals for enhancing the salaries of academics. Salaries must be increased to reverse the brain drain from the universities but those who get high salaries must deserve them. I propose that, like natural scientists, our social scientists too must agree to be evaluated according to the point-weightage system proposed here (or elsewhere) so that a fair and transparent system of measuring academic competence is evolved. Those who do not volunteer to be evaluated and ranked may continue to serve in the same salary structure as at present till they retire. New entrants must, however, get enhanced salaries along with a more exacting criterion of promotion based upon ranking as carried out for the scientists and the social scientists. This system may initially be applied only to two major universities while the others, whether private or public, will continue to give whatever salaries they do at present and need not rank their faculty according to the new scheme proposed above. These less exacting universities, concentrating upon teaching rather than research, may be affiliated to a Board of Higher Education which will ensure that minimum academic standards are met with and examinations are conducted in a uniform and fair manner by the  Board and not internally by the university itself. Only the two research universities with higher pay scales, better facilities and excellent scientists and scholars should have the autonomy to hold their own examinations and determine what they will teach and how. These steps will go a long way to improving the universities of Pakistan.

Dr. Tariq Rahman