Last Tuesday, McGill lost its claim as the top university in Canada to the University of Toronto, according to the 2013 Quacquarelli Symbols (QS) World University Rankings. With the release of more rankings approaching next month, the Tribune set out to understand what university rankings actually mean, and how students—both current and prospective—should approach them.

Ranking systems for higher education institutions have existed for over a century at regional, national, and international levels. According to Alenoush Saroyan, a professor in McGill’s department of educational and counseling psychology, students pay attention to university rankings because of the large financial investment involved in a university degree.

“There’s an absence of information about universities and a desire to have some kind of a comparison between institutions,” she said. “In the absence of any other framework that provides them with that information, the ranking exercise fills the gap.”

According to Associate Registrar of Recruitment at McGill, Jocelyne Younan, there are three major rankings that compare the world’s universities—the QS Rankings, the Times Higher Education World University Rankings, and the Shanghai Rankings. In each case, universities are given an overall score using quantitative data from the institutions, as well as qualitative measures of assessment like surveys.

“You have to look at what a particular ranking measures—publications, presence of Nobel-prize winners amongst faculty or alumni, research activity, teaching, etc.—to determine which ranking suits your particular academic interest,” Younan said.

As one of the most widely recognized international rankings, QS considers over 2,000 universities every year and ranks the top 800. Danny Byrne, editor of TopUniversities.com—the website that publishes QS rankings—said that their methodology is the result of a collaboration between journalists.

“QS World University Rankings were first launched in 2004 as a way to give a rapidly expanding contingent of internationally mobile students a more objective way of comparing universities around the world,” he said. “We wanted to produce an alternative ranking that took into account areas of more direct relevance to students, like academic reputation, employability and student-to-faculty ratios.”

However, international rankings such as QS have also been criticized for evaluating the quality of an educational experience based on broad and often subjective criteria such as academic reputation. McGill Principal Suzanne Fortier said that McGill’s drop from 18 to 21 in the QS Rankings this year is not “significant.”

“These aren’t very accurate scientific studies, so the margin of error is big,” Fortier told the Tribune. “However, we must watch to see whether this is a trend or a blip. And more importantly, we must look carefully at the data these rankings will provide us, and take advantage of these to see where we can put our efforts, particularly where it aligns with the goals of our university.”

For Saroyan, the methodology of international rankings is ultimately flawed for numerous reasons. One of these is that they base categories such as university reputation on survey results. For the Times World Rankings, these surveys only have a one per cent return rate. Additionally, the use of “proxy indicators” such as the ratio of students to faculty, assumes that a university fulfilling these criteria will automatically provide a good educational experience.

“If University X has a Faculty of Law [or a] Faculty of Medicine that has a very high reputation, that reputation overflows to other aspects of that university,” Saroyan said. “So even though Faculty of Religious Studies or Arts in that university may actually be pretty bad, it benefits from the overall reputation of the university.”

“There’s an absence of information about universities and a desire to have some kind of a comparison between institutions.”

Byrne, however, said that the measures they employ are deliberately generic.

“One of the major difficulties in compiling an international ranking is that many of the data sources that make sense on a national level—say, average exam grades of students admitted—aren’t always globally available or straightforwardly comparable,” he said. “We therefore have chosen to measure broader performance areas such as academic and employer reputation, that are of clear relevance to students and of importance to all universities, as opposed to narrow and prescriptive measures that reward a given university model or system at the expense of another.”

Byrne pointed to the QS World University Rankings by Subject as one way that QS has addressed the tendency of overall rankings to privilege large universities over specialist institutions. He said that ultimately students should take international university rankings with a grain of salt when deciding which university to attend.

“We would never recommend that anyone base their university choice purely on a ranking table,” he said. “But they can provide an invaluable starting point in identifying institutions around the world that are strong in a given field, or in an area that is of particular importance to you.”

While the methodology of university rankings may be widely debated, Post-Graduate Students’ Society (PGSS) Secretary-General Jonathan Mooney said students are also highly aware that the financial situation McGill currently faces is a cause for unease when considering the university’s ability to maintain its international reputation.

“McGill does not have the same level of funding as its peers, and cannot continue to offer a top-quality education with insufficient resources indefinitely,” he said. “I think Quebec society needs to come together and make clear that properly funding education should be a top priority.”

Ultimately, Younan said there is no need to “push panic buttons” when considering McGill’s lowered position in this year’s QS rankings. She  pointed to the fact that McGill’s QS score this year (90.6) was almost exactly the same as last year’s (90.43), and that McGill actually increased their score in the Shanghai Rankings from 63 to 58. However, she said McGill will continue to pay attention to their international rankings and seek to improve them as the university looks to the future.

“The competition for top student and profs (sic) is global and fierce, so we need to stay in the game and continue our focus on excellence in teaching and research for which we are known around the world,” Younan said.