Some of Canada’s largest research universities delivered a pointed message to the editors of Maclean’s magazine last month when they announced in an open letter that they would no longer assist the publication in preparing its annual university rankings issue.

The issue, popular with prospective students and their parents since its debut 15 years ago, has been controversial among top university officials. Principals and presidents have often complained about the magazine’s methodology and have had problems with its conclusions.

In their letter, the leaders of the 11 universities, including the Université de Montréal, University of British Columbia and top-ranked University of Toronto, cite the magazine’s reliance on surveys with low response rates, as well as its practice of lumping together diverse rankings to get a single rank, as reasons for their decision.

“Consider how such an approach might pervert one’s understanding of a general hospital that is ranked number one in obstetrics and number ten in cancer care,” the letter reads. “Averaging these rankings would result in this hospital being ranked number five overall.”

The letter goes on to point out that for a patient looking for care in just one of the areas, the ranking would be “useless at best, and misleading at worst.”

Last week, Concordia University joined Montreal neighbour U de M in withdrawing its cooperation with the ranking.

In an open letter published Aug 29 on Concordia’s Web site, the school acknowledged its participation in the rankings had benefited Maclean’s, but said “we maintain that the benefits to Concordia have been far less clear.”

Concordia Director of Institutional Planning Brad Tucker said that the university’s withdrawal would not make the eventual ranking less reflective of the university.

“They’ve never been reflective in the first place,” he said.

He also questioned the use of the Maclean’s issue as a recruiting device.

“Do we attract students? Our studies have shown our Web site to be far more a factor in recruitment than Maclean’s magazine.”

However, Tony Keller, the Maclean’s editor in charge of the ranking, disputes the universities’ reasons for withdrawing.

“They’re questioning the very idea of a ranking,” Keller said. “Universities engage in rankings all the time, always comparing themselves to other schools. They just don’t want journalists doing the ranking. They want to be the players and the referees.”

Keller has had lengthy conversations about the issue with most university presidents. He indicated that it is difficult to arrive at a consensus with so many schools making different and even contrary demands.

“It’s sort of like the federal government dealing with the premiers, except in this case we have 47 different universities with 47 different points of view.”

The schools boycotting Maclean’s also said the magazine was unresponsive to their suggestions and demands. Tucker indicated that Concordia has been involved in a cordial dialogue with the magazine about improving the methodology, but hasn’t seen any efforts on behalf of Maclean’s to make any serious adjustments.

Keller disagreed with Tucker’s characterization.

“It’s like they’re saying, ‘We want to negotiate with you about how you cover us,'” he said. “We’re journalists!”

Keller said that Maclean’s has made numerous changes over the years, incorporating many suggestions from post-secondary institutions. For example, they’ve introduced three new features at the request of various schools for the upcoming issue.

“[In the past] we never provided a score. The US News has always given you a score. We will now, and I believe it will be helpful for readers.”

The magazine is also introducing a graduate studies issue as well as an interactive section on its Web site where students can create personalized university rankings by taking into account factors that are most important to them.

McGill Deputy Provost Morton Mendelson indicated that the school will be participating in the rankings.

“We know there are problems with the methodology. We’ve been expressing them. However, there are problems with every kind of ranking.”

Mendelson pointed out that universities across the country take these rankings quite seriously. In fact, many schools go out of their way to improve on weak areas in efforts to boost their ranking.

“It’s important to be publicly accountable to our various constitutents,” he said.However, Mendelson admitted that the Maclean’s rankings are easy to take in stride because the university does well. Last year U of T and McGill tied for first place in the medical/doctoral category.