Reviewed by
Francisco Grimaldo Moreno
Departament d'Informatica, Universitat de València
At a social time marked by somehow insane competitiveness, this book conveniently collects an engaging subset of previous work in the rather old area of rating and ranking. In spite of including several new proposals by the authors, its main contribution may be that of bringing together a selection of handful algebraic methods and repeatedly applying them to a small running example, so as to make a clear comparison among them and to illustrate their strengths and weaknesses.Examples dealing with sports data abound throughout the text, an application domain that can possibly fall out of the scope of the reader's interest and/or research field. Though, besides making the reading more pleasant, they do not prevent the authors to also sketch how to rank other items such as: web pages, films, universities, countries, political candidates or research project proposals. Since these sort of entities often interact (or result from the interactions) within a simulation model, the techniques explained in this book can help us bridge the gap between the micro and the macro levels in artificial societies by going beyond the sometimes used basic statistical measures.
Chapter 1 opens the discussion with a reference to Kenneth Arrow's Impossibility Theorem, that proves the non-existence of a perfect voting system, and distinguishes between ranking (i. e. creating a permutation) and rating a number of items (i. e. assigning a numerical score to each of them from which a ranking can be derived). Then, even not explicitly mentioned, the book is divided into two parts: the first part introduces about eight rating and ranking methods on a chapter basis, while the second part mainly deals with method comparison, tuning and aggregation.
The rating and ranking methods covered along the book are as follows: Chapter 2 presents the Massey least square method to obtain general, offensive and defensive rating vectors, suitable to rate items and reduce ranking volatility; Chapter 3 shows Colley's method, a slight modification to the winning percentage formula that generates bias-free ratings and maintains an overall conservation of the total ranking; Chapter 4 explains how Keener's method exploits the Perron-Frobenius theory to construct rating vectors that perform well on hindsight prediction; Chapter 5 addresses the Elo rating system, an elegant highly-adaptable and near-perfect way to rate things by simple “this-or-that” pairwise comparisons; Chapter 6 depicts how to build Markov chains from which to get ratings by computing their stationary vectors ; Chapter 7 proposes a new iterative refinement process to calculate offensive and defensive ratings that can be flexibly combined to produce a global rating; and, finally, Chapter 8 exhibits how a ranking vector can be directly created by finding an optimal reordering of any data-differential matrix, an NP-hard problem only advisable for small and highly connected collections of items.
When it comes to compare, tune or aggregate rating and ranking methods, the following issues are considered: foresight accuracy can be increased by means of a rating system that reflects point spreads (Chapter 9), which can also be employed to rank items by user preference (Chapter 10); in the presence of ties in the input data, methods incorporating ties can outperform the predictive power of others that ignore them (Chapter 11); weighting input data can have positive effects as it enables to customize a method based on expert or application-specific information (Chapter 12); perturbation analysis allows to evaluate the sensitivity of rankings under different scenarios with minimal additional computation (Chapter 13); aggregated rankings usually outperform individual rankings and are helpful in on-time predictive situations (Chapters 14 and 15); and, last but not least, Kendall's tau and Spearman's footrule are two adequate statistical measures to compare ranking lists quantitatively (Chapter 16). The last two closing chapters (Chapters 17 and 18) barely deal with data gathering and other excluded rating-ranking methods such as Monte Carlo Simulations or Statistical Analysis.
As a little constructive criticism, I have missed a summing-up table or section giving a general overview of the main features offered by each method in the book, which would ease comparisons. On the other hand, a number of pros worth to be mentioned such as the profuse amount of bibliographic references for further reading, the authors remarks on computational aspects when introducing each method, the asides and historical notes scattered all along the text and the fact that chapters are mostly self-contained and, thus, well suited for teaching purposes.
In conclusion, this book is a call to consciousness on the relevance of rating and ranking as well as an enjoyable start-up guide from the point of view of algebraic methods, whereas yet to arrive are "Lies, damned lies, and statistics".
Return to Contents
of this issue