Measuring the work of ICC

The system of measurement of the ICC’s performance is the answer to the request of the 13th session of the International Criminal Court’s Assembly of States Parties, Resolution 5 (December 17, 2014), Annex I, para. 7 (available online). To invite the Court to follow the evidence-base policy is inspired by the suggestion to overcome the policy based on the mixture between theory, intuition, ideology and conventional wisdom (Banks, 2009). Actually, to apply the evidence-base policy to the management of the Court is also an ideology, because the worth of the empirical demonstration is equivalent to the other considerations both theoretical and operative (Davies, 2004). Furthermore, the application to the international penal court puts a challenge to the method chose, because the jurisprudential methodology is typically deductive, while the evidence-base policy has a strong inductive feature that moves toward an inferential approach when probability is introduced. Nevertheless it is important to remember the role of judicial statistics and nomostatistics in the history of the management of the state’s functions and the statistics themselves.

Statistics sing the saga of the many not the few; of case en masse, not of cases sui generis. Accordingly, the single case is shorn of its individuality and is merged—or submerged—in the herd. Whereupon the group phenomena are dissected by analysts armed with an assortment of averages, deviations, probable errors, ratios, indices, time-series, correlation coefficients, curve fits and other apparatus. (Jaffin 1935, pp 11–12)

The citation stresses two concepts: the single case is not object of statistics; statistics speak with the functions of data, but they must be a model for interpreting the meaning of these functions. Moreover, the problematic application of inferential model to justice issues depends on the difficult to build samples and panels really representative and respectful to the rules of sampling theory. Nonetheless the application of the probability to the evaluation of the probation in a process has carried out some significant hallmarks in the conceptualization of the uncertainty and its mathematical use (Hacking, 1990; Hald, 2004; Hacking, 2006; Sheynin, 2012), causing an important debate like that one about the Poisson distribution and the Law of Large Numbers.

Unfortunately, for the most part of these contributions are not useful for this measurement of ICC’s activities, currently an experience unique in the history of the law enforcement. The most disruptive consequence is the lack of information that is useful for building reliable statistics. In this way the preliminary study conducted by the Open Society Justice Initiative that provided several recommendations, often shared, has avoided an empirical approach that is useful to show what kind of the indicators could be better to adopt. For example, the smart advice to distinguish three categories of indicators (operational, Rome-statute, impact) would need a suggestion about the methodology of aggregation in accordance with the established set of meaningful measurement. The invitation to the creativity and the vision is associated with a form of self-consciousness that is more rhetorical than operative. Nonetheless the difficulties to draw a data generator process in this specific case has forced to exclude to create a specific codebook and a correspondent protocol. Starting from the first goal, the correspondent legal measures are counting and debatable when the report tries to provide a metric for the proper Law concepts like fair, whereas the choice made could be confuted. To the offset vagueness, the first goal is pursued detecting a large list of measures in correspondence to each stage of the trial. Calculating the indicators suggested in the second report, the attempt to obtain holistic indicators is really unreachable.

The second goal has set up measures closer to an effective indicator; instead, the third goal required a resume of two dimensions: Physical and asset security and Information security. Concerning the fourth goal the most effective methodology of measure is a survey of the victim population (UNODC-UNECE, 2010); nonetheless the context in which the victims acts may advise setting up a group of indirect measures through the evaluation of the degree of access to the Court. It is relevant to stress the strong will to guarantee a reliable measure of the role of ICC in relationship with one of the main reasons that caused the birth of the Court: to give justice to the people oppressed by Regimes or victims of Deprivation, Terror and War committed by States. Anyway the involvement of the victims needs a further effort to build a reliable system of information about its progressive development.

The entire proposed system of measurements shows the incapacity to provide a clear synthesis of the performance of ICC respect to each goal and the overall outcome attained. The tables in the annexes which lists the values of the indicators of the second report are remarkable, nonetheless the accuracy of the details does not help to formulate an assessment about the performance of Court regarding each goal. Moreover, the comparison among the different trials is out of reach, because the indicators do not have a range of variation.

How to improve the system?

It is historical evidence (Meitzen, 1891; Robert, 1977; Hacking, 1990; Mo, 1995; Rousseaux et al., 1998; Tixhon, 1999; Perelli D’Argenzio M. P., 2002; Bingham, 2006; Hacking 2006; Bolaños Guerra, 2011; Palermo, 2011; Zabell, 2011; ISTAT, 2013; Sheynin, 2012, 2014) that the use of judicial statistics was one of the most important driving the development of the statistical thinking and delivering the easiest way to measure actions (including their effects), events (including their likelihood) and outcomes proper of the law enforcement thorough the enumeration of the proceedings and crime events: a census methodology.

In the preface of the very famous book by Duff, How to Lie with Statistics, the author speaks about a father-in-law’s observation about the increased level of crime moving from Iowa to California after reading the newspaper as an informal statistical statement, pointing out the newspaper as his universe of reference for extracting the representative sample. Nowadays, how much is important to proceed in the use of a census approach when it is not possible to identify correctly the population or when the population is too small (less of 50 cases) is forgotten. Sir Kendall’s argument about the statistics totally different respect to “Political arithmetic” as the technique of counting “complete enumeration… to be a record of a situation” (Kendall, 1960), has convinced most of statistician to leave the search of measures of the Res Publica, including the Law enforcement, to the scholars of social and political sciences rather than methodological fellows. For this reason, in social context, the statistical mainstream is a sample anyway the universe at the instant it is not properly specified or fewer than 50 independent observable units . Statistics without inference is not acceptable and every social scientist knows it. And the system of ICC’s measurement seems to confirm this argument: the absence of indices statistically concerns statistically significant produces a list of counts, merely descriptive and bereft operational availment.

Nonetheless, the focus on the development of Court-wide performance indicators is an imperative of the normal activities of ICC and the proposal of the second report has to use a statistical answer in continuity with the heritage of nomostatistics (Jacobi, 1935; Jaffin 1939), because Statistics is not only the operative branch of the inductive methodology, but the mathematical language for understanding the reality and the count data are the first method of measurement. What Is To Be Done?

The suggestion is “data speak themselves”, insomuch that guides to a metric fitted to the features of the ICC. For the nature of the information the choice is the use of a ranking function applying to the frequency. This function classifies each measure respect to the whole universe detected. Through the rank these statistics classify the outcome of each counting out of the search of the range of variation, which is not possible to define for the most part of the measure proposed. Each measure looses the dimension of the absolute value for acquiring the dimension of position respect to own performance in ordinal graduation. In this way, the work accomplished by the Court will produce the position and the comparison has full meaning, whereas this operation is inner bench-marking, coherent with the uniqueness of the ICC.

Afterwards, the ranks respect to a different dimension of each goal could be gathered in a composite index could be built using a correspondent system of weight based of the order of the ranking, when it has a meaning. In the case where there is no precise meaning, the jurisprudential relations are taken into account with measures that have a weight and can be associated with the same weight. Other possibility we could apply a Delphi method for the estimation of the weight vector.

The purpose of the composite index causes the choice of the aggregation method: if they give more preference to a specific dimension than others the geometric method is better than linear one (OECD, 2008). As the composite index is also a rank, these indicators provide the performance of each dimension to everyone who wants to know how the Court runs. Moreover, the new suggested indicators may be displayed on the web site through a dashboard.

Conclusion

The measurement of the performance of ICC is harsh challenge for the statistician, because the pillars of the statistical job are very fragile. Nevertheless the need to measure the performance of the Court is a priority for its success. The proposed system is not able to provide an outstanding picture. The suggestion to pass a system based on the rank function could permit stakeholders to realize a dashboard in the web site.

References

Armatte M. (2011) L’observation scientifique est-elle un témoignage? Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online

Banks, G. (2009), “Evidence-based policy making: What is it? How do we get it?”, ANU Public Lecture Series, ANZSOG, Canberra, February

Bingham Nicholas H. (2006) Heroic periods, Mathématiques et sciences humaines [En ligne], 176 | Hiver 2006, mis en ligne le 28 juillet 2006, consulté le 05 décembre 2016; DOI: 10.4000/msh.3641 : available online

Bolaños Guerra B. (2011) Condorcet et l’épistémologie juridique. Lettres à Turgot de 1771—Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online

Davies, P. (2004), “Is Evidence-Based Government Possible?”, Jerry Lee Lecture 2004, 4th Annual Campbell Collaboration Colloquium, Washington D.C., 19 February

Hacking, Ian (2006) The Emergence of Probability Second Edition Cambridge University Press 1975–2006 ISBN 0-521-86655-3

Hacking, Ian (1990) The Taming of Chance Cambridge University Press 1990 ISBN 0 521380146 hard covers ISBN 0 521 38884 8 paperback

Hald, Anders (2004) A History of Parametric Statistical Inference from Bernoulli to Fisher, 1713 to 1935 Department of Applied Mathematics and Statistics—University of Copenhagen—ISBN 87-7834-628-2

Huff, Darrell (1954) How to lie with Statistics—Pictures by IRVING GELS w W’ NORTON & COMPANY· INC· New York, 1954 ISBN 0 393 05264 8

Kendall, M. G. (1960) Where shall the history of statistics begin? Biometrika, 47, 447–49 (1960)

ISTAT (2013) Il percorso storico della statistica nell’Italia unita Atti del workshop—Roma, 7 giugno 2011 ISBN 978-88-458-1734-2 (print) ISBN 978-88-458-1760-1 (digital)

Jacoby, Sidney B. (1939) Some Realism about Judicial Statistics Virginia Law Review Vol. 25, No. 5 (Mar. 1939), pp. 528–558, available online

Jaffin, George H. (1935) Prologue to the Nomostatistics Columbia Law Review Vol. 35, No. 1 (Jan. 1935), pp. 1–32, available online

Meitzen, August (1891) History, theory, and technique of statistics—Part 1 History of Statistics translated by Roland P. Falkner American Academy of Political and Social Science—available online

Mo, Y. (1995) History of China Statistics. Statistical Research (5) 68–76

National Academies of Sciences, Engineering, and Medicine. (2016). Modernizing Crime Statistics—Report 1: Defining and Classifying Crime. Washington, DC: The National Academies Press. Doi:10.17226/23492.

OECD (2008) Handbook on Constructing Composite Indicators: Methodology and User Guide ISBN 978-92-64-04345-9—© OECD 2008

Palermo, A. (2011) Logique juridique et logique probabiliste chez F. M. Pagano Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online

Perelli, D’Argenzio M. P. (2002) Storia della Statistica. I momenti decisivi in AA.VV. L’insegnamento della Matematica e delle Scienze Integrate Vol.25a-B N.6 Novembre–Dicembre 2002

Robert Philippe (1977). Les statistiques criminelles et la recherche. Réflexions conceptuelles. In: Déviance et société. 1977—Vol. 1—N°1. pp. 3–27; doi : 10.3406/ds.1977.935 available online

Rousseaux, Xavier; Stevens, Frédéric; Tixhon, Axel. Les origines de la statistique pénale en Belgique (1795–1835). In: Déviance et société. 1998—Vol. 22—N°2. pp. 127–153; doi : 10.3406/ds.1998.1658 available online

Sheynin, Oscar (2012) History of Statistics Berlin, 2012 ISBN 978-3-942944-20-5

Sheynin, Oscar (2014) (compiled and translated by) Studies in the History of Statistics and Probability Collected Translations: P. R. Montmort, Nic. Bernoulli, P. S. Laplace, S. D. Poisson and others vol. 4 Privately printed

Tixhon, Axel (1999) Contrôler la Justice, construire l’Etat et surveiller le crime au XIXe siècle. Naissance et développement de la statistique judiciaire en Belgique (1795–1901). In: Revue belge de philologie et d’histoire, tome 77, fasc. 4, 1999. Histoire medievale, moderne et contemporaine—Middeleeuwse, moderne en hedendaagse geschiedenis. pp. 965–1001; doi : 10.3406/rbph.1999.4394 available online

Tremper, Charles & Thomas, Sue & Wagenaar, Alexander. (2010). Measuring Law for Evaluation Research Evaluation review 34. 242–66. DOI 10.1177/0193841X10370018.

UNODC-UNECE United Nations Office on Drugs and Crime—United Nations Economic Commission for Europe (2010) Manual On Victimization Surveys—Geneva, 2010

Zabell, Sandy L. (2011) The Subjective and the Objective in Handbook of Philosophy of Science Volume 7 Philosophy of Statistics Edited by Bandyopadhyay Prasanta S. and Forster Malcom R. North Holland Elsevier B.V. 2011, ISBN 978-0-444-51862-0

International Criminal Court, Report of the Court on the Development of Performance Indicators for the International Criminal Court (Nov. 12, 2015), available online, archived.

International Criminal Court, Second Court’s Report on the Development of Performance Indicators for the International Criminal Court (Nov. 11, 2016), available online, archived.

International Criminal Court, Strategic Plan 2013–2017 (Jul. 24, 2015), available online.

Open Society Justice Initiative, Briefing Paper: Establishing Performance Indicators for the International Criminal Court (Nov. 2015), available online, archived.

How can the Performance of the ICC be Properly Assessed?, ICC Forum (Jul. 10, 2017), available online.