A Single Comment — Permalink
© ICCforum.com, 2010–2023. All rights reserved. Policies | Guidelines
- roccosan: Premise The uniqueness among the Judicial Systems of ICC is a challenge for the application of statistics to measurement about its activities. This is for two reasons: the absence of a reasonable term of comparison; the self-referentiality of the system of measurement. Both the features provide the occasion for a reflection about the meaning of measurement of an object that is strictly qualitative and hold an ontology bereft of... (more)
- chrisjlin: Prosecutorial Discretion in Investigations: A Balance Between Politicization and Independence Abstract I examine the possibility of enforcing a balance between discretion and independence, seeing as how the International Criminal Court (“ICC”) Prosecutor’s decision to investigate a specific country, individual or group is necessarily political, given the ICC’s method of functioning. Despite the occurrence of... (more)
- cgsanchez: The Al Mahdi Case Study: Establishing a Media Bias Baseline to Support Future Research Regarding how ICC’s Operations Affects Public Perception International media reaction to the case The Prosecution v. Ahmad Al Faqi Al Mahdi heard in the International Criminal Court reveals that the case’s timeline, a country’s governance structure, and whether or not the country is an ICC... (more)
- Hanni Maoz: Evaluation of the Court Performance: A Critical View of the International Criminal Court Indicators for Security Introduction In November 2015, the International Criminal Court (the ICC or the Court) released Report of the Court on the development of performance indicators for the International Criminal Court.1 This First Report contained qualitative and quantitative indicators that were supposed to allow... (more)
- Sebastian Barrios... Evaluating the performance of the ICC: the value of and challenges associated with measuring the expeditiousness of ICC’s proceedings I. Introduction One of the most persistent and often repeated criticisms of all international criminal tribunals has been that they cost too much money and take too long.1 After having been in operation for more than ten years, the International Criminal Court... (more)
- kbanafshe: Evaluating the Performance of the Court: To What Extent have the Actions of the ICC Transformed the Politics and Fostered Peace Within Central and Eastern Africa? The International Criminal Court (“ICC” or “the Court”) was established in 1998 to “exercise its jurisdiction over the most serious crimes of international concern.”1 The main mandate of the court is to bring wrongdoers to justice, however... (more)
- Mehrunisa Ranjh: Measuring Victim Access to the International Criminal Court: Peace as the Ultimate Goal of International Justice Introduction The creation of the International Criminal Court (“ICC”) was an attempt to traverse previously uncharted territory by setting up a permanent, truly international court that would remain impartial, expeditious, and transparent in the face of huge and often conflicting external pressures. The court was envisioned... (more)
- emrenslo: The Performance Indicator “Expeditiousness of Proceedings” Will Cause Unintended Negative Consequences Using “Expeditiousness of Proceedings” as a performance indicator, as currently envisioned by the ICC, has unintended negative consequences for the prosecutorial, judicial, and management aspects of the Court. I. Introduction The ICC released a... (more)
- pgsojo: Measuring Performance on Arrests and Visualizing a More Effective Way in Procuring Them Performance indicators to evaluate the success or failure of the International Criminal Court (ICC) in terms of its mandate of arresting suspects. It is clear how the world has always been in desperate need of a worldwide ruling court, where criminals of great atrocities would face justice internationally known and accepted. The ICC was... (more)
- aaacosta09: Measuring Performance: A Case Study of Positive Complementarity Catalyzed During the Preliminary Examination Stage in Colombia An effective way to evaluate the performance of the ICC is to measure the positive complementarity catalyzed by the Court during the preliminary examination stage in Colombia. Introduction The International Criminal Court (“ICC” or “the Court”) was... (more)
Comment on the Performance Question: “The ICC has established four key goals regarding, broadly, its proceedings, leadership, witness security, and victim access. What are the appropriate ways to measure the ICC’s progress towards those stated goals? How can the performance of the ICC as a whole be properly assessed?”
The uniqueness among the Judicial Systems of ICC is a challenge for the application of statistics to measurement about its activities. This is for two reasons: the absence of a reasonable term of comparison; the self-referentiality of the system of measurement. Both the features provide the occasion for a reflection about the meaning of measurement of an object that is strictly qualitative and hold an ontology bereft of correspondent foreseeable phenomenology for the singularity of a judicial job. Afterwards, the suggestion for applying a methodology consistent with the object and its purposes will answer to the questions posed by the second report.
Defining the object
The measurement the Law is the operation that overcomes the set of abstract concepts linked to a system of rules and principles. Therefore, the observable object becomes the enforcement of the Law and not the Law, inasmuch as the complexity of the interrelations, the causation links and the unforeseeable feedback among Law and many aspects of the society produce a manifold cognitive map with unknown outcome, usually not observable. Implementing the scheme suggested by Tremper et al. 2003, the observational process can be divided into three logical stages: Design, Testing, Production.
The first stage consists in the following procedures:
The second stage consists in the following procedures:
The third stage consists in the following procedures
The entire process is characterized by iterative steps and circularity; the goals are tuning in accordance to the evidences. These features could complicate the execution and cause a significant increase in the budget. Nonetheless the scheme is overall a guideline to provide a fitted benchmark for building a general system of measurement applied to the judicial issues whether for the appraisal of its effectiveness or for the analysis of the state of the law enforcement.
Measuring the work of ICC
The system of measurement of the ICC’s performance is the answer to the request of the 13th session of the International Criminal Court’s Assembly of States Parties, Resolution 5 (December 17, 2014), Annex I, para. 7 (available online). To invite the Court to follow the evidence-base policy is inspired by the suggestion to overcome the policy based on the mixture between theory, intuition, ideology and conventional wisdom (Banks, 2009). Actually, to apply the evidence-base policy to the management of the Court is also an ideology, because the worth of the empirical demonstration is equivalent to the other considerations both theoretical and operative (Davies, 2004). Furthermore, the application to the international penal court puts a challenge to the method chose, because the jurisprudential methodology is typically deductive, while the evidence-base policy has a strong inductive feature that moves toward an inferential approach when probability is introduced. Nevertheless it is important to remember the role of judicial statistics and nomostatistics in the history of the management of the state’s functions and the statistics themselves.
The citation stresses two concepts: the single case is not object of statistics; statistics speak with the functions of data, but they must be a model for interpreting the meaning of these functions. Moreover, the problematic application of inferential model to justice issues depends on the difficult to build samples and panels really representative and respectful to the rules of sampling theory. Nonetheless the application of the probability to the evaluation of the probation in a process has carried out some significant hallmarks in the conceptualization of the uncertainty and its mathematical use (Hacking, 1990; Hald, 2004; Hacking, 2006; Sheynin, 2012), causing an important debate like that one about the Poisson distribution and the Law of Large Numbers.
Unfortunately, for the most part of these contributions are not useful for this measurement of ICC’s activities, currently an experience unique in the history of the law enforcement. The most disruptive consequence is the lack of information that is useful for building reliable statistics. In this way the preliminary study conducted by the Open Society Justice Initiative that provided several recommendations, often shared, has avoided an empirical approach that is useful to show what kind of the indicators could be better to adopt. For example, the smart advice to distinguish three categories of indicators (operational, Rome-statute, impact) would need a suggestion about the methodology of aggregation in accordance with the established set of meaningful measurement. The invitation to the creativity and the vision is associated with a form of self-consciousness that is more rhetorical than operative. Nonetheless the difficulties to draw a data generator process in this specific case has forced to exclude to create a specific codebook and a correspondent protocol. Starting from the first goal, the correspondent legal measures are counting and debatable when the report tries to provide a metric for the proper Law concepts like fair, whereas the choice made could be confuted. To the offset vagueness, the first goal is pursued detecting a large list of measures in correspondence to each stage of the trial. Calculating the indicators suggested in the second report, the attempt to obtain holistic indicators is really unreachable.
The second goal has set up measures closer to an effective indicator; instead, the third goal required a resume of two dimensions: Physical and asset security and Information security. Concerning the fourth goal the most effective methodology of measure is a survey of the victim population (UNODC-UNECE, 2010); nonetheless the context in which the victims acts may advise setting up a group of indirect measures through the evaluation of the degree of access to the Court. It is relevant to stress the strong will to guarantee a reliable measure of the role of ICC in relationship with one of the main reasons that caused the birth of the Court: to give justice to the people oppressed by Regimes or victims of Deprivation, Terror and War committed by States. Anyway the involvement of the victims needs a further effort to build a reliable system of information about its progressive development.
The entire proposed system of measurements shows the incapacity to provide a clear synthesis of the performance of ICC respect to each goal and the overall outcome attained. The tables in the annexes which lists the values of the indicators of the second report are remarkable, nonetheless the accuracy of the details does not help to formulate an assessment about the performance of Court regarding each goal. Moreover, the comparison among the different trials is out of reach, because the indicators do not have a range of variation.
How to improve the system?
It is historical evidence (Meitzen, 1891; Robert, 1977; Hacking, 1990; Mo, 1995; Rousseaux et al., 1998; Tixhon, 1999; Perelli D’Argenzio M. P., 2002; Bingham, 2006; Hacking 2006; Bolaños Guerra, 2011; Palermo, 2011; Zabell, 2011; ISTAT, 2013; Sheynin, 2012, 2014) that the use of judicial statistics was one of the most important driving the development of the statistical thinking and delivering the easiest way to measure actions (including their effects), events (including their likelihood) and outcomes proper of the law enforcement thorough the enumeration of the proceedings and crime events: a census methodology.
In the preface of the very famous book by Duff, How to Lie with Statistics, the author speaks about a father-in-law’s observation about the increased level of crime moving from Iowa to California after reading the newspaper as an informal statistical statement, pointing out the newspaper as his universe of reference for extracting the representative sample. Nowadays, how much is important to proceed in the use of a census approach when it is not possible to identify correctly the population or when the population is too small (less of 50 cases) is forgotten. Sir Kendall’s argument about the statistics totally different respect to “Political arithmetic” as the technique of counting “complete enumeration… to be a record of a situation” (Kendall, 1960), has convinced most of statistician to leave the search of measures of the Res Publica, including the Law enforcement, to the scholars of social and political sciences rather than methodological fellows. For this reason, in social context, the statistical mainstream is a sample anyway the universe at the instant it is not properly specified or fewer than 50 independent observable units . Statistics without inference is not acceptable and every social scientist knows it. And the system of ICC’s measurement seems to confirm this argument: the absence of indices statistically concerns statistically significant produces a list of counts, merely descriptive and bereft operational availment.
Nonetheless, the focus on the development of Court-wide performance indicators is an imperative of the normal activities of ICC and the proposal of the second report has to use a statistical answer in continuity with the heritage of nomostatistics (Jacobi, 1935; Jaffin 1939), because Statistics is not only the operative branch of the inductive methodology, but the mathematical language for understanding the reality and the count data are the first method of measurement. What Is To Be Done?
The suggestion is “data speak themselves”, insomuch that guides to a metric fitted to the features of the ICC. For the nature of the information the choice is the use of a ranking function applying to the frequency. This function classifies each measure respect to the whole universe detected. Through the rank these statistics classify the outcome of each counting out of the search of the range of variation, which is not possible to define for the most part of the measure proposed. Each measure looses the dimension of the absolute value for acquiring the dimension of position respect to own performance in ordinal graduation. In this way, the work accomplished by the Court will produce the position and the comparison has full meaning, whereas this operation is inner bench-marking, coherent with the uniqueness of the ICC.
Afterwards, the ranks respect to a different dimension of each goal could be gathered in a composite index could be built using a correspondent system of weight based of the order of the ranking, when it has a meaning. In the case where there is no precise meaning, the jurisprudential relations are taken into account with measures that have a weight and can be associated with the same weight. Other possibility we could apply a Delphi method for the estimation of the weight vector.
The purpose of the composite index causes the choice of the aggregation method: if they give more preference to a specific dimension than others the geometric method is better than linear one (OECD, 2008). As the composite index is also a rank, these indicators provide the performance of each dimension to everyone who wants to know how the Court runs. Moreover, the new suggested indicators may be displayed on the web site through a dashboard.
The measurement of the performance of ICC is harsh challenge for the statistician, because the pillars of the statistical job are very fragile. Nevertheless the need to measure the performance of the Court is a priority for its success. The proposed system is not able to provide an outstanding picture. The suggestion to pass a system based on the rank function could permit stakeholders to realize a dashboard in the web site.
Armatte M. (2011) L’observation scientifique est-elle un témoignage? Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online
Banks, G. (2009), “Evidence-based policy making: What is it? How do we get it?”, ANU Public Lecture Series, ANZSOG, Canberra, February
Bingham Nicholas H. (2006) Heroic periods, Mathématiques et sciences humaines [En ligne], 176 | Hiver 2006, mis en ligne le 28 juillet 2006, consulté le 05 décembre 2016; DOI: 10.4000/msh.3641 : available online
Bolaños Guerra B. (2011) Condorcet et l’épistémologie juridique. Lettres à Turgot de 1771—Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online
Davies, P. (2004), “Is Evidence-Based Government Possible?”, Jerry Lee Lecture 2004, 4th Annual Campbell Collaboration Colloquium, Washington D.C., 19 February
Hacking, Ian (2006) The Emergence of Probability Second Edition Cambridge University Press 1975–2006 ISBN 0-521-86655-3
Hacking, Ian (1990) The Taming of Chance Cambridge University Press 1990 ISBN 0 521380146 hard covers ISBN 0 521 38884 8 paperback
Hald, Anders (2004) A History of Parametric Statistical Inference from Bernoulli to Fisher, 1713 to 1935 Department of Applied Mathematics and Statistics—University of Copenhagen—ISBN 87-7834-628-2
Huff, Darrell (1954) How to lie with Statistics—Pictures by IRVING GELS w W’ NORTON & COMPANY· INC· New York, 1954 ISBN 0 393 05264 8
Kendall, M. G. (1960) Where shall the history of statistics begin? Biometrika, 47, 447–49 (1960)
ISTAT (2013) Il percorso storico della statistica nell’Italia unita Atti del workshop—Roma, 7 giugno 2011 ISBN 978-88-458-1734-2 (print) ISBN 978-88-458-1760-1 (digital)
Jacoby, Sidney B. (1939) Some Realism about Judicial Statistics Virginia Law Review Vol. 25, No. 5 (Mar. 1939), pp. 528–558, available online
Jaffin, George H. (1935) Prologue to the Nomostatistics Columbia Law Review Vol. 35, No. 1 (Jan. 1935), pp. 1–32, available online
Meitzen, August (1891) History, theory, and technique of statistics—Part 1 History of Statistics translated by Roland P. Falkner American Academy of Political and Social Science—available online
Mo, Y. (1995) History of China Statistics. Statistical Research (5) 68–76
National Academies of Sciences, Engineering, and Medicine. (2016). Modernizing Crime Statistics—Report 1: Defining and Classifying Crime. Washington, DC: The National Academies Press. Doi:10.17226/23492.
OECD (2008) Handbook on Constructing Composite Indicators: Methodology and User Guide ISBN 978-92-64-04345-9—© OECD 2008
Palermo, A. (2011) Logique juridique et logique probabiliste chez F. M. Pagano Electronic Journal for History of Probability and Statistics Vol 7, n°1; Juin/June 2011 available online
Perelli, D’Argenzio M. P. (2002) Storia della Statistica. I momenti decisivi in AA.VV. L’insegnamento della Matematica e delle Scienze Integrate Vol.25a-B N.6 Novembre–Dicembre 2002
Robert Philippe (1977). Les statistiques criminelles et la recherche. Réflexions conceptuelles. In: Déviance et société. 1977—Vol. 1—N°1. pp. 3–27; doi : 10.3406/ds.1977.935 available online
Rousseaux, Xavier; Stevens, Frédéric; Tixhon, Axel. Les origines de la statistique pénale en Belgique (1795–1835). In: Déviance et société. 1998—Vol. 22—N°2. pp. 127–153; doi : 10.3406/ds.1998.1658 available online
Sheynin, Oscar (2012) History of Statistics Berlin, 2012 ISBN 978-3-942944-20-5
Sheynin, Oscar (2014) (compiled and translated by) Studies in the History of Statistics and Probability Collected Translations: P. R. Montmort, Nic. Bernoulli, P. S. Laplace, S. D. Poisson and others vol. 4 Privately printed
Tixhon, Axel (1999) Contrôler la Justice, construire l’Etat et surveiller le crime au XIXe siècle. Naissance et développement de la statistique judiciaire en Belgique (1795–1901). In: Revue belge de philologie et d’histoire, tome 77, fasc. 4, 1999. Histoire medievale, moderne et contemporaine—Middeleeuwse, moderne en hedendaagse geschiedenis. pp. 965–1001; doi : 10.3406/rbph.1999.4394 available online
Tremper, Charles & Thomas, Sue & Wagenaar, Alexander. (2010). Measuring Law for Evaluation Research Evaluation review 34. 242–66. DOI 10.1177/0193841X10370018.
UNODC-UNECE United Nations Office on Drugs and Crime—United Nations Economic Commission for Europe (2010) Manual On Victimization Surveys—Geneva, 2010
Zabell, Sandy L. (2011) The Subjective and the Objective in Handbook of Philosophy of Science Volume 7 Philosophy of Statistics Edited by Bandyopadhyay Prasanta S. and Forster Malcom R. North Holland Elsevier B.V. 2011, ISBN 978-0-444-51862-0
International Criminal Court, Report of the Court on the Development of Performance Indicators for the International Criminal Court (Nov. 12, 2015), available online, archived.
International Criminal Court, Second Court’s Report on the Development of Performance Indicators for the International Criminal Court (Nov. 11, 2016), available online, archived.
International Criminal Court, Strategic Plan 2013–2017 (Jul. 24, 2015), available online.
Open Society Justice Initiative, Briefing Paper: Establishing Performance Indicators for the International Criminal Court (Nov. 2015), available online, archived.
How can the Performance of the ICC be Properly Assessed?, ICC Forum (Jul. 10, 2017), available online.