The Importance of Perception

Comparative studies of courts internationally have shown that key performance indicators are useful in tracking a court’s performance.5 However, metrics developed to measure court performance often rely strictly on easily quantifiable indicators, which primarily focus on the efficiency of the court’s operation and neglect to account for less quantifiable variables to performance assessment (such as the perception of the judicial system in question). The ICC’s 2015 report, too, focuses almost exclusively on quantifiable indicators that measure operational efficiency to evaluate the Court’s progress. However, as noted by Dr. Edgardo Buscaglia and Maria Dakolias, Lead Counsel in the World Bank’s Legal Vice Presidency, in their study of performance indicators throughout different national court systems, measuring efficiency is a practical device for tracking court performance because it provides a quantifiable means of measuring performance.6 “However, justice has many other dimensions, and judicial reforms often strive for less quantifiable changes…It is important to keep this distinction in mind, because factors that affect efficiency do not always improve the overall quality of justice, and in some cases can even undermine it.”7 It is important, then, for the Court to recognize that an assessment of its performance as a judicial organization hinges on more than just qualitative outputs in its designated key performance areas. The Court must also keep track of how it is externally perceived, by those who use it, in order to develop a more complete evaluation of its performance.8

The Nebulous Key Performance Areas

Several of the key indicators proposed by the Court in its 2015 report may open themselves up to criticism, as well as skepticism, about their ability to be quantifiably measured. For instance, the Court has suggested that KPIs will be used to measure the fairness of the Court’s proceedings. Two metrics the Court has suggested as variables for measuring fairness would be: 1) the percentage of findings by Chambers confirming fair trial violations pursuant to the motions of the parties, and 2) the number of successfully argued appeals on grounds of fair trial violations.9 Further quantifiable measures could come in the form of the percentage of judicial findings confirming complaints by the parties to the Court, in regards to service delivery,10 or how many sanctions the Office of the Prosecutor has received for misconduct in its trials pursuant to Article 71 of the Rome Statute.11

However, a few possible issues arise from measuring the Court’s fairness in this manner. First, the proposed metrics only relate to procedural fairness once prosecutorial investigations have already begun. Much of the criticism the Court faces today in regards to fairness goes beyond whether the Court has followed the rules of litigation, but rather point to perceived unfairness in the Court before cases are even investigated. The Court has been accused of racial prejudice by States Parties in its selection of cases due to the fact that every defendant who has faced prosecution in the ICC has been of African descent.12 This assessment of the Court’s performance was used as the reason for several of the States Parties to withdraw, or threaten withdrawal, from the Court.13 Should, then, the ethnicity of the defendants be considered a key performance indicator of the Court’s fairness? Moreover, would such a metric actually be a quantifiable indication of whether the Court has been fair considering that many argue that the Court has justifiable grounds for its focus on the prosecution of African leaders?14

Furthermore, there are even less quantifiable aspects of fairness in regards to the Court’s performance when viewed from the differing cultural perspectives of States Parties. Though how a court is perceived is “essential to judicial reform efforts…it may be more difficult to compare on an international basis given that there are cultural and institutional differences.”15 The leaders of the Northern Ugandan tribes, for example, disapproved of the Court’s indictment of Joseph Kony for his massacres of thousands, because the cultural tradition of their people requires that all disputes end in reconciliation between the guilty party and those he has wronged.16 In effect, for some States Parties, entering a trial with intention to bestow punishment upon the defendant if he is found guilty could be viewed as unfair. These perceived differences in the concept of fairness could lead some States Parties to develop a negative perception of the Court resulting in a lack of trust and, thereby, decreasing support for the ICC.

Another key performance area that the Court has deemed to be critical to its success is the effectiveness of the Court’s leadership and management. While the Court has suggested two metrics as a starting-point for measurement (gender and regional balance of the staff and the annual rate of approved budget implementation by organ),17 these metrics fail to account for the complexities of what makes the Court’s leadership effective. The Court’s leadership affects the overall performance of the Court not only by its ability to generate productivity and efficiency within the court, but also by their choices outside of the walls of the ICC. Any misbehavior, or accusations of misbehavior, by an organization’s leader, even if it occurs in a situation wholly unrelated to the activities of the Court, can, in turn, diminish the Court’s reputation and the trust the States Parties have for the ICC, which should be reflected in an assessment of the overall effectiveness of the Court’s leadership. Furthermore, it is possible that a leader of an organization may be incredibly effective although objective measures of their performance, in regards to quantifiable outputs may indicate otherwise. For instance, if a new chief prosecutor entered the ICC following a predecessor who had an unsuccessful time in office, he may be inclined to bring on new staff members with whom he has previously worked and who are respected for the quality of their work. Assuming that the new prosecutor is familiar with the quality of these individuals’ work because they practiced together, they would likely be from the same region. So although the net quality of the Court’s work product could rise, this would reflect as ineffective performance by the Court’s leadership, since the regional diversity of the Court’s staff has decreased. Meanwhile, the overall perception of the Court’s leadership amongst the States Parties could be rising, thereby resulting in the Court’s performance indicators being an inaccurate portrayal of the Court’s actual performance.

In consideration of these nebulous interpretations of certain KPIs, having some indicator of the States Parties’ perceptions of the Court may give a more complete evaluation of the Court’s performance. The Court could accomplish this by either creating proxy measures to track the level of satisfaction the States Parties feel about the Court’s performance or by establishing a performance evaluation that directly monitors the States Parties’ assessment of the Court’s performance.

Proxy measures of the Court’s performance are closer in line with the current metrics the Court has suggested to evaluate its performance. The Court could create substitute variables that represent the States Parties opinions as to the Court’s performance. For example, the Court could track the number of withdrawals, and threats of withdrawal, which could then be used as indicators of the States Parties dissatisfaction with the Court. The benefits of proxy variables are that they allow for the Court to have easily quantifiable metrics. However, this comment will discuss the possible drawbacks of relying purely on quantitative data points while measuring general perceptions of the Court’s performance.

A second suggested method would be for the Court to apply a multi-method approach where, along with the proposed KPIs, the Court could collect performance evaluations from the States Parties. Performance evaluations could be given in several different ways, however, this comment will focus on the benefits of using an anonymous binary questionnaire that directly asks the States Parties how they would rate the Court’s performance in each proposed key performance area. There are several benefits to using the binary questionnaire method including: low cost of implementation, simplicity of use, the ability to give each respondent an equally valued voice in their opinion of the Court, and many other ancillary benefits to the Court. The possible drawback to this method is that some States Parties may, for a number of reasons, provide dishonest feedback, thereby leaving the Court with inaccurate information about how it is perceived by the States Parties.

Proxy Measures of Satisfaction

Proxy measures could be established that would serve as indicators of States Parties’ satisfaction or dissatisfaction with the Court’s performance. One possible proxy for satisfaction could come from monitoring how many States Parties have threatened withdrawal during a given period. So far, several States Parties that have withdrawn, or threatened withdrawal, have cited unfair treatment by the Court as well as conflicting interests between the Rome Statute System and the State Party’s domestic laws.18 Since the desire to withdraw from the ICC is an indication of dissatisfaction with some aspect of the Court and its performance, withdrawals and threats of withdrawal could act as an indicator of negative perceptions of the Court by the States Parties. Moreover, the Court could compare the withdrawal rates with the Court’s proposed KPIs and determine if there is any correlation to the withdrawals and underperformance by the Court in a key performance area. If causation of dissatisfaction can be linked to the Court’s underperformance, the Court will have a better opportunity to understand what areas it should focus on improving and develop strategies to fix insufficiencies that negatively impact perceptions of the Court.

Another proxy for satisfaction of States Parties is the repeat customer metric. In the private sector, a business can measure customer satisfaction by tracking the number of repeat customers it has had over a given period.19 This quantifies the degree of satisfaction by capturing the number of people who do business with the organization more than once on the assumption that if a consumer is dissatisfied with the organization’s performance, they would not use the service.20 The Court could emulate this method of measurement by tracking the number of referrals given by a State Party after a previous referral the State Party has made has come to completion either through trial or dismissal. If a State Party continues to use the Court’s services after it has seen the outcome of a prior referral, the Court could view this as positive feedback about its performance from the State Party.

A third possible proxy for satisfaction with the Court’s performance could be a measurement of positive and negative reaction to the activities of the Court and its organs through public communications made by States Parties. For instance, if a particular action by the Office of the Prosecutor spurs debate within a country’s parliament, and the debate results in a consensus that the State Party strictly opposes the activity, the Court can view this as a negative evaluation of its performance. Moreover, a group discussion between States Parties, such as those held by the African Union, could be seen as both positive and negative evaluations of the Court, with each State Party that supports withdrawal being a negative review and each State Party that opposes withdrawal being a positive review. The Court could also monitor speeches, interviews, and other communications made by States Parties and use positive and negative statements as proxies for the States Parties’ level of satisfaction.

There are, however, several downsides to relying on proxy measures of satisfaction. Proxy measures inherently force the Court to rely on measurement by inference, which inevitably avails itself to misinterpretation of the collected data.

Using withdrawal as a proxy measure of satisfaction with the Court’s performance, for instance, only gives voice to the States Parties that are dissatisfied with the Court. As of December 2016, there are one hundred and twenty-four States Parties. To date, three states have withdrawn and several other African countries, most notably Uganda and Kenya, have threatened withdrawal.21 Although withdrawals and threats of withdrawals are never insignificant, the vast majority of the States Parties have not vocalized their dissatisfaction with the Court’s performance. It would also be incorrect to interpret their silence as approval of the Court’s performance, since they may be dissatisfied with the Court but not voice their disapproval. Therefore, the withdrawal proxy would give disproportionate weight to the performance evaluation of the Court to only the most dissatisfied States Parties.

A similar critique could be made about the use of publicized opinions as proxies of the Court’s performance. Again, the performance of the Court would be evaluated by only the most vocal members of the Assembly of States Parties. Moreover, it would be incredibly costly (in terms of resources and man power) for the Court to monitor all publicized references to the Court. Considering the plethora of communication methods and mediums where a State Party could vocalize its opinion (e.g. social media, televised news reports, news publications, blogs, and podcasts), the Court would likely need dedicated staff specifically to locate and translate these communications; such costs could be seen as outweighing the benefit of collecting the information.

There is also an issue in determining how much value should be given to each publicized opinion of the Court. For instance, if a State Party restates the same complaint against the Court through multiple different mediums, should the Court weigh each complaint as having the same value in its metrics? Moreover, this could create perverse incentives for disgruntled States Parties to create false complaints in order to artificially lower the generally perceived satisfaction level with the Court.

Lastly, although repeated use of the Court could be an indication that a State Party is satisfied with the Court’s performance, there are two major drawbacks to this proxy. First, because the Court handles such a small number of cases, the length of time before any meaningful measurement can be made would be quite long. This means that the Court would not gain any immediate insights into the perception of its performance by using the repeat customer method. Second, repeated use of a service is traditionally applicable as a proxy measure of satisfaction where there are competing organizations to where a user can turn.22 The ICC, however, is a court of last resort.23 Because cases are referred to the ICC only when a national judicial system will not investigate a case, a referring State Party may be using the Court’s services in spite of its dissatisfaction with the Court’s performance, because it has no alternative. Satisfaction with the Court’s performance, therefore, may not necessarily be correlated with repeated usage of the Court’s services.

The Binary Questionnaire

There are other alternatives to the proxy measure method that the Court could use in order to gain an understanding of how the States Parties feel about the Court’s performance. Performance evaluations have been the standard in many organizations for stakeholders to evaluate their level of satisfaction with the activities of the organizations in both private and public institutions. In the United States, several states use questionnaires for judicial performance evaluations.24 Questionnaires provide a quick and cost effective method of collecting data; given the size and hectic schedules of the representatives of the States Parties, a questionnaire survey would be an efficient mechanism for gaining a collective and individual insight into the States Parties opinions about the performance and direction of the Court.

Moreover, using an anonymous binary response questionnaire to gauge whether a State Party is satisfied with the Court’s performance regarding key areas would provide immediate feedback to the Court regarding how its performance is viewed. An example of how these binary questionnaires would work in practice would be as follows: each State Party would be asked whether they found the Court’s proceedings to be fair (a key performance indicator of the Court). The State Party would then be allowed to choose either a yes or no response. The State Party would then be asked about their opinion on whether they believe the Court’s proceedings have been transparent (another key performance indicator of the Court), and again, the State Party would be allowed to answer in a yes or no response. The questionnaire would continue in this manner until the Court’s performance in each of the key performance areas deemed critical to the Court has been assessed by the State Party.

The binary performance measurement model has several advantages. First, because of its simplicity, it would not require a large investment, either in the form of capital or resources for the Court to develop and implement.

Moreover, a recent comparison of binary and, the more traditional, Likert-scale version of performance evaluations surveys (where participants are asked to rank the performance of an organization by their level of approval through either a numerical scale, e.g. 1 through 5; or level of satisfaction, e.g. strongly disagree to strongly agree) showed that binary models did not decrease the validity of the survey.25 The binary questionnaire method also eliminates some of the causes for distortion in the respondents answers by eliminating possible respondent biases such as central tendency bias26 (where respondents choose the median value, e.g. neither agree nor disagree, when presented with a scale ranging from strongly disagree to strongly agree, in order to avoid showing extremist views), because the respondents would only be given a yes or no response to choose from. Furthermore, the anonymity of the responses would allow States Parties to give more candid feedback, thereby lessening the chances of social desirability bias27 (where respondents choose to portray themselves in a manner that they believe will reflect them in either a more or less favorable manner to the examiner, and in society, rather than giving a truthful response). Questionnaires also give each State Party an equal voice, since each respondent’s opinion would hold the same relative value as all other respondent thus solving the issue of disproportionate attention being given to only the most vocal States Parties.

The States Parties should also be given the option to provide written feedback to explain their responses to the questionnaire. Doing this would provide two benefits: the Court would be able to better understand why States Parties may have a positive perception of the Court’s performance in certain areas and why they may have negative feelings about the Court’s performance in others. For example, assume a hypothetical situation where a significant number of State Parties may believe that the leadership and management of the Court are ineffective because some of the leaders chosen by the Court were found to have conducted themselves inappropriately while intoxicated during a vacation. Because, in this example, the actions by the Court’s leadership were taken outside of the organization, and do not directly relate to the Court’s operational efficiency, the Court may not initially have thought to, or have been able to, measure the performance of management outside of the Court through a KPI. But such behavior would nonetheless be a valid reflection of whether the Court’s leadership is effective, since impropriety, even outside of the organization, by the Court’s figureheads could negatively impact the legitimacy of the court and in turn hinder the Court’s performance. Moreover, written feedback from the States Parties would give the Court the ability to quickly troubleshoot inefficiencies in its organization and solve the problem (e.g. by either reprimanding the Court’s leaders or dismissing them for their behavior outside of the Court). The Court could also track changes of the States Parties attitudes and confidence in the Court over time by comparing and contrasting current findings with their previous research.28 Giving the Court an opportunity to measure the impact of new strategies or policy changes by the Court on the State’s Parties perceptions of the Court’s performance.

An additional benefit of giving the States Parties an opportunity to provide feedback in their performance evaluations of the Court is that States Parties who may otherwise not have vocalized any opinion with regards to the Court’s performance may take the opportunity to provide valuable feedback. The court can use the feedback it has gathered to evaluate the adequacy of its current KPIs, since a significant discrepancy between the States Parties and the Court regarding how the Court has performed in a key area would allow the Court to assess the efficacy of the existing KPI. Moreover, if there is overwhelming negative feedback in regards to a certain key performance area, the Court could consider developing a KPI that correlates specifically to that particular negative feedback in its internal evaluation. For instance, if there is widespread complaint that the Court has been unfairly targeting the leaders of a particular geographic region in its prosecutions, the Court could create a metric which evaluates whether the percentage of cases that have gone into investigation, by geography, in relation to all geographic locations where the Court has been referred (by the States Parties, the UN Security Council, or through the Office of the Prosecutor). The Court could then evaluate whether there has been a disproportionate amount of activity within a specific geographic area, which could be an indicator of unfairness in the Court.

The binary questionnaire, however, is not completely without pitfalls. First, a small but notable issue is that, in order to get a correct representation of how the Assembly of States Parties feels, as a collective, about the Court’s performance, each State Party must respond to the questionnaire. The court would, therefore, need to create some incentive to make sure that all States Parties participate in the performance evaluation.

A second, larger issue, is that arguments may arise as to the ability of the Court to keep the survey truly anonymous, which in turn could lead States Parties to be hesitant in providing candid feedback to the Court.

In light of recent leaks of classified government documents,29 private e-mails of political leaders being released,30 and other threats to anonymity and privacy, there have been growing concerns internationally about the ability to keep any collection of data secure. Some experts in information security have gone on the record to express that anything connected to the internet can be hacked.31 It is not without merit, then, to say that the previously mentioned value provided to the Court by States Parties offering anonymous feedback through binary questionnaires may be curtailed by fears that a State Party’s opinions could be traced back to the responding State Party, thereby disincentivizing honest feedback and giving reason for States Parties to behave strategically in evaluating the Court’s performance rather than candidly. It is possible, for instance, that out of fear that their opinions of the Court’s performance may become public, a State Party who actually believes that the ICC has done an insufficient job in offering adequate security for those at risk from involvement with the Court,32 may indicate otherwise in its performance evaluation of the Court in order to appease allies within the Assembly of States Parties.

There are, of course, measures that can be taken to create deterrence for both the States Parties or the Court from trying to unveil the anonymity of the performance evaluations. First, the Court could implement data encryption into the questionnaires in order to raise the cost for would be hackers who attempt to gain access into the performance evaluation. Although data encryption technology is not an impregnable defense against hacking, when looking at the vigorous attempts by many countries to block encryption of data and their requests for backdoor access into encrypted data services,33 this should give reason to believe that data encryption does pose some deterrent effect to countries who wish to easily access protected information. Therefore, so long as the cost of breaking the Court’s encryptions are sufficiently high, it could be a reasonable deterrent against unveiling the anonymity of the questionnaires.

The Court could also try using offline methods in order to collect the performance evaluations rather than having performance evaluations done online. Of course, just because information is not digital does not mean that it is secure. Certainly, physical copies of documents are just as vulnerable to theft as digital copies. However, the fact that an actor would have to physically engage in theft, rather than being able to anonymously steal the information through the internet, may be a sufficient deterrent as well.

Additionally, the Court could create criminal sanctions, which the Court itself would also be susceptible to (since the Court should be unaware of the source of the performance evaluation in order to preserve anonymity for the States Parties), for any actor who has either engaged or attempted to engage in determining how States Parties evaluated the performance of the Court. For States Parties, the cost-benefit of trying to gain information about how other States Parties evaluated the Court, in relation to opening themselves up to criminal proceedings, would likely favor not partaking in such behavior. For the Court, being caught behaving in a criminal manner would likely lead to a complete loss of credibility and legitimacy in the eyes of the international community (since its purpose is to punish criminal conduct) and such an action could possibly be a death knell for the Court itself, which should act a sufficient deterrent.

Implementing encryption services or using offline data collection methods, however, would raise the costs of the performance evaluations since encryption services would need to either be developed or purchased, and trying to collect physical performance evaluations from one hundred and twenty-four world leaders would take a tremendous amount of man power and time, both of which are at a premium for the ICC. Moreover, criminal sanctions would not be a sufficient deterrent for non-States Parties who are outside of the Court’s jurisdiction. These non-States Parties, including signatories, may have independent motivation to unveil the anonymity of the Court’s performance evaluations or be working alongside a State Party.

However, the possibility of the States Parties being motivated to behave strategically, rather than candidly, in their evaluations is likely not fatal to the idea of using a performance evaluation. Although the complex nature of realpolitik and the rising level of distrust and unhappiness with the ICC among several States Parties could lead one to believe that some actor(s) could use a State Party’s opinions of the Court’s performance against a State Party or the Court, an unscrupulous actor could just as easily hack into the e-mail accounts of other States Parties and use them for the same unethical purpose. If the Court is dissuaded from attempting new strategies that may benefit the ICC because of fears that it may be used against them, this would drastically reduce the amount of things the Court could do to improve its performance.

Even if the States Parties’ responses were marred by some form of social desirability bias, because they fear that their evaluations may be unveiled to the public, by and large, States Parties would be maintaining the same opinion of the Court that they currently publicly hold. Meanwhile, the feedback given to the Court by the States Parties would still be valuable to the Court, because, so long as the Court considers and responds to the assessment of the States Parties for where they would like to see improved performance, the Court is doing its part to work towards the success of the Rome Statute System and the States Parties would have less grounds to criticize the performance of the Court.

Finally, considering the efforts taken by the Assembly of States Parties to create the ICC, the vast majority of States Parties should be sufficiently motivated to behave in a manner that will help the Court succeed, and the States Parties should be trusted to act in good faith in their evaluation of the Court. Given that the Court is a manifestation of the States Parties’ collective desire to enforce international criminal justice and human rights, it is ultimately in the hands of the Assembly of the States Parties to nurture the continued growth and prosperity of the ICC. As previously discussed, attempts to hinder the progress of the Court can be made regardless of whether the Court’s performance evaluations are used against the Court’s interests, but, as the primary stakeholders in the ICC, disingenuous critiques of the Court ultimately hurt the States Parties the most. In the end, the Court, is to some degree, always at the mercy of the willingness of the States Parties to earnestly work towards the Court’s success (since the Court only has jurisdiction where the States Parties allow it). Therefore, trusting the Assembly of States Parties to evaluate the Court’s performance in an honest manner would most likely result in a net benefit to the Court, which outweighs not implementing it due to feelings of trepidation that anonymity may not be guaranteed.

Conclusion

The efforts by the ICC and the Assembly of States Parties to track the Court’s performance through qualitative and quantitative methods will benefit the Court greatly. It provides the Court with an opportunity to collect data about the Court’s operational efficiency, which in the future could be of great value to the Court in assessing its performance in relation to past performances in key performance areas. However, there are aspects to a judicial institution’s performance that are not strictly quantifiable. This is the nature of an organization whose primary objective is based on an ideology of subjective concepts, such as justice and fairness. The suggested metrics would, therefore, be greatly enhanced by including some form of measurement that tracks the States Parties perceptions of the Court. Measuring States Parties’ perceptions would provide the Court with an ability to see where the Court’s assessment of its own performance varies from that of the States Parties. The Court can then make efforts to appropriately align the States Parties views with that of the Court when needed.

This is not a recommendation for the Court to forgo its principle of being an autonomous organization, operating without political influence, but rather a note that the Court is still a partner to the Assembly of the States Parties. Like in any partnership, success depends upon periodic discussions of how satisfied each partner is. Forgetting to account for this crucial aspect of the Court’s partnership with the Assembly of States Parties would be a significant lapse.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   International Criminal Court, Report of the Court on the Development of Performance Indicators for the International Criminal Court, 2 (Nov. 12, 2015), [hereinafter First Report], available online, archived. ↩

2. 2.

   Id. at 1. ↩

3. 3.

   Id. ↩

4. 4.

   Maria Dakolias, Court Performance Around the World: A Comparative Perspective, 1 Yale Hum. Rts. & Dev. L.J. 87, 90–93 (1999), available online.

   (using public confidence in a court system as a key factor for evaluating the performance of the court). ↩

5. 5.

   See generally Edgardo Buscaglia & Maria Dakolias, Comparative International Study of Court Performance Indicators: a Descriptive and Analytical Account, Research Paper, Legal Department, The World Bank (Aug. 1999), available online, archived.

   (contending that qualitative surveys of judicial systems through performance indicators can help courts resolve cases in a fair and timely manner). ↩

6. 6.

   Id. at 2. ↩

7. 7.

   Id. ↩

8. 8.

   Edgardo Buscaglia & Maria Dakolias, Judicial Reform in Latin American Courts: The Experience in Argentina and Ecuador, Technical Paper, The World Bank (1996), available online.

   (noting that, generally, surveys and polls are needed in order to assess a population’s overall confidence in the judicial system). ↩

9. 9.

   First Report, supra note 1, at 5. ↩

10. 10.

    Id. ↩

11. 11.

    See Rome Statute of the International Criminal Court, Adopted by the United Nations Diplomatic Conference of Plenipotentiaries on the Establishment of an International Criminal Court, Jul. 17 1998, UN Doc. A/CONF.183/9 [hereinafter Rome Statute], Art. 71, available online. ↩

12. 12.

    See Chief Charles Achaleke Taku, International Politics and Policy Considerations for the Inappropriate Targeting of Africa by the ICC OTP, in Contemporary Issues Facing the International Criminal Court 338, 339 (Richard H. Steinberg ed., 2016), earlier version (Mar. 17, 2013) available online.

    (contending that the ICC’s exclusive focus on Africa is an indicator that the Court has become a neo-colonial weapon to silence opposition). ↩

13. 13.

    See, e.g., Sewell Chan & Marlise Simons, South Africa to Withdraw From International Criminal Court, N.Y. Times, Oct. 21, 2016, available online. ↩

14. 14.

    See, e.g., Margaret M. deGuzman, Is the ICC Targeting Africa Inappropriately? A Moral, Legal, and Sociological Assessment, in Contemporary Issues Facing the International Criminal Court 333, 334–337 (Richard H. Steinberg ed., 2016), earlier version (Mar. 17, 2013) available online.

    (addressing the fact that, as of 2016, the ICC has prosecuted six cases in the African continent with four being referred by the states in question, two by the UN security council, and only one case being opened by the Office of the Prosecutor). ↩

15. 15.

    Dakolias, supra note 4, at 92. ↩

16. 16.

    Benjy Steinberg, The Prosecutor and the Paramount Chief, Vimeo (Mar. 26, 2015), available online. ↩

17. 17.

    First Report, supra note 1, at 6. ↩

18. 18.

    See, e.g., Chan & Simons, supra note 13. ↩

19. 19.

    See Clinton W. Brownley, Multi-Objective Decision Analysis: Managing Trade-Offs and Uncertainty (2013). ↩

20. 20.

    Id. ↩

21. 21.

    See Somini Sengupta, As 3 African Nations Vow to Exit, International Court Faces Its Own Trial, N.Y. Times, Oct. 26, 2016, available online. ↩

22. 22.

    See Sadia Samar Ali & Rameshwar Dubey, Redefining Retailer’s Satisfaction Index: A Case of FMCG Market in India, 133 Procedia 280, 283 (2014), available online.

    (stating that repeat business is a viable method of measuring satisfaction of consumers so long as the consumer have a choice to go somewhere else). ↩

23. 23.

    International Criminal Court, Encyclo. Britannica, available online (last visited Dec. 13, 2016). ↩

24. 24.

    See, e.g., Judicial Performance Evaluation, NCSC (2004), available online. ↩

25. 25.

    Mario Grassi et al., Performance Comparison of Likert and Binary Formats of SF-36 Version 1.6 Across ECRHS II Adults Populations, 10 Value in Health 478 (2007) available online.

    (noting that a comparison of a binary and a Likert-scale version of a standardized health surveys led to the conclusion that replacing multi-category answer options with binary options did not decrease the validity or the component structure of the test, but the change did significantly reduce the time required to complete the questions, thus making it a better suited methodology for surveys administered in a clinical setting). ↩

26. 26.

    Central Tendency Bias, Oxford Reference, available online (last visited Dec. 11, 2016). ↩

27. 27.

    See Derek L. Phillips & Kevin J. Clancy, Some Effects of “Social Desirability” in Survey Studies, 77 Am. J. Sociology 921, 922 (Mar. 1972), University of Chicago paywall. ↩

28. 28.

    American Bar Association, Perceptions of the U.S. Justice System 11 (1999), available online.

    (following a survey of the American public’s perception of the U.S. judicial system, the American Bar Association specified several attitudinal areas that correlated strongly with confidence in the Court’s system, and should therefore be maintained, with other areas in which respondents showed a more negative attitude, where the judicial system was advised to find ways to improve its performance). ↩

29. 29.

    See, e.g., Kenneth Roth & Salil Shetty, Pardon Edward Snowden, N.Y. Times, Sep. 15, 2016, available online.

    (discussing the leaks of Edward Snowden regarding high-tech surveillance by the U.S. and the resulting discussions at the United Nations regarding the protection of digital privacy). ↩

30. 30.

    See, e.g., Sam Frizell, What Leaked Emails Reveal About Hillary Clinton’s Campaign, Time Magazine, Oct. 7, 2016, available online.

    (discussing the impacts of hacker group Wikileaks’ release of the private e-mails of Hillary Clinton’s campaign chairman, John Podesta, and the subsequent accusations by the White House that the Russian government had been involved with the hacking of the e-mails). ↩

31. 31.

    TED-Ed, All your devices can be hacked—Avi Rubin, YouTube (Jun. 12, 2013), available online; See also Bruce Schneier, Could Your Plane be Hacked?, CNN, Apr. 16, 2015, available online

    (stating that all computer systems are vulnerable because the engineering expertise to design and build perfectly secure computers and networks do not currently exist). ↩

32. 32.

    First Report supra note 1, at 2.

    (stating that protection of those at risk from their involvement with the court is one of the four key goals that are critical for assessing the performance of the ICC). ↩

33. 33.

    See e.g., Andrea Peterson, The Debate Over Government ‘Backdoors’ into Encryption isn’t Just Happening in the U.S., Wash. Post, Jan. 11, 2016, available online. ↩