Is Guessing Really Not Judging? A Reply to Eidenmüller and Hochgürtel

Horst Eidenmüller and Anna-Sophie Hochgürtel have recently written a post on this blog critiquing decentralised arbitration protocols. In their post, the co-authors argued that decisions produced by these systems reward participants for predicting what the crowd will decide rather than for reasoning toward the most justified answer, and should therefore not be treated as arbitral awards. We are grateful to the authors for their thoughtful critique. However, their argument rests on a few assumptions, which in what follows we hope to clarify. In this post, we argue that the dichotomies the critique relies on (no focal point versus a focal point, convergence versus deliberation, and guessing versus judging) are better viewed as existing on continua, particularly once the Schelling point is understood as a shared strategy of honest evaluation rather than an obvious verdict. 

The Focal Point Is a Strategy, Not an Outcome

In a well-known passage from The Strategy of Conflict (1981), Thomas Schelling argued that strangers would usually choose noon at the information booth in Grand Central Station as the most common meeting point in New York City. Nothing about this choice made it objectively superior to any other; it worked only because everyone expected everyone else to guess it too. This is what game theorists call a focal point.

Eidenmüller and Hochgürtel assume that for a Schelling point to work, a winning verdict must stand out the same way. In a genuinely hard dispute, no outcome ever does. However, this misidentifies the coordination problem crowdsourced systems ask their evaluators to solve.

Jurors do not vote in a vacuum; rather, they rely on a shared body of information. This includes rules established by the parties beforehand (‘dispute policies’); rules enacted by the Kleros community to structure general reasoning across different case types (‘court policies’); and the evidence submitted by the parties. This shared informational framework anchors the juror community in situations where common norms might otherwise be lacking.

What has to be true for the mechanism to work is not that a uniquely salient honest outcome exists. Rather, where a legal or contractual standard is provided, that standard acts as a shared informational anchor that makes honest, standard-applying judgment the focal strategy, even if individual assessments of the facts carry noise.

Persuading, Not Just Counting

The authors celebrate traditional tribunals for exchanging arguments and deliberating through reasons, contrasting this with decentralised justice systems where ‘votes are counted rather than arguments weighed’. They then argue that the justifications provided by sole-arbitrator tribunals provide a function that is similar to deliberation. 

Kleros jurors are also invited to provide justifications with their votes. In the event of an appeal, the appellate jurors see these justifications. As the incentives for the juror in the initial round depend on how the appellate jurors decide, early-stage jurors have an incentive to provide justifications that will persuade the appellate evaluators. Indeed, in addition to the parties to the dispute, the jurors themselves can also trigger appeals if they feel that the panel they were on incorrectly ruled against their position.

Is Guessing Different from Judging?

Eidenmüller and Hochgürtel’s critique ultimately seems to rely on a strict ontological divide: a decision is either ‘pure judging’ or ‘pure guessing’. But adjudication in the real world does not operate as a binary switch. It exists along a continuum.

At one extreme sits pure guessing: flipping a coin, picking numbers out of a hat, or speculating on a Keynesian beauty contest with zero external reference points. At the opposite extreme sits the idealized archetype of pure judging: Ronald Dworkin’s mythical Judge Hercules, a superhuman adjudicator, capable of discovering the single, perfectly seamless legal answer in every hard case.

Neither extreme describes real-world dispute resolution. To measure decentralised justice against Judge Hercules is to commit a classic Nirvana Fallacy: comparing a practical, real-world mechanism to an impossible philosophical ideal. 

Traditional courts and arbitral tribunals themselves operate squarely in the middle of this spectrum. Indeed, legal systems explicitly acknowledge the impossibility of absolute certainty by relying on probabilistic decision-making heuristics and standards of proof such as ‘preponderance of the evidence’ or ‘beyond a reasonable doubt’. Adjudicators are required not to discover absolute metaphysical truth, but to evaluate whether a claim crosses a given threshold of probability under incomplete information, cognitive biases, and practical (even budgetary) realities.

Kleros simply sits at a different point along that same continuum. When anchored by a clear contractual policy, evidence, and appellate risk, a crowdsourced system operates far from the ‘pure guessing’ pole. Moreover, the point that Kleros 1.0 occupies on this spectrum is not fixed by intrinsic limitations of decentralised justice systems. The increased modularity of Kleros 2.0, which is currently deployed in a beta version, allows for more complex dispute mechanisms. For example, a mechanism like the following:

• Jurors are required by court policies to provide a written justification for their vote. 
• Jurors receive rewards regardless of how they vote and are not penalised for votes that disagree with the majority.
• If another participant (such as a party to the dispute, an appellate juror, or a third party) reads the juror’s justification and deems it to be inadequate, that participant can place a deposit to ‘challenge’ the justification. 
• This challenge creates a secondary dispute to judge the quality of the juror’s justification. The jurors in this secondary dispute are incentivized via a Schelling point mechanism, where they are rewarded or penalised based on whether their vote agrees with the majority of jurors, possibly after appeal. 
• If the secondary dispute rules that the justification was inadequate, the challenger receives a reward drawn from a deposit lost by the juror who wrote that justification. 

Where would this system fall in the ontological dichotomy between ‘guessing’ and ‘judging’, if at all? The dynamics of the underlying dispute perhaps seem more familiar in a context of traditional arbitration. However, there are still game-theoretic aspects here as jurors need to anticipate what justifications will be found acceptable by the community. We would argue that this approach would be yet another point on the spectrum. 

The Historical Transition of Dispute Resolution

If arbitration is defined strictly through the twentieth-century paradigm codified by the 1958 New York Convention, then crowdsourced mechanisms indeed fit uncomfortably within the category. 

But we need to keep in mind that the framework was designed in a post-war environment specifically to address disputes arising from large-scale commercial transactions and cross-border investments between states and multinational corporations. Today, however, we inhabit a vastly different economic landscape transformed by internet technology and artificial intelligence. For specific categories of digital disputes where conventional arbitration is economically unviable, decentralised justice models offer a scalable and accessible solution.

Whether that counts as ‘judging’ in some philosophically pristine sense is, we suspect, the wrong question. The one that matters is whether the mechanism is reasoned enough, consistent enough, and resistant enough to manipulation to serve the disputes it was built for.

Federico Ast is the Founder and CEO of Kleros. 

William George is Research Lead at Kleros, focusing on game theory, mechanism design, and decentralised justice protocol architecture.

Facundo Trotz is an Attorney and Legal Researcher at Kleros, specializing in international arbitration, legaltech innovation, and decentralised dispute resolution frameworks.

The authors are members of the team at Kleros, an open-source protocol for decentralised dispute resolution discussed in this article.