Lighting Up the Darknet

Data and Tools are Necessary to Interpret High Data Volumes

The judgement in United States v. Sterlingov recognized that software and tools are acceptable when interpreting high volumes of data like blockchain, described by the Court as an “overwhelming mass” where “no jury could possibly discern whether a particular darknet site… had made significant use of a bitcoin mixer without the use of a tool”.

However, while the judgment recognizes the necessity of tools, it also references the importance of being able to verify the outcomes:

“Given the volume of transactions recorded on the blockchain, investigators frequently make use of proprietary software… Much of this work could be done manually given enough time, and …  it is possible to corroborate (or to challenge) the results generated by the software for particular clusters with the public blockchain data, a pad of paper, a pencil, and hours of work.”

The significance of this statement is that the blockchain analytics software is not a “black box.” In the Bitcoin Fog case, the Court noted that the defense could have performed its own tracing to assess the accuracy of the clustering results generated by the software and to “corroborate or challenge” the prosecution’s evidence. Another word for this is “deterministic.” In the context of digital assets, this means that when the blockchain analytics software is run on a fixed data set, it will produce the same results every time. This contrasts with “black box” artificial intelligence models which can produce different outputs based on learning patterns and evolving parameters.

Decision-Making & Traceability

In the Bitcoin Fog case, blockchain analytics software was used to associate digital asset wallet addresses with the activity of eight darknet markets – as the sources of the illicit funds – and mixer Bitcoin Fog. The software provided detailed information on which “heuristics,” meaning rules or methods used to cluster addresses, had been used. The three heuristics used in the Bitcoin Fog were:

• Heuristic 1 uses “co-spending”, meaning when a transaction contains multiple input wallet addresses, it is inferred that they are owned by the same individual or entity (described in more detail above).
• Heuristic 2 uses observations on an entity’s on-chain behaviors and patterns including “peel chains” (described in more detail above).
• Heuristic 3 is not truly a heuristic and instead refers to data collected from sources such as court documents, data leaks, and information sharing by exchanges.

The software could provide detailed data on how these heuristics were applied to identify the darknet markets, for example:

• Sheep Market: 100% of clustering used heuristic 1
• Abraxas Market: 79.52% used heuristic 1 and 20.48% used heuristic 2
• Pandora Market: 78.49% used heuristic 1 and 21.51% used heuristic 2
• Alpha Bay: used all three heuristics

Finally, to identify the addresses associated with Bitcoin Fog, the software used heuristic 1 (50.26%) and heuristic 2 (49.74%) to cluster addresses.

This level of “traceability” – used in this context to mean the steps in making a decision – ensures that the outputs could be scrutinized, challenged, and verified.

Blockchain Analytics: A “Minor Witness”

In the Bitcoin Fog case, blockchain analytics were used to establish the “order of magnitude” of the illicit proceeds that passed through Bitcoin Fog. The software clustered and attributed over 900,000 addresses and traced receipt of approximately 1,284,251 bitcoin to Bitcoin Fog and withdrawals of approximately 1,280,935 bitcoin, representing approximately $400 million in receipts and a similar amount in withdrawals.

The blockchain analytics data was just one element of the evidence presented against Sterlingov. In addition, evidence included materials found on his person when he was arrested, online forum posts, analysis of IP addresses, and traditional manual blockchain tracing. Use of blockchain analytics in this context was described as a “minor witness” because the way blockchain analytics were used did not require “precise line drawing.”

The Court recognized other situations where the use of blockchain analytics may be approached differently, while this did not apply in the Bitcoin Fog case. Firstly, in Bitcoin Fog, blockchain analytics were not used to identify whether a single address, or small number of addresses, had been correctly attributed using the software. The judgement recognized that this would be “very different” from order of magnitude use.

Secondly, the analysis of the admissibility of blockchain analytics evidence took place during a “Daubert trial,” which means that the Court was deciding whether the blockchain analytics could be presented to a jury, where it could be challenged. This contrasts with a decision on whether the blockchain analytics software produced accurate results.

“This is not a case in which the government’s theory that Sterlingov was the operator of Bitcoin Fog turns exclusively, or even primarily, on Scholl and Bisbee’s [the government experts] use of the Reactor software.”

Validation of Outcomes

The investigators for Bitcoin Fog used Chainalysis’ “Reactor” as the blockchain analytics tool. The judgement recognized that other analytics tools, including those provided by Elliptic and TRM Labs, are comparable with Chainalysis in the provision of blockchain analytics.

In addition to the ability to validate the results, described above, the Court recognized that the investigators had actually validated the results using TRM Labs’ blockchain analytics tool and – for five addresses – by manual analysis of the blockchain “by hand.”

Actions for Chief Compliance Officers

What can Chief Compliance Officers learn from the use of blockchain analytics data in the Bitcoin Fog case, that they can apply within their department?

We’ve previously outlined the actions a Chief Compliance Officer can take to fulfil their responsibilities when using automation and AI. Now we can use the Bitcoin Fog case to add three more best practices, including detail on how they apply to blockchain analytics.

• Tools and technology are appropriate when interpreting large and complex datasets like blockchain – but governance is required. Use reliable tools, ensure teams understand their strengths and limitations, and use them accordingly.

Example – Digital Assets: For blockchain analytics decision support tools, use well-established and reputable software. Compliance teams could consider evaluating several tools in parallel to identify which ones produce the most reliable and comprehensive results for the business’ specific client base, before making a decision.

• Be prepared to provide detail on the data and methodology used to reach a decision. It will help if teams can manually demonstrate step-by-step how a specific conclusion was reached – even if this is not the business-as-usual process. It will be more straightforward to fulfil this requirement using “determinative” tools compared with “generative” tools.

Example – Digital Assets: The results of blockchain analytics tools can be replicated using manual tracing to demonstrate that the analytics software produce accurate results – at least for a sample.

• Higher “fidelity” is required for more specific decisions. Tools used to make decisions in more specific situations must meet more stringent requirements than those used for “order of magnitude” decision making.

Example – Digital Assets: If you use blockchain analytics to inform a decision to exit (or retain) a specific customer, you should rigorously validate the data and methodology – rather than simply rely on an automated system. You could consider corroborating the results using a different blockchain tool. This contrasts with use of blockchain analytics to identify the risk profile at the aggregated level, for example for client segments.

While the Court’s findings on the use of blockchain analytics data as evidence have limitations, their admissibility represents a milestone. More broadly than the judicial context, it demonstrates another step in blockchain and crypto becoming established elements of the financial system.

The defense had contended that blockchain analytics is “junk science” and was not the product of reliable principles and methods. Giving the Court the final word, it was “unpersuaded” on these points and instead it was “persuaded by ample corroborating evidence and testimony that [blockchain analytics tool] Reactor’s reliability has been established by a preponderance of the evidence.”