On February 14, 2019, the nonprofit Open AI announced that they had developed a new text-generating Artificial Intelligence (AI) system called GPT-2 that was capable of autonomously performing a range of natural language processing (NLP) tasks such as question and answering, summarization, translation, and perhaps most interestingly, text generation. In an effort to curb the spread of fake news, Open AI opted for a staged release of their GPT-2 model, noting at that time that “Due to our concerns about malicious applications of the technology, we are not releasing the trained model […] As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper”.

Eventually, the complete 1.5 billion parameter GPT-2 model was fully released in November 2019. In May 2020, Open AI announced the public beta release of the next generation GPT-3 model with the capacity of 175 billion machine learning parameters. Finally, earlier this year, Google Brain open-sourced their Switch Transformer, a natural-language processing AI model that scales up to 1.6-trillion parameters.

The unprecedented rate in which natural language processing technology has grown in the past few years poses various ethical, legal, and security questions related to misuse and accountability. This is a development that will most certainly shape the public discourse in the coming years. However, how can anti-corruption domain experts benefit from this technology today?

Freely available text-generating AI systems like the GPT-2, enable domain experts to create, scrutinize, and probe their own hyperrealities or, to use Umberto Eco’s terminology, authentic fakes, under controlled laboratory conditions [1]. Unlike traditional computer simulations that often involve complex statistical models, text-generating AI systems serve output formats which are human-readable.

Thus, we can foresee at least two elementary scenarios were text-generating AI can help anti-corruption experts. First, the expert can instruct the AI to unconditionally simulate authentic fakes. Second, the expert instructs the AI to generate conditional authentic fakes by manually providing prompts. In both scenarios, the AI will generate authentic narratives that can be used by the expert in downstream diagnostic tasks, such as corruption mapping and risk assessment. To put this into perspective, I have created a minimal web application where the user can prompt the GPT-2 model.

In the first example below, I have simply used the first 20 words from a 2012 article by Roberto Saviano in the New York Times to prompt the AI (prompt: The global financial crisis has been a blessing for organized crime. A series of recent scandals have exposed the connection). The synthetic text generated by the system is shown below the search bar (Fig. 1). Right straight out of the box, the AI demonstrated impressive cognitive ability to create text with plausible linkages between different actors and activities. The second example (Fig. 2) shows the results after prompting the system with a headline from a blog post by Sean McGoey.

The synthetic text generated in the examples follows a specific genre, reflecting the style of the original text data used during the training phase of the model. The GPT-2 model used in this public demo has not been fine-tuned on a dedicated corruption corpus that would have drastically increased its coherence of the output and the potential for misuse. In a production environment, anti-corruption teams can fine-tune the model by using original texts from their domain, and generate a large volume of synthetic reports that will enable them to explore new corruption risk terrains and previously unobserved linages.

Figure 1. Text generation example from a New York times article.

Figure 2. Text generation example from a ICIJ blog post.

Endnotes:

[1] Travels in Hyperreality is an anthology of essays written by Eco that uses the theme of “faith in fakes” as a tool to critique modern Western culture. Eco goes “in search of instances where the American imagination demands the real thing and, to attain it, must fabricate the absolute fake” (p. 17). In Eco’s terms, the “absolute fake” is a full-scale authentic copy of real cultural artifact, buildings, people, and texts. It is a form of hyperreality that people perceive as an improved copy of the actual or “real” things.

Panos Kostakos, RIOC member