This document is a final methodological paper of WP2 of the DIGIWHIST project. It describes how the final database (DB henceforth) was developed starting with a high level description of each public procurement source that was processed, continuing with a description of the processes that led to the development of a structured database, followed by the processes involved in linking related data and creating a final database based on the
linked data. The last chapter contains the description of performance indicators (transparency, corruption risks and administrative capacity) and the conversion of the DIGIWHIST data template to the Open Contracting Data Standard (OCDS henceforth).

This methodology report describes the following steps in data processing:

● Data download – collection of HTML, XML, CSV and other content from government
sources
● Structuring data – conversion of each publication from its original format to a
uniform structured data template
● Formatting data – conversion of structured text to standard data types (numbers,
dates, enumeration values) including cleaning nonsensical values or ballast
information
● Linking related information – grouping information which describes one real world
tender together
● Data merging – putting information from all linked data records together to create
one final image of a public tender covering its whole tendering cycle

Within DIGIWHIST, 25 public procurement data sources were processed covering all 34
jurisdictions listed in the Grant Agreement. This total number consists of:

● TED
● 21 national web portals or open data sources
● Archives for UK, CZ
● Project partner’s DB of older Hungarian tenders

This paper describes the methodology and current output of the data validation process led by Datlab. The purpose of the process has been to check various properties of DIGIWHIST data on public procurement and provide continuous feedback to project partners. Thus the paper will:

1. Describe the validation methodology
2. Provide an overview of the work done
3. Summarize the current status of data quality

Datlab has used its extensive know-how in software development as well as public procurement data analytics to provide timely feedback to project partners. That led not only to a shift in data extraction strategy during the project, but more importantly towards transferring a large portion of procurement expertise towards technical staff. These steps have contributed largely to the good procurement data quality, which is paramount to the
project.

However, judging from the experience with Czech and Slovak data, it takes years of work to actually fine-tune data extraction even from a single procurement source. In fact, such a job is never finished because of amendments to the legal framework and consequent changes to the data structure and terminology. Based on this experience, DIGIWHIST’s goals are ambitious, even if we only aim to achieve moderate quality data within the project. Note that since the DIGIWHIST team aims to further improve the quality of the data during the sustainability period, the data quality results are to be taken rather as state-of-the-art. In fact, the methodology has been designed in such a way, that validation is possible on an ongoing basis. That implies, that updated versions of source-specific validation reports can be published upon major data releases.

The deliverable D5.4 presents the final EU dissemination conference in Brussels and the further international presence of DIGIWHIST researchers at events such as conferences, workshops, panel discussions and other formats throughout the project.

The deliverable D6.1 presents the Memorandum of Understanding on the Sustainability of the EU Horizon 2020 DIGIWHIST project (MoU).

In the framework of the Horizon 2020 Research Programme a consortium of six partner organizations that signed this MoU worked on the DIGIWHIST project (full title: “The Digital Whistleblower. Fiscal Transparency, Risk Assessment and Impact of Good Governance Policies Assessed” Grant Agreement n° 645852) from 01 March 2015 until 28 February 2018. The Consortium has conducted three years of research and tool development with the aim of increasing transparency and accountability in the field of public procurement in 35 jurisdictions (28 EU member states, Norway, the European Commission, Iceland, Switzerland, Serbia, Georgia and Armenia). The research results led to the creation of Data Collection Infrastructure and diverse tools (Opentender, EuroPAM, MET) for different actors and stakeholders interested in fair, transparent and efficient public procurement enabling the public to better monitor tenders and procedures.

The first objective of this MoU is to fulfil the DIGIWHIST grant agreement and create collaboration agreement among consortium partners in particular its clauses on maintaining the data collection infrastructure and the tools created as part of DIGIWHIST project (DIGIWHIST tools). The second objective of this MoU is to create a DIGIWHIST network on public procurement (hereinafter DIGIWHIST network) that will be based on the work conducted by DIGIWHIST project on Opentender.eu and will take into account its research results and important of continuing its sustainability. The DIGIWHIST network will include NGOs and stakeholders active in the field of public procurement.

The deliverable D6.2 presents the Memorandum of Understanding on the Establishment of the DIGIWHIST Network on Public Procurement (MoU).

“Concerned about the seriousness of threats posed by corruption in the public procurement sector to the stability and security of societies

Being aware that preventing and combating corruption in the area of public procurement can be substantially enhances by international co-operation through sharing knowledge and expertise

Having regard that fair, transparent and efficient public procurement enable the public to better monitor tenders and procedures

Reiterating the need to ensure transparency, accessibility, accountability, legitimacy, impartiality and integrity of public procurement data

Acknowledging Directive 2014/24/EU of the European Parliament and of the Council on public procurement

Acknowledging the General Data Protection Regulation (GDPR) (Regulation (EU) 2016/679) by which European Union (EU) institutions intend to strengthen and unify data protection for all individuals within the EU.

(…)

Based on the work conducted by DIGIWHIST project on Opentender.eu and taking into account its research results and importance of continuing its sustainability, the DIGIWHIST Consortium has decided to create a DIGIWHIST network on public procurement (hereinafter DIGIWHIST network) that will include organisations active in the field of public procurement.

DIGIWHIST network is determined to create a strong partnership that will generate synergies and enhance sharing of public procurement and expertise between the DIGIWHIST network and other stakeholders.”

The national workshops are part of the dissemination plan of the project Digiwhist. The workshops have been organised to disseminate the results, including the Digiwhist digital products, to the relevant stakeholders, namely

• EuroPAM
• Opentender
• MET

This deliverable D5.3 describes the scope and objectives of the workshops and includes the slides used during the workshops.

Monitoring European Tenders – MET (https://monitoringeutenders.eu) – is a risk assessment software for public authorities to assess the degree of integrity of European public procurement procedures. MET is based on data collected from the Tenders Electronic Daily (TED) for the period 2009-2017 and on the risk indicators developed by the project DIGIWHIST.
The aim of MET is to provide public authorities engaged in procurement activities with an easy-to-use software that can guide the users in the identification of risky contracts. Moreover, the MET software provides insights on the degree of integrity of the suppliers, based on their participation in opaque procedures and other characteristics at firm level.
To reach these goals, the software provides three main channels for data exploration. The first is the search by country. Data have been collected for 34 jurisdictions (28 EU Member States, Armenia, Switzerland, Georgia, Iceland, Norway and Serbia). The second way to explore data is on the basis of the actor (contracting authority or supplier) name. The third is by typing the tender id. In the following, more details are provided on the different data searching strategies.
Once accessed, the software provides a detailed tutorial introducing the exploration modes. Along with the tutorial, MET provides a glossary which defines the procurement indicators used (procurement integrity and administrative capacity indicators).
Besides static information, the software allows to customise the risk associated to each lot. The function allows to differentially weight indicators based on their relative importance attributed by the user.
After some filtering procedures, it is possible to download the selected data. This allows the users to integrate and process these data alongside with their own information.

Deliverable 4.3 “European transparency legislation observatory” or European Public Accountability Mechanisms (EuroPAM) is a website (http://europam.eu/) that presents the results of. D1.2 “Database of legal and regulatory norms” that was submitted by the Hertie School of Governance in the end of February 2016. EuroPAM is a database of legal and regulatory norms for 34 European countries.1 EuroPAM is an extension of the Public Accountability Mechanisms Initiative (PAM) of the World Bank, which is a primary data collection effort that produces assessments of in-law efforts to enhance the transparency of public administration and the accountability of public officials. The EuroPAM database serves as a European transparency legislation observatory that is based on the PAM indicators for financial disclosure, conflict of interest restrictions, and freedom of information, while also adding data on public procurement, and updating the International Institute for Democracy and Electoral Assistance (IDEA) database on political financing.

EuroPAM in-law data measures the comprehensiveness of a country’s legal framework in four spheres of administrative transparency and accountability: financial disclosure, conflict of interest, political finance, and freedom of information. Indicators for these mechanisms are based on internationally-accepted legal standards, established by organizations such as the World Bank, Article 19, Access Info Europe, Global Integrity, and the Institute for Democracy and Electoral Assistance. For public procurement, EuroPAM in-law data assesses both the extent of the procurement framework and its adherence to norms established by the European Commission.

To ensure the reliability of in-law data, a rigorous and systematic approach was applied to data collection and analysis. Researchers produced summaries of the legal provisions collected from primary source documents, in the original language where possible. In cases where further consultation was required to clarify legal codes, the data was sent to technical in-country experts for feedback on accuracy and relevance. The final data is released in both quantitative and qualitative form for policy and research purposes. Several rounds of data collection are envisioned from 2012 onwards. The exception to this is public procurement, for which data collection began in 2015.

On 29 January 2018 our DIGIWHIST Project Final Conference took place at the European Parliament (Altiero Spinelli building) in Brussels.

For everyone who missed the event or who wants to re-listen to the presentations and discussions here are the recordings of the event and the downloadable presentation slides of the respective conference speakers.

Part 1

Introduction:
● Benedek Jávor (MEP): Welcome from the host
● Ella McPherson (University of Cambridge): Welcome from DIGIWHIST

Session 1: Transparency and data availability in European public procurement. How far have we gotten and what needs to be done?
● Jiri Skuhrovec (DIGIWHIST): Data availability, transparency, and data
gaps. Presentation available here: Skuhrovec_session1_digiwhist final conf_180129
● Commentators: Gavin Hayman (Open Contracting Partnership),
Friedrich Lindenberg (OCCRP).
● Q&A

Session 2: Snapshot of government favoritism in European public procurement
● Mihaly Fazekas (DIGIWHIST): Risk assessment methods, risk areas and policy lessons. Presentation available here: Fazekas_session2_digiwhist final conf_180128
● Commentators: Ana Gomes (MEP), Frank Michlik (OLAF). Presentation available here: Michlik_20180129 Snapshot of government favoritism in European public procurement
● Q&A

Part 2

Session 3: How to reform to address problems, the interaction between legal arrangements and civil society activism (digital whistleblowing)
● Aram Khaghaghordyan (DIGIWHIST): Europam. Presentation available here: Khaghaghordyan_session3_EuroPAM_180122
● Mara Mendes (DIGIWHIST): opentender. Presentation available here: Mendes_session3_Opentender_180122
● Francesco Calderoni (DIGIWHIST): risk assessment software. Presentation available here: Calderoni_session3_METsoftware29012018
● Commentators: Gustavo Piga (University of Rome). Presentation available here: Piga_Session3, Carl Dolan (TI Europe)
● Q&A

Session 4: The EU’s role: What should Europe do next?
● Intro & moderation: Alina Mungiu-Pippidi (DIGIWHIST)
● Concluding panel: Monica Macovei (MEP), Pascal Boijmans (DG REGIO), Nikita Stampa (DG GROW),
Jean-François Junger (DG Connect)

This document has been prepared for the European Commission as part of a series of short papers on regional research and indicators produced by the Directorate-General for Regional Policy (WP 12/2017)

Summary

Based on previous DIGIWHIST research, we use a unique database of the EU-wide Tenders Electronic Daily (TED) which describes public procurement activities across the whole EU-28 between 2006-2015, through more than 4 million records. Each dimension of good governance as well as a composite score are calculated and their validity tested by comparing them to widely used regional indicators such as GDP/capita, European Quality of Government Index (EQI), or public service meritocracy. All tests confirm that the indicators proposed, based on prior academic and policy literature, are valid.

Public procurement plays a crucial role in economic development and the quality of government across the European Union (EU): on average, it amounts to about 13 % of GDP or 29 % of government spending (European Commission, 2016; OECD, 2015). It is a genuinely cross-cutting government function concerning virtually every public body, and is also one of the principal means by which governments can influence growth rates and the quality of public services. However, our understanding of the quality of public procurement processes and outcomes is very much in its infancy, which limits governments’ capacity to intervene in pursuance of
specific public procurement as well as broader developmental objectives.

In order to enhance prosperity, human well-being and the territorial cohesion of the EU, the quality of governance (or quality of institutions) is a fundamental precondition. High-quality institutions are characterised by “the absence of corruption, a workable approach to competition and procurement policy, an effective legal environment, and an independent and efficient judicial system”, as well as “strong institutional and administrative capacity, reducing the administrative burden and improving the quality of legislation” (European Commission, 2014, p. 161). Such a broad understanding of institutional quality is also underpinned by
influential academic thinking focusing on impartial policy implementation rather than the content of policies or democratic decisionmaking processes (Rothstein & Teorell, 2008). Building on this focus on policy implementation, good governance in public procurement is assessed according to four main dimensions:

• Transparency (e.g. amount of information published in procurement announcements);
• Competition (e.g. average number of bidders);
• Administrative efficiency (e.g. length of decision-making period); and
• Corruption (e.g. the use of non-open, opaque procedure types).

The new indicators enable a detailed analysis of the quality of NUTS 3 and NUTS 2 regional public procurement governance according to the four above-mentioned dimensions, while changes over the last 10 years can also be explored. We find a mixed picture of regional convergence between 2006-2015 in the EU. While some Central and Eastern European regions have converged to the EU average, many Mediterranean regions have strongly diverged and, surprisingly, some well-governed Western and Northern European regions have also experienced a strong deterioration in governance quality. Overall, governance quality and competition in particular have deteriorated across the whole EU.

Based on novel findings, a small number of tentative policy recommendations are proposed:

1. Increase competition in public procurement by encouraging market entry of both local and non-local firms; for example, through: a) better use of e-procurement and especially the complete implementation of various electronic tools, such as e-submission, e-invoicing, or e-contract monitoring; b) a more extensive use of central purchasing bodies as well as framework agreements for homogenous, standard goods; c) improving auction and tender design by better accommodating bidder characteristics, such as the needs of SMEs; and d) reducing bureaucratic controls on public procurement processes combined with better monitoring of outcomes or incentives for administrators better aligned with public goals (e.g. pay for performance).
2. Understand the broader political and institutional antecedents of governance decay and design tailored solutions; for example, through: a) increasing pay for civil servants and political office holders and improving meritocracy in the public service; b) improving political competition; and c) better regulating political finance, campaign contributions, and personal connections between bidding firms and political office holders.
3. Understand better the contribution of procurement governance quality to the effectiveness of EU funds and regional convergence to boost critical functions.
4. Improve data quality and availability to support wider data use in ongoing policy implementation and design, through:
   a) improving data scope and quality via better legislation as well as investment in IT systems; b) combining TED data with national public procurement datasets in cases where the latter are of sufficient scope and quality; and c) encouraging the regular use of public procurement analytics in EU and national policy implementation and design (e.g. Tableau).