Digitalization improves public sector performance. But to what extent does it also enhance performance measurement? To what extent can digitalization give us what we need to properly measure government performance?
These are questions that I will try to answer in this and two subsequent blog pieces. But before doing so, let’s be clear about what we are asking. “Digitalization” means the use of digital tools to support or automate government business processes, including service delivery. (This is what is widely referred to as GovTech, although this much-abused term is now used in so many other ways that it is arguably better to avoid it entirely.) When business processes are digitalized, administrative data – that is, data on activities carried out and other information routinely collected during service delivery – is collected in digital form. Once in digital form, it can be readily drawn upon for performance measurement purposes. There is at present enormous enthusiasm in some quarters for “government analytics” based principally on this data, and this makes it important to be clear about both its potential and limits.
The biggest gains for performance measurement arising from digitalization lie in the area of output indicators and intermediate service indicators, including indicators of the volume of services delivered to citizens and the time taken to deliver services. This is obvious when we remind ourselves that “business processes” refer to the processes by which government organizations transform inputs (labor, equipment etc) into intermediate services (services such as payments processing and procurement which support government operations) and then into outputs (services to or for citizens, such as health treatments and environmental protection interventions).
As important as outputs and intermediate services are, the most important dimension of government performance is outcomes. Outcomes are about the effectiveness of government services – the extent to which they achieve goals such as saving lives and improving education and employment levels. Here, the potential contribution of digitalization to performance measurement is more limited. This is because information about many of the outcomes which government seek to achieve is not – and in many cases cannot be – collected as part of the service delivery process and is therefore not part of the administrative data to which digitalization provides enhanced access. For example, information about the effectiveness of preventative health public information campaigns – such as their impact in reducing rates of smoking – cannot by definition be collected as part of the process of delivering these campaigns. Similarly, it is not possible as part of the process of organizing export promotion activities to collect information about the increased exports which result from those activities. Such information has in both cases to be collected separately and at a later stage.
Nevertheless, digitalization can make certain significant contributions to the measurement of the effectiveness of government. Some of these contributions pertain to measuring outcomes, and some to the measurement of output quality.
In a Latin American country that I was advising recently, the health ministry publishes, as its sole indicator of antenatal care services, a measure of the number of women who received a minimum of four antenatal checkups prior to the birth of their child. This is an output quantity indicator, and a very useful one. However, it says nothing about the effectiveness of the service, and this led me to recommend the parallel development of effectiveness indicators. In this case, when the birth occurs – generally in hospital – key outcome information is routinely recorded (birth weight, specific health problems affecting the child, and delivery complications). By matching that outcome information to the output information – something which in this case would require making two separate databases (one maintained by the antenatal care service, the other by the hospitals) communicate with one another – it is possible to derive highly meaningful outcome indicators for this important service. This is precisely the type of process that digitalization facilitates.
The general point this example highlights is that in the case of government services which are delivered to clients who directly and immediately benefit from them, outcomes are in some cases recorded as part of the administrative data. In such cases digitalization directly helps improve outcome measurement. It is important that such opportunities to use digitalization to improve outcome indicators are exploited to the full.
Nevertheless, this only gets us so far in measuring outcomes. It remains the case that many important outcomes are not measured as part of the administrative data of government organizations. This means that for the purposes of outcome measurement, it is necessary to go well beyond administrative data. Surveys (e.g. to measure post-graduation employment rates of university students), physical sampling (e.g. air-quality and atmospheric CO2 levels), testing (e.g. PISA education level indicators) and other methods of obtaining outcome data are all essential.
There is, however, one other major contribution that digitalized administrative data can make with respect to effectiveness. This is in helping measure service quality. I will turn to this topic in the next blog piece.