Emerging Lessons From National Statistical Office Performance During The Covid-19 Pandemic
Written by Grant Cameron*
At the onset of the COVID-19 pandemic, Lauren Gardner, a professor of civil and system engineering at John Hopkins University and Ensheng Dong, her graduate student, created a dashboard and an interactive map to track the spread of the disease. Little did they know that when they launched it on January 22, it would quickly become the world’s most authoritative source on the latest coronavirus numbers and trends, with the dashboard’s site receiving upwards of 1.2 billion requests daily.
Source: JHU
The popularity of the John Hopkins’ dashboard underscores how the COVID-19 pandemic has boosted the demand for reliable, real-time statistics across all sectors. Such data has become invaluable for informing policymakers on the fast-developing situation and keeping the public, politicians, and other stakeholders abreast of evolving conditions. What made this dashboard successful was the team’s ability to determine needs, quickly launch the platform, provide an intuitive and compelling user experience, modify their offerings as the pandemic evolved, and draw data from existing trusted sources.
Many of the National Statistics Offices (NSOs) that have coped well during the pandemic share several of the same characteristics as the John Hopkins’ initiative. These NSOs recognized early on that the challenge of saving lives while fending off economic collapse put policymaking counterparts under incredible strain. Bureaucrats have never had to deploy interventions so rapidly, get instantaneous feedback to understand what is working (and what is not), and quickly adapt.
The speed and breadth of policymaking is an experiment in and of itself, and the breadth of the response has been remarkable. In a matter of weeks, many governments have expanded social protection schemes, deferred taxation, ramped up resources for public health, and provided loans and grants to large enterprises and small businesses. In pre-COVID times, these interventions would be developed over months to ensure cross-program coherence, benefit from rounds of stakeholder consultations, and be robustly validated. Developing these interventions has required existing data to be used in new ways and new statistical products and analysis to be developed rapidly.
Successful NSOs quickly brought their respective strengths to this experiment in real-time policy and program development. Policymakers relied on NSOs for expertise to deduce trends and discover new ways to measure the pandemic’s impact. NSOs have improved their visibility within the public domain through collaborations with health data systems, including producing weekly death statistics with minimal lag time, establishing COVID dashboards, and expanding visualizations and tools to present data in local areas.
What we have learned so far is that successful NSOs share many characteristics. For instance, their infrastructure and production processes were already in place to enable staff to continue working from home shortly after lockdowns began, and their prioritization of deliverables ensured that the essential statistics were compiled and released on schedule. The NSO leadership has also maintained a constant dialogue with the staff to ensure all employees feel connected and a part of the organization’s mission through the pandemic. Further, these NSOs stayed connected with policymaking processes to gauge shifting needs and adapt products to respond to new challenges. Close collaborations with other data providers allowed NSOs to provide real-time data to the fast-changing demands. For example, Colombia’s NSO, DANE, combined its census data with health system information to produce heat maps of vulnerable populations in Bogotá at fine levels of granularity.
Successful NSOs have also developed new approaches to gathering information on labor markets, businesses, and household sentiment with surveys that limit the number of questions to encourage participation and to expedite processing. Ex-post calibration using higher-quality data collected before the crisis allows new methods, like crowdsourcing, to contribute to understanding the state of the pandemic.
Unfortunately, there is evidence that a significant proportion of statistical offices in middle and low-income countries are struggling to be effective during the pandemic. Half of NSOs experienced funding cuts, many lacked basic internet infrastructure and home computing equipment, and about a third were unable to produce essential monthly and quarterly statistics and meet international reporting requirements. These challenges are likely to have grave effects on the data and statistical ecosystem, including derailments to core statistics products, such as household, business, and price surveys. As statistical production of these offices falls behind, it not only threatens their capacity to support policymakers, but their standing with the public is likely to suffer as well.
If “inequality” between successful and less successful NSOs increases, will it have important, but unpredictable effects on the global statistical eco-system? The potential exists for the “shining star” NSO examples to have a positive spillover effect for struggling NSOs. Observing the effectiveness of NSOs that have informed pandemic-fighting policy measures might encourage governments with less capable NSOs to increase funding, support, and their expectations for their national statistical system. But the opposite effect may occur. If an NSO’s capabilities gap becomes so glaring, politicians may decide to give up on their NSO altogether and rely solely on untested, often untrusted, alternatives.
However, one impact of the pandemic is predictable: a renewed international effort to provide guidance and recommendations to under-capacitated NSOs to be better prepared for the next crisis. However, it will be important to avoid, as the saying goes, “preparing to fight the last war.”
NSOs that have successfully managed the effects of the pandemic avoided “fighting the last war” by building on past reforms to modernize processes and capabilities to become more open, adaptive, and responsive. For example, many have taken steps to play the role of the Nation’s Data Steward: helping other ministries develop strategies to better manage their data, which has led to efficiencies, additional insights, and better decisions. The UNECE’s Modernization of Official Statistics’ work program has led the development of tools and guidance that have supported many of these reforms.
As the UNECE and others in the global statistical community develop new guidance, I believe three areas deserve special attention to further support NSOs’ modernization: (1) the development of a business continuity framework; (2) the documentation of successful processes and pathways for real-time NSO engagement with policymakers; and (3) strategies that reinforce modernization post-Covid-19. By focusing on these areas, it will enable NSOs and other government organizations to become more resilient and provide a foundation for less-developed NSOs to increase their effectiveness in future crises.
1. Develop a Business Continuity Framework for National Statistical Systems.
Experience-sharing on how successful NSOs developed resilient production processes, with an eye towards establishing a business continuity framework for official statistics, would benefit many NSOs who lack these capabilities. The framework should include a number of items, such as a description of how to establish priority products and services; identification of staff associated with producing these products and services; the NSO’s working culture, and staff’s adaptability to work from home and quickly develop new products (or scale pilot projects); a risk assessment of the data supply chains, working environments, and technical infrastructure capabilities; and a description of governance arrangements and accountabilities for business continuity operationalization. The framework should also provide guidance on how successful NSOs gathered information on their performance during the pandemic to plan improvements to processes, systems, and staff capabilities. Of course, this guidance should reflect the NSO-specific requirements within the broader context of overall government business continuity plans, where they exist.
2. Document pathways and processes of successful real-time NSO engagement with policymakers.
We are learning that successful NSOs were connected to whole-of-government processes responsible for coordinating and developing policies and programs. Senior NSO officials were in constant contact with high-level policymakers to identify new uses of existing data and identify innovations in new products that could be brought to scale. NSO crisis response teams worked with Health Ministries to develop real-time indicators on infections, death, and hospitalizations that addressed the needs of health policymakers, and informed Cabinet officials on daily developments. And these NSO-policymaker interactions were coordinated through a rigorous management process to ensure the NSO prioritized its work and provided consistent information. Framing these pathways, managerial and technical, into a generic structure/activity model (such as the Generic Activity Model for Statistical Organization) or assessment tools (such as PARIS21’s STEP) would facilitate replicating these channels in NSOs where engagement with policymakers remains weak.
3. Develop strategies to reinforce “modernization” principles when the crisis is over.
During Covid-19, the real-time engagement between successful NSOs and policymakers has created new experiences for official statisticians, and I argue that the immediacy between official statistics and decision-making has never been greater. Never have we seen so many statistical products and services been eagerly awaited, and many statisticians are likely seeing how their work directly impacts decision-making for the first time in their careers. NSOs can use this once-in-a-generation opportunity to develop strategies to further entrench their working culture, emphasizing user responsiveness and collaboration rather than only focusing on statistical production. This involves creating incentives for sustaining new partnerships (but also being cognizant of risks), creating secondment opportunities for official statisticians to work in line ministries, and developing a community of practices between statisticians and policymakers in data-intensive domains.
The impact of Covid-19 has been profound in all areas, but especially within the data and statistical ecosystem. New guidance and standards are necessary for meeting the evolving and growing demands posed on NSOs. Existing guidance – such as the UN’s Official Statistics Strategic Communications Framework, the Role of NSOs in Measuring Hazardous Events and Disasters – provide a starting point, but greater specificity on improving processes, pathways to user engagement, and fostering resilience in supplying products and services is necessary to meet user expectations in the years to come.
*About Grant Cameron
Grant Cameron is an independent consultant and most recently worked as a manager at the World Bank for 15 years. While there, he led teams supporting developing countries to improve their capacity to produce data and statistics and to make them more accessible and useful. Before joining the Bank, Grant worked for the Government of Canada in policy development (tax) and at Statistics Canada.