BLOG

How a new hybrid methodology combining data science and expert consultation is transforming electricity regulatory assessment across the continent

Since 2018, the African Development Bank's Electricity Regulatory Index (ERI) has served as the definitive benchmark for assessing regulatory performance across the continent. Now, a comprehensive methodological review is underway to enhance its scientific rigor and policy relevance.

The ERI evaluates regulatory frameworks across three fundamental pillars: the Regulatory Governance Index (examining institutional and legal design), the Regulatory Substance Index (assessing concrete regulatory actions and decisions), and the Regulatory Outcome Index (measuring tangible effects on utilities and consumers). With 43 participating countries and nearly 270 questions, the index has become an essential tool for identifying regulatory gaps and recommending evidence-based solutions.

As ERI has grown in influence, so too has the importance of ensuring that its methodology remains transparent, robust, and fit for purpose. A comprehensive methodological review currently underway marks a significant step forward — strengthening ERI’s scientific foundations while preserving the credibility and comparability that policymakers and regulators rely on.

But the need for a critical methodological adjustment has been recognized to address skewed assumptions: while no explicit weights were assigned to any indicator, the unequal number of questions within each category inadvertently introduced implicit weights into the calculation—weights that may not necessarily reflect the actual significance of different regulatory dimensions.

BOX 1: WHY THIS MATTERS

The Electricity Regulatory Index (ERI) is used by governments, regulators, utilities, development partners and investors to understand regulatory performance and reform priorities across Africa. Strengthening the methodology behind ERI ensures that country scores are not only comparable but also credible, transparent, and grounded in both data and professional judgment.

The Challenge of Hidden Assumptions

Equal weighting—assigning identical importance to all indicators—is transparent and easy to implement. However, it embeds a strong assumption that all dimensions matter equally, which rarely holds in practice. When indicators are grouped into dimensions with varying numbers of components, equal weighting at the indicator level leads to unequal weighting at the dimension level, potentially distorting country rankings and policy priorities.

"We believe that regulatory indicators have varied significance and impact. To accurately reflect this in the ERI, explicit weights should be determined for the underlying indicators and sub-indices."

This recognition prompted a comprehensive review examining alternative scientific approaches to assigning appropriate weights—approaches that are both statistically acceptable and practically viable within the African regulatory context.

A Hybrid Solution: Data Meets Expertise

The solution lies in combining two complementary approaches. At the question level, Principal Component Analysis (PCA)—a statistical technique that identifies patterns of variance in data—can objectively determine which questions contribute unique information versus which are redundant. At higher aggregation levels, structured country expert consultation captures the normative priorities and domain knowledge that statistics alone cannot reveal.

56 Expert Participants  •  24 African Countries  •  35% Potential Question Reduction

The expert consultation process employed two rigorous methodologies: the Analytic Hierarchy Process (AHP), which breaks complex weighting decisions into manageable pairwise comparisons, and the Budget Allocation Process (BAP), where experts distribute 100 points across indicators based on their perceived importance. This dual approach ensures weights represent robust consensus rather than artifacts of any single elicitation technique.

Credibility Through Inclusive Process

What sets this methodology apart is its foundation in broad stakeholder engagement. The consultation brought together regulatory professionals from electricity authorities and utilities representing 24 countries across the continent, through both synchronous workshop sessions and asynchronous follow-up rounds.

Key Expert Priorities Revealed

• Independence emerged as the most valued dimension, increasing from 12.5% to 18.2% weight—reflecting experts' emphasis on regulatory autonomy
• Financial Independence and Decision-Making Independence were prioritized over procedural dimensions
• Quality of Service and Capacity Building gained importance within regulatory substance
• Experts systematically prioritized institutional fundamentals over procedural elements

This pattern reveals a conception of regulatory effectiveness centered on "getting the fundamentals right"—establishing autonomous, well-resourced institutions with clear legal mandates as prerequisites to effective implementation of specific policies or achievement of sector outcomes.

Stability with Refinement

A crucial finding: transitioning to the new methodology produces refinements rather than revolutionary changes to country rankings. Statistical correlation analysis shows rank-order agreement exceeding 97% between equal-weighted and expert-weighted approaches. Countries that perform well under the current system continue to do so under the new methodology.

This stability means the ERI's historical comparability is preserved while its scientific foundation is strengthened. The hybrid approach introduces meaningful differentiation where it matters most, while maintaining the fundamental structure of ERI that stakeholders trust.

Importantly, the methodological refinements preserve stability at both ends of the ranking. Top performers in 2024—including Senegal, Kenya, Uganda, Namibia, and Tanzania—maintain their leading positions within the High-Level category, while bottom performers such as Guinea and São Tomé and Príncipe show minimal variation within the Low Level. Changes concentrate primarily among middle-ranked countries, with a large majority of the shifts limited to one to three positions, while at the same time, most countries remain within their original regulatory development category. This pattern confirms that the new methodology enhances analytical precision without disrupting the fundamental hierarchy that stakeholders rely upon

Looking Forward

The transition from implicit to explicit weights represents a significant methodological advancement. By making hidden assumptions transparent and grounding them in both statistical evidence and professional consensus, the enhanced ERI will serve as an even more powerful instrument for promoting regulatory effectiveness across Africa's electricity sector.

The message is clear: robust methodology and inclusive stakeholder engagement aren't competing priorities—they're complementary foundations for credible, actionable regulatory assessment. As Africa's electricity sector evolves, so too must the tools we use to measure and improve it.