Optimism, pessimism, and chaos worldwide: A comparative study of Energy Management Performance with robust benchmarking methods

06/26/2025

The assessment of national Energy Management Performance (EMP) is a critical yet complex endeavor, often yielding varied and sensitive results—a phenomenon this thesis conceptualizes as methodological 'chaos'. This research confronts this challenge by developing and applying an integrated, multi-method framework to robustly evaluate the EMP of 174 countries (c. 2022), moving beyond single-score assessments. The core approach utilizes Benefit-of-the-Doubt (BoD) modeling, extended with Hit-and-Run (H&R) sampling for weight sensitivity analysis and complemented by outlier-robust Order-m efficiency models (both unconditional and conditional). Initially, the study established the performance spectrum using optimistic and pessimistic BoD models, revealing significant performance specialization and a strong dependence of country rankings on the evaluative perspective. The subsequent H&R analysis quantified a substantial optimism bias inherent in standard BoD scores, providing more stable, weight-robust performance averages. Order-m models were employed to investigate outlier sensitivity and contextual influences. This revealed that outlier-robust efficiency scores (Order-m) often diverged considerably from both standard BoD and H&R-adjusted scores, indicating that different robustness techniques capture distinct facets of performance stability. Finally, comparative analyses were conducted across methodologies. These highlighted that sensitivity to weighting and sensitivity to model form/outliers are largely uncorrelated dimensions of instability. A key contribution was the identification of two primary EMP archetypes—"Pessimistically Robust, Lower Performers" and "Optimistically Advantaged, Higher Performers"—through cluster analysis of multi-method performance profiles. This research underscores that a comprehensive understanding of EMP necessitates a multi-perspective approach, leveraging diverse robust methods to navigate methodological variability and uncover meaningful performance patterns.

Authors in the community:

• 

  F

  Filipe Ribeiro de Almeida Farinhas

  ist187504

Supervisors of this institution:

• 

  D

  Diogo Filipe da Cunha Ferreira

  ist158548

environmental-engineering - Environmental engineering

por - Portuguese

Instituto Superior Técnico