Information about http://www.oecd.org/dataoecd/26/37/21687644.pdf
For Official Use …
Tags: cla, committees, comparability, ifie, international experts, oecd statistics, quality framework, reputation, ss, statistical activities, timeliness,Pages: 6
Language: english
Created: Tue Dec 16 11:56:17 2003
Display cached document
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
For Official Use STD/QFS(2003)1
Organisation de Coopération et de Développement Economiques
Organisation for Economic Co-operation and Development 17-Oct-2003
___________________________________________________________________________________________
English - Or. English
STATISTICS DIRECTORATE
For Official Use
STD/QFS(2003)1
d
ifie
QUALITY FRAMEWORK AND GUIDELINES FOR OECD STATISTICAL ACTIVITIES
Version 2003/1
ss
cla
De
English - Or. English
JT00151752
Document complet disponible sur OLIS dans son format d'origine
Complete document available on OLIS in its original format
STD/QFS(2003)1
FOREWORD
The OECD has a high reputation for both the quality of its analytical work and for the statistics that
underpin that work. In some areas, OECD statistics are internationally recognised as the "best" in terms of
coverage, timeliness, and comparability. The network of national and international experts created,
through the Committees and Working Parties, gives to the Organisation the possibility to launch research
activities (and data collections) on new topics, play a leading role in producing new statistics and develop
new definitions, classifications and standards.
Nevertheless, the improvement of quality is a continuous process and one of the main objectives of
the Statistics Strategy launched by the OECD in 2001 is the enhancement of the quality of OECD statistics.
All OECD statisticians devote a large part of their time to improve quality, but, as the experience of other
national and international organisations clearly has demonstrated, the adoption of a formalised approach to
quality for statistics can bring important benefits. First, it provides a systematic mechanism for ongoing
identification and resolution of quality problems, maximising the interaction between experts in different
fields (statisticians, IT experts, final users, etc.); second, it gives greatly increased transparency to the
processes used by the OECD to assure quality; and third, it reinforces the political role of the OECD in the
context of an information society, increasing the credibility of the Organisation as one of the "first class"
data providers.
It is important to note that to develop and implement a successful approach to quality, independently
from its technical characteristics, the human factor is absolutely crucial. From this point of view I would
like to stress how the OECD approach to quality in statistics has developed through the deep involvement
of statisticians working in all Directorates, and a process that has achieved, step by step, a growing
consensus on its characteristics, usefulness and importance. This process has also promoted a common
view among OECD statisticians of the dimensions of quality, the core values of OECD statisticians and
their role vis-à-vis stakeholders and civil society at large. A stronger sense of partnership between OECD
statisticians is already an important result of the "quality initiative".
As already said, the improvement of quality is a "never ending story", also partly because of the
continuous evolution of statistical and IT techniques. This means that the current version of the quality
framework and guidelines (version 2003/1) will necessarily be reviewed in the near future, as such
developments take place. The framework already contains a procedure for its periodic review and
updating, in particular, significant revisions are expected in 2004, after the conclusion of the first wave of
quality reviews and the full development of the new OECD statistical information system. The "dynamic"
nature of the document is an important characteristic, necessary to provide OECD statisticians with a
useful and up-to-date tool for improving the efficiency and effectiveness of the statistical activities they
manage.
Even though the development of the "Quality framework and guidelines for OECD statistical
activities" has been supported by the contribution of a number of OECD statisticians, I would like to
acknowledge the role played by M. Colledge, D. Ward and G. Salou, who have worked with me in
designing the system and writing this document. Special thanks also goes to E. Capponi, E. Espinasse, M.
Linster, P. Lübkert, P. Richardson and C. Webb, who took part in the task-force established to develop the
quality framework. A. Signora and E. Matthews contributed to the preparation and the finalisation of the
2
STD/QFS(2003)1
document. Important comments on an earlier version of the framework were provided by G. Brackstone
(Statistics Canada) and other national experts. Finally, I would like to thank Donald Johnston, the
Secretary General of the OECD, for his decision to give high priority to the Statistics Strategy and for his
continuous support to its implementation.
Enrico Giovannini
Chief Statistician of the OECD
September 2003
3
STD/QFS(2003)1
TABLE OF CONTENTS
INTRODUCTION 5
PART 1. QUALITY DIMENSIONS, CORE VALUES FOR OECD STATISTICS AND PROCEDURES
FOR PLANNING AND EVALUATING STATISTICAL ACTIVITIES 7
1.1 Definition and dimensions of data quality 7
1.2 Core values for OECD statistics 11
1.3 Procedures for assuring the quality of OECD statistical activities: an overview 13
1.4 The development and the update of Quality Guidelines 15
PART 2. QUALITY GUIDELINES FOR OECD STATISTICS 21
2.1 Introduction 21
2.2 Guidelines for different phases of statistical activities 22
2.2.1 Phase 1: Definition of data requirements in general terms 24
2.2.2 Phase 2: Evaluation of other data currently available and integration with other OECD
datasets 26
2.2.3 Phase 3: Design and planning of statistical activity 28
2.2.4 Phase 4: Usage of data from databases within and external to OECD 31
2.2.5 Phase 5: Implementation of specific data collection mechanism 35
2.2.6 Phase 6: Data and metadata verification, compilation, storage, analysis and evaluation 44
2.2.7 Phase 7: Data and metadata dissemination 50
PART 3. PROCEDURES AND CHECKLISTS FOR OECD STATISTICAL ACTIVITIES 56
3.1 Quality checklist for existing statistical activities 56
3.2 Quality checklist for new statistical activities 57
QUESTIONNAIRE 1 - QUALITY CHECKLIST FOR EXISTING OECD STATISTICAL ACTIVITIES 59
QUESTIONNAIRE 2 - SUMMARY INFORMATION TO SPG MEMBERS ABOUT A NEW
STATISTICAL ACTIVITY 87
ANNEXES 90
ANNEX 1. THE UNITED NATIONS FUNDAMENTAL PRINCIPLES OF OFFICIAL STATISTICS 91
ANNEX 2. THE INTERNATIONAL STATISTICAL INSTITUTE DECLARATION ON
PROFESSIONAL ETHICS 93
ANNEX 3. TOWARDS THE NEW OECD STATISTICAL INFORMATION SYSTEM 96
ANNEX 4. A LAYOUT FOR THE NOTES SECTION OF OECD DATA SETS 106
ANNEX 5. AN EXAMPLE FOR THE PREPARATION OF THE SELF-ASSESSMENT 110
4
STD/QFS(2003)1
INTRODUCTION
1. Improvement in the quality of OECD statistics is one of the main objectives of the current OECD
Statistics Strategy (OSS). OECD statisticians devote a significant part of their effort to quality
improvement at an individual level, and the Organisation already applies techniques that are used in total
quality management frameworks without having adopted a formalised approach to quality. Internal quality
assurance processes are already in place in several Directorates. These include treatment and validation of
questionnaire replies, cross-checking with national publications, compilation of additional information
from OECD and other international sources, preparation of draft publication, referral of data queries and
other quality issues back to source agency, interpretation, etc.
2. Several projects have been launched in the framework of the OSS and considerable
improvements have been achieved in 2001-2002. These initiatives will enhance quality, though there is
still need for a common framework which can be used to systematically assess, compare and further
improve OECD statistics. A quality framework that is applied across the Organisation will:
· provide a systematic mechanism for the ongoing identification and resolution of quality
problems;
· significantly increase the transparency of processes used by the OECD to assure quality; and
· reinforce the political role of the OECD in the context of an information society.
3. A lot of work has been done in recent years to apply the concept of quality to statistical data. For
example, the IMF, Eurostat, Statistics Canada and other national statistical offices (NSOs) have identified
various sets of data quality components and have adopted quality frameworks to improve their
organisations and the quality of data produced. The OECD quality framework benefits from this work and
we have avoided "reinventing the wheel" by adapting existing definitions and approaches to the OECD
context.
4. For an international organisation, the quality of statistics disseminated depends on two aspects,
the quality of national statistics received, and the quality of internal processes for collection, processing,
analysis and dissemination of data and metadata. In several fields, national statistics are developed closely
in accordance with international standards. On the other hand, statistical processes at international level are
often derived from best practices developed at national level. Thus, there is a clear inter-dependence
between the two aspects.
5. The OECD Quality Framework outlined in this document focuses on improving the quality of
data collected, compiled and disseminated by the OECD through an improvement in the Organisation's
internal statistical processes and management, though there will also be a positive spillover effect on the
quality of data compiled at the national level. Thus, in a sense, the OECD quality initiative is similar to
those developed by Statistics Canada and other NSOs, which also encompass statistical managerial and
technical processes.
5
STD/QFS(2003)1
6. The OECD Quality Framework has four elements:
· a definition of quality and its dimensions;
· a procedure for assuring the quality of proposed new statistical activities;
· a procedure for evaluating the quality of existing statistical activities on a regular basis;
· a set of broad principles on which OECD statistical activities are to be conducted and quality
guidelines covering all phases of the statistical production process.
7. This document is organised as follows. Part 1 provides definitions of quality dimensions, together
with a brief description of procedures for assuring the quality of new statistical activities and for evaluating
the quality of existing statistical activities. In addition, it presents a set of broad basic principles on which
OECD statistical activities have to be conducted. Part 2 outlines quality guidelines for different phases of
the statistical production process. Finally, Part 3 presents a description of specific steps to be followed in
planning and carrying out statistical activities, together with examples of good practice currently in use in
OECD Directorates.
6