Aynur SULTANOVA
State Statistical Committee of the Republic of Azerbaijan
Advisor of the Quality Management and Metadata Department
 E-mail: sultanova.aynur@stat.gov.az
ORCID: 0000-0002-4318-4546

Abstract. The purpose of this research is studying of some aspects of application of the Quality Management System (QMS) in State Statistical Committee of the Republic of Azerbaijan (Committee) and analyze the possibilities for improving the internal audits carried out. As a result of the research also presented the practice of traditional inspections in Committee, comparative experience of quality management documentation of other countries’ statistical offices, some shortcomings of internal audits carried out in Committee, proposals for their elimination, as well as a brief overview of the positive effect of audits for the QMS.

The analysis of the experience of internal audits’ organization in Committee presented in the article can serve as a basis for revision of the working plans in the field of quality, applied theoretical and practical audit documents and their implementation into practice not only in statistical offices but also in other institutions.

The article reflects the real experience and possible development capacity of the QMS in the statistical service of Azerbaijan, the practical and theoretical aspects of assessment stage of the QMS, or rather the conduct of internal audits aimed at identifying inconsistencies in the processes and their results.

Keywords: quality, Quality Management System, audit, standard, assessment

Introduction

Since 2000s observing a trend of strengthening the role of quality topics at the global level among producers of official statistics. Development of this issue in organization is possible by providing involvement of all processes and elements of its system. Hence various quality systems [3] developing at the same time one of which is being introduced in management processes of the Committee. A quality system can be thought as a set of requirements for documentation, planning, resources, procurement, evaluation, etc., as well as recommendations that contribute to the effective operation of an organization. The article attempts to reveal the potential of fulfilling one of these requirements in the Committee or rather the tradition of conducting internal audits, the advantages of internal audits for the system as a whole, as well as measures for their further improvement.

For controlling or improving performance in addition to internal audits in Committee conducing total control of regional statistical offices (RSOs), expert and analytical analyzes and global assessments.

Internal audits are carried out in Committee within the framework of the Quality Management System (QMS) implemented in 2010 based on ISO 9000 family of standards [16]. In 2020 as a result of conducted audits the validity of the certificate of conformity of the QMS of the Committee to the international standard ISO 9001:2015 was extended until 2023 and the number of certified RSOs was increased to 30.

Currently the system is used in 51 RSOs. It should also be noted that Committee was the first among the national statistical services of the countries of the Commonwealth of Independent States (CIS) and state bodies of the second category (administrations of ministries, committees, etc.) of the Republic of Azerbaijan to receive a certificate of compliance of the QMS with the requirements of the international standard ISO 9001:2015.

Note that the statistics of certification of QMS compliance with the requirements of the ISO 9001 standard using in calculation of the Global Innovation Index [10]. According to the report submitted in 2021 on indicator in terms of QMS certification, i.e. 6.2.4. Azerbaijan occupies the 94th place in the rating, among the neighboring countries Armenia is on the 114th position, Georgia 74th, Iran 87th, Russia 105th, Turkey 70th position.

Internal audits of the Committee along with ISO quality management principle [15] Evidence-based decision making also indirectly related to principles of Customer focus, Leadership, Engagement of people, Process approach, Improvement and Relationship management which is associated with close interconnections of processes within the system. In accordance with ISO recommendations for implementation of these principles attention is paid to the evaluation and monitoring of performance. Therefore, we propose to observing the processes of internal audits in Committee and possibilities for their improvement.

1. The practice of conducting internal audits

Committee researching the practice of internal audit and developing the relevant guidance documents since 2010. Committee conducts internal audits in accordance with principle 4 Commitment to quality of the Code of Practice [6] and sub article 9.2 Internal audit of Article 9 Performance evaluation of the ISO 9001:2015 QMS standard.

Certification audits in Committee were first held in 2013. While preparation for external audits internal audits were organized in statistical and administrative departments of the Administration, then in selected RSOs that implemented QMS.

The Committee implemented preventive, corrective actions and process improvement measures which became possible due to the analysis by middle and top management of nonconformities discovered during internal and external audits.

In first year for internal auditing of 15 departments of Administration were provided 9 auditors and in 2019 totally 42 audits (20 departments of Administration and 22 RSOs) were conducted by 22 auditors. It is noticeable that in the years under review the average workload of auditors is increasing.

As a result of participation in trainings on ISO 9001 and ISO 19011 standards [13] conducted by Moody Intertek, SGS LTD, etc., 16 specialists who form groups of auditors of the Committee were certified. Additional training helps to improve the effectiveness of internal audits. It’s also possible to provide certificates and awards to active auditors.

In the workload analysis of the Committee for the 2nd quarter of 2018 noted that 21.7% of the work performed by the Quality Management and Metadata Department is related to the measurement and evaluation of the effectiveness of the QMS. We think that taking into account contribution to this activity as well as the performance of the specialists from other departments will serve the accuracy of the data.

2. Theoretical aspects of internal audits

Documents that provide theoretical basis for organizing internal audits The documented procedure “Internal audits” as well as the Memo book for conducting internal audits of the quality management system [14] were prepared in 2010 and 2013. For preparation of these documents the requirements of the ISO 19011 standard, the practice of the Bureau of National Statistics of the Republic of Kazakhstan and the Handbook on internal quality audits of statistical processes and products in the National Statistical Institute (NSI) of Bulgaria [12] were considered. In order to avoid partial duplication and to increase the recourse in practice to a single document there is a growing need to eliminate one of them which in turn will remove the burden of adapting these documents to new editions of the ISO 19011 standard. The need to update the relevant documentation is also obvious in the NSI of the Republic of Bulgaria.

Note that the phrase “documented procedure” was inherent in international standard ISO 9001 of 2008 version and in 2015 version this phrase along with the concepts of “documentation”, “quality instructions”, “notes” were replaced by the phrase “documented information”. In our opinion renaming the Documented Procedure “Internal Audits” into Instructions for Internal Audits or Manual for Internal Audits will not change the essence of the document and does not require using of phrase “documented procedure” as evidence of documenting the procedure, practice of audit. Similar practice is also observed in the National Statistical Committee of the Republic of Belarus [20] and the Bureau of National Statistics of the Republic of Kazakhstan [21].

Theoretically the stages of internal audit are planning and preparation (selection of auditors, formation of an audit team and appointment of a chief auditor, preparation of an annual audit program, training of auditors, preparation of an audit plan, initial audit), auditing (holding an opening meeting, checking the availability of QMS documentation and compliance with established requirements, holding intermediate and closing meetings), formation of results and control over the elimination of identified inconsistencies.

As feedback to the audit process and opportunities for improvement beside the chief auditor’s analysis the auditee’s assessment of the auditors and the audit is taken into account.

Audit documents (audit programs and plans, checklists, nonconformity acts, reports, etc.) are archived in paper and electronic forms and also serve as metadata (template and practice) in the next audit process.

3. Comparison of internal audit with total controls

As before total controls are also currently being carried out in accordance with the annual Plan for reviewing the activities of regional statistical offices and providing practical assistance which is approved by the decision of the collegium and determines participants, time and leaders of the review teams. For these checks more detailed process map and accessible instructions can be provided.

In accordance with the Program for familiarization and inspection of the work of regional statistical offices control is exercised over planning and execution of work, ensuring the reliability of data and reporting discipline, preparing economic and statistical data, maintaining the state register of statistical units, using technology, personnel management, etc. For example, in 2018 total controls were organized in eight RSOs with participation of 4-5 inspectors in each group. Four of the checked RSOs had implemented QMS and two of them were also the subject of audits in the same year.

Taking into account the presence in the practice of the Committee of total controls conducting internal audits in accordance with recommendations of the international standard ISO 19011 is understood by us as creation of a new system within the existing mature system. Perhaps at the beginning of the work on modernization of the Committee’s management total controls in accordance with recommendations of the international standard ISO 19011 could play the role of a practical basis for the planned internal audits. Does this mean that the QMS created and developed since 2010 has already existed in the Committee for many years?

Let’s look at some of the differences between total controls and internal audits. In the first one verification process supervised by the top management of the Committee, shortcomings are eliminated, state officers who make mistakes in their work can be held accountable in accordance with the law and in the last one inconsistencies are analyzed but the idea of continuous improvements does not provide for punishment. In addition shortcomings (inconsistencies) discovered during total controls are discussed at board meetings but not listed in single electronic database of inconsistencies.

The similarity of these control methods observing in annual planning (the audit program is approved by the chairman), assessing by a group of controllers (auditors), bringing to the attention of the audited parties the conclusions to be considered in the report.

The Committee may take the risk of integrating these control methods and organizing single evaluation processes. It is observed that the QMS of the Committee overlapping in some matters with existing system which is fraught with additional costs of human, financial and time resources which will increase the need for full integration and increased coordination. However these problems are the subject of another study.

4. Technical and other shortcomings of internal audit

As noted above, internal and certification audits are also organized in the RSOs. Since the RSOs provide Committee with statistical information the Committee is its consumer and regulator. Accordingly internal audits in the RSOs conducted by the auditors of the Committee’s Administration can be considered as external audits. This note may be reflected in the audit program.

In our opinion the audit process needs technical amendments in order to organize it more effectively. For example, replacing the paper format of checklists with electronic ones, reducing the amount of information entered into them by referring to the Non-conformity Act. By implementing the above its possible to save time spent on compiling checklists – a kind of “navigator” of auditors, equipment and materials (cartridge, paper) for printing and physical archiving space. Saving time is more important for auditors who are seconded to RSOs.

It should be noted that different checklists are considered for audits in statistical, administrative departments and RSOs. The Committee uses four types of checklists: those compiled in accordance with the requirements of the ISO 9001:2015 standard for all departments, for the RSOs, in accordance with the standards and process maps for administrative departments and in accordance with the processes of the Generic Statistical Business Process Model (GSBPM) [17] that facilitates description, comparison and control of processes for statistical departments. The figure below helps to understand the relationship of the GSBPM with the existing processes in the practice of the Committee.

Picture. Integration of GSBPM into the management processes of the Committee (with some changes from the author). Source: [4].

The use of two assessment tools, i.e., GSBPM-based checklists, as well as self-assessment questionnaire considering the Development of Self-Assessment Program (DESAP) [18] are likely to increase the administrative burden. On the other hand, since both questionnaires cover GSBPM processes it would be appropriate to practice one of them.

The checklist based on the national version of the GSBPM (rev.4) contains 105 questions. For comparison we note that in the audit manual of the NSI of the Republic of Bulgaria the checklist is represented by 52 generalized questions related to GSBPM processes as well as 21 sub-questions for calculating quality and performance indicators. This checklist is less heavy and probably more convenient to use. The NSI of the Republic of Bulgaria dedicates 5 days to field work, in statistical departments of the Committee this process lasts 3 days. It can be concluded that in the NSI under study more time and attention is paid to the analysis of audit findings.

The DESAP questionnaire consider taking into account possible risks, suggestions for improvement and evaluation of statistical activities within the GSBPM stages. From this point of view, we suggest as a pilot project completion of the DESAP questionnaire by statistical departments during internal audits, further assess the progress on diagram obtained in accordance with the responses in the follow-up audit. Using the questionnaire for the first time is a more voluminous process, in the future only key questions can be taken into account. In this case more time will be required for field work and analysis of audit evidence.

Evaluation by the audited party of the audit process and auditors not in paper but in electronic format, linking to the assessment data in the report on the results of the audit will serve to digitalized the audit documentation. There is a need to provide uniform numbers for audits and follow-up audits, i.e., assigning them the same codes.

The structure of the audit documentation (plans, reports) of the SGS LTD certification organization which approved the compliance of the QMS of the Committee with the requirements of ISO 9001:2015 can serve for technical refinement of the internal audit documentation.

Considering the delicacy of the audit process it is preferable to involve the most experienced specialists in audits and “newcomers” to train for four years which on the one hand will not contradict the Law on “Internal Audit” of the Republic of Azerbaijan [8] (which however is more prone to financial audit rather than the QMS), on the other hand would eliminate the risk of giving the impression of inobjectivity of the audit to auditees.

5. Benefits of conducting internal audits in the Committee

Describing the experience gained as conducting audits in the Committee over the past years we will consider some of their advantages. As A.Veliyev and Y.Yusifov noted in their papers this list can include the identification of shortcomings in the work of departments, the possibility of disseminating positive experience, strengthening discipline in the performance of work in accordance with regulatory requirements, etc. [5, 2].

At the European Conference on Quality in Official Statistics held in Krakow in 2018 the Committee experts also considered audit as a tool that contributes to implementation of one of the 8 stages of the GSBPM – Assessment, which give possibilities to make an objective decision for further development of the system [11].

Internal audit helps to systematically identify and control the elimination of inconsistencies in processes. In addition, the staff of the Committee and most of the RSOs get acquainted with the achievements of modern management, master innovations and experience in training and implementing the system.

It is known that in Azerbaijan several dozens of government units (state bodies, enterprises) have certified the QMS on the basis of the ISO 9000 family of standards [3]. In these units the practice of internal audit is developing which create interest to get acquainted with this experience. The Committee can initiate organization of discussions, seminars and other events with its active participation which is likely to contribute to exchange of effective decisions and successful methods.

Conclusions

It is obvious that the audit process itself aimed at identifying gaps in the activities of the Committee also has its own gaps which may not be limited to the above mentioned. In our opinion these shortcomings are related to the lack of consultations with consulting organizations during the creation of the QMS in the Committee. Although external auditors are expected to timely identify gaps in the QMS, but it seems to us that in practice certification organizations often seek to promote their brand, generate income and expand the client base for the trainings they organize which however does not contradict the market economy. We do not deny the fact that each finding of external (certification) audits helps to improve the QMS in the Committee. Let us give an example of a similar problem observed in the NSI of the Republic of Bulgaria. Despite the certification of the system for compliance with the requirements of the ISO 9001:2015 standard [19] which first requires conducting internal audits which according to the information provided on the official website of the NSI are organized using the Handbook on internal quality audits of statistical processes and products in the National Statistical Institute of Bulgaria prepared on the basis of outdated versions of ISO standards.

There is a growing need to review the feasibility of introducing structural units with a systematically small number of inconsistencies into the audit programs, taking into account the risks of losing the relevance of the QMS documentation, as well as continuous improvement of the competence of auditors in connection with the release of new versions of standards. Further development in the Committee of the trend of digitalization of production processes, rejection of paper-based information will help to continuously and remotely conduct audits.

Considering the experience of the Committee in the field of quality [3, 5, 11] observing its desire to develop the QMS based on the standards of the ISO 9000 family which leads to elimination of shortcomings in internal audit, bringing it into line with the new version (2018) of the ISO 19011 standard [7 ], as well as the implementation of other improvements. As a result of the amendments the time saved during the audit can be spent on new research, dissemination of innovations and achievements. The training on new version of the standard of all those participating in internal audits of the Committee should be continued.

In Global Assessment of the National Statistical System of Azerbaijan conducted in 2016-2017 with the financial support of the Statistical Office of the European Union, international experts recommended the Committee during the audits (if necessary and with the participation of external experts) to focus on relevant methodology of statistical production and processes [9]. The implementation of the above proposals, as well as calls for further development, noted by the Chairman of the Committee T. Budagov in his manuscript [1] will contribute to the improvement of audit practice.

Analysis of the advantages and disadvantages of internal audit as an integral part of the QMS also leads to the following idea. In new organizations the creation of a QMS is a simpler task than, for example, in statistical offices with a practice that developed over the years which actually meets the requirements of the ISO 9001 standard. The question is whether statistical offices are ready to modernize traditional administrative processes, replace the “old” to “new”, otherwise a conflict of many years and rich statistical experience with a seemingly “new” implementation of the QMS is inevitable.

Having achieved good results as in the case of the international model GSBPM developed on the basis of the practice of the work processes of the New Zealand NSI the Committee can also try to develop and promote its audit model in the global statistical community.

Bibliography

In Azerbaijan language:

1. Budaqov T. Y. Azərbaycanda rəsmi statistikanın əsas inkişaf istiqamətləri.//“Statistika Xəbərləri” elmi-praktik jurnalı, №1, Bakı, 2015, s. 9-15.
2. Yusifov Y.X. Keyfiyyətin təmin olunmasının və rəsmi statistikanın əsas prinsiplərinin milli statistikada nəzərə alınması məsələləri.//“Statistika Xəbərləri” elmi-praktik jurnalı, №1, Bakı, 2016, s. 24-32.
3. Sultanova A.A. Azərbaycanın dövlət idarəçiliyi sistemində keyfiyyət menecmentinin müasir vəziyyəti.//“Strateji Təhlil” analitik jurnalı/say 3-4 (25-26), 2018, s. 161-176.

 In Russian language:

4. Султанова А.А. GSBPM как инструмент качественной модернизации производственной системы официальной статистики Азербайджана.//“Statistika Xəbərləri” elmi-praktik jurnalı, № 3, Bakı, 2018, s. 82-91.
5. Велиев А.М. Создание системы управления качеством: от теории к практике.//Научно-информационный журнал “Вопросы статистики”, № 4, Москва, 2014, с. 69-71.

Internet resources:

6. Azərbaycan Respublikası Dövlət Statistika orqanlarının Normalar Məcəlləsi, 2011. – https://www.stat.gov.az/menu/2/quality/az/regulations_SK.zip
7. Beynəlxalq ISO 19011 standartının 2018-ci ilin redaksiyasına baxış. – https://www.facebook.com/keyfiyyet.mektebi/posts/1426108244211875
8. Daxili audit haqqında Azərbaycan Respublikasının Qanunu – http://www.e-qanun.az/framework/13241
9. Global Assessment of the National Statistical System of Azerbaijan, 2017. – https://www.stat.gov.az/source/others/G_Q_en.pdf
10. Global Innovation Index 2021. – https://www.wipo.int/edocs/pubdocs/en/wipo_pub_gii_2021.pdf
11. GSBPM and ISO: as quality management system tools in official statistics (Azerbaijan experience). – https://www.q2018.pl/papers-presentations/?drawer=Sessions*Session%2003*Yusif%20Yusifov
12. Handbook on internal quality audits of statistical processes and products in the National Statistical Institute of Bulgaria. – https://www.nsi.bg/sites/default/files/files/pages/Quality_e/1.6.%20Handbook%20%20Internal%20Audit_EN_28.11.2011_corr.pdf
13. ISO 19011:2018 Guidelines for auditing management systems. – https://www.iso.org/standard/70017.html
14. Keyfiyyətin idarə edilməsi sisteminə dair metodoloji sənədlər. – https://www.stat.gov.az/menu/2/quality/#collapseOne
15. Quality management principles. – http://www.iso.org/iso/pub100080.pdf
16. Selection and use of the ISO 9000 family of standards. – https://www.iso.org/publication/PUB100208.html
17. Statistik biznes prosesin ümumi modeli. – https://www.stat.gov.az/menu/2/quality/az/00_7.pdf
18. Анкета DESAP (Разработка программы самооценки) для оценки качества наблюдений в Европейской статистической системе. – https://ec.europa.eu/eurostat/documents/64157/4373903/07-Checklist-for-Survey-Managers_DESAP-EN.pdf/ec76e3a3-46b5-409e-a7c3-52305d05bd42
19. Сертификат № 32639/15/S. – https://www.nsi.bg/sites/default/files/files/images/Sertificat_ISO_9001_2021.pdf
20. Система менеджмента качества. – https://www.belstat.gov.by/o-belstate_2/sistema-menedzhmenta-kachestva/
21. Система менеджмента качества Бюро. – https://stat.gov.kz/search/item/WC16200012345

Aynur Sultanova

AZƏRBAYCANDA STATİSTİKA SİSTEMİNİN ÜMUMİ AUDİT MODELİNİN İNKİŞAF POTENSİALI

 Xülasə

Tədqiqatın məqsədi Azərbaycan Respublikasının Dövlət Statistika Komitəsində (Komitə) keyfiyyətin idarə edilməsi sisteminin (KİS) tətbiqinin bəzi aspektlərini öyrənmək və Komitə tərəfindən keçirilən daxili auditlərin təkmilləşdirilməsi imkanlarını təhlil etməkdir. Tədqiqatda həmçinin Komitədə ənənəvi yoxlamaların aparılması təcrübəsi, digər ölkələrin statistika idarələrində keyfiyyətin idarə edilməsi üzrə sənədləşdirilmənin müqayisəli təcrübəsi, Komitədə keçirilən daxili auditlərin bəzi çatışmazlıqları, onların aradan qaldırılması üçün təkliflər, həmçinin auditlərin KİS-ə müsbət təsirinin qısa icmalı təqdim edilmişdir.

Məqalədə təqdim edilmiş Komitənin daxili auditlərinin təşkili təcrübəsinin təhlili statistika idarələri ilə yanaşı digər qurumlarda da keyfiyyət sahəsində iş planlarının, tətbiq edilən nəzəri və praktiki audit sənədlərinin yenidən baxılması və onların praktikada tətbiqi üçün əsas ola bilər.

Məqalədə Azərbaycanın statistika xidmətində KİS-in inkişafının real təcrübəsi və mümkün potensialı, KİS-in qiymətləndirmə mərhələsinin, daha doğrusu, proseslərdə və onların nəticələrində uyğunsuzluqların aşkar edilməsinə yönəlmiş daxili auditlərin aparılmasının praktiki və nəzəri aspektləri öz əksini tapmışdır.

Açar sözlər: keyfiyyət, keyfiyyətin idarə edilməsi sistemi, audit, standart, qiymətləndirmə

Айнур Султанова

ПОТЕНЦИАЛ РАЗВИТИЯ ТИПОВОЙ МОДЕЛИ АУДИТА СТАТИСТИЧЕСКОЙ СИСТЕМЫ В АЗЕРБАЙДЖАНЕ

 Резюме

Целью данного исследования является изучение некоторых аспектов применения системы менеджмента качества (СМК) в Государственном комитете по статистике Азербайджанской Республики (Комитет) и анализ возможностей улучшения проводимых им внутренних аудитов. В результате исследования представлена также практика традиционных проверок в Комитете, сравнительный опыт документаций менеджмента качества статистических управлений других стран, некоторые недостатки проводимых в Комитете внутренних аудитов, предложения по их устранению, а также краткий обзор положительного эффекта аудитов для СМК.

Представленный в статье анализ опыта организации внутренних аудитов в Комитете может послужить основанием для пересмотра планов работ в области качества, применяемых теоретико-практических документов аудита и внедрения их в практику не только в статистических управлениях, но и других учреждениях.

В статье нашли свое отражение реальный опыт и возможный потенциал развития СМК в статистической службе Азербайджана, практические и теоретические аспекты этапа оценки СМК, а точнее ведения внутренних аудитов, направленных на выявление несоответствий в процессах и их результатах.

Ключевые слова: качество, система менеджмента качества, аудит, стандарт, оценка