Customers’ satisfaction assessment in water laboratories

Water quality is a set of physical, chemical, and biological characteristics defined according to water uses such as for human consumption, industries, and agriculture. The parameters evaluated can reveal physical, chemical, and biological threats when exceeding values higher than those established for a particular use. The concern with the evaluation of the water quality parameters has been growing in Portugal and with the accredited laboratories. Accreditation by ISO/IEC 17025 (ISO/IEC 2017) aims to improve employee performance in determining water quality parameters and to help in complying with the current Portuguese regulation. Decree-Law 152/2017 establishes the drinking water quality regime in accordance with European legislation, promotes human health from the effects of harmful water contamination, and ensures the availability of clean and balanced water (Portuguese Government 2017). According to this law, the water laboratories are required to have accreditation in accordance with ISO/IEC 17025. In other words, all Portuguese laboratories that perform water analyses must work in accordance with internationally approved procedures and use validated methods. In fact, quality is the main factor that differentiates organizations and is considered a fundamental aspect that can influence the survival of the organization in an increasingly competitive market. Quality defines the service level of the organization and determines customer satisfaction. Thus, the evaluation of customer satisfaction is essential for companies that seek to obtain a competitive advantage in the market, including water laboratories. In Portugal, although 96% of the population is served by public water supply systems, in rural areas, this percentage is only 80% (PORDATA 2021). However, regardless of the existence of public water supply, in rural areas, there is a high number of private abstractions from which water is used (e.g., human consumption, recreational waters, swimming pools). Thus, the owners rely on water laboratories to assess the quality of their water. In fact, some water laboratories, besides physical-chemical and microbiological analysis, also offer services of collection and transport of water samples (e.g., wells, boreholes, and/or pools water).

Assessing customer satisfaction is critical for companies looking to gain competitive advantage in the market. Therefore, customer satisfaction is tied to the standards that apply to different areas of activity, including water laboratories. The present work aims to assess the satisfaction of the customers of water laboratories based on parameters that influence the management requirements and the technical requirements of the ISO/IEC 17025 standard. To achieve this general goal, artificial intelligence methods and techniques have been used to develop a Decision Support System. For this reason, this article introduces a formal method of customer satisfaction assessment based on a computational system geared toward ANNs (Haykin 2009; Vicente et al. 2012).

The perception of quality is inherent in humans and extends to different political domains, including the most common and automatic actions in our daily life (Christian & Drilling 2010). Quality is the set of attributes and characteristics of a product or service that affect the ability to meet a person's needs. If the main characteristic of quality is the continuous improvement of company processes and, on the other hand, the intention to satisfy the customer and everyone involved in the functioning of the organizations, then the concept of Overall Quality Management (OQM) arises (Hoyle 2017). Quality management was of the utmost importance as it included efforts to maintain and improve quality in all areas of the company and to create products and services with the greatest possible savings (Kotler et al. 2016). OQM aims to continuously improve all processes, products, and services (Kotler et al. 2016). It is a philosophy that came up in the 1950s and 1960s that led to a change in attitudes where quality became a business problem (Hoyle 2017). In the context of organizations, there are various definitions of quality, all of which relate to customer satisfaction with the products or services offered by companies (Kotler et al. 2016). The European Foundation for Quality Management (EFQM) claims that the model of excellence is not a requirement, because there are many ways to achieve organizational excellence (Protzman et al. 2015). The EFQM Excellence Model (EFQM-EM) is based on nine criteria, five of which are enablers and four results. Activation criteria include what an organization does and how it does it. The criteria of the results cover what an organization achieves. The model assumes that excellent organizations achieve superior performance results in a sustainable manner and thus meet or exceed the expectations of all stakeholders (EFQM 2013). According to EFQM-EM, leadership outcomes that motivate employees, develop strategies and guidelines, build partnerships with their suppliers, and analyze the organization as a process can help to remove obstacles between different departments (EFQM 2013). The results are caused by the enablers and the enablers are improved by the feedback of the results (Figure 1). In Figure 1, the arrows illustrate the dynamics of the model and also show that learning and innovation support the improvement of enablers and also leads to better results (EFQM 2013).

In the literature, employee satisfaction studies are more common than customer satisfaction studies. These were carried out in areas such as health (Allen 2013; Boev et al. 2015; Borishad et al. 2018; Liu et al. 2019), tourism (Francesco & Roberta 2019; Gerdt et al. 2019), and call centers (Walsh et al. 2012; Chicu et al. 2019). To our knowledge, only one customer satisfaction study has been carried out in water laboratories (Kaynar et al. 2018). The authors assessed the customers’ satisfaction of water laboratories accredited within the scope of TS EN ISO/IEC 17025, using a ten-question survey with a one to five scale. The authors evaluated the average and standard deviation of responses obtaining 4.83 ± 0.07, corresponding to a percentage of satisfaction of 96.6%. According to the authors, the customers’ satisfaction was higher for water laboratories accredited within the scope of TS EN ISO/IEC 17025. Furthermore, the results obtained allowed corrective actions to improve the laboratory service (Kaynar et al. 2018).

Regarding health care, Boev et al. (2015) examined the relationship between job satisfaction among healthcare workers and infections in adults in intensive care units. Wage inequality, dissatisfaction with the salary, and lack of recognition are the most important aspects (Boev et al. 2015). Allen (2013) examined the effects of the introduction of a quality management system based on ISO 15189 on the improvement of security and customer satisfaction. The author emphasized that the implementation of such systems led to an improvement in quality and an increase in customer security. Borishad et al. (2018) examined the relationship between customer experience and customer satisfaction in four private hospitals in Nigeria. The data were recorded on a questionnaire with a 5-point Likert scale and a total of 365 participants. The authors developed a model that describes customer satisfaction as a function of the reliability of services, the correct diagnosis of diseases, the correct treatment of diseases, the effectiveness of verbal communication and non-verbal communication. The model was able to explain 43.1% of the observed variance in customer satisfaction (Borishad et al. 2018). Liu et al. (2019) examined the relationship between workplace violence and patient safety. A questionnaire was administered in 23 Chinese hospitals in Guangdong Province, with 1502 nurses participating. The data were collected between December 2013 and August 2014. According to this study, workplace violence was associated with a higher incidence of burnout, lower job satisfaction, and lower patient safety. The authors also found strong associations between nursing staff burnout and less patient safety. Conversely, a higher level of job satisfaction among nursing staff was directly linked to greater patient safety. The authors also point out that burnout and job satisfaction among nurses played a mediating role in workplace violence and patient safety (Liu et al. 2019).

The artificial neural networks (ANNs) approach to data processing enables not only that data are processed in a system context but also to capture complex or unknown relationships between inputs and outputs (Vicente et al. 2012; Vilhena et al. 2017). ANNs simulate the structure of the human brain, being populated by artificial neurons, with a valuable set of activation functions (Haykin 2009). The Multilayer Perceptron (MLP) is one of the widespread ANN architectures, in which the artificial neurons are organized in layers and only forward connections occur (Haykin 2009). MLP design is usually done by trial-and-error and various automated techniques have been suggested to find the ‘best’ MLP network. Typically used is a hill-climbing approach that begins with an initial architecture that is adjusted with the purpose of decreasing the internal measure of error (e.g., the mean squared error) (Haykin 2009). In recent years, several studies have demonstrated the effectiveness of ANNs in multiple contexts such as environment (Ahmed 2017; Ruben et al. 2018), health (Vilhena et al. 2017; Neves et al. 2018), and education (Figueiredo et al. 2020).

This study was carried out in the Water Laboratory of Santiago do Cacém Municipality, located in Alentejo Litoral, southern Portugal. This laboratory is accredited according to the ISO/IEC 17025 standard. It is known for its technical proficiency in sampling and analytical tests. The water laboratory offers services in the collection, transport, and physicochemical and microbiological analysis of samples of treated and non-treated water (e.g., wells, boreholes, pools water, and whirlpools) for municipalities and private customers. The laboratory has a team of 18 workers, consists of 1 laboratory manager, 1 quality manager, 1 administration employee, 4 sample technicians, 5 microbiology technicians, and 6 physicochemical technicians.

This study included 253 participants (water laboratory customers) aged between 23 and 76 with an average age of 43. The gender distribution was 46% for men and 54% for women. Eighty-eight percent of the participants came from rural areas and 12% from urban.

Considering the advantages and disadvantages of the possible data collection methods (McMillan & Schumacher 2009; DeKetele & Roegiers 2016; Cohen et al. 2017), the questionnaire survey was selected. The reasons for this choice are related to the fact that this tool has a clearly defined structure and makes it possible to convert the qualitative information provided by the respondents into quantitative data. For this study, a customer satisfaction questionnaire was created that was used for a cohort of 253 customers. The questionnaire was divided into three sections, the first of which contained general questions on age, gender, location, academic qualifications, and type of acquaintance with the laboratory. The second contained statements on the services provided, namely customer service, quality of service, support documentation, and handling of complaints. In the third section, respondents were asked to express their overall satisfaction with the services provided by the laboratory. In the first part of the questionnaire, the answers were descriptive, while in the others, the Likert scale was used with four levels: very dissatisfied, slightly dissatisfied, satisfied, and very satisfied (Supplementary Material, Annex 1). The questionnaire was validated according to the rules of Bell (2010). Therefore, the questionnaire was evaluated by a group of experts who suggested some corrections. After an expert analysis, the questionnaire was modified and applied to a limited group of customers who were not included in the sample to assess its validity and identified difficulties in interpreting the questionnaire.

Data collection was carried out between January 2019 and September 2020. To prevent potential hidden errors related to ostensibly random sampling methods that can lead to biased results (Shang 2019), all customers were asked to fill out the questionnaire. The response rate was 86% (253 questionnaires received in 294 administered).

Conversion of the qualitative information collected through the questionnaire into a quantitative form was conducted using the method proposed by Fernandes et al. (2016). According to this method, a set of n issues regarding a particular subject is itemized into a unitary area circle split into n slices, where the marks in the axis correspond to each one of the possible answers, as described in the Customer Satisfaction Assessment section.

The software used to implement ANNs was the Waikato Environment for Knowledge Analysis (WEKA), while maintaining the standard software parameters (Hall et al. 2009; Frank et al. 2016). Aiming to guarantee statistical significance of the results, 20 experiments were applied in all tests. In each simulation, the database was randomly split into two mutually exclusive partitions, the training set with two thirds of the cases and the test set made up of the remaining examples.

The respondents took notice of the goals of the questionnaire and participated voluntarily, without any pressure or coercion. The study was conducted in compliance with the relevant laws and institutional guidelines and was approved by the quality manager of the Water Laboratory of Santiago do Cacém Municipality. Furthermore, the participants gave an informed consent to participate in the study.

Respondents were divided into age groups as shown in Figure 2(a) (i.e., age less than 20 years, 20–30, 31–50, 51–70, and more than 70 years old). Seventy-one percent of the participants were 50 years old or less, while 29% were 51 years old or more (Figure 2(a)). On academic qualification, 34% of the cohort stated to have basic education, 39% finished secondary education, 20% completed a degree, and 7% have some postgraduate education (Figure 2(b)). Regarding the way customers became familiar with the laboratory, 49% declared that they got acquainted through the internet (web or social networks), 16% said that they discovered the laboratory on site, 15% declared to be existing customers, and 13% affirmed that they knew the laboratory through close people (Figure 2(c)).

Figure 3 shows the results of the second part of the questionnaire, in which the respondents expressed their opinion on Customer Service and Quality of Service Provided. Regarding Customer Service, the analysis of the results shows that the majority of respondents were satisfied (87, 73, 57, 62, and 59% for S1, S2, S3, S4, and S5, respectively). In addition, more than 20% of the respondents were very satisfied with the availability of services (S3), the effectiveness of the answers to questions (S4), and the clarity of the answers to questions (S5). Conversely, 8% of those surveyed stated that they were very dissatisfied with the clarity of the answers to questions (S5). In relation to the other statements, only a small percentage, less than 4% of the participants, stated that they were very dissatisfied.

Regarding the Quality of Services Provided, the analysis of the results shows that the majority of respondents were satisfied (80, 86, 52, 70, and 60% for S6, S7, S8, S9, and S10, respectively). In addition, more than 20% of the respondents were very satisfied with the reliability of results (S8), appropriateness of delivery times (S9) and delivery deadlines (S10). In contrast, 7% of participants were very dissatisfied with the reliability of the results (S8).

Figure 4 shows the results of the Support Documentation. A review of Figure 4 shows that the majority of respondents were satisfied (74 and 64% for S11 and S12, respectively). Moreover, 24% of respondents were very satisfied with the presentation of commercial proposals (S11), whereas 20% were very satisfied with the presentation of analysis reports (S12). Conversely, 13% of the respondents were slightly dissatisfied with the presentation of analysis reports (S12). A possible explanation of this dissatisfaction can be related to both the age of the respondents and their academic qualifications. In fact, 29% of respondents were older than 50 years and 34% only had basic training (Figure 2).

Only 11% of respondents replied to the group of statements related to Complaint Handling. Although this percentage can be considered low in terms of the cohort, 44, 48, and 56% of respondents who responded to this group were slightly dissatisfied with resolution of their complaints (S13), response time (S14), and effectiveness (S15), respectively. Furthermore, 33, 30, and 22% of respondents who responded to this group ticked the option very dissatisfied in statements S13, S14, and S15, respectively. The high percentage of negative opinions on complaints handling suggests that the organization should take these points into account and strive for necessary improvements to address these shortcomings.

Figure 5 shows the results of the third part of the questionnaire, in which the respondents expressed their opinion on the overall assessment of the services. A vast majority of the respondents had a positive opinion, 56% were satisfied and 32% were very satisfied. Conversely, 12% responded negatively; 10% were slightly dissatisfied; and 2% were very dissatisfied.

Figure 6 shows the answers of the first respondent to the second part of the questionnaire. In order to quantify the qualitative information presented in Figure 6, the method proposed by Fernandes et al. (2016) was followed. For each dimension, Customer Service, Quality of Services Provided, and Support Documentation, the correspondent answers were itemized into a unitary area circle. The marks in the axis correspond to the possible answer, very dissatisfied, slightly dissatisfied, satisfied, and very satisfied. Taking as an example the dimension Customer Service, the answer of the first respondent to statements 1 and 5 (S1 and S5) was satisfied and the correspondent areas are computed as ⁠. In statements 2 and 4 (S2 and S4), the alternative very satisfied was chosen and the corresponding areas are ⁠. Finally, for statement 3 (S3), the answer was slightly dissatisfied and the area is ⁠. The total area (0.67) is the sum of the partial ones, being the quantitative value regarding the dimension Customer Service for the first respondent (Figure 7). For the remaining dimensions, the procedure is similar and the results are shown in Table 1.

The data presented in Table 1 were used as input variables in the training of ANNs to obtain the output variable, i.e., the Customers’ Satisfaction Assessment (Figure 8).

The assessment of ANN models' performance can be done using the confusion matrix (Vilhena et al. 2017). Table 2 presents the confusion matrix for the ANN model shown in Figure 8 (the values refer to an average concerning 20 experiments). The values shown in Table 2 allow computing of the model accuracy for training set (93.5%, i.e., 158 well classified in 169) and for test set (90.5%, i.e., 76 well classified in 84). Therefore, the predictions of customers’ satisfaction using the ANN model are satisfactory, achieving accuracies higher than 90%. The confusion matrix also allows to compute the Sensitivity, Specificity, Positive Predictive Value (PPV), and the Negative Predictive Value (NPV) of the model (Vilhena et al. 2017). The sensitivity and specificity for training and test sets were 94.0, 88.9, 90.3, and 91.7%, respectively. Regarding PPV, the values obtained were 98.6 and 98.5%, respectively, for training and test sets. Concerning NPV, the values obtained were 64.0 and 61.1%, respectively. These results seem to reveal a good overall ability of the model to discriminate between the output classes. Thus, the ANN model exhibits a good performance in the assessment of customers’ satisfaction.

Sensitivity analysis examines the model output response to variations in its input variables. Sensitivity according to variance (Kewley et al. 2000) was used to compute the relative Importance (I) of the input variables, that is, the effects of each input variable on the ANN output. The outcomes suggest that the ANN output is more influenced by Customer Service dimension (I = 0.44), followed by the dimensions Support Documentation (I = 0.29) and Quality of Services Provided (I = 0.27). These outcomes are corroborated by the results shown in Figures 3 and 4. The dimension Customer Service exhibit a higher frequency of very dissatisfied and slightly dissatisfied responses and, consequently, a small variation in the answers given by respondents to the statements S1 to S5 should have great repercussion in the output (the customers’ satisfaction assessment).

This study was carried out in a water laboratory that implements the ISO/IEC17025 standard. The results show that the vast majority of respondents rated the overall performance of the laboratory as positive. However, the results also revealed that the statements S5 (clarity of the answers to questions), S8 (reliability of the results), and S12 (presentation of the analysis reports) were the ones that obtained high percentages of negative opinions. This work also introduced an intelligent decision support system to assess customer satisfaction using ANN paradigm. The proposed approach processed information gathered through questionnaire surveys and enabled the full integration of factors related to Customer Service, Service Quality, and Support Documentation, used as input to the ANN. It provided reasonable answers with an overall accuracy of more than 90%. Moreover, the results of this study allowed to identify the weaknesses of the laboratory organization and to design and develop future improvement measures to promote customers’ satisfaction. Therefore, the focus of improvements should be put on the presentation of the analysis reports and on the reliability of the results. Furthermore, it is important to train staff to improve customer service. It should be noted that this kind of model can be used in any company. Thus, it is necessary to carry out an assessment of the factors that can influence customers’ satisfaction and adapt the questionnaires. Moreover, this study can be extended to assess the laboratory performance. For this purpose, feedback from both staff and suppliers should be considered. Thus, for each stakeholder, specific questionnaires should be developed according to the ISO/IEC 17025 standard requirements. Considering that the sample size was not large enough to evaluate demographic effects, future work will consider expanding the study to a larger sample to validate the results as well as to study the effects of demographic characteristics.

This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope UIDB/00319/2020 and UIDB/50006/2020.

All relevant data are included in the paper or its Supplementary Information.