Visualisation of the combinatorial effects within evolutionary algorithms: the compass plot

Applications of evolutionary algorithms (EAs) to real-world problems are usually hindered due to parameterisation issues and computational efficiency. This paper shows how the combinatorial effects related to the parameterisation issues of EAs can be visualised and extracted by the so-called compass plot. This new plot is inspired by the traditional Chinese compass used for navigation and geomantic detection. We demonstrate the value of the proposed compass plot in two scenarios with application to the optimal design of the Hanoi water distribution system. One is to identify the dominant parameters in the well-known NSGA-II. The other is to seek the efficient combinations of search operators embedded in Borg, which uses an ensemble of search operators by auto-adapting their use at runtime to fit an optimisation problem. As such, the implicit and vital interdependency among parameters and search operators can be intuitively demonstrated and identified. In particular, the compass plot revealed some counter-intuitive relationships among the algorithm parameters that led to a considerable change in performance. The information extracted, in turn, facilitates a deeper understanding of EAs and better practices for real-world cases, which eventually leads to more cost-effective decision-making.


GRAPHICAL ABSTRACT INTRODUCTION
Using evolutionary algorithms (EAs), especially multi-objective evolutionary algorithms (MOEAs), for the optimal design of water distribution systems (WDSs) is a prominent and active field in the domain of hydroinformatics (Mala-Jetmarova et al. ). The root of the complexity associated with a WDS design problem is mainly due to its highly combinatorial nature; thus, it is impossible to visit the entire search space systematically. There exist varying types of problem formulations, including single-objective, also known as the least-cost design problems (Savic & Walters ); multiobjective, i.e., concerning the trade-off among a set of goals usually competing with each other (Halhal et al. ); and even many-objective, i.e., considering four or more objectives simultaneously (Fu et al. ). In recent years, in the real world, which are typically associated with complex constraints and huge search spaces. In such a case, it is beneficial to understand which parameters are dominant factors requiring fine-tuning and in which directions (or scales) those parameters should be adjusted. Note that the fine-tuning process is usually problem-specific, which implies that a group of efficient settings of parameters for one case may be less efficient for another. Consequently, a sophisticated method of analysing parameters' sensitivity is necessary, especially when it is computationally intensive to solve a problem. However, this aspect is beyond the scope of this paper.
Currently, researchers in the hydroinformatics community rely mostly on the trial-and-error approach to analyse the sensitivity of the parameters or simply follow the recommended settings (Wang et al. b). One reason for such a gap lies in the lack of effective tools, preferably a visualisation method, for understanding the hidden mechanism of MOEAs responsible for their behaviour and success. That is, how their parameterisation can impact on the success of solving the problem at hand. In essence, it is another form of parameterisation issuedeciding whether a specific operator is employed as a parameter.
This paper seeks to narrow the knowledge gap between experts of MOEAs and non-expert users by developing a visualisation method that can intuitively present the complex combinatorial effects related to MOEAs. The key question is how a visualisation tool can assist in identifying the dominant factor(s) and appropriate setting of algorithm parameters for a given problem. It will be shown that the compass plot, which stems from the ancient Chinese ingenuity, can help deal with the above question.

Conceptual origin
Feng Shui, also known as Chinese geomancy, is a Chinese art that is based on the belief that how things are arranged within a room can affect one's life (e.g., luck and fortune).
The creation of the compass plot was mainly inspired by the device widely used in this domain. Figure 1  the days, weeks, and months for 40 years. Rotating the brass dial tells the corresponding dates given the year and month.
A Chinese Feng Shui compass is usually composed of Tianchi (the centric compass), an inner disc, and an outer square tray (deliberately omitted from Figure 1(a)). The inner disc can be rotated to align with the centric compass.
There are many concentric annuli carved on the inner disc, one annulus is called one layer, and it represents a specific and provide comprehensive explanations to guide his/her client(s) in furniture placement.
As mentioned above, Feng Shui containing multi-dimensional information can be recorded and presented in a series of concentric annuli. In other words, we can use such an annular object to illustrate any type of phenomena that are influenced by different factors. Below, we will explain the development of the compass plot based on the ideas that originated from the ancient Chinese way of thinking.

Functional design
Inspired by the Chinese Feng Shui compass, we developed a six-step procedure to generate and utilise the proposed compass plot for analysing the combinatorial effects associated with multiple factors. Without loss of generality, here we refer to the independent variables considered under a specific situation as 'impact factors,' and the dependent variables influenced by these factors as 'outcome performance.' As shown in Figure 2, the procedure has six steps that are categorised into three stages (shown in rounded rectangles). At the first stage, the analytical approach (e.g., optimisation or simulation) is chosen when a wellstructured problem is formulated (Step 1). This involves investigating the effects of different combinations of 'impact factors' on the 'outcome performance.' Then, the key impact factors are identified, and their candidate values are determined (Step 2). At the second stage, the problem is solved by the analytical approach specified in Step 1, and the corresponding results are obtained for each combination of impact factors (Step 3). At the last stage, in which the compass plot is generated and interpreted, results gained at the previous step are evaluated either based on the raw result values or by using a suitable performance indicator (Step 4). Next, the input for producing the compass plot is prepared, mainly including the specification of colours for 'impact factors' and 'outcome performance' and the layout (i.e., number and order of annuli) for the compass plot (Step 5). Finally, the compass plot is generated via the pseudo-code detailed below, and the important information is extracted based on the patterns observed in both the radial and annular directions of the compass plot (Step 6).

Interpretation of the compass plot
The merit of the proposed compass plot lies in the fact that the patterns identified in both the radial and annular directions reveal essential messages. In the radial direction, the first few slices (depending on the scope of the top grey level in the outermost annulus) in the counter-clockwise direction (the default order in a pie chart produced by MATLAB ® ) demonstrate the combinations of factors that have a great impact on outcome performance. Furthermore, within these slices, the most significant factor(s) and its corresponding value(s) can be visualised if one candidate value appears frequently in a certain annulus, especially if it happens consecutively.
Conversely, the insensitive factor(s) and/or its corresponding value(s) are more likely to be scattered around the annulus. Water is supplied to a flat region (elevation equal to zero) comprised of 31 demand nodes from one reservoir with a fixed head of 100 m. The required minimum pressure head at each node is 30 m. There are six commercially available pipe sizes, and their corresponding unit costs are listed in Table 1. As a result, the search space of the HAN problem contains 6 34 ≈ 2.87 × 10 26 discrete combinations.

Problem formulation
Two kinds of problem formulations for WDS design purposes were considered in this paper. In the first definition, Definition 1: a relaxed form of the least-cost design of WDSs The first definition is aimed at minimising both the capital cost and the total head deficit across the network. This transforms the hard constraints considered in the least-cost design of WDSs to soft ones. The mathematical definition of the relaxed least-cost design is given in Equation (1): where np is the total number of pipes in a network; nn is the total number of nodes in a network; C(D i ) is the unit cost of pipe i with a diameter size of D i ; L i is the length of pipe i; H j is the total head at node j; H min j is the minimum head required at node j; H d is the total head deficit across nodes; and A is the set of available pipe sizes in the local market.

Definition 2: the multi-objective design of WDSs
The second definition is aimed at minimising the capital cost and maximising I n , approaching the trade-off between cost and reliability. The mathematical definition of the bi-objective design is given in Equation (2): (2) where C j is the uniformity of node j which is defined in Equation (3); Q j is the demand at node j; nr is the number of reservoirs; Q k and H k are outflow and head at reservoir k, respectively; npu is the number of pumps; P m is the power supplied by pump m; γ is the specific weight of water; np j is the number of pipes connected to node j; and D i j is the diameter of pipe i connected to node j.

Scenarios
Two scenarios were demonstrated in this paper to show how the proposed compass plot can assist in exploring the   DI m and DI c turn out to be more critical parameters compared with P c and P m . This is probably surprising and inspiring as most previous studies using NSGA-II did not fine-tune DI m and DI c (Wang et al. b). When both parameters were set to 1, all the related combinations (eight in total) found the best-known solution at a Freq higher than 40%. Seven combinations out of eight were able to achieve a Freq higher than 50%, and three combinations had a Freq higher than 60%. In particular, for the best combination with DI m and DI c set to 1 (i.e., the second slice counter-clockwise), P c and P m were equal to 0. counter-intuitive as users who are non-experts in MOEAs are inclined to follow the default settings with application to problems they are interested in. Note that the differences in terms of HV can be increased when larger design problems are considered. This implies that using the default combination of operators in Borg can be suboptimal or even misleading. On the other hand, it is necessary to investigate the overall usage of each group of operators over 50 independent runs because Borg applies only one group per iteration for the recombination process. It is interesting to know whether a frequently used operator is an essential ingredient in Borg.

CONCLUSIONS
This paper proposes an innovative visualisation tool for investigation of the challenging combinatorial effects commonly found in many fields. The tool is called the compass plot and inspired by the traditional Chinese Feng Shui compass.
We demonstrate how the compass plot can be used to extract The compass tool applied to the least-cost design of the Hanoi problem reveals that PS is the most critical parameter of NSGA-II, followed by DI m and DI c . In contrast, P c and P m , which are generally regarded as key parameters by most previous studies, play a secondary role. More interestingly, when applied to investigate the synergistic effect of the auto-adaptive multi-operator strategy in Borg, this tool identifies the paradox hidden inside the advanced Borg framework. That is, weak operators with a minor contribution to the archive have a significant impact on the performance of Borg. In contrast, strong operators are not as essential as expected. As a result, for a specific type of problem such as the multi-objective design of WDSs, using the default combination of operators can be suboptimal or even misleading. The key messages identified by the proposed compass plot contribute new knowledge to the research community; thus, it can guide users who rely on these MOEAs for coping with more challenging problems in the real world.
Note that the compass plot is not restricted to EAs that are closely connected with optimisation. It can also be useful to demonstrate the hidden and complex interactions among various parameters associated with the calibration of simulation models where a set of best calibration parameters is needed. Despite the value of the compass plot, it is worth noting that users can still suffer the 'dimensional explosion,' in which many impact factors and their candidate values (e.g., more than two) are involved. It can lead to computational overheads that are too high to bear or far more complex patterns in the compass plot to interpret.
In such a case, users have to resort to alternatives for eliminating less important factors and limiting the number of candidate values. The compass plot is envisaged, however, to benefit the hydroinformatics community in deciding the effective values for key parameters for a given type of pro- blem. An online version of such a tool, which provides real-time feedback on the efficiency of parameterisation, may also contribute to the dynamic or interactive parameter settings to further boost the power of MOEAs.

DATA AVAILABILITY STATEMENT
All relevant data are available from an online repository or repositories (http://dx.doi.org/10.17632/cgh6bpzjfs.1).