A methodology for project portfolio selection under criteria prioritisation, uncertainty and projects interdependency – combination of fuzzy QFD and DEA

Resources of an organisation (people, time, money, equipment, etc) are never endless. As such, a constant and continuous challenge for decision makers is to decide which projects should be given priority in terms of receiving critical resources in a way that the organisation’s productivity and profitability is best guarantied. Previous literature has already developed a plenitude of project portfolio selection methodologies ranging from simple scoring to complex mathematical models. However, most of them too often fail to propose one integrated and seamless method that can simultaneously take into account three important elements: 1) prioritisation of selection criteria over each other, 2) uncertainty in decision-making, and 3) projects interdependencies. This paper aspires to fill this gap by proposing an integrated method that can simultaneously address all these three aspects. The proposed method combines Quality Function Development (QFD), fuzzy logic, and Data Envelopment Analysis (DEA) to accounts for prioritisation, uncertainty and interdependency. We then apply this method in a numerical example from a real word case to illustrate the applicability and efficacy of the proposed methodology.


Introduction
In every organisation there are many good project ideas hanging around waiting for an attention from the organisation executives to become a real living project. However, in a word of limited resources, each project is in a fierce competition with others to get a place in the top section of the priority list of the organisation. Not every project is viable and among those that are, the challenge is left for top managers /decision makers to judiciously decide which project should first come onboard to get the scarce resources of the organisation (staff, time, budget, equipment, etc). The challenging task of determining the combination of the projects which can collectively create the maximum business value for the organisation is referred to as "Project Portfolio Management" (Jiménez et al., 2017). Danila (1989) defines portfolio management as "selecting an investment from a list of candidate investments in order to maximize some objectives without violating constraints" (Ghapanchi et al., 2012).
Numerous theoretical and practical methods have been proposed for developing most profitable portfolio of projects, ranging from simple weighed scoring to sophisticated mathematical modelling. Some of those proposed methods use a single technique, such as weighted average (Eilat et al., 2006), fuzzy AHP (Huang et al., 2008, Tiryaki andAhlatcioglu, 2009), fuzzy DEA (Ghapanchi et al., 2012), multi-objective algorithms (Altuntas and Dereli, 2015), or goal programming (Kucukbay andAraz, 2016, Jiménez et al., 2017), while some use a combination of several techniques, such as DEA 1 and AHP 2 (Conka et al., 2008), fuzzy AHP and TOPSIS 3 (Sivzattian and Nuseibeh, 2001), Fuzzy DEA and TOPSIS (Tavana et al., 2015), or QFD 4 and AHP (Sivzattian and Nuseibeh, 2001). Most of these studies, however, have predominantly devoted their efforts towards developing models for identifying the best portfolio based on a given set of selection criteria; yet they do not focus on how such selection criteria themselves should be identified and prioritized in the first place. For example, Ghapanchi et al. (2012) suggest using Fuzzy Data Envelopment Analysis (DEA) for selecting project portfolio, but, as acknowledged in the paper itself (p.798), they specify the selection criteria (the feed to the DEA model) arbitrarily without rigorous analysis of the criteria to identify the high priority items systematically. Another example is Tavana et al. (2015) in which the authors introduce a hybrid model combining DEA, TOPSIS and integer programing for prioritising IT projects. While their hybrid model develops a quite sophisticated and comprehensive algorithm for project prioritisation based on any set of given selection criteria, they do not provide any guideline for determining such criteria. They totally leave it to the organisation to choose which selection criteria are critical for them. This is the case in a great majority of the papers in the area of project selection/prioritisation. A number of such studies are listed in Table 1. As can be seen in the table, many studies simply pick an 1 Data Envelopment Analysis 2 Analytic Hierarchy Process 3 Technique for Order of Preference by Similarity to Ideal Solution 4 Quality Function Deployment arbitrary set of selection criteria without proposing a solution for filtering or prioritising the many criteria that usually exist during the process of project selection. Some others consider prioritisation of the criteria but only as part of the project selection algorithm (i.e, AHP) (which makes it very limited to that particular algorithm). The lack of a focus on criteria selection is not due to unawareness of the importance of the criteria selection, but because the development of the project prioritisation models and algorithms is already a complex task. Thus, they have put aside the prerequisite step of criteria determination and have considered it out of scope. Nevertheless, provided that in the real word a large number of competing factors do exist when selecting projects over each other, it is necessary to have a mechanism in place to compare the criteria against each other and prioritise them.
To address this gap in the literature, in this paper we propose using Quality Function Development (QFD), in fuzzy form, for a systematic and rigorous identification and prioritisation of the various criteria involved in the selection of IT projects. QFD is a total quality management technique developed back in 1960s with the purpose of designing a new product in a way that the requirements of the customers are best taken into consideration, leading to a higher customer satisfaction (Akao & Mazur, 2003). While QFD was originally introduced in the manufacturing industry for new product development, it has been successfully spread to other applications (e.g. for supplier selection (Lima-Junior & Carpinetti, 2016)). Also, in order to account for the inherit uncertainty involved in the real-word problems and decision-makings, researchers have expanded QFD with fuzzy logic, known as fuzzy QFD (FQFD). FQFD stands as a suitable approach for modelling imprecise data (Lima-Junior and Carpinetti, 2016) and has been extensively used in many research domains (e.g., for supplier selection, ERP 5 selection and so on) but, as far as our literature review indicates, it has been rarely employed in the context of project selection and portfolio management.
For example, searching a wide search phrase of [project and ("fuzzy QFD" or FQFD or "F-QFD" or "Fuzzy Quality Function Deployment")] in Scopus (in title, abstract and keywords 6 ) return just 20 entries. Among the results, only Roghanian & Bazleh (2011) is about project selection; which is a conference paper in the thermal power energy and has used FQFD to rate some technical thermal factor and not for criteria selection and prioritisation. The others 19 papers are about either new product development (Kannan, 2008;Lee, Kang, Lin, & Chen, 2017;C.-T. Lin, Chang, & Mi, 2017;Shi & Xie, 2009;Van Luu, Kim, Truong, & Ogunlana, 2009;Vinodh, Manjunatheshwara, Sundaram, & Kirthivasan, 2017;Zhang & Chu, 2009), construction (Shi & Xie, 2009, Raut & Mahajan, 2015Y. Q. Yang, Wang, Dulaimi, & Low, 2003), six sigma (M. S. Yang, Li, Liu, & Gao, 2010), ERP selection (Sofyalıoğlu, 2012), selection of process in BPR 7 (Hakim et al., 2016), or project scheduling management (Liu & Yang, 2010). This is in spite the fact that project prioritisation is a highly complex task and a great deal of uncertainty and 5 Enterprise Resource Planning 6 As of 18 November 2017 7 Business Process Re-engineering ambiguity is involved (Ghapanchi et al., 2012) which makes FQFD a better choice than crisp QFD. In the present paper, we suggest employing fuzzy QFD to systematically determine and prioritise the criteria for selection of IT projects.
Another gap in the current literature of project selection is a lack of simultaneously considering two important factors: (i) uncertainties in the projects, and (ii) interactions among the projects, both of which are very common in real-world practice. While some researchers have catered for uncertainty in the real life, they have ignored the projects interdependencies (Chen and Cheng, 2009, Tiryaki and Ahlatcioglu, 2009, Huang, 2017. On the other end, while some studies have incorporated project interaction into their portfolio management methods, they have solely focused on deterministic environments without considering situations where uncertainty is involved (Basso and Peccati, 2001, Wang and Hwang, 2007, Bardhan et al., 2004. This limitation (i.e., considering either uncertainty or interdependency) has limited the applicability of such models in practice when uncertainty and interdependency do exist, resulting in no widespread use of those models in real-word situation (Eilat et al., 2006). An exception is Ghapanchi et al. (2012)'s study that has considered project uncertainty and project interdependency in one single model. In the present study, we adopt the approach proposed by Ghapanchi (2012) to accounts for both interdependency and uncertainty, and then extend it with fuzzy QFD to enable a systematic and rigorous identification of the selection criteria.
Given the above, this study is an effort to address two gaps in the literature of project portfolio management: (i) a lack of thoughtful, rigorous and systematic identification of high priority requirements/criteria for selecting the most profitable projects, and (ii) a lack of studies that take both uncertainty and project interdependency into consideration at the same time. To achieve these objectives, we use Data Envelopment Analysis (DEA) in combination with fuzzy Quality Function Development (FQFD). Our paper contributes to the literature of in three ways: Firstly, a great majority of the prior studies focus on proposing algorithms and models for projects prioritisation based of a given set of criteria (with no indication of how such criteria can/should be first identified). Our study will expand that understanding, and propose a systematic and rigorous approach for identifying and prioritising the criteria in the first place, using QFD. Secondly, by employing fuzzy logic and combining it with QFD, this paper will address high levels of uncertainty and ambiguity, which arguably exist in real-word situation when managers compare and choose between the projects. Thirdly, by using DEA, we take project interdependency into account. We apply the proposed model in a real anonymous IT organisation with 30 projects to illustrate the applicability of the approach. This paper is organised as follows: we first provide the essential background on the topic of research (including project portfolio selection, fuzzy QFD, and DEA). Then, the proposed model is presented and discussed, followed by explaining the numerical example. The paper will conclude with a discussion on the findings as well as outlining the limitations and providing directions for future research.

Project portfolio management and project ranking
According to the Project Management Institute, "portfolio, program and project management are all aligned and driven by organizational strategies, however, each one contributes differently to the achievement of strategic goals" (PMI, 2016). This critical role of portfolio sparked the appearance of the term "portfolio management" and an stream of research on project ranking and project selection started more than six decades ago when Markowitz used this term in the field of finance in 1952 (Markowitz, 1952) . The first use of portfolio management in IT projects goes back to 1981 by McFarlen. Since then, many theoretical and applied methods have been proposed to enable selection of a portfolio of projects with maximum benefits. Many of these works -similar to our work in this study -falls into the broad category of multi-criteria decision-making (MCDM) where the focus is on constrained optimization methods in which the goal is to select a subset of candidate projects to maximise a certain objective functions as per a given set of selection criteria, without violating the constraints (Danila, 1989).
As previously discussed in the introduction section of the paper, most of prior studies however only focus on proposing models and algorithms for project prioritisation (with the purpose of maximising the benefits) but do not argue how the selection criteria should be first identified and prioritised before the prioritisation process starts. Also, except to Ghapanchi et al. (2012), simultaneous consideration of both uncertainty and projects interdependency remains as a gap in the literature. Table 1 summarizes a list of prior publications with an indication on how the topic of criteria identification is dealt with in those papers, plus whether or not the paper accounts for uncertainty and interdependency. The table implies that no single study simultaneously accounts for the three aspects of 1) systematic and rigorous determination of the selection criteria, 2) uncertainty, and 3) interdependency, in one seamless solution.

Fuzzy set theory
Fuzzy Set Theory, introduced by Zadeh (1965), has been widely used to support decision making in situations where a decision is made under uncertainty and with imprecise data. In fuzzy logic, linguistic terms are used to qualitatively express the value of the variables which are then converted to quantitative values using fuzzy sets membership functions (Lima-Junior andCarpinetti, 2016, Zadeh, 1978).
Any fuzzy set has a membership function, and members of this set have a different membership grade. ! " # as membership function of fuzzy set maps each element of this set to real number in the interval 0,1 . For example, when the grade of membership is 0, it means that the element does not belong to that set. When the grade of membership for x is 1, it reflects that the element belongs completely to the fuzzy set. Ambiguous cases are assigned values between 0 and 1. As shown in Figure 1, a triangular fuzzy number can be determined by a triplet ' ( . ' * . ' , . The parameters that describe a fuzzy event are ' ( ,' * and ' , respectively, indicating the minimum possible value, the most probable value, and the maximum possible value (Zadeh, 1978, Zimmermann, 2010. In the following, we briefly discuss some definitions and properties on fuzzy set theory. The membership function of the fuzzy number ! " (.) is defined as: Let 9 = ' ( . ' * . ' , and ; = < ( . < * . < , be two triangular fuzzy numbers. Then the algebraic operational of these two triangular fuzzy numbers are defined as follows: (1) Addition and subtraction of two triangular fuzzy numbers 9 ± ; = (' ( ± < ( . ' * ± < * . ' , ± < , ) (2) Multiplication of two triangular fuzzy numbers: (3) Division of two triangular fuzzy numbers: (4) Multiplication of two triangular fuzzy number by a constant r:

Fuzzy QFD
QFD is a comprehensive and widely known quality management method, which was developed in 1960s and 70s to translate customer requirements into the characteristics of new services and products (Akao and Mazur, 2003) . In a QFD approach, customers' needs (usually called the voice of the customer) are converted into a set of detailed qualitative and quantitative requirements. These requirements help to engineer the features and characteristics of a product or service. QFD has been widely used in different areas such as project portfolio selection or supplier selection to determine the criteria for decision-making (Lima-Junior and Carpinetti, 2016, Chen et al., 2017). In general, QFD is applicable when the intention is to prioritize a list of objectives (HOWs) based on a list of requirements (WHATs).
QFD approach is based on a semantic visualization called "House of Quality (HOQ)" (see Figure  2) which facilitates transforming customer needs (WHATs) to design specification (HOWs). HOQ consists of several blocks as explained in the following (refer to Figure 2) (Brown, 1991, Chen et al., 2017. The QFD process begins with the QFD team (i.e., the experts) identifying the customer requirements (CR) (voice of customer, or WHATs) (block A) and determining the relevant importance score of each requirement (block B), via listening to the customers. Drawing on their judgment and experience, the QFD team translates the customer requirements into a number of design specification (DS) (or HOWs) (block C). In the next step, the QFD team judges which DS (WHAT) impacts which CR (HOW) and to what extent (block D). Also, the correlation between design specifications (HOWs) is identified by the team (block E). Using the data seated in block A to E, the importance score of the design specifications is calculated (block F), which is the aim of HOQ process. These scores will then be used in the process of developing the new product or service by informing which DSs should receive more resources to ensure that the relevant high importance customer requirement(s) are best satisfied.

Figure 2. House of Quality
Traditionally, all importance scores and weights in HOQ matrix are set as crisp values (Cohen, 1995, Chan and Wu, 2005, Terninko, 1997, but decision-making in the real word is always tied to vagueness and imprecision. In practical circumstances, the human judgment mechanism is uncertain and decision-makers may be reluctant or even unable to allocate crisp values to the comparison decisions Kumar, 2007, Amiri, 2010). Limiting the judgement to crisp values is one of the problematic challenges in the evaluation process. One reason is that people are usually more comfortable if they can state their judgement in the form of an interval choice rather than picking a single numeric option (Amiri, 2010). Another other reason is the mathematical models built upon crisp numbers. Those models are unable to adequately cater for the decisionmakers' ambiguities, uncertainties, vagueness and impreciseness which cannot be handles by crisp values (Shyur andShih, 2006, Amiri, 2010). Therefore, considering the inherent fuzziness in decision-making, fuzzy methods can greatly help tackling such limitations of QFD applications in real probelms (Chen et al., 2017), allowing to incorporate unquantifiable information, incomplete information; non-obtainable information and partially ignorant facts into decision model (Chan and Kumar, 2007). As a result, prior researchers have introduced different fuzzy QFD approaches (Temponi et al., 1999, Fung et al., 1999, Karsak, 2004, Juan et al., 2009, Dursun and Karsak, 2013, Lu et al., 2017 among which triangular fuzzy numbers and algebraic operations is a very popular and widely used approach. In this approach, fuzzy numbers are applied to model the linguistic judgments of the decision makers. For instance, the linguistic terms such as "very low, low, medium, high and very high" can be used to evaluate the degree of importance of a requirement and also the relationship between a requirement and a criterion (Lima-Junior and Carpinetti, 2016, Bevilacqua et al., 2006). We use this approach (fuzzy numbers and algebraic operations) in our study, which will be explained later in the proposed method section.

DEA
Data envelopment analysis (DEA) (Charnes, 1978) is a non-parametric multi-criteria decisionmaking technique based on linear programming. DEA computes the performance of a set of homogeneous decision-making units (DMU) by considering multiple inputs and outputs (Adler et al., 2002). In DEA, the optimal (i.e., the most efficient) DMU is achieved when no other DMU can generate more outputs with the same or lesser inputs. The efficiency of each DMU is calculated as the weighted sum of the outputs of that DMU divided by its weighted sum of the inputs, and the objective function (z in Equation 6) is to maximize this ratio. The constraint in Equation 6 controls that the weighting system for the relevant DMU cannot exceed the value of 1.
According to Equation 6, which is the basic model for DEA introduced by Charnes (1978) and known as Charnes, Cooper and Rhodes (CCR) model, the efficiency value of each DMU is computed by executing the model n times. In each execution, a different DMU is assessed and its possible efficiency is calculated in the situation where the efficiency of no DMU is greater than 1. If a DMU achieves score of 1, it is considered efficient, otherwise it's inefficient to the amount of distance to 1.

Proposed method
The model proposed in this study integrates fuzzy QFD and DEA methods ( Figure 3). The model consists of three main phases: (i) modeling the problem, (ii) HOQ construction for prioritization, and (iii) evaluation of maximal portfolio. In the first phase, we determine the criteria that affect IT project selection (WHATs) as well as the benefits of project portfolio selection (HOWs). In the second phase, we construct HOQ structure to priorities the different benefits of project portfolio selection through fuzzy QFD process. In the third phase, we use DEA to determine the most optimized project portfolio. Each step is composed of several steps, which are described in the following:  Step one starts with choosing the decision makers (DMs), or the experts, participating in the process of selecting the best project portfolio for the organization. In our proposed method, these DMs are responsible for first, refining and confirming the project selection criterion and project portfolio benefits for their organization (in phase 1), second, determining the required importance factors to build the HOQ matrix (in phase 2), and third, evaluating the benefits of the projects for DEA analysis (in phase 3).

Step 1b. Identifying initial list of WHATs and HOWs from literature
Various factors influence the selection of IT projects. There is no lack of research in the literature elaborating on such factors, with a great deal of overlap and consensus among them. For this paper, we use the factors addressed by Meskendahl (2010) Table 2). The reason for choosing these sources is that they either have aggregated a list of important factors from previous studies or have been well accepted and cited by other researchers. These factors will serve as the HOWs (project selection criteria) later on in the process when we construct HOQ structure for fuzzy QFD analysis. We should also determine the expected benefits of the IT project portfolio implementation which will serve as the WHATs in the HOQ. In this paper we use the aggregated list of benefits proposed by Meskendahl (2010), Caron et al. (2007), Cooper et al. (2002), DeLone and McLean (1992), 8 Note that the "dependency to other projects (B6)" factor in Table 2 is just an input factor that came out from the literature review when we were searching for DEA inputs (similar to all other factors listed in that Table). This factor doesn't have anything to do with the interdependency concept that we have incorporated procedurally into our proposed method. They are very different in nature. The former is just one of the DEA inputs (which happens to be about dependency) while the latter is a concept imbedded into the proposed model (calculated via Equation 14 to 16). Kendall and Rollins (2003), Beaujon et al. (2001), De Reyck et al. (2005 (see the 7 factors in Table 3).

Step 1c. Refining and confirming WHATs and HOWs
While an initial list of WHATs and HOWS were identified in the two previous steps, some of them may not be applicable in the specific situations of a particular company (Tavana et al., 2015). At the same time, there might be some additional factors that are of interest to a specific organization due to its unique internal or environmental conditions. Thus, in our proposed method, the factors listed in Table 2 and 3 are presented to the DMs to make sure that a valid and reliable list of WHATs and HOWs is used in the portfolio selection. The DMs may cross out some factors or add others considering the characteristics of their organization.
The output of phase 1 is a screened and validated list of WHATs and HOWs tailored to the needs of the organization, ready to be fed into the next phase to generate the house of quality (HOQ) matrix.

Phase 2. Fuzzy QFD analysis to determine criteria priority
In phase 2, the house of quality structure is built to calculate the relevant importance of the selection criteria and the benefits, and prioritize them based on the judgment of the DMs and the fuzzy QFD algorithm.

Step 2a. Linguistic evaluation of the importance of WHATs by the DMs
When the benefits (WHATs) and the selection criteria (HOWs) have been determined (in phase 1), then the decision makers judge the important of the benefits using fuzzy logic. In our proposed method, we use the linguistic terms very high, high, medium, low and very low (Lima-Junior and Carpinetti, 2016, Bevilacqua et al., 2006).

Step2b. Prioritizing HOWs as per Fuzzy QFD algorithm
Once the linguistic importance of the WHATs are judged by the DMs, we use fuzzy set logic, as explained below, to translate the linguistic terms into quantified fuzzy values. Then fuzzy QFD procedure is applied to prioritize the benefits of project portfolio (HOWs). This is the key contribution of our proposed method in which, unlike other studies (e.g., Ghapanchi et al., 2012, Tavana et al., 2015 , the criteria are prioritized before feeding into the MCDM step to determine the optimum portfolio. Table 4 and Figure 4 shows the triangular fuzzy membership function used in this paper. The reason for choosing a triangular fuzzy function is that it is intuitively easy to use and compute for the decision makers. Moreover, modeling with triangular fuzzy members has proven to be an effective modeling approach where the decisions are made relying on subjective and imprecise information (Chang et al., 2007, Zimmermann, 2010, Amiri, 2010, Kahraman et al., 2004. In practical applications, the triangular membership function has been the most frequent form of fuzzy numbers being used (Xu andChen, 2007, Amiri, 2010).  Assume 9 R be the fuzzy number of WHATs (benefits of project portfolio selection) that shows the experts opinion about the S TU requirement (i=1,2, . . .,m).
' R,( , ' R,* , ' R,, Let @ XY be the linguistic expert judgment of the relationship between the S TU benefit (i=1,2,…,m) and Z TU criterion (j=1,2…,n), made by the 0 TU decision maker (x = 1, 2, . . ., k). The aggregated judgments of the decision makers regarding the relationship between the S TU benefits and the Z TU criterion, represented by @ XY , is calculated by: in which [ XY is the aggregated judgment of the decision makers, and x refers to the DMs.
\ ] * is a fuzzy number that is called. The weighted mean of the Z TU criterion is calculated as: After that the absolute weight of the Z TU criterion, \ ] * , is calculated as: After defuzzification, the absolute weight of the Z TU criterion is converted to the relative weight cd ] according to:

Phase 3. Evaluation of maximal portfolio
So far up to this point in our proposed method, the expected benefits from the optimum project portfolio are identified and prioritized. Now, these prioritized benefits can be used as a set of criteria to determine the most effective portfolio of the projects for the organization. To do so, in phase 3 of our opposed model, we use DEA technique to assess the maximal portfolio.

Step 3a. Categorize HOWs into DEA inputs and outputs
DEA technique consists of input and output criteria or variables. Thus, before feeding the HOW variables into the DEA algorithm, they should be categorized into inputs and outputs as per the nature of the variable. DEA Literature suggests if the criterion is about the objectives that the organization seeks to achieve out of the decision making process, it is an output. On the other hand, if the criterion is about characteristics or solutions towards achieving the objectives, then it is an input (Lim et al., 2014). For example, criteria relevant to revenue or profit are usually considered as output (as they are among the major objectives of any firm) whereas criteria relevant to planning or operation strategies are categorized as input (as they are about the ways to achieve the objectives) (Lim et al., 2014) .

Step 3b. Cut off the input and output criteria (if needed)
DEA algorithm is sensitive to the number of input and output criteria in the model. Too many input or output variables will reduce the usefulness of the ranking in practice as the calculated numbers become too close. It is suggested that the combined number of input and output variables should be smaller than the number of DMUs divided by 3 (see Equation 13) (Bowlin , 1998). In the context of our study, it means that the total number of the project selection criteria (input plus output) fed into the DEA algorithm should be three times smaller than the number of projects. Thus, if the number of selection criteria is high, those ones that are found to be less important (as the result of fuzzy QFD analysis carried out in phase 2) should be left out in a way that Equation 13 is satisfied. This is another benefit of using fuzzy QFD analysis for prioritizing the selection criteria to cut-off the less important factors thoughtfully. The DMs collectively decide which criteria to drop in accordance to the outcome of the fuzzy QFD analysis. In the last step, DEA is used to rank and priorities the projects based on the input and output criteria established previously. In real word situation, some of the criteria may be crisp data such as implementation training cost or return of investment. Others may need to be estimated by the expert DMs. The data is then fed into the DEA procedure (explained in section 2.4) to calculate the optimal portfolio of the projects.
As mentioned previously, this paper takes project interactions into account using an input (Equation 14) and output (Equation 15) accumulation function. We customized the equations proposed by Ghapanchi et al. (2012). In this paper, interactions are classified into three class: input interactions, output interactions and possibility interactions. The interactions among the inputs, outputs and probability of success are included in the interaction matrices. Matrix n R represents the i th input interaction (in which M ]V R is the value interaction between project j and project k for input i). Matrix o N represents the r th output interaction (in which Q ]V N is the value interaction between project j and project k for output r). Finally, matrix P represents probability (in which p ]V shows the marginal likelihood of the success of project j when project k is participating in a portfolio comprising project j. The crisp value of these matrixes are collected from the DMs and the averages of the input, output and success likelihood are computed. After computing the efficiency scores of each project (comes from Equation 6), input and output interaction as well as the probability are applied to finalize the ranking of the projects (Equation 16).
Expected value for each project:

Numerical example
Similar to the majority of the cutting-edge studies on project portfolio selection methods, this paper uses a numerical example to demonstrate an application of the proposed method in a real word anonymous scenario. This example relates to a large scale Tele-Communication Company (called TCC hereafter) with more than 60 million active users. The Project Management Office (PMO) of the company constantly faces the challenge of choosing the most efficient portfolio of the projects subject to the monitory limitations of the company. In the following, we illustrate how the proposed method can systematically assist with such decision-making. For the scope of this example, 30 IT projects (i.e., DMUs) of the company are considered.

Step 1a. Forming the team of decision makers (DMs)
Phase 1 begins with forming up the team of expert decision makers. Four expert were chosen in consultation with the Project Management Office. The reason for selecting those four people was that they were fully aware of the digital strategy of the organization and understood the 30 projects well. They all had postgraduate degrees (Masters or PhD) with at least 5 years of work experience in TCC in middle to top managerial positions.

Step 1b and 1c. Identifying initial list of WHATs and HOWs from literature and refining them
Once the DMs were identified and briefed, the next action is to identify the list of WHATs and HOWs that suit the situation of the organization. Each DM was given the list of project selection criteria (HOWs) and the project portfolio benefits (WHATs) extracted from the literature (Table 2 and 3) and were asked if these lists represent the needs and situation of their company correctly. The DMs confirmed that the lists are both OK and did not suggest adding or removing any factors.

Step 2b. Prioritizing HOWs as per Fuzzy QFD algorithm
This step starts with determining the HOWs and WHATs correlations. To do so, we asked each decision maker to express their opinion of the impact of each HOW on each WHAT using same linguistic variables (very high to very low in five intervals). We then applied the same triangular fuzzy membership function to quantify the linguistic terms. The outcome is depicted in Table 7. The fuzzy numbers obtained for each decision-maker were then aggregated by means of the following equation, as presented in Table 8.  Finally, the relative weight , \e ] , of each item is calculated according to absolute weight of the j th criteria (Table 11).

Phase 3. Evaluation of maximal portfolio
The weighted list of benefits obtained at the end of Phase 2 (Table 11) forms a solid foundation for Phase 3 in which DEA algorithm is applied to determine the optimum portfolio of the project that can create the maximum value for the organization (as per identified benefits). In the following, we take our example organization through the three steps of Phase 3 to identify the best project portfolio for this organization.

Step 3a. Categorize HOWs into DEA inputs and outputs
Amongst the 14 project selection criteria listed in Table 1, we categorized four of them, namely, alignment with strategic objectives (B1), ROI (B8), implementation cost (B12), and net present value (B14) as outputs because they are primarily about revenue or profitability which is a major objective criterion for a firm (Lim et al., 2014). We categorize the other ten factors as input because they are more relevant to projects characteristics and operation towards achieving revenue or income.

Step 3b. Cut off the input and output criteria
As stated in section 3.3.2, DEA is sensitive to the number of input and output variables and the best DEA analysis happens when there is a right balance between the number of variables and the number of decision-making units (see Equation 13). Noting that in our numerical example, the number of DMUs (i.e., projects) is 30, thus, as per Equation 13, the total number of input and output variables should not exceed 9, but we do have 14. Accordingly, in consultation with the expert panels from the organization, we excluded 1 output (B14) and 4 input (B4, B5, B6, B7) variables, leaving us with 9 criteria to get into the DEA analysis (3 outputs and 6 inputs). The excluded factors were the least important items as per FQFD analysis in Phase 2 (in each category).

4.3.3.
Step 3c. Evaluating the maximal portfolio using DEA Finally, using a questionnaire, the four DMs (the same experts engaged in the previous steps) were asked to score the 30 IT projects (P1 to P30) against the nine criteria (3 outputs and 6 inputs). Fivepoint Likert scale was employed to collect experts' opinion. We then used GAMS software to perform DEA analysis and calculate the efficiency score of each decision-making unit (i.e., project). The result is presented in Table 12. It is worth mentioning the fact that two-third of the projects in Table 12 have received efficiency factor of 1. This is a typical outcome in many DEA analyses, especially when the number of input and output variable is relatively large (e.g., 9 in our example) (Eilat et al., 2008). In such cases, when a large number of alternatives receive the maximum efficiency score, there is a number of solutions to further differentiate between the alternatives. One is to use Fuzzy DEA instead of crisp DEA to increase the sensitivity of the analysis (Ghapanchi et al., 2012). Another is to use techniques complimentary to DEA analysis such as Anderson-Peterson ranking (Andersen and Petersen, 1993). Taking these two approaches however is not in-line with the purpose of our work and we consider it out of scope. However, this issue is irrelevant in our research because our proposed method considers the project interdependency which will further differentiate between the value of the projects for a better decision-making, as explained in the following: In our proposed method, the interactions among the input, outputs and probability of success for each pair-wise project are calculated and applies as per Equation 14 to 16. Here we explain one sample for illustration. Let's consider the 'risk of project' (input B2, i=2), and 'implementation cost' (output B12, r=12), for project P1 and P10 (j=1 and k=10). We asked the DMs to judge M ]V R and Q ]V N . The average of DMs opinion was that the risk of the portfolio will be increased two times if project P1 and P10 are selected simultaneously (hence M (,(á * = 2) but no interaction for implementation cost (hence Q (,(á (* = 0). Similarly, for P2 and P7, there was no interaction of risk but the cost would increase 6.2 units (thousand dollars) if the two projects are included in the portfolio at the same time (i.e., M *,à (* = 1 and Q *,à (* = 6.2). Table 13, as an example, shows the interactions among the risk input (i=2) and cost output (r=12) and the success probability of a number of pair-wise projects. Finally, the impact of interdependency on the efficiency scores is calculated using Equation (16) resulting in the final expected value for each project as depicted in Table 14. The highest values are the best candidates to form the most profitable portfolio (i.e., P17, P26, and so on).

Discussion and recommendations for future studies
This paper adds to the literature of project portfolio selection by introducing a new approach based on a combination of fuzzy QFD and DEA methods. The proposed method comprises three phases, modelling the problem, prioritizing selection criteria, and calculating the maximal portfolio, as well as several distinct steps within each phase (refer to Figure 3). In our approach, fuzzy QFD is used to prioritise the many project selection criteria that typically exist in any practice of project investment decision-makings in organisations. Data envelopment analysis technique is then applied to this set of prioritised criteria to determine the best portfolio of the projects that can bring in the maximum value for money.
Prior studies have developed various methodologies for portfolio selection under the situations where there are dependencies between the projects but in stochastic environments. At the same time, others have proposed selection techniques when an uncertainly exists during the project selection decision-making process, but projects are isolated. This reflects a lack of prior research on simultaneously considering both 'uncertainty' and 'projects interdependency' in one single method (Ghapanchi et al, 2012). Moreover, prior methods have largely focused on the selection technique itself as the focal point of their method, and tended to ignore explaining how the selection criteria should be determined and prioritised in the first place. For example, in their proposed approach based on fuzzy DEA, Ghapanchi et all (2012) states that their selection model assumes that a couple of selection criteria is simply picked by the decision makers. They do acknowledge that this is an oversimplification and urge future studies to develop seamless and integrated methodologies that not only recommend selection algorithms but also propose solutions for determining and prioritising the selection criteria in the first place.
This paper therefore contributes to the literature of project portfolio selection by responding to two calls: first, the call for more studies that concurrently incorporated both projects interdependency and uncertainty in one single approach, and second, the call for expanded models that consider solutions for selection criteria prioritisation before entering the decision making process itself. In our model, the proposed DEA approach caters for project interdependencies, the fuzzy logic accounts for uncertainty, and QFD enables prioritisation of the selection criteria.
Having applied the proposed method to a practical case in a telecommunication company, we observed that 'alignment with the organisation strategic plan' factor sat on the top of the list when the decision makers determined the importance of the project selection criteria. This is perfectly in line with a well-approved premise in the IT literature stating that IT investment creates value and competitive advantage only when it is aligned with business strategy, and not just an amusement with technology (Luftman, 2004, Maes et al., 2017. At the same time, the importance of other factors such as support from top managers and acceptance should not be overlooked as they sat right in the next spots of the priority list in our sample case, with no big difference in importance scores. All these findings from our numerical example matches the principles of project selection in IT literature, however we should highlight that, in our paper, they are all comes out from one single case only. Any generalisation should be made carefully and thoughtfully as per the situation of each organisation. Determining the most important factors for portfolio selection is closely tied to the contextual and organisational characteristics of each organisation and thus a significant part of our proposed method (phase 2) is devoted to determining the importance and priority of decision factors (to be made carefully and thoughtfully by the company's panel of experts, i.e., DMs).
Along with its contributions, this study poses limitations that open opportunities for future research. Firstly, we put together a list of project selection factors and portfolio benefits (Table 2 and 3) from a handful of prior studies in the literature. The factors are not coming out from a systematic literature review. However, we consider this an acceptable list as the main contribution of this paper was meant to be introducing a quantitative methodology for project selection based on a set of prioritised criteria. A comprehensive and systematic review of literature was not the purpose and the focus of our work. We mitigated this limitation by including a step in our proposed method in which the decision makers of the company review and refine the list of selection factors as per the needs of their organisation (step 1c). This will reduce the risk of missing any important factor. Nevertheless, there is an opportunity for future research to bring together a more comprehensive list of factors using systematic review approach or similar.
Secondly, unlike some prior works, we used crisp DEA rather than fuzzy DEA. The reason was to ovoid making our method overly complex. Our primary intention was to showcase how QFD (in fuzzy form) and DEA can collate together to address our research objective. Readers interested in fuzzy DEA can refer to, for example, Ghapanchi et al (2012) to learn how DEA can be used in a fuzzy manner (instead of crisp) in phase 3 of our proposed method. The logic and process of our method is still perfectly applicable.
Thirdly, we do not argue that our hybrid proposed method is the best match among all alternative approaches. Numerous MCDM techniques are available (e.g., AHP, TOPSIS, linear programming, and so on). Comparing between all these techniques was not the purpose of our research and requires a separated study in a different setting and with its own objective.