Product Selection using AHP

(An old paper of mine)

VALUATING PRODUCTS FOR ARCHITECTURE DESIGNS

Problem Statement:

How do we decide on the product for a particular architecture service?

Assumptions:

The following assumptions are made pertaining to the problem statement-

· It is assumed that there is a provision in the project schedule for the selection activity and it is based on the customer needs

· The set of products that are shortlisted are agreed upon with the customer before the evaluation commences

· The Requirements document is ready and available to the architect

· Business process understanding and corresponding document is available

· List of architectural services is well defined

Inputs Required

The following inputs are needed to aid in the decision making -

· Requirements document

· Architectural services

· Business process diagram

· Document on the existing customer architecture

· Short listed products to be evaluated

Parameters:

The parameters should be considered along the following dimensions for arriving at a decision –

1) Technology dimension (pertaining to technical requirements)

2) Feature dimension (pertaining to Customer/project/application specific requirements)

3) Vendor dimension (pertaining to Vendor Quality)

Each of these dimensions are given an overall weightage. In the example illustrated in this document, the weightages given are as follows –

Technology dimension – 40%, Feature dimension – 40%, Vendor dimension – 20%

These weightages can be altered based on the scenario considered.

The detailed parameters along each dimension are as follows:

1) Parameters pertaining to the technology:

· Platform

· Integration capabilities

· Security

· Scalability

· Customization policy

· Enhancement policy

· Training availability

· Documentation availability

2) Parameters pertaining to feature

· Elementary requirements – customer requirements, project specific requirements, application specific requirements (Example of feature parameters – the application is a banking application)

3) Parameters pertaining to Vendor dimension:

· Financial stability

· Implementation experience

· Support

· Willingness to work with other vendors

The weightages for the parameters within any dimension could also be altered depending on a case-to-case basis. The weightages shown in the example in the next section are only illustrative.

Decision making technique applied:

The decision making technique that is considered is a simple weightage and ranking method to quantify the benefits from various candidate products and arrive at the best option.

The parameters to be considered are all listed down and appropriate weightages are given considering the customer’s specific needs. The parameters are considered along three dimensions – vendor, technology and features. Each of these dimensions is assigned a different weightage.

The candidate products are ranked on each of the parameters considered on a 10-point scale based on the data available.

The score with respect to any parameter is the product of the parameter ranking and its weightage. The cumulative score is the summation of all the parameter scores. The product that scores the most could be a likely choice. The final decision has to be made by the architect responsible for the decision making. The decision making technique only facilitates the architect to eliminate the alternatives that are not viable.

The technique can be modeled using a template scorecard. The user interface would be as represented below –

Dimension	Criteria	Weight (in %)	Score	Weighted score	Weighted Total
Vendor (20% of total)	Financial stability	10	9	0.9
	Implementation experience	30	6	1.8
	Support	40	8	3.2
	Willingness to work with other vendors	20	9	1.8
			Sub total	7.7	1.54 (20% of 7.7)
Technology (40% of total)	Platform	20	9	1.8
	Integration capabilities	15	7	1.05
	Security	25	8	2
	Scalability	25	9	2.25
	Customization policy	2.5	6	0.15
	Enhancement policy	2.5	8	0.2
	Training availability	7.5	8	0.6
	Documentation availability	2.5	6	0.15
			Sub total	8.20	3.28
Features (40% of total)	Elementary requirements (based on customer/project/application specific requirements)	Give weightage to each of the features. Alternatively, give an overall rating	8	8
			Sub total	8	3.2

Overall product score = 8.02

Alternatives:

If two or more candidate products have the same overall score, we need to make a call based on the parameters that are prioritized by the customer.

In case the customer inputs are not available, there could also be a cost benefit analysis done for the products that score similarly. The costs would involve the initial investment cost and the maintenance cost.

Evaluation Criteria:

The evaluation criteria to select the optimum product are based on the overall product score obtained by the candidate products.

Risk:

The following are the risks identified in using the stated decision making technique –

· Eliciting of the proper requirements with priority

· Eliciting of the customer ranking for technical requirement

Evaluating Products for an Architecture –

An AHP Based Approach

Product selection using AHP

Brief About Analytic Hierarchy Process (AHP)

AHP provides a means of decomposing the problem into a hierarchy of sub-problems which can more easily be comprehended and subjectively evaluated. The subjective evaluations are converted into numerical values and processed to evaluate each alternative on a numerical scale. The detailed methodology of AHP can be explained in following steps:

Step 1: Problem is decomposed into a hierarchy of Goal, alternatives, criteria and sub criteria.

Step 2: Data is collected from experts corresponding to the hierarchic structure, in pair wise comparison of alternatives on a qualitative scale as described below. Expert can rate the comparison as equal, marginally strong, strong, very strong, and extremely strong. The opinion can be collected in a specially designed format as shown in Fig 1.

A								X		B
Extremely Strong		Very Strong	Strong	Marginally strong	Equal	Marginally strong	Strong	Very Strong	Extremely Strong

Fig 1: Format for pair wise comparisons

‘X’ in the column marked very strong indicates that B is Very Strong compared to A with respect to the criterion on which the comparison is being made. The comparisons are made for each criterion and converted into quantitative numbers as per Table 1.

Table 1: Gradation Scale for quantitative comparison of alternatives

Option	Numerical Index
Equal	1
Marginally Strong	3
Strong	5
Very Strong	7
Extremely	9

Step 3: The pair wise comparisons of various criteria generated at step 2 are organized into a square matrix. The diagonal elements of the matrix are 1. The criteria in the ith row is better than criteria in the jth column if the value of element (i,j) is more than 1; otherwise criteria in jth column is better than criteria in ith row. The (j,i) element of the matrix is reciprocal of (i,j) element.

Step 4: The principal eigen value and the corresponding right eigen vector of the comparison matrix gives the relative importance of various criteria being compared. The elements of the normalized eigen vector are termed weights with respect to the criteria or sub criteria and ratings with respect to alternatives.

Step 5: The consistency of the matrix is then evaluated. It is often the case that people’s judgments are not always consistent. Comparisons made by this method are subjective and AHP tolerates inconsistency through the amount of redundancy in the approach. If this consistency index fails to reach a required level then answers to comparisons may be re-examined. The consistency index, CI is calculated as

CI = (l_max – n)/ (n-1)

Where, l_max is the maximum eigen value of the judgment matrix. This CI can be compared to that of a random matrix, RI. The ratio derived, (CI /RI) is termed the consistency ratio (CR). Saaty [1] suggests the value of CR should be less that 0.1.

Step 6: The ratings of each alternative is multiplied by the weights of the sub-criteria and aggregated to get local ratings with respect to each criterion. The local ratings are then multiplied by weights of the criteria and aggregated to get global ratings.

AHP produces weight values for each alternative based on the judged importance of one alternative over another with respect to a common criterion. In the next section we will use AHP to define a framework for Product Evaluation.

AHP for the Problem Stated in the document

The Hierarchy for the problem is shown below.

The evaluations at each level of the hierarchy are given below.

At Level 1, the product ratings depend upon three parameters – Vendor(Ve), Technology(Te) and Features(Fe). The client is asked to pairwise compare the three parameters in terms of importance to be given to each parameter with respect to product rating. The format for asking these questions may vary. A typical format is given below with ticks obtained from client in the hypothetical example.

Subject: Importance of Vendor, Technology or Features should be given _______ more, while evaluating products for an architecture

	E	V	S	M	EQ	M	S	V	E
Ve								Ö		Te
Ve								Ö		Fe
Te					Ö					Fe

E : Extremely

V: Very Strongly

S: Strongly

M : Marginally

EQ : Equal

The above comparison chart is converted to the following reciprocal matrix as per the established scale.

Parameters	Ve	Te	Fe	Normalized Eigen Vector (NEV) (Relative weights)
Vendor	1	1/7	1/7	0.066
Technology	7	1	1	0.467
Features	7	1	1	0.467
	Eigen Value	3.01	C.I. & C.R.	0.0 & 0.0

At the second level also the client is asked to give his preferences. Let the matrix for Parameter - Vendor comes out to be

Relative Importance of Factors affecting the parameter- Vendor
	Financial Stability	Experience	Support	Willingness	NEV
Financial Stability	1	1/3	1/4	1/2	0.095
Experience	3	1	1/2	2	0.277
Support	4	2	1	3	0.467
Willingness	2	1/2	1/3	1	0.160
	Eigen Value	4.03	C.I. & C.R.	0.01 & 0.011

Let us assume that the relative importance of criterion in Technology and Features comes out to be as given in Tables below.

Relative Importance of Factors affecting the parameters- Technology and Features
Technology		Features
Platform	0.25	ER A	0.25
Integration Capability	0.35	ER B	0.35
Security	0.40	ER C	0.40

Next Step is to compare alternatives (P1, P2 and P3) with respect to leaf parameters of the hierarchy, i.e., for Financial Stability, …, Elementary Requirement C, client is asked to compare P1,P2 and P3.

Let these comes out to be as shown in the Table below. Table also shows the calculations to combine all these weights at various levels of the Hierarchy.

Parameter	Weights	Criteria	Weights	P1	P2	P3
Vendor	0.066	Financial Stability	0.095	0.35	0.3	0.35
		Experience	0.277	0.4	0.4	0.2
		Support	0.467	0.1	0.2	0.7
		Willingness	0.16	0.2	0.3	0.5
Local Rating (Vendor)				0.223	0.281	0.496
Technology	0.467	Platform	0.25	0.2	0.3	0.5
		Integration Capability	0.35	0.3	0.4	0.3
		Security	0.4	0.3	0.2	0.5
Local Rating (Technology)				0.275	0.295	0.430
Features	0.467	Elementary Feature A	0.25	0.5	0.3	0.2
		Elementary Feature B	0.35	0.2	0.2	0.6
		Elementary Feature C	0.4	0.5	0.35	0.15
Local Rating (Features)				0.395	0.285	0.320
FINAL RATINGS				0.328	0.289	0.383

In this hypothetical scenario P3 is almost 0.383/0.328 = 1.169 i.e. around 17% better than P1 and 32% better than P2.

*****