Measurement Methodological reflection on Case Study 2

Success of the product innovation project was determined with a ques- tionnaire that was filled out by the project manager of that project. Project managers rated, for a number of success indicators on a five- point scale, whether there had been a “disappointing” performance or one “well beyond expectations”. Successful projects were defined as projects with an average score of three equal to expectations or higher. The 15 successful projects in which the hypothesis was tested were selected according to this criterion: their performance had not disappointed the project managers. Several questions could be raised regarding the measurement validity of success, when success is meas- ured by the “degree to which expectations have been met”: the problem of measurement validity of success is briefly addressed in 5.4.11, where reasons for not confirming the hypothesis are discussed. Case Study 2 also provides a quite detailed description of how type of innovation was determined. It is clear that this categorization was not achieved by a straightforward application of a set of clear-cut decision rules. The values of the four dimensions of the organizational configuration coordination integration, ownership integration, task integration, and knowledge integration were derived from a qualitative interview with the project manager and, for two dimensions, compared with the project manager’s rating on a five-point scale in a questionnaire. No major deviations were found between these two assessments.

5.5.8 Data presentation

Case Study 2 does not give detailed descriptions of the different projects such as provided in Case Study 1, which would enable experts who know one or more projects to evaluate the correctness of, for example, the categorization of type of innovation or the estimation of success. For all 15 successful projects, all relevant data for testing are provided in Table 5.5, i.e. its type of innovation and the organizational configuration in terms of the values of the four organizational dimensions.

5.5.9 Data analysis

The hypothesis-testing consisted of comparing the “observed” typolo- gies with the predicted ones see Table 5.5. The four values for the organizational dimensions formed an observed “pattern” that could be compared with the ideal typical configuration. This test was performed for each case separately and each test result rejection or confirm- ation was evaluated on a case-by-case basis.

5.5.10 Implications for the theory

The hypothesis that successful product innovation projects possessed ideal typical organizational configurations was rejected. None of the six ideal types proved to be a necessary condition for initially expected project success in these 15 cases. Case Study 2 suggests, as one possible implication for the theory, that the proposition itself might not be correct. However, assuming that the exploration phase at the very beginning of the research project was conducted in a serious manner and that, thus, the proposition that was formulated and tested was based on sound practical and theoretical insights, such a conclusion would be a significant one that cannot be drawn lightly, and other possible reasons for the rejection of the hypothesis in all 15 cases should be evaluated. Below we elaborate on the evaluation as presented in 5.4.11. 1. Case Study 2 rejects the likelihood that the test results were the result of a too-strict test procedure. The reason for this rejec- tion was that “a large number of the successful projects deviate not marginally but rather substantially from the predicted pro- file”. However, if something was wrong with the measurement of the four organizational dimensions, this would have had a direct effect on the test. If it is assumed that the measurement of these dimensions was too unreliable for justifying the mid- dle value M and if, for that reason, the researchers would be forced to decide whether the value is H or L, it is possible that a number of tests would have resulted in a confirmation of the hypothesis. Repeating the same test on the same cases with dif- ferent measurement procedures might yield different results. 2. It might be that the ideal type itself not the test was too strict. Why should it be necessary for a success to occur that an organizational configuration is exactly as prescribed on all four dimensions, and for all types of product innovation? Could it be possible that having an ideal typical organizational configuration consisting of only three dimensions is a neces- sary condition for success for one type of product innovation, and an ideal type consisting of specific values of another set of