Candidate cases Case Study 2: Theory-testing research: testing a necessary condition

5.4.8 Measurement

For checking whether the case innovation project was successful and therefore could be included in the study, success was determined with a questionnaire that was filled out by the project manager of that pro- ject. Items on project performance in our questionnaire asked for spe- cific judgements regarding: meeting the time-to-market deadline; adherence to interim project deadlines; quality of the project; and budget performance of the project. A control item asking for an over- all judgement of project performance was also included. For each indi- cator we measured actual performance relative to expectations as perceived by the project managers on a five-point scale ranging from “very disappointing performance” to “a performance level well beyond expectations”. First, the average score for the first four items was cal- culated. Next, to reduce measurement error even further, we averaged the score for “overall project performance” with the average for the four items. Successful projects were defined as projects with a score of three which means that the project performed in line with expect- ations or higher. From the 30 projects that we analysed, we identified 15 successful projects; hence, our cases. For each case, the type of innovation was determined based on the qualitative project descriptions that we had collected. Additionally, the project manager filled out a questionnaire to determine a project’s degree of interface change using a four-point rating scale about “the degree of uncertainty regarding the interfaces to connect the applica- tion to the network” and “the degree of standardization of the plat- form to which the application was connected”. This latter scale ranged from “no standards” to “highly standardized”. Usually, newly intro- duced networks employ tailor-made platforms, whereas over time stand- ardized platforms emerge that manage the development and interconnection of applications. To rate a project’s degree of component technology change, we used a rating scale for “the uncertainty regarding the costs to develop this application”. For the distinction between core and peripheral projects we also primarily drew on the interview data with the project manager. We followed Gatignon et al. 2002 who characterize core components as strategically important to the firm andor tightly coupled to the larger system. During the interviews, we assessed the strategic importance of the application to the operator. We could cor- roborate these findings using data on the questionnaire item asking for “the urgency felt by the network operator to introduce this appli- cation quickly”. We hypothesized that operators experience high urgency for strategically important applications in order to build