Strategy for testing a proposition that expresses a deterministic relation

The survey might be used to test a deterministic relation as well. For a test in a survey, any population can be selected from the domain. The statistical analysis could compute for each pair of instances in the sample or in the population whether an observed difference in the values of the dependent concept B in the two instances of the pair corresponds in the way predicted by the proposition with the differ- ence in the values of the independent concept A. It is tested in this population whether the frequency of occurrences of pairs of instances in which B does not follow A in the predicted direction is zero as expected if the proposition is true in the population or is very small according to a “pragmatic determinist” criterion, as discussed in Box 8. The hypothesis is rejected if the proportion of instances with the values A absentB present is larger than zero or larger than the proportion specified. The same argument about inefficiency of the survey as dis- cussed above for the use of the survey for testing a sufficient or a nec- essary condition applies here as well.

4.4.4 Strategy for testing a proposition that expresses a probabilistic relation

The experiment is the preferred research strategy for testing a prob- abilistic relation. The effect of an independent concept causal factor A is investigated by comparing the change in value of a dependent concept effect B in an experimental group which was exposed to the causal factor A with the change in value of B in a control group which was in the same condition as the experimental group but with- out the independent concept A causal factor. Different experimen- tal conditions, with different values of A, might be created and the range of values of B in each of these conditions is measured. Differences in the values of B between the different experimental groups are analysed, usually statistically, in order to draw a conclusion about how the values of B co-vary probabilistically i.e. on average with the values of A. If such an experiment is not feasible, the survey is the next best strat- egy for testing a probabilistic relation. In a survey, the co-variation between the values of two or more concepts is observed in a group of real life non-experimental instances. These are usually cross- sectional measurements i.e. at one point in time, but sometimes it is possible to design a prospective and longitudinal survey, allowing the researcher to observe how changes in the dependent concept follow in time upon changes in the independent concept. If a survey is not feasible, a comparative case study is the next best option see Box 9. In this type of case study the principles of a good survey are followed as closely as possible “quasi-survey case study”. This implies that a population is specified in which the proposition is tested, and that the sample is representative for that population and should be selected randomly. Box 9 How the survey can become a case study An essential characteristic of any survey is probability sampling, e.g. random sampling of instances from the population in which each instance of the population has an equal chance of being selected. This is the only guarantee that a co-variation that is observed in the group of observed instances in the sample also exists in other instances than those included in the sample. Probability sampling is only possible if the sampling frame is specified, i.e. if there is a list of members of a population or a set of directions for identifying each of them. Because there is never or very rarely a sampling frame for all members of an entire theoretical domain, a theory-testing survey is always conducted in a specified population of instances from within that domain. The propo- sition is tested in that population and this test will be followed by other tests in other populations in a replication strategy, in order to achieve generalizability to other parts of the domain. If no population of instances can be identified in the domain no sampling frame is available, it is not possible to test the proposition with a survey. However, this problem can be solved by specifying a smaller population within a domain for which a frame for probability sampling can be defined. It is, for instance, not likely that there is a sampling frame list of innovation projects in general, or of such projects in Europe, or in an economic sector in a country, but it is likely that there is a list of projects for which an EU subsidy was requested or a list of projects within a large company. Such often small populations are not “representative” of the domain, but no population ever is. A consumer behaviour theory, for instance, is always tested in a specific population of consumers say Rotterdam housewives or Toronto students and then replicated in other populations see Chapter 3.2.3. Another problem may then arise with such strategy: the number of available instances from the domain is too small for conducting a statistical analysis of the data, which is the main characteristic of a survey. This problem exists, for instance, in the field of compar- ative politics research when propositions about nations with specific characteristics