28
dummy variable for adopting the ISSC Info sharing, but reports a general increase in sales and attendance across treatment and control stores after the ISSC was introduced Post.
Furthermore, the table shows a positive and significant correlation between the salespeople’s access to the system and sales. Many control variables are significantly associated with Sales,
Attendance, or both measures, and in many instances are also associated with our Info sharing and Active access variables, suggesting a multivariate analysis would yield more accurate
insights than the correlation table. Similarly, Table 2, Panel B does not show any association between the proxies for the quality of creative work and Info Sharing, but it shows a significantly
positive association between the quality of creative work measures and salespeople’s access to the ISSC. Several statistically significant correlations between the control measures and both, the
dependent variables and the ISSC variables Info sharing and Active access, support the use of multivariate analyses.
24
4.3. Regression Analyses Testing Our Hypotheses
We test our hypotheses by estimating the following difference-in-differences regression model at the store i–brand j level, using robust standard errors clustered by store:
Outcome
ijt
= β
+ β
1
Info sharing
i
+ β
2
Info sharing
i
Post
it
+ β
3
Post
it
+ β
n
Controls
ijt
+ Brand fixed effects + ε
ijt
1 where Outcome
ijt
is measured using a the natural logarithm of sales for brand j at store i in week t, b the quality of the creative work displayed at store i in month 3, 4, 6, or 8 based on
24
Matching each poster to the average weekly sales data ranging from two weeks before a given poster collection to two weeks before the next poster collection, we find that both the value and the novelty of creative work are positively and significantly
correlated with financial performance.
29
customer panel evaluations of the posters, or c promoter attendance for brand j at store i in week t.
25
The variable Info sharing is equal to one if the store is treated i.e., is part of group A and zero otherwise. Hypotheses 1 to 3 predicting significant effects of the ISSC on the quality
of the creative work, job engagement, and financial performance are tested by examining the direction and significance of
β
2
. The controls
ijt
include the store and brand promoter characteristics described in Tables 1 and 2, as well as brand fixed effects since the brands have different types of products spanning
different price ranges. Appendix 4 defines the variables used in our analyses. We first evaluate the average treatment effect of the ISSC on sales, the financial outcome Table
3, column 1. Then, we run similar regressions to evaluate the average treatment effect of the ISSC on important intermediary outcomes, that is, the quality of the creative work, measured
based on the value and novelty of the posters Table 3, columns 2 and 3, and job engagement, measured based on weekly promoter attendance Table 3, column 4. Our results in Table 3
suggest that the introduction of the ISSC did not have a statistically significant effect on any of the outcomes of interest. Based on our power analyses, we are 80 confident 1-
β=0.8, using a 10 significance level,
α=0.1 that, on average, the mere introduction of the ISSC did not change our creative measures by 0.18 points or more on a 1-5 point scale; the promoters’
attendance by 0.23 or more days per week; or sales by 13 or more.
26
We conduct a series of
25
For stores with two or more promoters for a given brand which make up less than 10 of the store-brands in a given week, the attendance variables and promoter-level characteristics are an average value across the promoters in the same store-brand.
26
These power analyses are based on regressions where we use brand and store fixed effects as our control variables and rely on 2015 data. These estimations also suggest that, with a probability of 90 1-
β=0.9, the ISSC did not change our creative measures by 0.21 points or more on a 1-5 point scale; the promoters’ attendance by 0.28 or more days per week; or sales by 15 or more.
30
tests to ensure that our results are robust to alternative specifications. We rerun our analyses making the following changes, one at a time:
a. We substitute all of our control variables for brand and store fixed effects the results of these tests are presented in Table 4.
b. We winsorize our sales variable at the 1st and 99th percentiles. c. We re-include store-brand-weeks when the sales are equal to zero, as long as a brand
promoter attended the store in those weeks. d. We re-include in our attendance measure cases when the promoter attended a different
store than hisher home store for 2 days or less in a month or when the new promoter did not last more than a week with the company.
e. We re-include the month of November into the analyses which we had excluded from our main specification due to disruptions caused by the demonetization in India.
All, except the first, of these five specifications yield coefficients for the Info sharing x Post variable that are statistically insignificant and similar in magnitude to the coefficients reported in
Table 3. The only analysis that produces somewhat different results the first in the list above is presented in Table 4. In this case, the Info sharing x Post coefficient for the “Value of creative
work” variable becomes statistically significant. The coefficient in this case is only slightly
bigger than that in Table 3, and suggests that the introduction of MPR’s ISSC resulted in a marginally significant increase in the value of the poster ratings of 0.18 points in a 1-5 scale.
27
27
The size of this coefficient is roughly equal to the minimum detectable effect size that we estimated for this analysis before running the field experiment, assuming a significance level of 10 and power 1-
β equal to 80, and using 2015 data.
31
In summary, our results suggest that the ISSC did not significantly affect financial performance, quality of creative output, or job engagement at the treatment stores. We conjecture that the lack
of results may be due to either a the limited extent to which salespeople accessed the ISSC
28
or b the inherent nature of the ISSC, which could have simultaneously provided useful
information to produce high quality posters but also triggered free-riding or other unintended behaviors. To test our first conjecture, in section 4.4.1, we examine whether salespeople that
accessed the system more often were significantly impacted by the introduction of the ISSC. To test our second conjecture, in section 4.4.2, we examine factors that could have increased or
decreased the benefits of the ISSC.
4.4. Additional Analyses
