For the SIRUP project we conducted a total of three preliminary studies in order to try to identify successful indicators for complexity, familiarity, and conflict. In addition to the measures for these three factors, we included measures to enable us to test the theoretical model we developed based on the work of Daniel E. Berlyne and David Sylvia (see Figure 1).
Figure 1. SIRUP theoretical model
First of all, it was hypothesized that novelty would have a positive effect on interest. However, novelty can be perceived as threatening and as a result hinder interest. The extent to which novelty is perceived as a threath depends on the individual. Therefore, it was argued that an individual’s ability to cope with novelty (i.e. coping potential) is also related to interest. The relationship between coping potential and interest was expected to be curvilinear, such that interest is highest at a midlevel of coping potential. This means that some novelty is required in order for interest to occur, but not too much novelty.
Based on Berlyne’s work, we argued that novelty is determined by complexity, familiarity, and conflict. It was hypothesized that complexity and conflict have a positive effect on novelty (i.e. higher complexity and conflict predict higher novelty), whereas familiarity has a negative effect on novelty (i.e. lower familiarity predicts higher novelty).
For each of the programs in the three preliminary studies, participants were not only asked to rate the perceived complexity , familiarity, conflict of the program, but also the programs’ perceived novelty and the level of interest as triggered by the program description. Novelty and interest were coded on a 7-point semantic scale, with “familiar/boring” on the left and “novel/interesting” extreme on the right. Furthermore, participants were asked to rate how much they would find it pleasant, relevant, and unexpected if a recommender system were to provide them with this particular recommendation (measurement of the three defined components of serendipity). Answers were coded on a 5-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). Lastly, a simple measure for coping potential was included by asking participants how many hours of television they watch on an average day. The measures are displayed in Figure 2 .
Figure 2. Measures of the theoretical model test
The results of the testing of the theoretical model were very consistent across the three preliminary studies. As was predicted, complexity and conflict had a positive effect on the perceived novelty of a television program. Although it was expected that familiarity would negatively relate to novelty, the results showed that familiarity positively predicted the perceived novelty of the program. This indicates that programs that were perceived as more familiar by participants were rated more novel. This is remarkable considering that familiarity is opposite of novelty. However, this finding may be attributed to the fact that familiarity was conceptualized as popularity (i.e. programs perceived as more popular were rated as more novel).
The results also showed that novelty positively predicted interest, supporting our expectation. Considering the measure for coping potential, the results suggested that coping potential had a positive influence on interest.
Looking at the measure for serendipity, the results consistently showed that interest had a strong positive association with pleasant and relevant. However, unexpectedness had a strong negative association with interest, suggesting that a recommendation should not be unexpected in order to evoke serendipity.
In sum, the results showed that the theoretical model seems correct and may be a useful framework to help induce serendipity in users. However, we have not yet succeeded in identifying the indicators for complexity, familiarity, and conflict. In the future we should define the new indicators or use another test method for the factors we defined.
The results can be found in more detail here: SIRUP results theoretical model testing – Results.