Session-aware recommendation: A surprising quest for the state-of-the-art

- Session-aware recommendation: A surprising quest for the state-of-the-art.
- This makes it challeng- ing to understand what represents the state-of-the-art.
- One reason for the success of RS lies in their ability to personalize the item suggestions based on the preferences and observed past behavior of the individual users.
- Today, the research literature is how- ever still scattered, which makes it difficult to understand what represents the state-of-the-art in this area.
- With this work, our goal is to close the research gap regarding the state-of-the-art in session-aware recommendation.
- For this purpose, we have conducted extensive experimental evaluations in which we compared five recent neural models for session-aware recommendation with (i) a set of existing neural and non-neural approaches to session-based recommenda- tion, and (ii) heuristic extensions of the session-based techniques that, e.g., make use of reminders [20] or consider interac- tions that were observed immediately before the ongoing session..
- In this original collaborative filtering prob- lem setting, the order of the ratings or the time when they were provided were not considered in the algorithms.
- Soon, how- ever, it turned out that these aspects can matter, leading to the development of time-aware recommender systems [5], e.g., in the form of the timeSVD++ algorithm as used in the Netflix Prize [24]..
- The prediction task in this setting is to predict the next user action, given only the interactions of the current session.
- In the CASER method, Tang and Wang [45] embedded a sequence of recent items into latent spaces as an ‘‘image” in time, and proposed to learn sequential patterns as local features of the image with the help of convolutional filters.
- [29] benchmarked several of the mentioned neural methods against conceptually simpler session-based algorithms which, for example, rely on nearest-neighbor tech- niques.
- While the work in [20] relied on deep learning for the final predictions in one of their models, the core of the proposed technical approach was based on feature engineering and the use of side information about the items to recommend.
- One of the earliest ‘‘pure” deep learning techniques for session-aware recommendation was proposed by [36].
- Moreover, we provide details about the experimental configuration in terms of the evaluation protocol and the used datasets.
- The user-level GRU initializes the hidden state of the session-level GRU and optionally propagates the user representation in the input to the session-level GRU to personalize its recommendations..
- At the beginning of every ses- sion, the final output of the inter-session RNN initializes the hidden state of the intra-session RNN..
- The second attention layer returns the final user representation by combining the long-term user model and the embeddings of the items in the short-term item set..
- (iii) the joint context encoder to integrate the information of the intra-session context and the inter-session context for item prediction..
- It supports different ways of integrating user long-term preferences with the session patterns, where user embeddings can either be integrated with the input or the output of the session RNNs.
- Besides being actually a work on session-aware recommendation according to the above definition, we required that the source code of the method was publicly available and could be integrated into our Python-based evaluation framework.
- Our analysis furthermore shows that researchers use a variety of baselines in their experiments, which contributes to the difficulty of understanding what represents the state-of-the-art..
- It utilizes a hybrid encoder with an attention mechanism to model the sequential behavior of users and capture their main intent of the current session.
- It considers the order of the items in a session as well as the distance between them when scoring the items..
- This method considers the order of the items while computing both session similarities and item scores.
- Overview of the baseline techniques that each neural session-aware approach was originally compared to.
- GRU4REC2 : the improved version of the GRU4REC model [14].
- Methods like SR and VSKNN , for example, return scores based on item co-occurrences and the positions of the co-occurring items, as described in [27].
- Technically, we look up each item that is recommended by the underlying method and check if it occurred in the interaction history of the current user at least once.
- In all of them, the reminder list is created by adding items of the user’s last p sessions, which is a hyperparameter to be determined on the validation set.
- In this strategy, we use a decay function to score reminder items, i.e., items in the interaction history of the user, based on the time of the user’s last interaction with them:.
- Here, T c is the timestamp of the current session, and T i is the timestamp of the last interaction of the user with the item i..
- In this strategy, the items of the last P sessions of the user are scored based on the similarity score(s) of the current session and the previous session(s) that they belong to.
- Here, S c denotes the current session, and S p denotes a session in the set of the last P past sessions of the user.
- Sim is the similarity function of the nearest-neighbor algorithm, and the indicator function 1 p ð i Þ returns 1 if the session p contains item i and 0 otherwise..
- XING: A dataset published in the context of the ACM RecSys 2016 Challenge that contains interactions of job postings on a career-oriented social networking site, XING, from about three months.
- Furthermore, we randomly sampled 10% of the users of this large dataset because of scalability issues of some of the neural methods..
- Many previous works on session-aware recommendation use a single training-test split of the whole dataset or a sample of it for evaluation.
- To have about the same number of events for each slice, we skipped the first 500 days of the LASTFM dataset..
- Table 2 shows the average characteristics of the slices for each dataset after the pre-preprocessing phase..
- We finally filter out the items from the validation and test sets of each of the five slices that did not appear in the training set of that slice..
- We use standard classification and ranking measures to evaluate the performance of the recommenda- tion algorithms.
- Second, when all items of the current session are assumed to be relevant to the user, we consider all the remaining items of an ongoing session as the target item.
- A strong popularity bias, on the other hand, indicates that an algorithm is not focusing much on the long-tail of the item catalog, which, however, can be desirable in practice.
- 9 In the implementation of the NCSF method, we found that the authors use a context window, which includes items before and after a given target item t to make the prediction.
- To compare the computational complex- ity of the various methods, we measured the time that is needed by each algorithm, both for the training and the prediction phases.
- Note that we do not report all possible combinations of the proposed extensions discussed in Section 3.1.3 for the sake of conciseness.
- For the sake of clarity, for each model and dataset, we report (i) the combination of the extensions that led to the best performance according to MAP@20 and (ii) the results of the original models without extensions.
- Moreover, the heuristic extensions of the session-based algorithms further improve their accuracy performance in all but one cases (MRR for VSKNN _ EBR.
- Characteristics of the datasets.
- 10 We also tried MAP@20 as the optimization target for some approaches, but this did not lead to a different ranking of the algorithms in terms of accuracy..
- Probably even more surprising is that we find the session-aware methods at the very end of the performance ranking.
- Reminders for example help to improve the perfor- mance of the neural session-based techniques ( GRU4REC , NARM ) to an extent that they sometimes outperform the basic nearest- neighbor techniques.
- However, when the extensions are also considered for the neighborhood-based techniques, their accu- racy results are again much higher than those of the neural techniques..
- Results of the performance comparison on the RETAIL dataset with the focus on the comparison of simple (non-neural) methods and neural session-aware ones.
- Such modifications of the original algorithms are however not in the scope of our work..
- The best results are obtained by IIRNN , which is however one of the earliest session-aware methods.
- 14 Therefore, we only report the results of the original neural session-based methods ( GRU4REC and NARM.
- Results of the performance comparison on the XING dataset with the focus on the comparison of simple (non-neural) methods and neural session-aware ones.
- Tables 3–6 also report the values of coverage and popularity bias of the algorithms.
- Results of the performance comparison on the COSMETICS dataset with the focus on the comparison of simple (non-neural) methods and neural session-aware ones.
- Results of the performance comparison on the LASTFM dataset with the focus on the comparison of simple (non-neural) methods and neural session-aware ones.
- What can be observed is that the achieved level of coverage and the ranking of the algorithms in this respect seems to depend on the datasets..
- We made all the measurements on the validation slice (i.e., the largest slice in terms of the number of events) for each dataset.
- We report running times of the session-based algo- rithms (neural and non-neural) both for the best-performing extension and the corresponding method..
- The performance of the session-aware methods, finally, exhibit a large spread.
- The running times for the simple SR method are in the range of the neural methods.
- For the XING dataset, for example, the prediction times are in the range of the neural methods.
- On the other hand, for the LASTFM dataset, where, on average, there is a large number of sessions in the history of the users, the time needed for creating one recommendation list can go up to a few hundred milliseconds..
- Generally, the use of the proposed extensions ( BOOST , EXTEND , REMIND ) leads to increased computation times for training and predicting.
- Note, however, that the more efficient basic versions of the nearest-neighbor techniques already outperform all neural session-aware methods in all cases, except one..
- Another related problem can also lie in the choice of the baselines.
- Our experiments so far however indicate that the ranking of the family of algorithms is quite consistent across experiments, with today’s neural approaches to session-aware recommendation not being among the top-performing techniques, and non-neural techniques working well across datasets..
- Finally, regarding running times—in particular for datasets where there is a larger set of past sessions per user—further performance enhancements of the nearest-neighbor methods might be achieved through additional engineering efforts..
- As such, our experimental evaluation can only represent a snapshot of the state-of-the-art in the area.
- Finally, we observed that the size of the datasets used in the experiments, both in the original papers and in our work, are small compared to the amounts of data that are available in real-world deployments.
- The current limitations of the inves- tigated deep learning models might therefore be a result of these dataset limitations, see also [21].
- We see the reasons for these unexpected phenomena in methodological issues that are not limited to session-aware rec- ommendation scenarios, in particular in the choice of the baselines in experimental evaluations or the lack of proper tuning of the baselines.
- We are grateful to Zhongli Filippo Hu for his contributions to the integration of the algorithms.
- (For SR , VSKNN , STAN and VSTAN ) Tune together with other hyperparameters of the algorithm.
- (For VSKNN , STAN and VSTAN ) Tune together with other hyperparameters of the algorithm.
- hyperparameters of the algorithm weight_IRec Integer 0–9 10.
- hyperparameters of the algorithm weight_Rel Integer 1–10 10.
- However, ranges for these hyperparameters can be set for each dataset separately based on its mean average length of the sessions.
- 20 for XING because of the computational complexity.
- Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM pp.
- [8] Diksha Garg, Priyanka Gupta, Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, Sequence and time aware neighborhood for session-based recommendations: stan, in: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR pp.
- [14] Balázs Hidasi, Alexandros Karatzoglou, Recurrent neural networks with top-k gains for session-based recommendations, in: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM pp.
- Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, 2017, pp.
- [18] Dietmar Jannach, Lukas Lerche, Michael Jugovac, Adaptation and evaluation of recommendations for short-term shopping goals, in: Proceedings of the 9th ACM Conference on Recommender Systems, RecSys pp.
- [19] Dietmar Jannach, Malte Ludewig, When recurrent neural networks meet the neighborhood for session-based recommendation, in: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys pp.
- [22] Santosh Kabbur, Xia Ning, George Karypis, FISM: Factored item similarity models for top-n recommender systems, in: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD pp.
- [24] Yehuda Koren, Collaborative filtering with temporal dynamics, in: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD pp.
- [25] Lukas Lerche, Dietmar Jannach, Malte Ludewig, On the value of reminders within e-commerce recommendations, in: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, UMAP pp.
- [26] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, Jun Ma, Neural attentive session-based recommendation, in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM pp.
- [28] Malte Ludewig, Noemi Mauro, Sara Latifi, and Dietmar Jannach, Performance comparison of neural and non-neural approaches to session-based recommendation, in: Proceedings of the 13th ACM Conference on Recommender Systems, RecSys pp.
- [36] Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi, Personalizing session-based recommendations with hierarchical recurrent neural networks, in: Proceedings of the Eleventh ACM Conference on Recommender Systems, 2017, pp.
- Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI pp.
- Proceedings of the Web Conference, WWW pp.
- [40] Massimiliano Ruocco, Ole Steinar Lillestøl Skrede, Helge Langseth, Inter-session modeling for session-based recommendation, in: Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems, DLRS pp.
- [45] Jiaxi Tang, Ke Wang, Personalized top-n sequential recommendation via convolutional sequence embedding, in: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM pp

Xem thử không khả dụng, vui lòng xem tại trang nguồn
hoặc xem Tóm tắt