Title: Representation Learning with Large Language Models for Recommendation

URL Source: https://arxiv.org/html/2310.15950

Published Time: Thu, 12 Dec 2024 01:31:52 GMT

Markdown Content:
(2024)

###### Abstract.

Recommender systems have seen significant advancements with the influence of deep learning and graph neural networks, particularly in capturing complex user-item relationships. However, these graph-based recommenders heavily depend on ID-based data, potentially disregarding valuable textual information associated with users and items, resulting in less informative learned representations. Moreover, the utilization of implicit feedback data introduces potential noise and bias, posing challenges for the effectiveness of user preference learning. While the integration of large language models (LLMs) into traditional ID-based recommenders has gained attention, challenges such as scalability issues, limitations in text-only reliance, and prompt input constraints need to be addressed for effective implementation in practical recommender systems. To address these challenges, we propose a model-agnostic framework RLMRec that aims to enhance existing recommenders with LLM-empowered representation learning. It proposes a recommendation paradigm that integrates representation learning with LLMs to capture intricate semantic aspects of user behaviors and preferences. RLMRec incorporates auxiliary textual signals, employs LLMs for user/item profiling, and aligns the semantic space of LLMs with collaborative relational signals through cross-view alignment. This work further demonstrates the theoretical foundation of incorporating textual signals through mutual information maximization, which improves the quality of representations. Our evaluation integrates RLMRec with state-of-the-art recommender models, while also analyzing its efficiency and robustness to noise data. Implementation codes are available at [https://github.com/HKUDS/RLMRec](https://github.com/HKUDS/RLMRec).

Large Language Models, Recommendation, Alignment

††journalyear: 2024††copyright: acmlicensed††conference: Proceedings of the ACM Web Conference 2024; May 13–17, 2024; Singapore, Singapore††booktitle: Proceedings of the ACM Web Conference 2024 (WWW ’24), May 13–17, 2024, Singapore, Singapore††doi: 10.1145/3589334.3645458††isbn: 979-8-4007-0171-9/24/05††ccs: Information systems Recommender systems
1. Introduction
---------------

Recommender systems have evolved to provide personalized item recommendations based on user interactions, with deep learning and graph neural networks playing a significant role(Chang et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib5); Wu et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib40)). Graph-based recommenders like NGCF(Wang et al., [2019](https://arxiv.org/html/2310.15950v5#bib.bib36)) and LightGCN(He et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib12)) have demonstrated impressive capabilities in capturing complex user-item relationships, making them state-of-the-art approaches.

However, it’s important to note that recent graph-based recommenders heavily rely on ID-corresponding information for learning. The training data in this line only consists of mapped user/item indices, and their interactions are represented in an interaction matrix using binary values (1 for interaction and 0 for no interaction). While this data arrangement has shown effectiveness, one limitation is its primary reliance on ID-based information, potentially overlooking other valuable data such as rich textual information associated with users and items. The absence of this additional information can lead to reduced informativeness in the learned representations. Furthermore, it is worth noting that a substantial portion of the data in these graph-based recommenders consists of implicit feedback(Rendle et al., [2012](https://arxiv.org/html/2310.15950v5#bib.bib28); Wang et al., [2021a](https://arxiv.org/html/2310.15950v5#bib.bib34)), which can introduce noise from false negatives or bias (e.g., misclicks(Wang et al., [2021b](https://arxiv.org/html/2310.15950v5#bib.bib35)) or popularity bias(Chen et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib6))). Consequently, the learned representations of these GNN-based models heavily rely on the inherent quality of the data. This heavy reliance on the data quality poses a potential challenge as it can lead to detrimental representations that hinder the effectiveness of recommendation systems, especially when the data contains noise.

In recent times, there have been several endeavors to leverage diverse data modalities in order to enhance traditional ID-based recommenders(Yuan et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib47); Li et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib18); Geng et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib10)). Particularly interesting is the emergence of large language models (LLMs) like GPT-4(OpenAI, [2023](https://arxiv.org/html/2310.15950v5#bib.bib25)) and LLaMA(Touvron et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib33)), which have demonstrated impressive capabilities in neural language understanding tasks. This development has sparked significant interest among researchers, who are actively exploring how LLMs, with their proficiency in handling textual content, can extend the capabilities of recommendation systems beyond the original data(Fan et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib8); Lin et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib19); Liu et al., [2023b](https://arxiv.org/html/2310.15950v5#bib.bib22)). A primary focus of current research in this field revolves around aligning recommendation approaches with the characteristics of language models through prompt design. Methods like InstructRec(Zhang et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib48)) structure the recommendation task in an instruction-question-answering format, enabling LLMs to simultaneously address the recommendation objective and respond to intricately designed questions(Geng et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib10); Bao et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib2)). However, these methods still lag behind existing recommenders in terms of efficiency and precision. This can be attributed to inherent shortcomings associated with this approach, including the following key aspects:

i) Scalability issues in practical recommenders. Utilizing large language models (LLMs) in personalized user behavior modeling requires significant computational resources. As the scale of user behavior data grows, so do the computational demands and associated inference time costs. For instance, in TALLRec(Bao et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib2)), where recommendations are generated based on an instruction-question-answering format, the response time for LLaMA2-13B to provide recommendations to individual users is approximately 3.6 seconds, with an input size of around 800 tokens (equivalent to approximately 5 users). However, this poses significant challenges when attempting to scale up the approach for practical recommender systems with a substantial user base and extensive item catalog.

ii) Limitations stemming from text-only reliance. LLMs have the potential to generate text answers that may include recommendations for non-existent items due to hallucination issues(Liu et al., [2023a](https://arxiv.org/html/2310.15950v5#bib.bib21)). This poses a challenge in ensuring the accuracy and reliability of the generated recommendations. Additionally, the limited capacity of prompt inputs, constrained by the maximum number of tokens (e.g., 2048 tokens for LLaMA), hinders the effective modeling of comprehensive collaborative signals with global user dependencies.

![Image 1: Refer to caption](https://arxiv.org/html/2310.15950v5/x1.png)

Figure 1. LLM’s performance for recommendation reranking when dealing with different sizes of candidate items.

To validate the aforementioned limitations, we evaluate the effectiveness of directly using LLMs in enhancing the re-ranking task(Hou et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib13); Sun et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib32)) for recommendation on the Amazon dataset. Specifically, we utilize LightGCN(He et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib12)) as the backbone model, which retrieve a ranking list of 50 candidate items for each user. To further refine the recommendations, we integrate the textual information of each item with our custom prompts (for details, please refer to Appendix[A.3](https://arxiv.org/html/2310.15950v5#A1.SS3 "A.3. Analysis on the Reranking Task with LLMs ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation")). These prompts are then processed by ChatGPT (i.e., gpt-3.5-turbo). The objective of this task is to re-rank the item list for each user and identify the Top-10&20 most relevant items.

It is evident from the results in Figure[1](https://arxiv.org/html/2310.15950v5#S1.F1 "Figure 1 ‣ 1. Introduction ‣ Representation Learning with Large Language Models for Recommendation") that the recommendations refined by the ChatGPT perform worse than the original results provided by LightGCN. This indicates limitations when blindly using LLMs to improve the re-ranking process in recommendation. These limitations can be attributed to three factors: i) The hallucination issue of LLMs, suggesting items not in the candidate set; ii) The lack of a comprehensive global text-based collaborative relationship input due to token limits; iii) Additionally, it is worth noting that the reranking process using LLM takes several hours to complete, which poses a challenge when dealing with large-scale data in real-world recommendation scenarios. Due to page constraints, we provide detailed analysis and examples in the Appendix to demonstrate the phenomenon of hallucination.

Contributions. In light of the aforementioned limitations, we aim to leverage the power of LLMs to seamlessly enhance existing recommender systems. To accomplish this, we propose a model-agnostic framework called RLMRec(R epresentation Learning with Large L anguage M odels for Rec ommendation). The core idea of RLMRec is to utilize representation learning as a bridge between ID-based recommenders and LLMs. Our new recommendation paradigm aims to preserve the accuracy and efficiency of existing recommenders while harnessing the powerful text comprehension capabilities of LLMs to understand the intricate semantic aspects of user behaviors and preferences. To begin, we lay the theoretical groundwork by modeling the benefits of incorporating auxiliary textual signals for representation learning. This involves transforming the textual signals into meaningful representations and establishing a theoretical foundation for maximizing mutual information within general recommendation models. Moreover, we develop a user/item profiling paradigm empowered by LLMs, enhancing representation expressiveness by incorporating comprehensive semantic understanding from the global knowledge space of LLMs.

Furthermore, we propose to align the semantic space of LLMs and the representation space of collaborative relational signals through a cross-view alignment framework. This alignment is achieved through a cross-view mutual information maximization scheme, which allows us to find a common semantic subspace where the textual and collaborative relational embeddings are well aligned from the contrastive and generative modeling, respectively. In a nutshell, our main contributions can be summarized as follows:

*   •This work aims to explore the potential of enhancing the recommendation performance of existing recommender systems, by leveraging LLMs and aligning their semantic space with collaborative relation modeling for better representation learning. 
*   •We propose a model-agnostic representation learning framework called RLMRec, which is guided by our theoretical findings. This framework leverages contrastive or generative modeling techniques to enhance the quality of learned representations. 
*   •We establish a theoretical foundation to demonstrate the effectiveness of incorporating textual signals in enhancing the representation learning. By utilizing mutual information maximization, we show how textual signals can improve the representation quality. 
*   •We integrate RLMRec with various state-of-the-art recommender models and validate the effectiveness of our method. Additionally, we analyze the framework’s robustness to noise and incomplete data, showcasing its ability to handle real-world challenges. 

2. Related Work
---------------

GNN-enhanced Collaborative Filtering. Collaborative Filtering (CF), a fundamental technique in recommendation systems, has been extensively studied(Su and Khoshgoftaar, [2009](https://arxiv.org/html/2310.15950v5#bib.bib31); Koren et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib17)). An emerging direction is to use historical user-item interactions to create a bipartite graph and employ Graph Neural Networks (GNNs) to capture high-order collaborative relationships. These graph-based methods, such as NGCF(Wang et al., [2019](https://arxiv.org/html/2310.15950v5#bib.bib36)), GCCF(Chen et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib7)), LightGCN(He et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib12)), have demonstrated state-of-the-art performance, improving recommendation effectiveness. However, the sparsity and noise in implicit feedback data pose challenges to graph-based methods. To address this, researchers have explored the use of self-supervised learning (SSL) techniques as auxiliary learning objectives to enhance robustness in recommendations(Yu et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib46); Yang et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib44)). Among various SSL techniques, contrastive learning has emerged as a prominent solution in collaborative filtering models. Methods like SGL(Wu et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib38)), SimGCL(Yu et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib45)), NCL(Lin et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib20)), LightGCL(Cai et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib4)) leverage contrastive data augmentation to improve recommendation performance. In this work, we take a step further by integrating LLMs with existing CF models to effectively align the knowledge and reasoning abilities of LLMs with the collaborative relation learning for enhancing recommendation performance.

Large Language Models for Recommendation. leveraging LLMs for recommendation systems has gained interest(Fan et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib8); Lin et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib19); Liu et al., [2023b](https://arxiv.org/html/2310.15950v5#bib.bib22); Wu et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib39)). Several studies have leveraged LLMs as inference models by designing prompts that align them with recommendation tasks. For example, P5(Geng et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib10)) converts the user interaction data into textual prompts using item indexes, which are then used for language model training. Chat-REC(Gao et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib9)) builds a conversational recommender by transforming user profiles and interactions into prompts for LLMs to generate recommendations. InstructRec(Zhang et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib48)) and TALLRec(Bao et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib2)) employ instructional designs to define recommendation tasks and fine-tune LLMs to align with these instructions for generating recommendations. However, using LLMs directly for recommendation tasks faces challenges like high computational costs and slow inference times. To address this, our approach adopts mutual information maximization to align LLMs knowledge with collaborative relation modeling, enabling scalable and effective recommendations.

3. Methodology
--------------

### 3.1. Theoretical Basis of RLMRec

Collaborative Filtering. In our recommendation scenario, we have a set of users 𝒰=u 1,…,u I 𝒰 subscript 𝑢 1…subscript 𝑢 𝐼\mathcal{U}={u_{1},...,u_{I}}caligraphic_U = italic_u start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_u start_POSTSUBSCRIPT italic_I end_POSTSUBSCRIPT and a set of items 𝒱=v 1,…,v J 𝒱 subscript 𝑣 1…subscript 𝑣 𝐽\mathcal{V}={v_{1},...,v_{J}}caligraphic_V = italic_v start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_v start_POSTSUBSCRIPT italic_J end_POSTSUBSCRIPT. The observed user-item interactions are represented by 𝒳 𝒳\mathcal{X}caligraphic_X. In learning-based recommenders, each user and item is assigned initial embeddings x u subscript x 𝑢\textbf{x}_{u}x start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT and x v subscript x 𝑣\textbf{x}_{v}x start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT. The goal is to learn user and item representations e u,e v subscript e 𝑢 subscript e 𝑣\textbf{e}_{u},\textbf{e}_{v}e start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT through a recommender model (i.e., e u,e v=ℛ⁢(x u,x v)subscript e 𝑢 subscript e 𝑣 ℛ subscript x 𝑢 subscript x 𝑣\textbf{e}_{u},\textbf{e}_{v}=\mathcal{R}(\textbf{x}_{u},\textbf{x}_{v})e start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = caligraphic_R ( x start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT , x start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT )) that maximizes the posterior distribution shown below:

(1)p⁢(𝐞|𝒳)∝p⁢(𝒳|𝐞)⁢p⁢(𝐞).proportional-to 𝑝 conditional 𝐞 𝒳 𝑝 conditional 𝒳 𝐞 𝑝 𝐞\displaystyle p(\mathbf{e}|\mathcal{X})\propto p(\mathcal{X}|\mathbf{e})p(% \mathbf{e}).italic_p ( bold_e | caligraphic_X ) ∝ italic_p ( caligraphic_X | bold_e ) italic_p ( bold_e ) .

In practical recommendation scenarios, the observed user-item interactions 𝒳 𝒳\mathcal{X}caligraphic_X often contain noise, including false positives (e.g., misclicks or interactions influenced by popularity bias) and false negatives (e.g., users do not interact with unseen but interested items). As a result, the learned representation 𝐞 𝐞\mathbf{e}bold_e can also be affected by this noise, which negatively impacts recommendation accuracy. In this work, we introduce a hidden prior belief 𝐳 𝐳\mathbf{z}bold_z that is inherently beneficial for recommendation. This prior belief helps identify the true positive samples in 𝒳 𝒳\mathcal{X}caligraphic_X. Hence, the generation of representation 𝐞 𝐞\mathbf{e}bold_e involves a combination of the advantageous prior belief 𝐳 𝐳\mathbf{z}bold_z and the unavoidable noise present during the learning process.

Text-enhanced User Preference Learning. To mitigate the impact of irrelevant signals on the representation, it is necessary to incorporate auxiliary informative cues. One approach is to introduce textual information, e.g., user and item profiles, which provide insights for user preference learning. These profiles can be encoded using language models to generate representations 𝐬∈ℝ d s 𝐬 superscript ℝ subscript 𝑑 𝑠\mathbf{s}\in\mathbb{R}^{d_{s}}bold_s ∈ blackboard_R start_POSTSUPERSCRIPT italic_d start_POSTSUBSCRIPT italic_s end_POSTSUBSCRIPT end_POSTSUPERSCRIPT that effectively capture the semantic aspects of user preferences. Importantly, both 𝐬 𝐬\mathbf{s}bold_s and 𝐞 𝐞\mathbf{e}bold_e capture shared information that is relevant to the aspects associated with user-item interactions. This shared information is crucial as it indicates the inclusion of beneficial aspects for recommendation, aligning with the prior belief 𝐳 𝐳\mathbf{z}bold_z.

![Image 2: Refer to caption](https://arxiv.org/html/2310.15950v5/x2.png)

Figure 2. The type of directed graph model under consideration. As the alignment between CF-side representation and LLM-enhanced representation, the noisy effects in the learned representations e 𝑒 e italic_e are alleviated in RLMRec.

![Image 3: Refer to caption](https://arxiv.org/html/2310.15950v5/x3.png)

Figure 3. The overall framework of our proposed LLM-enhanced representation learning framework RLMRec.

With the collaborative-side representation 𝐞 𝐞\mathbf{e}bold_e and textual-side representation 𝐬 𝐬\mathbf{s}bold_s, both of which contain recommendation-beneficial information generated from 𝐳 𝐳\mathbf{z}bold_z, our objective is to learn the optimal value of 𝐞 𝐞\mathbf{e}bold_e denoted as 𝐞∗superscript 𝐞\mathbf{e}^{*}bold_e start_POSTSUPERSCRIPT ∗ end_POSTSUPERSCRIPT, by maximizing the conditional probability:

(2)𝐞∗=arg⁡max 𝐞 𝔼 p⁢(𝐞,𝐬)⁢[p⁢(𝐳,𝐬|𝐞)].superscript 𝐞 subscript 𝐞 subscript 𝔼 𝑝 𝐞 𝐬 delimited-[]𝑝 𝐳 conditional 𝐬 𝐞\displaystyle\mathbf{e}^{*}=\mathop{\arg\max}\limits_{\mathbf{e}}\mathbb{E}_{p% (\mathbf{e},\mathbf{s})}[p(\mathbf{z},\mathbf{s}|\mathbf{e})].bold_e start_POSTSUPERSCRIPT ∗ end_POSTSUPERSCRIPT = start_BIGOP roman_arg roman_max end_BIGOP start_POSTSUBSCRIPT bold_e end_POSTSUBSCRIPT blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT [ italic_p ( bold_z , bold_s | bold_e ) ] .

The underlying intuition behind maximizing the conditional probability is to ensure that the learnable representation 𝐞 𝐞\mathbf{e}bold_e from recommender models incorporates purer information generated from the prior belief 𝐳 𝐳\mathbf{z}bold_z and the shared information with the semantic representation 𝐬 𝐬\mathbf{s}bold_s. By doing so, the relevance and benefits of the learned representations 𝐞 𝐞\mathbf{e}bold_e for recommendation are enhanced.

Theorem 1. Maximizing the posteriori probability 𝔼 p⁢(𝐞,𝐬)⁢[p⁢(𝐳,𝐬|𝐞)]subscript 𝔼 𝑝 𝐞 𝐬 delimited-[]𝑝 𝐳 conditional 𝐬 𝐞\mathbb{E}_{p(\mathbf{e},\mathbf{s})}[p(\mathbf{z},\mathbf{s}|\mathbf{e})]blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT [ italic_p ( bold_z , bold_s | bold_e ) ] given the hidden prior belief 𝐳 𝐳\mathbf{z}bold_z, is equivalent to maximizing the mutual information I⁢(e;s)𝐼 𝑒 𝑠 I(e;s)italic_I ( italic_e ; italic_s ) between the CF-side relational representation 𝐞 𝐞\mathbf{e}bold_e and LLM-side semantic representation 𝐬 𝐬\mathbf{s}bold_s.

Proof. It is important to note that since the profiles of users and items are fixed, the probability p⁢(𝐬)𝑝 𝐬 p(\mathbf{s})italic_p ( bold_s ) remains constant during the learning process. Therefore, we can deduce the following:

(3)𝔼 p⁢(𝐞,𝐬)⁢[p⁢(𝐳,𝐬|𝐞)]subscript 𝔼 𝑝 𝐞 𝐬 delimited-[]𝑝 𝐳 conditional 𝐬 𝐞\displaystyle\mathbb{E}_{p(\mathbf{e},\mathbf{s})}[p(\mathbf{z},\mathbf{s}|% \mathbf{e})]blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT [ italic_p ( bold_z , bold_s | bold_e ) ]∝𝔼 p⁢(𝐞,𝐬)⁢log⁡[∫𝐳 p⁢(𝐳,𝐬|𝐞)p⁢(𝐬)⁢𝑑 𝐳]proportional-to absent subscript 𝔼 𝑝 𝐞 𝐬 subscript 𝐳 𝑝 𝐳 conditional 𝐬 𝐞 𝑝 𝐬 differential-d 𝐳\displaystyle\propto\mathbb{E}_{p(\mathbf{e},\mathbf{s})}\log[\int_{\mathbf{z}% }\ \frac{p(\mathbf{z},\mathbf{s}|\mathbf{e})}{p(\mathbf{s})}\,d\mathbf{z}\,]∝ blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT roman_log [ ∫ start_POSTSUBSCRIPT bold_z end_POSTSUBSCRIPT divide start_ARG italic_p ( bold_z , bold_s | bold_e ) end_ARG start_ARG italic_p ( bold_s ) end_ARG italic_d bold_z ]
(4)=𝔼 p⁢(𝐞,𝐬)⁢log⁡[∫𝐳 p⁢(𝐳,𝐞|𝐬)⁢𝑑 𝐳 p⁢(𝐞)]absent subscript 𝔼 𝑝 𝐞 𝐬 subscript 𝐳 𝑝 𝐳 conditional 𝐞 𝐬 differential-d 𝐳 𝑝 𝐞\displaystyle=\mathbb{E}_{p(\mathbf{e},\mathbf{s})}\log[\frac{\int_{\mathbf{z}% }\ p(\mathbf{z},\mathbf{e}|\mathbf{s})\,d\mathbf{z}\,}{p(\mathbf{e})}]= blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT roman_log [ divide start_ARG ∫ start_POSTSUBSCRIPT bold_z end_POSTSUBSCRIPT italic_p ( bold_z , bold_e | bold_s ) italic_d bold_z end_ARG start_ARG italic_p ( bold_e ) end_ARG ]
(5)=𝔼 p⁢(𝐞,𝐬)⁢log⁡[p⁢(𝐞|𝐬)p⁢(𝐞)]=I⁢(𝐞,𝐬).absent subscript 𝔼 𝑝 𝐞 𝐬 𝑝 conditional 𝐞 𝐬 𝑝 𝐞 𝐼 𝐞 𝐬\displaystyle=\mathbb{E}_{p(\mathbf{e},\mathbf{s})}\log[\frac{p(\mathbf{e}|% \mathbf{s})}{p(\mathbf{e})}]=I(\mathbf{e},\mathbf{s}).= blackboard_E start_POSTSUBSCRIPT italic_p ( bold_e , bold_s ) end_POSTSUBSCRIPT roman_log [ divide start_ARG italic_p ( bold_e | bold_s ) end_ARG start_ARG italic_p ( bold_e ) end_ARG ] = italic_I ( bold_e , bold_s ) .

Let’s consider 𝐞 𝐞\mathbf{e}bold_e and 𝐬 𝐬\mathbf{s}bold_s as data samples, assuming that we have N 𝑁 N italic_N pairwise corresponding elements of 𝐞 𝐞\mathbf{e}bold_e and 𝐬 𝐬\mathbf{s}bold_s, forming the sets 𝐄={𝐞 1,…,𝐞 i,…,𝐞 N}𝐄 subscript 𝐞 1…subscript 𝐞 𝑖…subscript 𝐞 𝑁\mathbf{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{i},\ldots,\mathbf{e}_{N}\}bold_E = { bold_e start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , … , bold_e start_POSTSUBSCRIPT italic_N end_POSTSUBSCRIPT } and 𝐒={𝐬 1,…,𝐬 i,…,𝐬 N}𝐒 subscript 𝐬 1…subscript 𝐬 𝑖…subscript 𝐬 𝑁\mathbf{S}=\{\mathbf{s}_{1},\ldots,\mathbf{s}_{i},\ldots,\mathbf{s}_{N}\}bold_S = { bold_s start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , … , bold_s start_POSTSUBSCRIPT italic_N end_POSTSUBSCRIPT }, respectively. Based on this, we optimize the mutual information as follows.

Theorem 2. By introducing the density ratio to preserve mutal information(Oord et al., [2018](https://arxiv.org/html/2310.15950v5#bib.bib24))f⁢(𝐬,𝐞)∝p⁢(𝐬|𝐞)/p⁢(𝐬)proportional-to 𝑓 𝐬 𝐞 𝑝 conditional 𝐬 𝐞 𝑝 𝐬 f(\mathbf{s},\mathbf{e})\propto p(\mathbf{s}|\mathbf{e})/p(\mathbf{s})italic_f ( bold_s , bold_e ) ∝ italic_p ( bold_s | bold_e ) / italic_p ( bold_s ), the maximization of I⁢(𝐞 i;𝐬 i)𝐼 subscript 𝐞 𝑖 subscript 𝐬 𝑖 I(\mathbf{e}_{i};\mathbf{s}_{i})italic_I ( bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ; bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) can be reformulated as maximizing the following lower bound:

(6)𝔼⁢log⁡[f⁢(𝐬 i,𝐞 i)∑𝐬 j∈𝐒 f⁢(𝐬 j,𝐞 i)].𝔼 𝑓 subscript 𝐬 𝑖 subscript 𝐞 𝑖 subscript subscript 𝐬 𝑗 𝐒 𝑓 subscript 𝐬 𝑗 subscript 𝐞 𝑖\displaystyle\mathbb{E}\log[\frac{f(\mathbf{s}_{i},\mathbf{e}_{i})}{\sum_{% \mathbf{s}_{j}\in\mathbf{S}}f(\mathbf{s}_{j},\mathbf{e}_{i})}].blackboard_E roman_log [ divide start_ARG italic_f ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG ∑ start_POSTSUBSCRIPT bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ∈ bold_S end_POSTSUBSCRIPT italic_f ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ] .

Proof. Based on the property of mutual information, we have I⁢(e i,s i)=I⁢(s i,e i)𝐼 subscript e 𝑖 subscript s 𝑖 𝐼 subscript s 𝑖 subscript e 𝑖 I(\textbf{e}_{i},\textbf{s}_{i})=I(\textbf{s}_{i},\textbf{e}_{i})italic_I ( e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_I ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ). With this in mind, we make the deductions as:

(7)I⁢(𝐬 i,𝐞 i)𝐼 subscript 𝐬 𝑖 subscript 𝐞 𝑖\displaystyle I(\mathbf{s}_{i},\mathbf{e}_{i})italic_I ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT )≥I⁢(𝐬 i,𝐞 i)−log⁡(N)=−𝔼⁢log⁡[p⁢(𝐬 i)p⁢(𝐬 𝐢|𝐞 i)⁢N]absent 𝐼 subscript 𝐬 𝑖 subscript 𝐞 𝑖 𝑁 𝔼 𝑝 subscript 𝐬 𝑖 𝑝 conditional subscript 𝐬 𝐢 subscript 𝐞 𝑖 𝑁\displaystyle\geq I(\mathbf{s}_{i},\mathbf{e}_{i})-\log(N)=-\mathbb{E}\log[% \frac{p(\mathbf{s}_{i})}{p(\mathbf{{s}_{i}}|\mathbf{e}_{i})}N]≥ italic_I ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) - roman_log ( italic_N ) = - blackboard_E roman_log [ divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT bold_i end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG italic_N ]
(8)≥−𝔼⁢log⁡[1+p⁢(𝐬 i)p⁢(𝐬 𝐢|𝐞 i)⁢(N−1)]absent 𝔼 1 𝑝 subscript 𝐬 𝑖 𝑝 conditional subscript 𝐬 𝐢 subscript 𝐞 𝑖 𝑁 1\displaystyle\geq-\mathbb{E}\log[1+\frac{p(\mathbf{s}_{i})}{p(\mathbf{{s}_{i}}% |\mathbf{e}_{i})}(N-1)]≥ - blackboard_E roman_log [ 1 + divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT bold_i end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ( italic_N - 1 ) ]
(9)=−𝔼⁢log⁡[1+p⁢(𝐬 i)p⁢(𝐬 𝐢|𝐞 i)⁢(N−1)⁢𝔼 𝐬 j∈𝐒 n⁢e⁢g⁢p⁢(𝐬 j|𝐞 i)p⁢(𝐬 j)]absent 𝔼 1 𝑝 subscript 𝐬 𝑖 𝑝 conditional subscript 𝐬 𝐢 subscript 𝐞 𝑖 𝑁 1 subscript 𝔼 subscript 𝐬 𝑗 subscript 𝐒 𝑛 𝑒 𝑔 𝑝 conditional subscript 𝐬 𝑗 subscript 𝐞 𝑖 𝑝 subscript 𝐬 𝑗\displaystyle=-\mathbb{E}\log[1+\frac{p(\mathbf{s}_{i})}{p(\mathbf{{s}_{i}}|% \mathbf{e}_{i})}(N-1)\mathbb{E}_{\mathbf{s}_{j}\in\mathbf{S}_{neg}}\frac{p(% \mathbf{s}_{j}|\mathbf{e}_{i})}{p(\mathbf{s}_{j})}]= - blackboard_E roman_log [ 1 + divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT bold_i end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ( italic_N - 1 ) blackboard_E start_POSTSUBSCRIPT bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ∈ bold_S start_POSTSUBSCRIPT italic_n italic_e italic_g end_POSTSUBSCRIPT end_POSTSUBSCRIPT divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ) end_ARG ]
(10)≈−𝔼⁢log⁡[1+p⁢(𝐬 i)p⁢(𝐬 𝐢|𝐞 i)⁢∑𝐬 𝐣∈𝐒 n⁢e⁢g p⁢(𝐬 j|𝐞 i)p⁢(𝐬 j)]absent 𝔼 1 𝑝 subscript 𝐬 𝑖 𝑝 conditional subscript 𝐬 𝐢 subscript 𝐞 𝑖 subscript subscript 𝐬 𝐣 subscript 𝐒 𝑛 𝑒 𝑔 𝑝 conditional subscript 𝐬 𝑗 subscript 𝐞 𝑖 𝑝 subscript 𝐬 𝑗\displaystyle\approx-\mathbb{E}\log[1+\frac{p(\mathbf{s}_{i})}{p(\mathbf{{s}_{% i}}|\mathbf{e}_{i})}\sum_{\mathbf{s_{j}}\in\mathbf{S}_{neg}}\frac{p(\mathbf{s}% _{j}|\mathbf{e}_{i})}{p(\mathbf{s}_{j})}]≈ - blackboard_E roman_log [ 1 + divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT bold_i end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ∑ start_POSTSUBSCRIPT bold_s start_POSTSUBSCRIPT bold_j end_POSTSUBSCRIPT ∈ bold_S start_POSTSUBSCRIPT italic_n italic_e italic_g end_POSTSUBSCRIPT end_POSTSUBSCRIPT divide start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT | bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG italic_p ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ) end_ARG ]
(11)=𝔼⁢log⁡[f⁢(𝐬 i,𝐞 i)∑𝐬 j∈𝐒 f⁢(𝐬 j,𝐞 i)].absent 𝔼 𝑓 subscript 𝐬 𝑖 subscript 𝐞 𝑖 subscript subscript 𝐬 𝑗 𝐒 𝑓 subscript 𝐬 𝑗 subscript 𝐞 𝑖\displaystyle=\mathbb{E}\log[\frac{f(\mathbf{s}_{i},\mathbf{e}_{i})}{\sum_{% \mathbf{s}_{j}\in\mathbf{S}}f(\mathbf{s}_{j},\mathbf{e}_{i})}].= blackboard_E roman_log [ divide start_ARG italic_f ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG ∑ start_POSTSUBSCRIPT bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ∈ bold_S end_POSTSUBSCRIPT italic_f ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ] .

Here, 𝐒 neg subscript 𝐒 neg\mathbf{S}_{\text{neg}}bold_S start_POSTSUBSCRIPT neg end_POSTSUBSCRIPT represents the negative samples when considering the i 𝑖 i italic_i-th sample (i.e., 𝐒 n⁢e⁢g=𝐒∖s i subscript 𝐒 𝑛 𝑒 𝑔 𝐒 subscript 𝑠 𝑖\mathbf{S}_{neg}=\mathbf{S}\setminus{s_{i}}bold_S start_POSTSUBSCRIPT italic_n italic_e italic_g end_POSTSUBSCRIPT = bold_S ∖ italic_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT). Up to this point, we have derived, from a theoretical perspective, how to alleivate noisy effects in representations by introducing external knowledge. However, this approach also presents two challenges: i) Challenge 1: How to obtain effective descriptions of users and items that capture their interaction preferences. ii) Challenge 2: How to involves effectively modeling the density ratio f⁢(𝐬,𝐞)𝑓 𝐬 𝐞 f(\mathbf{s},\mathbf{e})italic_f ( bold_s , bold_e ) to maximize the mutual information between 𝐞 𝐞\mathbf{e}bold_e and 𝐬 𝐬\mathbf{s}bold_s. In the following sections, we discuss potential solutions to address these two challenges.

### 3.2. User/Item Profiling Paradigm

In our previous derivation, we emphasize the importance of obtaining textual descriptions, referred to as profiles, for users and items. These profiles play a crucial role in mitigating the impact of noise in the learned representations of recommenders and enable a semantic understanding of users’ and items’ interaction preferences. Ideally, user and item profiles should exhibit the following characteristics:

*   •User profile: should effectively encapsulate the particular types of items that users are inclined to favor, allowing for a comprehensive representation of their personalized tastes and preferences. 
*   •Item profile: It should eloquently articulate the specific types of users that the item is apt to attract, providing a clear representation of the item’s characteristics and qualities that align with the preferences and interests of those users. 

In some cases, the original data may include textual properties related to users and items. For example, in the Yelp dataset, users provide reviews for visited businesses, and businesses have attributes such as location and category. However, such textual data often contains extraneous noise, leading to common predicaments: i) Missing Attributes: Some attributes of certain items or users may be missing; ii) Noisy Textual Data: The text itself may be contaminated with a plethora of noise that is irrelevant to users’ preferences. For instance, in the Steam dataset, user reviews for games may contain numerous special symbols or irrelevant information. These challenges make it difficult to distill useful user and item profiles from text. As a result, prevailing models often convert low-noise attributes into one-hot encodings without effectively leveraging the semantic information present in the textual data.

Fortunately, recent advancements in Large Language Models (LLMs) have unleashed their remarkable text processing capabilities, enabling them to address a wide range of NLP tasks, including text denoising and summarization. This pivotal development opens up new possibilities for generating user and item profiles from the noisy textual features inherent in the dataset. Leveraging the tremendous potential of LLMs, we propose a paradigm for profile generation that capitalizes on collaborative information. Considering that datasets often contain a higher proportion of textual descriptions for item attributes compared to user attributes, our approach takes an item-to-user perspective, as outlined below.

#### 3.2.1. Profile Generation via Reasoning.

Recent research has demonstrated the effectiveness of incorporating process reasoning in LLMs to mitigate hallucination and improve the quality of generated outputs. Building upon these findings, we have meticulously designed the system prompt 𝒮 u/v subscript 𝒮 𝑢 𝑣\mathcal{S}_{u/v}caligraphic_S start_POSTSUBSCRIPT italic_u / italic_v end_POSTSUBSCRIPT as part of the input provided to LLMs. The objective is to clearly define its functionality in generating user profile for user u 𝑢 u italic_u or item profile for item v 𝑣 v italic_v by precisely specifying the input-output content and desired output format. Importantly, we explicitly emphasize the inclusion of reasoning processes as an integral part of the generated output. By combining this system prompt with user/item profile generation prompts 𝒬 u subscript 𝒬 𝑢\mathcal{Q}_{u}caligraphic_Q start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT and 𝒬 v subscript 𝒬 𝑣\mathcal{Q}_{v}caligraphic_Q start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT, we can leverage LLMs to generate accurate profiles. The specific process is outlined as follows:

(12)𝒫 u=L⁢L⁢M⁢s⁢(𝒮 u,𝒬 u),𝒫 v=L⁢L⁢M⁢s⁢(𝒮 v,𝒬 v)formulae-sequence subscript 𝒫 𝑢 𝐿 𝐿 𝑀 𝑠 subscript 𝒮 𝑢 subscript 𝒬 𝑢 subscript 𝒫 𝑣 𝐿 𝐿 𝑀 𝑠 subscript 𝒮 𝑣 subscript 𝒬 𝑣\displaystyle\mathcal{P}_{u}=LLMs(\mathcal{S}_{u},\mathcal{Q}_{u}),\,\,\,\ % \mathcal{P}_{v}=LLMs(\mathcal{S}_{v},\mathcal{Q}_{v})caligraphic_P start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT = italic_L italic_L italic_M italic_s ( caligraphic_S start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT , caligraphic_Q start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT ) , caligraphic_P start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = italic_L italic_L italic_M italic_s ( caligraphic_S start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT , caligraphic_Q start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT )

#### 3.2.2. Item Prompt Construction.

we categorize the textual information of an item v∈𝒱 𝑣 𝒱 v\in\mathcal{V}italic_v ∈ caligraphic_V into four types: title α 𝛼\alpha italic_α, original description β 𝛽\beta italic_β, dataset-specific attributes 𝜸=γ 1,…,γ|𝜸|𝜸 subscript 𝛾 1…subscript 𝛾 𝜸\bm{\gamma}={\gamma_{1},...,\gamma_{|\bm{\gamma}|}}bold_italic_γ = italic_γ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_γ start_POSTSUBSCRIPT | bold_italic_γ | end_POSTSUBSCRIPT, and a collection of n 𝑛 n italic_n reviews from users 𝐫=r 1,…,r n 𝐫 subscript 𝑟 1…subscript 𝑟 𝑛\mathbf{r}={r_{1},...,r_{n}}bold_r = italic_r start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_r start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT. Based on these categories, we can formally outline the arrangement of the input prompt 𝒬 v subscript 𝒬 𝑣\mathcal{Q}_{v}caligraphic_Q start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT for item-profile generation as follows:

(13)𝒬 v=f v⁢(x)⁢w.r.t.x={[α,β],if⁢β⁢exists,[α,𝜸,r^⊂𝐫],other wise.formulae-sequence subscript 𝒬 𝑣 subscript 𝑓 𝑣 x 𝑤 𝑟 𝑡 x cases 𝛼 𝛽 if 𝛽 exists delimited-[]𝛼 𝜸^r 𝐫 other wise\displaystyle\mathcal{Q}_{v}=f_{v}(\textbf{x})\,\,\,\,w.r.t.\,\,\,\,\textbf{x}% =\begin{cases}[\alpha,\beta],&\text{if }\beta\,\,\text{exists},\\ [\alpha,\bm{\gamma},\hat{\textbf{r}}\subset\mathbf{r}],&\text{other wise}.\end% {cases}caligraphic_Q start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = italic_f start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT ( x ) italic_w . italic_r . italic_t . x = { start_ROW start_CELL [ italic_α , italic_β ] , end_CELL start_CELL if italic_β exists , end_CELL end_ROW start_ROW start_CELL [ italic_α , bold_italic_γ , over^ start_ARG r end_ARG ⊂ bold_r ] , end_CELL start_CELL other wise . end_CELL end_ROW

In our approach, we use a function f v⁢(⋅)subscript 𝑓 𝑣⋅f_{v}(\cdot)italic_f start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT ( ⋅ ) specific to each item, which combines various text features into a single string. If the original description β 𝛽\beta italic_β is missing, we randomly sample a subset of reviews r^^r\hat{\textbf{r}}over^ start_ARG r end_ARG and combine them with the attributes for input. By incorporating item descriptions or user reviews, our prompts provide precise information to Large Language Models, ensuring that the generated item profiles accurately reflect appealing characteristics.

#### 3.2.3. User Prompt Construction.

To generate the profile of user u 𝑢 u italic_u, we leverage collaborative information, assuming that we have already generated the item profiles beforehand. Specifically, we consider the items interacted with by user u 𝑢 u italic_u as ℐ u subscript ℐ 𝑢\mathcal{I}_{u}caligraphic_I start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT and uniformly sample a subset of items ℐ^u⊂ℐ u subscript^ℐ 𝑢 subscript ℐ 𝑢\hat{\mathcal{I}}_{u}\subset\mathcal{I}_{u}over^ start_ARG caligraphic_I end_ARG start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT ⊂ caligraphic_I start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT. For each item v 𝑣 v italic_v in ℐ^u subscript^ℐ 𝑢\hat{\mathcal{I}}_{u}over^ start_ARG caligraphic_I end_ARG start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT, we concatenate its textual attributes as c v=[α,𝒫 v,r u v]subscript c 𝑣 𝛼 subscript 𝒫 𝑣 superscript subscript 𝑟 𝑢 𝑣\textbf{c}_{v}=[\alpha,\mathcal{P}_{v},r_{u}^{v}]c start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = [ italic_α , caligraphic_P start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT , italic_r start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_v end_POSTSUPERSCRIPT ], where r u v superscript subscript 𝑟 𝑢 𝑣 r_{u}^{v}italic_r start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_v end_POSTSUPERSCRIPT represents the review provided by user u 𝑢 u italic_u. The input prompt 𝒬 u subscript 𝒬 𝑢\mathcal{Q}_{u}caligraphic_Q start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT for user-profile generation can be defined as follows:

(14)𝒬 u=f u⁢({𝐜 v|v∈ℐ^u}).subscript 𝒬 𝑢 subscript 𝑓 𝑢 conditional-set subscript 𝐜 𝑣 𝑣 subscript^ℐ 𝑢\displaystyle\mathcal{Q}_{u}=f_{u}(\{\mathbf{c}_{v}|v\in\hat{\mathcal{I}}_{u}% \}).caligraphic_Q start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT = italic_f start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT ( { bold_c start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT | italic_v ∈ over^ start_ARG caligraphic_I end_ARG start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT } ) .

The function f u⁢(⋅)subscript 𝑓 𝑢⋅f_{u}(\cdot)italic_f start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT ( ⋅ ) serves a similar purpose to f v⁢(⋅)subscript 𝑓 𝑣⋅f_{v}(\cdot)italic_f start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT ( ⋅ ) by organizing the textual content into a coherent string. Each textual attribute c v subscript c 𝑣\textbf{c}_{v}c start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT includes user reviews, which authentically reflect their genuine opinions. This construction of the user prompt provides valuable insights into their true preferences. Due to space constraints, we have included the detailed design of the prompt, including 𝒮 𝒮\mathcal{S}caligraphic_S, 𝒬 𝒬\mathcal{Q}caligraphic_Q, and f u/v⁢(⋅)subscript 𝑓 𝑢 𝑣⋅f_{u/v}(\cdot)italic_f start_POSTSUBSCRIPT italic_u / italic_v end_POSTSUBSCRIPT ( ⋅ ), along with sample examples in Appendix[A.2](https://arxiv.org/html/2310.15950v5#A1.SS2 "A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation").

### 3.3. Density Ratio Modeling for Mutual Information Maximization

In this section, we outline the process of modeling the density ratio, denoted as f⁢(s i,e i)𝑓 subscript s 𝑖 subscript e 𝑖 f(\textbf{s}_{i},\textbf{e}_{i})italic_f ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ), with the objective of maximizing the mutual information I⁢(s i,e i)𝐼 subscript s 𝑖 subscript e 𝑖 I(\textbf{s}_{i},\textbf{e}_{i})italic_I ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ). First of all, it is important to note that we have previously generated user/item profiles 𝒫 u/v subscript 𝒫 𝑢 𝑣\mathcal{P}_{u/v}caligraphic_P start_POSTSUBSCRIPT italic_u / italic_v end_POSTSUBSCRIPT that showcase their interaction preferences. As such, it is logical to encode the semantic representation s based on these profiles as follow:

(15)s u=𝒯⁢(𝒫 u),s v=𝒯⁢(𝒫 v).formulae-sequence subscript s 𝑢 𝒯 subscript 𝒫 𝑢 subscript s 𝑣 𝒯 subscript 𝒫 𝑣\displaystyle\textbf{s}_{u}=\mathcal{T}(\mathcal{P}_{u}),\,\,\,\,\textbf{s}_{v% }=\mathcal{T}(\mathcal{P}_{v}).s start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT = caligraphic_T ( caligraphic_P start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT ) , s start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = caligraphic_T ( caligraphic_P start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT ) .

Here, 𝒯⁢(⋅)𝒯⋅\mathcal{T}(\cdot)caligraphic_T ( ⋅ ) refers to a cutting-edge technology known as a text embedding model(Su et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib30); Izacard et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib15)), which has been shown to effectively transform diverse text inputs into fixed-length vectors that retain their inherent meaning and contextual information.

According to(Oord et al., [2018](https://arxiv.org/html/2310.15950v5#bib.bib24)), the density ratio f⁢(s i,e i)𝑓 subscript s 𝑖 subscript e 𝑖 f(\textbf{s}_{i},\textbf{e}_{i})italic_f ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) can be interpreted as a positive real-valued score measurement function that captures the similarity between s i subscript s 𝑖\textbf{s}_{i}s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT and e i subscript e 𝑖\textbf{e}_{i}e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT. A more accurate modeling of the density ratio(Sasaki and Takenouchi, [2022](https://arxiv.org/html/2310.15950v5#bib.bib29)) can have a positive impact on the alignment between CF-side rational representations and LLMs-enhanced semantic representations, helping to mitigate the influence of noisy signals in representation learning. In this context, we propose two types of modeling approaches that are well-suited for achieving this alignment. The first approach is contrastive modeling, which has been extensively validated(Wu et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib38); Khosla et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib16)) for effectively aligning different views bidirectionally, such as through pull and push pairs. The second approach is mask-reconstruction generative modeling, which is widely used as a self-supervised mechanism for reconstructing the partially masked input from data itself(He et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib11); Hou et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib14)). By employing CF-side representations to reconstruct the semantic representations, we can effectively align these two forms of information.

#### 3.3.1. Contrastive Alignment

As depicted in Fig[3](https://arxiv.org/html/2310.15950v5#S3.F3 "Figure 3 ‣ 3.1. Theoretical Basis of RLMRec ‣ 3. Methodology ‣ Representation Learning with Large Language Models for Recommendation") (b), we denote the specific implementation of f⁢(s i,e i)𝑓 subscript s 𝑖 subscript e 𝑖 f(\textbf{s}_{i},\textbf{e}_{i})italic_f ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) as contrastive alignment.

(16)f⁢(s i,e i)=e⁢x⁢p⁢(s⁢i⁢m⁢(σ↓⁢(s i),e i)).𝑓 subscript s 𝑖 subscript e 𝑖 𝑒 𝑥 𝑝 𝑠 𝑖 𝑚 subscript 𝜎↓subscript s 𝑖 subscript e 𝑖\displaystyle f(\textbf{s}_{i},\textbf{e}_{i})=exp(sim(\sigma_{\downarrow}(% \textbf{s}_{i}),\textbf{e}_{i})).italic_f ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_e italic_x italic_p ( italic_s italic_i italic_m ( italic_σ start_POSTSUBSCRIPT ↓ end_POSTSUBSCRIPT ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) ) .

The function s⁢i⁢m⁢(⋅)𝑠 𝑖 𝑚⋅sim(\cdot)italic_s italic_i italic_m ( ⋅ ) represents the cosine similarity, while σ↓subscript 𝜎↓\sigma_{\downarrow}italic_σ start_POSTSUBSCRIPT ↓ end_POSTSUBSCRIPT denotes a multi-layer perception that maps the semantic representation s i subscript s 𝑖\textbf{s}_{i}s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT into the feature space of e i subscript e 𝑖\textbf{e}_{i}e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT. In our contrastive alignment, we treat e i subscript e 𝑖\textbf{e}_{i}e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT and s i subscript s 𝑖\textbf{s}_{i}s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT as positive sample pairs. During the learning process, these pairs are pulled towards each other to align their representations. In the specific implementation, the objective is to bring positive sample pairs closer within a batch while considering the remaining samples as negatives.

#### 3.3.2. Generative Alignment

Taking inspiration from recent research on the masked autoencoder (MAE), which is considered a paradigm of generative self-supervised learning, we propose an additional modeling approach for the density ratio within the MAE.

(17)f⁢(s i,e i)=e⁢x⁢p⁢(s⁢i⁢m⁢(s i,σ↑⁢(e u^)))⁢w.r.t.e i^=ℛ⁢({x}∖x i).formulae-sequence 𝑓 subscript s 𝑖 subscript e 𝑖 𝑒 𝑥 𝑝 𝑠 𝑖 𝑚 subscript s 𝑖 subscript 𝜎↑^subscript e 𝑢 𝑤 𝑟 𝑡^subscript e 𝑖 ℛ x subscript x 𝑖\displaystyle f(\textbf{s}_{i},\textbf{e}_{i})=exp(sim(\textbf{s}_{i},\sigma_{% \uparrow}(\hat{\textbf{e}_{u}})))\,\,\,\,w.r.t.\,\,\,\,\hat{\textbf{e}_{i}}=% \mathcal{R}(\{\textbf{x}\}\setminus\textbf{x}_{i}).italic_f ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) = italic_e italic_x italic_p ( italic_s italic_i italic_m ( s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , italic_σ start_POSTSUBSCRIPT ↑ end_POSTSUBSCRIPT ( over^ start_ARG e start_POSTSUBSCRIPT italic_u end_POSTSUBSCRIPT end_ARG ) ) ) italic_w . italic_r . italic_t . over^ start_ARG e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_ARG = caligraphic_R ( { x } ∖ x start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) .

We employ σ↑subscript 𝜎↑\sigma_{\uparrow}italic_σ start_POSTSUBSCRIPT ↑ end_POSTSUBSCRIPT as a multi-layer perception model to map the representations to the semantic feature space. x∖x i x subscript x 𝑖{{\textbf{x}}}\setminus\textbf{x}_{i}x ∖ x start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT represents the initial embedding of the i 𝑖 i italic_i-th sample with masking applied. The generative process follows a single-direction reconstruction approach, focusing on reconstructing the semantic representations exclusively for the masked samples. Specifically, the masking operation involves replacing the initial embedding with a designated mask token (i.e., [M⁢A⁢S⁢K]delimited-[]𝑀 𝐴 𝑆 𝐾[MASK][ italic_M italic_A italic_S italic_K ]), and a random subset of users/items is masked and subsequently reconstructed. This allows us to explore the reconstruction capabilities within the semantic feature space.

With our contrastive and generative alignment method, we effectively align the knowledge of the LLM with the domain of understanding user preferences. This is achieved by combining id-based collaborative relational signals with text-based behavior semantics. We have given the names RLMRec-Con and RLMRec-Gen to our two proposed modeling approaches, respectively. In our experiments conducted on real-world data, we will comprehensively evaluate the performance of these two models across various tasks, each showcasing its unique advantages and disadvantages.

### 3.4. Model-agnostic Learning

Up until this point, our focus has been on optimizing the CF-side relational representation e and LLM-side semantic representation s. Any model that can perform representation learning for users/items can undergo the optimization process described earlier. Hence, our approach is model-agnostic and can seamlessly enhance existing collaborative filtering recommenders. Assuming that the optimization objective of the recommender ℛ ℛ\mathcal{R}caligraphic_R is denoted as ℒ ℛ subscript ℒ ℛ\mathcal{L}_{\mathcal{R}}caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT, our overall optimization function ℒ ℒ\mathcal{L}caligraphic_L can be formulated as follows:

(18)ℒ=ℒ ℛ+ℒ i⁢n⁢f⁢o⁢w.r.t.ℒ i⁢n⁢f⁢o=−𝔼⁢log⁡[f⁢(𝐬 i,𝐞 i)∑𝐬 j∈𝐒 f⁢(𝐬 j,𝐞 i)],formulae-sequence ℒ subscript ℒ ℛ subscript ℒ 𝑖 𝑛 𝑓 𝑜 𝑤 𝑟 𝑡 subscript ℒ 𝑖 𝑛 𝑓 𝑜 𝔼 𝑓 subscript 𝐬 𝑖 subscript 𝐞 𝑖 subscript subscript 𝐬 𝑗 𝐒 𝑓 subscript 𝐬 𝑗 subscript 𝐞 𝑖\displaystyle\mathcal{L}=\mathcal{L}_{\mathcal{R}}+\mathcal{L}_{info}\,\,\,\,w% .r.t.\,\,\,\mathcal{L}_{info}=-\mathbb{E}\log[\frac{f(\mathbf{s}_{i},\mathbf{e% }_{i})}{\sum_{\mathbf{s}_{j}\in\mathbf{S}}f(\mathbf{s}_{j},\mathbf{e}_{i})}],caligraphic_L = caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT + caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT italic_w . italic_r . italic_t . caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT = - blackboard_E roman_log [ divide start_ARG italic_f ( bold_s start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG start_ARG ∑ start_POSTSUBSCRIPT bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT ∈ bold_S end_POSTSUBSCRIPT italic_f ( bold_s start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT , bold_e start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) end_ARG ] ,

Minimizing the overall optimization function ℒ ℒ\mathcal{L}caligraphic_L corresponds to maximizing the mutual information mentioned earlier.

4. Evaluation
-------------

Table 1. Recommendation performance Imprvement of all backbone methods on different datasets in terms of Recall and NDCG. The superscript * indicates the Imprvement is statistically significant where the p-value is less than 0.05 0.05 0.05 0.05.

This section presents the experimental evaluation of our RLMRec on multiple datasets to address the following research questions:

*   •RQ1: Does our proposed RLMRec improve upon existing state-of-the-art recommenders across various experimental settings? 
*   •RQ2: Do the LLM-enhanced semantic representations contribute to the recommendation performance improvement? 
*   •RQ3: Does our proposed framework effectively tackle the issue of noisy data through cross-view semantic alignment? 
*   •RQ4: What is the potential of our model as a pre-training framework for enhancing the performance of recommender systems? 
*   •RQ5: How does our RLMRec perform w.r.t training efficiency? 

### 4.1. Experimental Settings

#### 4.1.1. Datasets

We conduct evaluations of our RLMRec on three public datasets: Amazon-book: This dataset contains user ratings and corresponding reviews for books sold on Amazon. Yelp: This dataset is a user-business dataset that provides extensive textual category information about various businesses. Steam: This dataset consists of textual feedback given by users for electronic games available on the Steam platform. Following the similar settings in(Wang et al., [2019](https://arxiv.org/html/2310.15950v5#bib.bib36); Xia et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib43); Yu et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib45)) for data preprocessing, we filter out interactions with ratings below 3 for both the Amazon-book and Yelp data. No filtering is applied to the Steam dataset due to the absence of rating scores. We then perform k-core filtering and divided each dataset into training, validation, and testing sets using a 3:1:1 ratio. Please refer to Table [5](https://arxiv.org/html/2310.15950v5#A1.T5 "Table 5 ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") in Appendix for a summary of the dataset statistics.

Table 2. Comparison with LLMs-enhanced Approaches.

#### 4.1.2. Evaluation Protocols and Metrics

To ensure comprehensive evaluation and mitigate bias, we adopt the all-rank protocol(He et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib12); Wu et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib38); Wang et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib37)) across all items to accurately assess our recommendations. We use two widely adopted ranking-based metrics: Recall@N and NDCG@N, which measure the model effectiveness.

#### 4.1.3. Base Models

We evaluate the effectiveness of our RLMRec by integrating it with state-of-the-art representation-based recommenders based on SSLRec(Ren et al., [2023a](https://arxiv.org/html/2310.15950v5#bib.bib26)).

*   •GCCF(Chen et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib7)): It simplifies graph-based recommender design by re-evaluating the role of non-linear operations in GNNs. 
*   •LightGCN(He et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib12)): It creates a lightweight recommender by streamlining redundant neural modules in graph message passing. 
*   •SGL(Wu et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib38)): It utilizes node/edge dropout as a data augmentator to generate diverse perspectives for contrastive learning. 
*   •SimGCL(Yu et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib45)): It enhances recommendation performance by introducing an augmentation-free view generation technique. 
*   •DCCF(Ren et al., [2023b](https://arxiv.org/html/2310.15950v5#bib.bib27)): It captures intent-wise relationships for recommendation purposes using disentangled contrastive learning. 
*   •AutoCF(Xia et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib42)): It is a self-supervised masked autoencoder to automate the process of data augmentation for recommendation. 

#### 4.1.4. Implementation Details

The dimension of representations (i.e.,x and e) is set to 32 for all base models. We determine the hyperparameters for each model through grid search. To generate user and item profiles, we leverage the ChatGPT model (specifically, gpt-3.5-turbo) provided by OpenAI. We use the text-embedding-ada-002(Neelakantan et al., [2022](https://arxiv.org/html/2310.15950v5#bib.bib23)) to generate semantic representations s. During training, all methods are trained with a fixed batch size of 4096 and a learning rate of 1e-3 using the Adam optimizer. We adopt the early stop technique based on the model’s performance on the validation set.

### 4.2. Performance Comparison (RQ1)

Model-agnostic Performance Gain. To demonstrate the effectiveness of RLMRec in improving recommendation performance, we integrate it into six state-of-the-art collaborative filtering models. We conduct experiments using 5 random initializations and report the average results in Table[1](https://arxiv.org/html/2310.15950v5#S4.T1 "Table 1 ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation"). The evaluation results reveal several interesting observations, as outlined below:

*   •Overall, we consistently observe that integrating RLMRec with the backbone recommenders leads to improved performance compared to the original versions. This provides compelling evidence for the effectiveness of RLMRec. We attribute these improvements to two key factors: i) RLMRec enables accurate user/item profiling empowered by LLMs, enhancing the representation of rich semantic information from user interaction behaviors. ii) our cross-view mutual information maximization facilitates the cooperative enhancement of CF-side relational embeddings and LLM-side semantic representations, effectively filtering out irrelevant noise in the recommendation features. 
*   •It is clear that both contrastive and generative modeling approaches generally improve performance. However, it is important to note that the contrastive approach exhibits superior performance when combined with various backbones like GCCF and SimGCL. Conversely, when applied to AutoCF, which involves masked reconstruction, RLMRec-Gen shows more significant improvements. We speculate that the mask operation functions as a form of regularization, leading to better results when used in conjunction with methods that employ a generative approach. 

Superiority over LLM-enhanced Approach. In addition, we conduct a comparative evaluation of the effectiveness of RLMRec in comparison to KAR(Xi et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib41)), a recent LLM-enhanced user behavior modeling approach. KAR aims to generate textual user/item descriptions to enhance the learning of user preferences for the CTR task. To ensure a fair comparison, we utilized the same semantic representation as in our approach and employed two classic methods (LightGCN and SGL) as the backbone models. This could be attributed to the fact that, while KAR incorporates textual information into the learning of user preferences, it treats the semantic representation as input features for the model. As a result, it may not effectively align the textual knowledge with the user behavior representations and could be more susceptible to irrelevant noise from either user behaviors or the LLM knowledge base.

![Image 4: Refer to caption](https://arxiv.org/html/2310.15950v5/x4.png)

Figure 4. Ablation study on variant text embedding models conducted on the Amazon-book dataset. Shuffling involves reordering user/item embeddings.

### 4.3. Ablation Study (RQ2)

In this section, we examine the impact of integrating semantic representations on performance. To do this, we shuffle the acquired semantic representations, creating a misalignment with collaborative relational representation and LLM’s knowledge. We use the default semantic encoding model, text-embedding-ada-002(Brown et al., [2020](https://arxiv.org/html/2310.15950v5#bib.bib3)), and also experiment with advanced models like Contriever(Izacard et al., [2021](https://arxiv.org/html/2310.15950v5#bib.bib15)) and Instructor(Su et al., [2023](https://arxiv.org/html/2310.15950v5#bib.bib30)). We evaluate our approach on four backbone methods (i.e., LightGCN, GCCF, SimGCL, and DCCF). The results are summarized in Figure[4](https://arxiv.org/html/2310.15950v5#S4.F4 "Figure 4 ‣ 4.2. Performance Comparison (RQ1) ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation"), leading to two key observations.

*   •After randomly rearranging the semantic representations to disrupt the correlation between collaborative and semantic signals, we observe a decrease in performance for both RLMRec-Con and RLMRec-Gen on the evaluated backbone models. This indicates that the shuffled representations introduce noise due to the mismatch between semantic and collaborative information. It provides evidence that accurate alignment between the LLM’s semantic knowledge and collaborative relationships among users is crucial for enhancing recommendation performance. 
*   •When we utilize variant text embedding models like Contriever and Instructor, our RLMRec still enhances the base performance, similar to the default setting with text-embedding-ada-002. This indicates that our RLMRec can effectively leverage an appropriate text encoder capable of transferring textual semantics into preference representations to improve the performance of the recommender backbone. Moreover, the ability of text embedding models to capture semantic information with higher accuracy can lead to even more significant improvements. 

### 4.4. In-depth Analysis of RLMRec(RQ3 – RQ5)

![Image 5: Refer to caption](https://arxiv.org/html/2310.15950v5/x5.png)

Figure 5. Comparing performance on different noise ratios in the Amazon-book dataset with LightGCN as the base model.

#### 4.4.1. Performance w.r.t. Noisy Data (RQ3)

We assess the robustness of RLMRec to data noise by adding non-existent interactions to the original training data. Noise levels range from 5% to 25% relative to the training set size. Using the Amazon dataset, we compare the performance of vanilla LightGCN with LightGCN enhanced by our RLMRec-Con/Gen. Key findings from Fig[5](https://arxiv.org/html/2310.15950v5#S4.F5 "Figure 5 ‣ 4.4. In-depth Analysis of RLMRec (RQ3 – RQ5) ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation") are:

*   •(i) Both RLMRec-Con and RLMRec-Gen consistently outperform the LightGCN backbone model at all noise levels. This highlights the advantages of incorporating semantic information and leveraging mutual information to filter out irrelevant data, resulting in improved recommendations and robustness over noise. 
*   •(ii) RLMRec-Con has shown better resistance to data noise compared to RLMRec-Gen. This is likely due to the inherent noise introduced by the generative method through node masking. In contrast, contrastive methods encounter less noise, leading to superior performance under the same noise ratio. 

Table 3. Performance comparison with different initialized parameters from various pre-training methods on the Yelp.

#### 4.4.2. Performance in Pre-training Scenarios (RQ4)

We explore the potential of our semantically involved training mechanism as a pre-training technique for downstream models. Using the Yelp dataset, we utilize data from 2012 to 2017 for pre-training and divide the data from 2018 to 2019 into a training set, a validation set, and a test set (the downstream dataset). Both datasets contain the same users and items. We train vanilla LightGCN and our model on the pre-training dataset. The learned parameters are used to initialize the embeddings for vanilla LightGCN, which is then trained on the downstream dataset. Key findings from Table[3](https://arxiv.org/html/2310.15950v5#S4.T3 "Table 3 ‣ 4.4.1. Performance w.r.t. Noisy Data (RQ3) ‣ 4.4. In-depth Analysis of RLMRec (RQ3 – RQ5) ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation") are:

*   •Pre-training with parameters yields superior results compared to no pre-training, regardless of whether it was done with the base model or our RLMRec. This suggests that the pre-training dataset contains valuable collaborative information that helps predict user/item preferences and benefits downstream tasks. 
*   •Both RLMRec-Con and RLMRec-Gen provide better pre-training benefits compared to pre-training with the base model alone, with RLMRec-Gen achieving the best results. This highlights the advantage of incorporating semantic information and the effectiveness of generative methods in pre-training scenarios, potentially due to the regulatory function of the mask operation, preventing overfitting on the pre-training dataset. 

Table 4. RLMRec’s efficiency with various recommenders.

#### 4.4.3. Analysis of Training Efficiency (RQ5)

We analyze the time complexity of using RLMRec. The theoretical time complexity of the multi-layer perception (σ↑subscript 𝜎↑\sigma_{\uparrow}italic_σ start_POSTSUBSCRIPT ↑ end_POSTSUBSCRIPT and σ↓subscript 𝜎↓\sigma_{\downarrow}italic_σ start_POSTSUBSCRIPT ↓ end_POSTSUBSCRIPT) for both RLMRec-Con and RLMRec-Gen is 𝒪⁢(N×d s×d e)𝒪 𝑁 subscript 𝑑 𝑠 subscript 𝑑 𝑒\mathcal{O}(N\times d_{s}\times d_{e})caligraphic_O ( italic_N × italic_d start_POSTSUBSCRIPT italic_s end_POSTSUBSCRIPT × italic_d start_POSTSUBSCRIPT italic_e end_POSTSUBSCRIPT ). For RLMRec-Con, the loss computation introduces an additional complexity of O⁢(N 2×d)𝑂 superscript 𝑁 2 𝑑 O(N^{2}\times d)italic_O ( italic_N start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT × italic_d ). For RLMRec-Gen, the time complexity is O⁢(M×d+M×N×d)𝑂 𝑀 𝑑 𝑀 𝑁 𝑑 O(M\times d+M\times N\times d)italic_O ( italic_M × italic_d + italic_M × italic_N × italic_d ), where the masking operation accounts for M×d 𝑀 𝑑 M\times d italic_M × italic_d, with M 𝑀 M italic_M representing the number of masked nodes. In Table[4](https://arxiv.org/html/2310.15950v5#S4.T4 "Table 4 ‣ 4.4.2. Performance in Pre-training Scenarios (RQ4) ‣ 4.4. In-depth Analysis of RLMRec (RQ3 – RQ5) ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation"), we present the epoch time of training on a server with an Intel Xeon Silver 4314 CPU and an NVIDIA RTX 3090 GPU. The results show that the time cost of RLMRec-Gen is consistently lower than that of RLMRec-Con. This is primarily because the value of N 𝑁 N italic_N in RLMRec-Con is determined by the batch size, which tends to be larger than the number of masked nodes M in RLMRec-Gen. Additionally, for larger models with improved performance, the additional time complexity is only around 10% to 20% compared to the original time.

### 4.5. Case Study

We explore the integration of LLM-enhanced semantics to capture global user relationships that are not easily captured through direct message passing. Figure[6](https://arxiv.org/html/2310.15950v5#S4.F6 "Figure 6 ‣ 4.5. Case Study ‣ 4. Evaluation ‣ Representation Learning with Large Language Models for Recommendation") presents a case study where the distance between user u 1998 subscript 𝑢 1998 u_{1998}italic_u start_POSTSUBSCRIPT 1998 end_POSTSUBSCRIPT and u 227 subscript 𝑢 227 u_{227}italic_u start_POSTSUBSCRIPT 227 end_POSTSUBSCRIPT exceeds 3 hops. To evaluate the models’ ability to capture their relationship, we examine the similarity of user representations. We compared LightGCN and RLMRec-Con, both using the same backbone. Two metrics were introduced: a relevance score for user u 1998 subscript 𝑢 1998 u_{1998}italic_u start_POSTSUBSCRIPT 1998 end_POSTSUBSCRIPT and the ranking of its long-distance neighbors (>>> 3 hops) based on the score. By incorporating semantic information derived from LLMs, such as shared interests between u 1998 subscript 𝑢 1998 u_{1998}italic_u start_POSTSUBSCRIPT 1998 end_POSTSUBSCRIPT and u 227 subscript 𝑢 227 u_{227}italic_u start_POSTSUBSCRIPT 227 end_POSTSUBSCRIPT (e.g., friendly service), we observed an increase in both the relevance score and ranking. This suggests that the learned representations from RLMRec capture global collaborative relationships beyond ID-based recommendation techniques.

![Image 6: Refer to caption](https://arxiv.org/html/2310.15950v5/x6.png)

Figure 6. Case study on capturing global user dependencies.

5. Conclusion
-------------

This paper presents RLMRec, a model-agnostic framework that leverages Large Language Models (LLMs) to improve the representation performance of recommender systems. We introduce a collaborative profile generation paradigm and a reasoning-driven system prompt, emphasizing the inclusion of reasoning processes in the generated output. RLMRec utilizes contrastive and generative alignment techniques to align CF-side relational embeddings with LLM-side semantic representations, effectively reducing feature noise. The framework combines the strengths of general recommenders and LLMs, supported by robust theoretical guarantees, and is extensively evaluated on real-world datasets. Our future investigations will focus on advancing LLM-based reasoning results in recommender systems by providing more insightful explanations.

References
----------

*   (1)
*   Bao et al. (2023) Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. _arXiv preprint arXiv:2305.00447_ (2023). 
*   Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In _NeurIPS_, Vol.33. 1877–1901. 
*   Cai et al. (2023) Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2023. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In _ICLR_. 
*   Chang et al. (2021) Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021. Sequential recommendation with graph neural networks. In _SIGIR_. 378–387. 
*   Chen et al. (2023) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions. _ACM Transactions on Information Systems (TOIS)_ 41, 3 (2023), 1–39. 
*   Chen et al. (2020) Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In _AAAI_, Vol.34. 27–34. 
*   Fan et al. (2023) Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang, and Qing Li. 2023. Recommender systems in the era of large language models (llms). _arXiv preprint arXiv:2307.02046_ (2023). 
*   Gao et al. (2023) Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. _arXiv preprint arXiv:2303.14524_ (2023). 
*   Geng et al. (2022) Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In _RecSys_. 299–315. 
*   He et al. (2022) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In _CVPR_. 16000–16009. 
*   He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In _SIGIR_. 639–648. 
*   Hou et al. (2023) Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023. Large language models are zero-shot rankers for recommender systems. _arXiv preprint arXiv:2305.08845_ (2023). 
*   Hou et al. (2022) Zhenyu Hou, Xiao Liu, Yuxiao Dong, Chunjie Wang, Jie Tang, et al. 2022. Graphmae: Self-supervised masked graph autoencoders. _arXiv preprint arXiv:2205.10803_ (2022). 
*   Izacard et al. (2021) Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense information retrieval with contrastive learning. _arXiv preprint arXiv:2112.09118_ (2021). 
*   Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. _NeurIPS_ (2020). 
*   Koren et al. (2021) Yehuda Koren, Steffen Rendle, and Robert Bell. 2021. Advances in collaborative filtering. _Recommender systems handbook_ (2021), 91–142. 
*   Li et al. (2023) Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. 2023. Text Is All You Need: Learning Language Representations for Sequential Recommendation. _arXiv preprint arXiv:2305.13731_ (2023). 
*   Lin et al. (2023) Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, et al. 2023. How Can Recommender Systems Benefit from Large Language Models: A Survey. _arXiv preprint arXiv:2306.05817_ (2023). 
*   Lin et al. (2022) Zihan Lin, Changxin Tian, Yupeng Hou, and Wayne Xin Zhao. 2022. Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In _WWW_. 2320–2329. 
*   Liu et al. (2023a) Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, and Yan Zhang. 2023a. Is chatgpt a good recommender? a preliminary study. _arXiv preprint arXiv:2304.10149_ (2023). 
*   Liu et al. (2023b) Peng Liu, Lemei Zhang, and Jon Atle Gulla. 2023b. Pre-train, prompt and recommendation: A comprehensive survey of language modelling paradigm adaptations in recommender systems. _arXiv preprint arXiv:2302.03735_ (2023). 
*   Neelakantan et al. (2022) Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. Text and code embeddings by contrastive pre-training. _arXiv preprint arXiv:2201.10005_ (2022). 
*   Oord et al. (2018) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. _arXiv preprint arXiv:1807.03748_ (2018). 
*   OpenAI (2023) OpenAI. 2023. GPT-4 Technical Report. _arXiv preprint arXiv:2303.08774_ (2023). 
*   Ren et al. (2023a) Xubin Ren, Lianghao Xia, Yuhao Yang, Wei Wei, Tianle Wang, Xuheng Cai, and Chao Huang. 2023a. SSLRec: A Self-Supervised Learning Framework for Recommendation. _arXiv preprint arXiv:2308.05697_ (2023). 
*   Ren et al. (2023b) Xubin Ren, Lianghao Xia, Jiashu Zhao, Dawei Yin, and Chao Huang. 2023b. Disentangled Contrastive Collaborative Filtering. In _SIGIR_. 1137–1146. 
*   Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. _arXiv preprint arXiv:1205.2618_ (2012). 
*   Sasaki and Takenouchi (2022) Hiroaki Sasaki and Takashi Takenouchi. 2022. Representation learning for maximization of MI, nonlinear ICA and nonlinear subspaces with robust density ratio estimation. _JMLR_ (2022). 
*   Su et al. (2023) Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. One Embedder, Any Task: Instruction-Finetuned Text Embeddings. In _Findings of the ACL_. 1102–1121. 
*   Su and Khoshgoftaar (2009) Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. _Advances in artificial intelligence_ 2009 (2009). 
*   Sun et al. (2023) Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent. _arXiv preprint arXiv:2304.09542_ (2023). 
*   Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. _arXiv preprint arXiv:2302.13971_ (2023). 
*   Wang et al. (2021a) Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021a. Denoising implicit feedback for recommendation. In _WSDM_. 373–381. 
*   Wang et al. (2021b) Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, and Tat-Seng Chua. 2021b. Deconfounded recommendation for alleviating bias amplification. In _KDD_. 1717–1725. 
*   Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In _SIGIR_. 165–174. 
*   Wang et al. (2020) Yifan Wang, Suyao Tang, Yuntong Lei, Weiping Song, Sheng Wang, and Ming Zhang. 2020. Disenhan: Disentangled heterogeneous graph attention network for recommendation. In _CIKM_. 1605–1614. 
*   Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised graph learning for recommendation. In _SIGIR_. 726–735. 
*   Wu et al. (2023) Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al. 2023. A Survey on Large Language Models for Recommendation. _arXiv preprint arXiv:2305.19860_ (2023). 
*   Wu et al. (2022) Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. 2022. Graph neural networks in recommender systems: a survey. _ACM Computing Surveys (CSUR)_ 55, 5 (2022), 1–37. 
*   Xi et al. (2023) Yunjia Xi, Weiwen Liu, Jianghao Lin, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, Rui Zhang, and Yong Yu. 2023. Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models. _arXiv preprint arXiv:2306.10933_ (2023). 
*   Xia et al. (2023) Lianghao Xia, Chao Huang, Chunzhen Huang, Kangyi Lin, Tao Yu, and Ben Kao. 2023. Automated Self-Supervised Learning for Recommendation. In _WWW_. 
*   Xia et al. (2022) Lianghao Xia, Chao Huang, Yong Xu, Jiashu Zhao, Dawei Yin, and Jimmy Huang. 2022. Hypergraph contrastive collaborative filtering. In _SIGIR_. 70–79. 
*   Yang et al. (2023) Yuhao Yang, Chao Huang, Lianghao Xia, Chunzhen Huang, Da Luo, and Kangyi Lin. 2023. Debiased Contrastive Learning for Sequential Recommendation. In _WWW_. 1063–1073. 
*   Yu et al. (2022) Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. 2022. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In _SIGIR_. 1294–1303. 
*   Yu et al. (2023) Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2023. Self-supervised learning for recommender systems: A survey. _TKDE_ (2023). 
*   Yuan et al. (2023) Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to go next for recommender systems? id-vs. modality-based recommender models revisited. In _SIGIR_. 2639–2649. 
*   Zhang et al. (2023) Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2023. Recommendation as instruction following: A large language model empowered recommendation approach. _arXiv preprint arXiv:2305.07001_ (2023). 

Appendix A Supplementary Material
---------------------------------

In the supplementary materials, we provide the training procedure of our proposed framework, RLMRec, through pseudocode. We also offer detailed insights into the design of prompts, accompanied by examples, to visualize the profile generation process within our item-to-user generation paradigm. Finally, we present experiment details for the reranking task mentioned in Section[1](https://arxiv.org/html/2310.15950v5#S1 "1. Introduction ‣ Representation Learning with Large Language Models for Recommendation"), where we analyze specific examples within the task.

Table 5. Statistics of the experimental datasets.

input :Base model

ℛ ℛ\mathcal{R}caligraphic_R
, implicit feedback

𝒳 𝒳\mathcal{X}caligraphic_X
, semantic representation s for each user & item and learning rate

η 𝜂\eta italic_η

Result:Trained model parameters

𝚯 𝚯\mathbf{\Theta}bold_Θ

1 repeat

2 uniformly sample batch data ℬ={(u,v p⁢o⁢s,v n⁢e⁢g)}∈𝒳 ℬ 𝑢 subscript 𝑣 𝑝 𝑜 𝑠 subscript 𝑣 𝑛 𝑒 𝑔 𝒳\mathcal{B}=\{(u,v_{pos},v_{neg})\}\in\mathcal{X}caligraphic_B = { ( italic_u , italic_v start_POSTSUBSCRIPT italic_p italic_o italic_s end_POSTSUBSCRIPT , italic_v start_POSTSUBSCRIPT italic_n italic_e italic_g end_POSTSUBSCRIPT ) } ∈ caligraphic_X;

3 inference collaborative-side representation e u/v subscript e 𝑢 𝑣\textbf{e}_{u/v}e start_POSTSUBSCRIPT italic_u / italic_v end_POSTSUBSCRIPT with ℛ ℛ\mathcal{R}caligraphic_R;

4 calculate model optimization objective ℒ ℛ subscript ℒ ℛ\mathcal{L}_{\mathcal{R}}caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT based on ℬ ℬ\mathcal{B}caligraphic_B;

5 calculate L i⁢n⁢f⁢o subscript 𝐿 𝑖 𝑛 𝑓 𝑜 L_{info}italic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT w.r.t. Eq([16](https://arxiv.org/html/2310.15950v5#S3.E16 "In 3.3.1. Contrastive Alignment ‣ 3.3. Density Ratio Modeling for Mutual Information Maximization ‣ 3. Methodology ‣ Representation Learning with Large Language Models for Recommendation")&[18](https://arxiv.org/html/2310.15950v5#S3.E18 "In 3.4. Model-agnostic Learning ‣ 3. Methodology ‣ Representation Learning with Large Language Models for Recommendation")) for all u/v 𝑢 𝑣 u/v italic_u / italic_v in ℬ ℬ\mathcal{B}caligraphic_B;

6

ℒ=ℒ ℛ+ℒ i⁢n⁢f⁢o ℒ subscript ℒ ℛ subscript ℒ 𝑖 𝑛 𝑓 𝑜\mathcal{L}=\mathcal{L}_{\mathcal{R}}+\mathcal{L}_{info}caligraphic_L = caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT + caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT
;

7

𝚯←𝚯−η⁢∇𝚯 ℒ←𝚯 𝚯 𝜂 subscript∇𝚯 ℒ\mathbf{\Theta}\leftarrow\mathbf{\Theta}-\eta\nabla_{\mathbf{\Theta}}\mathcal{L}bold_Θ ← bold_Θ - italic_η ∇ start_POSTSUBSCRIPT bold_Θ end_POSTSUBSCRIPT caligraphic_L
;

8

9 until _convergence_;

Algorithm 1 Training Procedure in RLMRec-Con

input :Base model

ℛ ℛ\mathcal{R}caligraphic_R
, implicit feedback

𝒳 𝒳\mathcal{X}caligraphic_X
, semantic representation s for each user & item, learning rate

η 𝜂\eta italic_η
and masking ratio

α 𝛼\alpha italic_α

Result:Trained model parameters

𝚯 𝚯\mathbf{\Theta}bold_Θ

1 repeat

2 uniformly sample batch data ℬ={(u,v p⁢o⁢s,v n⁢e⁢g)}∈𝒳 ℬ 𝑢 subscript 𝑣 𝑝 𝑜 𝑠 subscript 𝑣 𝑛 𝑒 𝑔 𝒳\mathcal{B}=\{(u,v_{pos},v_{neg})\}\in\mathcal{X}caligraphic_B = { ( italic_u , italic_v start_POSTSUBSCRIPT italic_p italic_o italic_s end_POSTSUBSCRIPT , italic_v start_POSTSUBSCRIPT italic_n italic_e italic_g end_POSTSUBSCRIPT ) } ∈ caligraphic_X;

3 randomly sample a subset of users & items with ratio α 𝛼\alpha italic_α;

4 replace initial embeddings of masked u/v 𝑢 𝑣 u/v italic_u / italic_v with [M⁢A⁢S⁢K]delimited-[]𝑀 𝐴 𝑆 𝐾[MASK][ italic_M italic_A italic_S italic_K ];

5 inference collaborative-side representation e u/v subscript e 𝑢 𝑣\textbf{e}_{u/v}e start_POSTSUBSCRIPT italic_u / italic_v end_POSTSUBSCRIPT with ℛ ℛ\mathcal{R}caligraphic_R;

6 calculate model optimization objective ℒ ℛ subscript ℒ ℛ\mathcal{L}_{\mathcal{R}}caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT based on ℬ ℬ\mathcal{B}caligraphic_B;

7 calculate L i⁢n⁢f⁢o subscript 𝐿 𝑖 𝑛 𝑓 𝑜 L_{info}italic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT w.r.t. Eq([17](https://arxiv.org/html/2310.15950v5#S3.E17 "In 3.3.2. Generative Alignment ‣ 3.3. Density Ratio Modeling for Mutual Information Maximization ‣ 3. Methodology ‣ Representation Learning with Large Language Models for Recommendation")&[18](https://arxiv.org/html/2310.15950v5#S3.E18 "In 3.4. Model-agnostic Learning ‣ 3. Methodology ‣ Representation Learning with Large Language Models for Recommendation")) for masked u/v 𝑢 𝑣 u/v italic_u / italic_v in ℬ ℬ\mathcal{B}caligraphic_B;

8

ℒ=ℒ ℛ+ℒ i⁢n⁢f⁢o ℒ subscript ℒ ℛ subscript ℒ 𝑖 𝑛 𝑓 𝑜\mathcal{L}=\mathcal{L}_{\mathcal{R}}+\mathcal{L}_{info}caligraphic_L = caligraphic_L start_POSTSUBSCRIPT caligraphic_R end_POSTSUBSCRIPT + caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT
;

9

𝚯←𝚯−η⁢∇𝚯 ℒ←𝚯 𝚯 𝜂 subscript∇𝚯 ℒ\mathbf{\Theta}\leftarrow\mathbf{\Theta}-\eta\nabla_{\mathbf{\Theta}}\mathcal{L}bold_Θ ← bold_Θ - italic_η ∇ start_POSTSUBSCRIPT bold_Θ end_POSTSUBSCRIPT caligraphic_L
;

10

11 until _convergence_;

Algorithm 2 Training Procedure in RLMRec-Gen

### A.1. Pseudocode of RLMRec

This section introduces the pseudocode for our model-agnostic RLMRec framework implementations, namely RLMRec-Con and RLMRec-Gen. The focus is on the training procedure of these implementations. Prior to training, user and item profiles are preprocessed, and their semantic embeddings s are generated using text models. Algorithm[1](https://arxiv.org/html/2310.15950v5#alg1 "In Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") presents the training procedure for RLMRec-Con, while Algorithm[2](https://arxiv.org/html/2310.15950v5#alg2 "In Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") outlines the process for RLMRec-Gen.

The difference between RLMRec-Con and RLMRec-Gen is that RLMRec-Gen randomly masks a portion of users/items before the base model encodes the CF-side relational representations. The objective function ℒ i⁢n⁢f⁢o subscript ℒ 𝑖 𝑛 𝑓 𝑜\mathcal{L}_{info}caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT for mutual information maximization is then computed based on the representations of the masked users and items. In contrast, RLMRec-Con models the density ratio in a contrastive manner and calculates the ℒ i⁢n⁢f⁢o subscript ℒ 𝑖 𝑛 𝑓 𝑜\mathcal{L}_{info}caligraphic_L start_POSTSUBSCRIPT italic_i italic_n italic_f italic_o end_POSTSUBSCRIPT objective for all users and items in the batch data, including both positives and negatives.

### A.2. Details of Profile Generation

In this section, we offer a comprehensive explanation of the generation process for both user and item profiles. Real examples from the Amazon-book dataset are used to illustrate this process, as depicted in Figure[7](https://arxiv.org/html/2310.15950v5#A1.F7 "Figure 7 ‣ A.2.2. Example of the Generated User Profile ‣ A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") and Figure[8](https://arxiv.org/html/2310.15950v5#A1.F8 "Figure 8 ‣ A.2.2. Example of the Generated User Profile ‣ A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation"). We adopt a general interaction paradigm with large language models (LLMs), where the system prompt serves as an instruction to guide the profile generation task. While the Amazon-book dataset is specifically showcased, the overall generation process remains consistent for the Yelp and Steam datasets as well, with minor differences in the instructions provided to represent the data information.

#### A.2.1. Example of the Generated Item Profile

Figure[7](https://arxiv.org/html/2310.15950v5#A1.F7 "Figure 7 ‣ A.2.2. Example of the Generated User Profile ‣ A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") showcases an example of item profile generation specifically for the Amazon-book dataset. The instruction prompt provided to the language models for all items remains consistent, directing them to summarize the types of books that would appeal to users, thus offering valuable information for recommendation purposes. The input information consists of the book’s title and original description from the dataset. To maintain consistency and facilitate parsing, we enforce the requirement that the output of the language models adhere to the JSON format. Furthermore, it is essential for the language models to provide their reasoning behind the generated profile, ensuring high-quality summarization while preventing any potential hallucinations. The generated results demonstrate that the language model, in this case ChatGPT, accurately captures from the book description that the book is likely to attract readers interested in mental health and women’s experiences.

#### A.2.2. Example of the Generated User Profile

Figure[8](https://arxiv.org/html/2310.15950v5#A1.F8 "Figure 8 ‣ A.2.2. Example of the Generated User Profile ‣ A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation") illustrates the process of generating user profiles using the Amazon-book dataset as an example. Our approach adopts an item-to-user generation paradigm, which allows us to leverage the previously generated profiles that describe the interaction preferences of items. To accomplish this, our prompt methodology incorporates not only users’ feedback information on items but also the profiles of the items themselves. By utilizing both sources of information comprehensively, large language models are empowered to capture users’ true preferences with enhanced accuracy. In the presented example, leveraging both the book descriptions and users’ review text, the language models distill the user’s preference for young adult fiction that seamlessly combines paranormal or supernatural elements.

![Image 7: Refer to caption](https://arxiv.org/html/2310.15950v5/x7.png)

Figure 7. Case study on item profile generation in Amazon-book data.

![Image 8: Refer to caption](https://arxiv.org/html/2310.15950v5/x8.png)

Figure 8. Case study on user profile generation in Amazon-book data.

![Image 9: Refer to caption](https://arxiv.org/html/2310.15950v5/x9.png)

Figure 9. Case study on LLMs-based reranking. The candidate items are retrieved by LightGCN.

### A.3. Analysis on the Reranking Task with LLMs

In this section, we explore real-world scenarios where LLMs are employed for reranking tasks on the Amazon-book dataset, as introduced in Section[1](https://arxiv.org/html/2310.15950v5#S1 "1. Introduction ‣ Representation Learning with Large Language Models for Recommendation"). Firstly, as depicted in Figure[9](https://arxiv.org/html/2310.15950v5#A1.F9 "Figure 9 ‣ A.2.2. Example of the Generated User Profile ‣ A.2. Details of Profile Generation ‣ Appendix A Supplementary Material ‣ Representation Learning with Large Language Models for Recommendation"), we carefully design the instruction and input prompt. We utilize the item ID as a key to index the items, along with their textual information such as the book title. This approach aims to enhance the accuracy of the generated outputs by avoiding direct generation of item titles that may not precisely match the original titles.

In the prompt, we include the historical purchased books of the user as a reference, along with a list of 40 candidate items retrieved by LightGCN. The goal is for the LLM to select the top 20 items for recommendation. However, upon examining the output results, it becomes apparent that the language models have recommended some non-existent items (highlighted in green) within the provided list. Despite utilizing item IDs for indexing, this occurrence is common in many reranking examples, and the presence of non-existent items can undoubtedly impact the overall reranking performance.

Additionally, the number of correctly recommended items from the language models is lower than the retrieved items (highlighted in red). This discrepancy is primarily attributed to the limited textual information available for the language models to effectively exploit users’ preferences. Moreover, the retrieved item list, learned by the state-of-the-art method LightGCN, benefits from collaborative information beyond just the textual content. This collaborative information contributes to the improved performance of the retrieval process compared to the language models’ recommendations.

Incorporating other raw textual information from the datasets to improve performance may have some anticipated limitations: i) The limitation of input token numbers may constrain the size of candidate items, as many raw descriptions can be excessively lengthy. ii) Raw descriptions may be missing or contain substantial noise in certain datasets. The absence of descriptions or the presence of noisy information can hinder the language models’ comprehension of users’ preferences. iii) Including a larger amount of input data, such as additional raw textual information, can increase the computational cost, which impacts the system’s scalability.