A Gentle Introduction to Bayesian Personalized Ranking from Implicit Feedback

The Concept of Learning to Rank

In collaborative filtering–based and content-based recommender systems, the model typically scores each user–item–context vector independently. The usual goal is to predict how much a user might like an item, often in the form of a rating.

Ranking, however, is a different problem. In ranking, the goal is not to predict the score of each item in isolation, but to recommend an ordered list of items to the user. Given a particular context, we assume that there is an underlying ordering of items that reflects the user’s preferences. The goal of learning to rank is to learn to estimate this ordering as accurately as possible.

This is precisely where standard collaborative filtering and content-based models can fall short. Since they score items independently, they do not directly optimize for the relative order of items in a list. A good learning-to-rank model, in contrast, learns to score items in a way that preserves the correct relative ordering of items.

In this article, we will explore a simple and intuitive ranking algorithm called Bayesian Personalized Ranking.

The first step in learning to rank is to construct an ordered set of preferences that reflects what a user likes. In practice, however, users do not explicitly tell us how they want items to be ranked. They do not provide a complete ranking of items for us to use as training data. Instead, we must infer these preferences indirectly from user behavior and other forms of implicit feedback. In the next section, we will see how this preference data can be constructed.

---

Constructing a User Preference List from Implicit Feedback

Let us assume a scenario in which we want to recommend a list of hotels to a user who intends to book a hotel through a website or app. When the user first visits the platform, they are presented with a search bar containing inputs such as location, number of adults, number of children, from_date, and to_date.

When the user clicks the search button, they are taken to the listing page, where they are shown an ordered list of hotels. On the listing page, each hotel is typically displayed with summary information such as its price, a thumbnail image, the property type, and its rating.

If the user finds a hotel interesting, they may click on it, which takes them to the detail page. On this page, the user can see more specific information about the hotel, such as its location, room types, whether breakfast is included, and the list of amenities.

If the user likes the hotel and feels that it matches their needs, they may proceed to the review page before booking. This page usually shows the user details and the stay details for confirmation before the final booking step.

Finally, if the user is satisfied, they complete the booking.

At any point during the session, the user may return to the listing page and click on or review multiple hotels before making a final decision. This sequence of actions is the implicit feedback we observe from the user.

Now the question is: how do we construct a user preference list from this activity data? The basic idea is fairly intuitive. We start by assigning relevance scores to different user actions. If a hotel is booked, we can reasonably say that the user strongly preferred it, so we may assign it a high relevance score, say 3. If the user reaches the review page for a hotel, we can infer that they liked it to some extent, so we may assign it a relevance score of 2. If the user merely clicks on a hotel, we can say that they found it somewhat interesting, so we may assign it a relevance score of 1. Finally, if a hotel was shown on the listing page but was never clicked, we may treat it as the least preferred option and assign it a relevance score of 0.

Once these relevance scores have been assigned, we can sort the hotels by relevance to obtain an ordered preference list for that user within that session. Repeating this process across users and sessions gives us the preference data needed for training a ranking model.

Before moving on, it is worth making one practical note. In real-world systems, a user session may span multiple days. For example, a user might begin searching for a hotel today, leave the platform, and return the next day to continue the same booking journey. Production systems and data pipelines should therefore be robust enough to group such events into a single relaxed session when they correspond to the same underlying booking intent.

---

Bayesian Personalized Ranking

Let $>_u$ denote the underlying preference structure of user $u$. The goal of BPR is to learn a model parameter vector $\theta$ such that the posterior probability of $\theta$ given $>_u$ is maximized. In simple terms, we want the model governed by $\theta$ to assign high probability to the true preference structure $>_u$ for each user.

Let $\pi(\theta \mid >_u)$ denote the posterior probability of $\theta$ given $>_u$. By Bayes’ rule, we have

$$\pi(\theta \mid >_u) \propto f(>_u \mid \theta) \cdot \pi(\theta)$$

Here, $f(>_u \mid \theta)$ is the likelihood of observing the preference structure $>_u$ under model parameters $\theta$, and $\pi(\theta)$ is the prior over the parameter vector.

We place a normal prior on $\theta$. Since $\theta$ is a vector, we assume that it follows a multivariate normal distribution:

$$\pi(\theta) \sim \operatorname{MVN}(0, \Sigma_n)$$

Here, $\Sigma_n$ is the variance-covariance matrix of $\theta$. For simplicity, we assume that the mean of $\theta$ is zero. Under this assumption, the prior density becomes

$$\pi(\theta) = \frac{1}{(2\pi)^{k/2}\sqrt{\det \Sigma_n}} \exp\left\{-\frac{1}{2}\theta^T \Sigma_n^{-1}\theta\right\}$$

where $k$ is the dimension of $\theta$. In principle, we are free to choose any prior we find appropriate. To simplify the calculations, let us assume that $\Sigma_n$ is a diagonal matrix of the form

$$\Sigma_n = \frac{1}{2\lambda}I,$$

where $\lambda > 0$ is a scalar and $I$ is the identity matrix. Substituting this into the previous expression gives

$$\pi(\theta) = \frac{1}{(2\pi)^{k/2}\sqrt{\det \Sigma_n}}\exp\left\{-\lambda \|\theta\|^2\right\}$$

Notice that the term $\lambda \|\theta\|^2$ in the exponent looks exactly like the familiar $L_2$ regularization term commonly used in machine learning.

Next, we turn to the likelihood function. The term $f(>_u \mid \theta)$ denotes the likelihood of observing the preference structure $>_u$ under model parameters $\theta$.

Now, $>_u$ denotes the preference structure of a single user. Since our goal is to model the preference structures of all users, the overall likelihood is obtained by taking the product of the individual likelihoods across users:

$$f(>_{u_1} \mid \theta) \cdot f(>_{u_2} \mid \theta) \cdots f(>_{u_n} \mid \theta)$$

Now, how do we model the likelihood $f(>_u \mid \theta)$? In practice, it is difficult to construct and train a model that takes an entire ranked list as input and directly outputs the probability of observing that ranking. To simplify the problem, we instead build a simpler model that scores each user-item-context vector independently, with the restriction that items with positive feedback should receive higher scores than negative or unknown items. If this condition is satisfied, then the predicted preference ordering will align with the true preference ordering. Based on this idea, BPR makes the following assumption:

$$f(>_{u_k} \mid \theta)\approx \prod_{(i,j)\in D_{u_k}} p(i >_{u_k} j \mid \theta),$$

where

$$D_{u_k} = \{(i,j)\mid i\in I_{u_k}^+, \, j\notin I_{u_k}^+\}$$

This assumption says that instead of modeling the full preference structure directly, we model the probability of each pairwise preference $p(i >_{u_k} j \mid \theta)$, which is the probability that user $u_k$ prefers item $i$ over item $j$. The overall likelihood $f(>_{u_k} \mid \theta)$ is then approximated by multiplying these pairwise probabilities across all relevant pairs. In other words, the core assumption of BPR is that the probability of observing a user’s true preference structure can be approximated by the product of the probabilities of all pairwise preferences derived from that structure.

Now let us look at $D_{u_k}$. For user $u_k$, the set $D_{u_k}$ is constructed by taking each positive item and pairing it with every item that is not positive. In our hotel recommendation example, this means that if a booked hotel is treated as positive, then it can be paired with hotels that were only reviewed, clicked, or merely shown on the listing page. Similarly, if a reviewed hotel is treated as positive relative to clicked or merely displayed hotels, then those pairs can also be included. Thus, $D_{u_k}$ is simply the set of all such preference pairs for user $u_k$.

Across all users, we combine these into a single training set:

$$D_s = \{(u,i,j)\mid i\in I_u^+ \wedge j\in I\setminus I_u^+\}.$$

That is, $D_s$ contains all triples $(u,i,j)$ such that user $u$ interacted positively with item $i$, while item $j$ was not positive for that user. Using this notation, the likelihood becomes

$$\prod_{(u,i,j)\in D_s} p(i >_u j \mid \theta).$$

A small practical note is worth mentioning here. In a hotel recommendation system, we usually define an ordered preference list for each user-session pair rather than for each user alone. A single user may have multiple sessions, and each session may correspond to a different travel intent. So in practice, $D_s$ is often constructed session-wise. We keep the notation $D_u$ only because it is standard in the BPR literature.

Next, let us discuss how to model the pairwise probability $p(i >_u j \mid \theta)$. A simple and effective approach is to first build a model that assigns a score to each user-item-context vector separately, and then use the difference of these scores to represent pairwise preference. In particular, $\hat r_{ui} - \hat r_{uj}$ measures the extent to which the model prefers item $i$ over item $j$, where $\hat{r}_{ui}$ and $\hat{r}_{uj}$ are the scores assigned to items $i$ and $j$, respectively. To convert this score difference into a probability between 0 and 1, we pass it through the sigmoid function:

$$p(i >_u j \mid \theta)=\sigma(\hat{r}_{ui}-\hat{r}_{uj}),$$

Now our likelihood becomes:

$$\prod_{(u,i,j)\in D_s}\sigma(\hat r_{ui}-\hat r_{uj}).$$

Finally, substituting both the likelihood and the prior into the posterior expression, we get

$$\pi(\theta \mid D_s) \propto \frac{1}{(2\pi)^{k/2}\sqrt{\det \Sigma_n}} \left( \prod_{(u,i,j) \in D_s} \sigma(\hat{r}_{ui}-\hat{r}_{uj}) \right) \exp\left\{-\lambda \|\theta\|^2\right\}$$

Since we are working under proportionality, we can drop all terms that do not depend on $\theta$. Therefore, this simplifies to

$$\pi(\theta \mid D_s) \propto \left( \prod_{(u,i,j)\in D_s} \sigma(\hat{r}_{ui}-\hat{r}_{uj}) \right) \exp\left\{-\lambda \|\theta\|^2\right\}.$$

From the posterior distribution of $\theta$, we obtain the maximum a posteriori (MAP) estimate by finding the value of $\theta$ that maximizes the posterior. Since the posterior is proportional to

$$\left( \prod_{(u,i,j)\in D_s} \sigma(\hat{r}_{ui}-\hat{r}_{uj}) \right)\exp\{-\lambda \|\theta\|^2\},$$

it is enough to maximize this right-hand side expression with respect to $\theta$.

We can instead maximize the logarithm of the above expression to make the calculations simpler:

$$\begin{align*} L &= \log \left[ \left( \prod_{(u,i,j)\in D_s} \sigma( \hat{r}_{ui} - \hat{r}_{uj} ) \right) \exp\{-\lambda \|\theta\|^2\} \right] \\[8pt] &= \log \left[ \left( \prod_{(u,i,j)\in D_s} \sigma( \hat{r}_{ui} - \hat{r}_{uj} ) \right) \right] + \log \left[ \exp\{-\lambda \|\theta\|^2\} \right] \\[8pt] &= \sum_{(u,i,j)\in D_s} \log \sigma( \hat{r}_{ui} - \hat{r}_{uj} ) - \lambda \|\theta\|^2 \end{align*}$$

The gradient of this objective with respect to $\theta$ can now be computed as follows:

$$\begin{aligned} \nabla_\theta L &= \nabla_\theta \left[ \sum_{(u,i,j)\in D_s} \log \sigma(\hat r_{ui}-\hat r_{uj}) - \lambda |\theta|^2 \right] \\[6pt] &= \sum_{(u,i,j)\in D_s} \nabla_\theta \log \sigma(\hat r_{ui}-\hat r_{uj}) - \lambda \nabla_\theta |\theta|^2 \\[6pt] &= \sum_{(u,i,j)\in D_s} \frac{1}{\sigma(\hat r_{ui}-\hat r_{uj})} \sigma'(\hat r_{ui}-\hat r_{uj}) \nabla_\theta (\hat r_{ui}-\hat r_{uj}) - 2\lambda \theta \\[6pt] &= \sum_{(u,i,j)\in D_s} \frac{1}{\sigma(\hat r_{ui}-\hat r_{uj})} \sigma(\hat r_{ui}-\hat r_{uj})\bigl(1-\sigma(\hat r_{ui}-\hat r_{uj})\bigr) \nabla_\theta (\hat r_{ui}-\hat r_{uj}) - 2\lambda \theta \\[6pt] &= \sum_{(u,i,j)\in D_s} \bigl(1-\sigma(\hat r_{ui}-\hat r_{uj})\bigr) \nabla_\theta (\hat r_{ui}-\hat r_{uj}) - 2\lambda \theta \end{aligned}$$

Now we can estimate $\theta$ using gradient descent. Starting from some initial value $\theta_0$, we repeatedly update

$$\theta_{k+1} = \theta_k - \alpha \nabla_\theta L,$$

for $k = 0,1,2,\ldots$, until convergence, where $\alpha > 0$ is the learning rate.

Note that in practice, we do not iterate through all pairs in $D_s$ at every update step. Instead, we sample pairs randomly and update the parameters using those sampled pairs.

---

Conclusion

I hope you enjoyed this article on BPR. We began by discussing the motivation behind learning to rank and why it differs from traditional scoring-based recommendation approaches. We then looked at how a training dataset can be constructed from implicit feedback in a hotel recommendation setting, and finally explored the BPR algorithm in depth.