Explicit, Implicit, and Negative Feedback

Explicit feedback asks users to express a judgment. The people who answer are a small, motivated subset. In the LaunchCast study below, 64.85% of the participants analysed said their preference for a song affects whether they choose to rate it at all. Implicit feedback observes behavior at scale instead. But the action may reflect availability, interface friction, habit, price, social context, or incomplete attention.

Negative feedback is the least uniform of the three. One hide can mean dislike. Or fatigue with the repetition. Or that the content was sensitive, that the timing was wrong, that the user is trying to reshape the profile. Those five readings are not one label. A schema that stores them as one has discarded the difference before any model sees it.

Spotify's public session dataset does the opposite. It keeps a skip in four graded columns rather than a single flag. The rest of this lesson is about which of those two habits a team is in, and about what the published measurements say it costs to be in the wrong one.

Spotify's public session dataset covers 160 million listening sessions and metadata for about 3.7 million tracks. The 2019 release describes itself in one line: “The dataset consists of 160 million sessions with lengths varying between 10 to 20 interactions.” What matters for a feedback schema is its Table 1. A skip is not stored as one binary. It is four graded fields: skip_1, “whether the track was only played very briefly”; skip_2, “whether the track was only played briefly”; skip_3, “whether most of the track was played”; and not_skipped, “whether the track was played in its entirety”. Beside them sit context_switch, no_pause/short_pause/long_pause, num_seekfwd and num_seekbk. The completions stay “relatively constant at a non-skip rate between 34% and 35%”.

That schema is vindicated by what a later group found inside it. Meggetto and colleagues clustered session-level skipping on the Spotify data in 2021, and the clusters did not come out as one behaviour: “Experimental results on the real-world music streaming dataset (Spotify) indicate four main types of session skipping behaviour.” The four types differ by weekday versus weekend, by time of day and by playlist type. They are not an artefact of session length: “A subsequent analysis of short, medium, and long listening sessions demonstrate that these session skipping types are consistent across sessions of varying length.” The premise of the work is the one this whole lesson rests on: “such implicit feedback signal can be considered a measure of users' satisfaction (dissatisfaction or lack of interest)”. Four types is the count a team collapses when it writes one negative column.

The explicit control is a piece of interface, not a window onto taste, and there is a published measurement of that too. Netflix announced on 16 March 2017 that five-star ratings were going, replaced by a thumbs up/down control. Its “Goodbye Stars, Hello Thumbs” post followed on 5 April 2017, and the change rolled out worldwide over the following weeks. The reason given: “Netflix has had star ratings for much of our history, but we've learned through over a year of testing that while we've used stars to help you personalize your suggestions, many of our members are confused about what they do.” Then the effect, from testing with hundreds of thousands of members during 2016: “when we tested replacing stars with thumbs we saw an astounding 200% increase in ratings activity.” The same population. The same taste. A different widget.

Abundance and meaning pull against each other along this row. The high-volume implicit signal arrives constantly and says little about satisfaction. The low-volume explicit signal is closer to an articulated preference, and it comes from the small share of people willing to give it — 64.85% of the LaunchCast participants told the researchers their opinion decides whether they speak at all. The operational outcome is what both are standing in for. It is also the slowest of the three to arrive.

The reason to keep the slow one is documented. A Bing bug once showed users very poor results, and two key organizational metrics improved: “distinct queries per user went up over 10%, and revenue per user went up over 30%!” Kohavi and colleagues at Microsoft reported that in 2012. Degraded algorithmic results force people to issue more queries and to click more on ads, so both proxies rise while the product gets worse. The authors' own test of whether such a metric can be a goal: “If they did, we would intentionally degrade quality to raise query share and revenue!” Their recommendation follows — “we recommend that distinct queries alone not be used as an OEC for search experiments” — and they say the same of revenue per user for search and ad experiments without other constraints. A proxy that can be improved by breaking the product is not standing in for the outcome. It is standing in front of it.

Read the table from the most deliberate signal a user gives to the least pleasant. A rating or preference choice is offered on purpose. A skip, a hide, a return or a complaint is a family of refusals, and the rows do not mean the same thing even when one column stores them all. Two of these rows are not assertions at all. They are production decisions with papers attached.

The click row. YouTube's ranker optimises expected watch time per impression rather than click-through rate, and Google published why in 2016: “Ranking by click-through rate often promotes deceptive videos that the user does not complete (“clickbait”) whereas watch time better captures engagement”. The mechanism is a weighting: “the positive (clicked) impressions are weighted by the observed watch time on the video. Negative (unclicked) impressions all receive unit weight.” The click was not deleted from the system. It was demoted to a carrier for a duration.

The dwell row. Article length predicts dwell time almost linearly, on desktop, on tablet and on mobile: “it is very clear that the length of the article has good linear correlation with the average dwell time across devices”. That measurement is Yahoo Labs, 2014. For the longest desktop items the relationship stops behaving: “for desktop when the article is longer than 1,000 words, the plot has big variance”. The warning is explicit: “we need to be careful of the bias of dwell time based user engagement measurements towards long length content stories”. Their reason for going past the click is YouTube's reason too: “users may have clicked on an item by mistake or because of link bait, but are truly not engaged with the content being presented.” The collaborative-filtering experiment in that paper ran on a three-month sample of a Yahoo property: 147,069 distinct users, 11,535 distinct items, 4,358,066 training events and 199,420 test events. The dwell row's caveat is not a thought experiment about a small panel.

Each entry in the dictionary starts as an observable event and ends as a claim validated against outcomes. In between, the team fixes what the event may mean, how long it persists, and how severe it is.

The published work supplies a model answer for two of those steps. Naming the observable event and separating severity is what Spotify's Table 1 does: skip_1 for a track “only played very briefly”, skip_3 for one where “most of the track was played”, not_skipped for a full play. Pause, seek and context-switch fields are recorded next to them rather than folded in. Listing plausible meanings is what the 2021 skipping study forces, because one skip column holds four main types of session skipping behaviour that vary with weekday versus weekend, time of day and playlist type. And validation against outcomes is what the Bing bug enforces: distinct queries per user rose over 10% while the results shown to users got worse. A signal that moves is not yet a signal that predicts the result the product intended.