Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Programming Science

Recommendation Algorithm Wants To Show You Something New 90

Posted by ScuttleMonkey from the try-it-you'll-like-it dept.
Several sources are reporting on a new metric that computer scientists are going after with respect to recommender systems — recommendation diversity. "In a paper that will be released by PNAS, a group of scientists are pushing the limits of recommendation systems, creating new algorithms that will make more tangential recommendations to users, which can help expand their interests, which will increase the longevity and utility of the recommendation system itself. Accuracy has long been the most prized measurement in recommending content, like movies, links, or music. However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer."
This discussion has been archived. No new comments can be posted.

Recommendation Algorithm Wants To Show You Something New More

Recommendation Algorithm Wants To Show You Something New

Comments Filter:

  • That's called an "contextual ad engine". (Score:4, Insightful)

    by Animats ( 122034 ) on Friday February 26, 2010 @03:25PM (#31290078) Homepage

    creating new algorithms that will make more tangential recommendations to users, which can help expand their interests,

    The advertising industry already has that technology. Their idea of "expand interests" usually involves shopping, of course.

    • Re: (Score:3, Insightful)

      by blahplusplus ( 757119 )

      The real issue is people's finite attention, I notice even with recommendation systems on amazon.com there is no way I could ever read everything they recommended to me and still have a life. It may be neat for movies but even then I'm sure there will be a list of recommended stuff that you simply can't get aroudn to.

      I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time, a lot of stuff is over-produ

      • Re: (Score:1, Offtopic)

        by wjousts ( 1529427 )

        The real issue is people's finite attention, I notice even with recommendation systems on amazon.com there is no way I could ever read everything they recommended to me and still have a life.

        I think you're missing the point. The point is to try and sell you stuff. You're not obligated to read (or even buy) something just because Amazon recommended it.

      • I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time

        Now there's a useless metric. Stuff is produced to satisfy the spectrum of interests. And the "average person" (who does not exist) would consume some small percentage of everything.

      • Re:That's called an "contextual ad engine". (Score:5, Insightful)

        by Kjella ( 173770 ) on Friday February 26, 2010 @04:10PM (#31290650) Homepage

        I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time, a lot of stuff is over-produced and it would be good if someone was out there modelling how many products you could possibly want to experience over a yearly period.

        But that makes good recommendations more important, not less. If you go into a library it's highly unlikely you'll be able to read every book in it, but does that matter? You just want to read the good books about things that interest you. If Spotify was on full shuffle and you could get everything from death metal to yodeling in the next song, you wouldn't want it - you'd go back to your own favorites. On the other hand, if everything is interpolated you only get more and more of the same. People don't work like that, you may have your favorite food but it's not something you want to perfect and have every day.

        A good search helper should be something in between - keeping to things you're reasonably likely to like but on the other hand challenge you a little to explore and listen to things a little outside your normal repertoire. Yes of course I realize the marketing potential here in sending the masses to their new hit wonder but I don't think the concept is that unreasonable. Think about how your friends are influencing your music taste, they're not interpolating they're gently pulling in the direction they like. If they hit the right mix this would be a real asset because you go to that site because of the good recommendations and that's not such an easy thing to copy.

        • Re: (Score:3, Interesting)

          by Skreems ( 598317 )
          Last.fm used to have a setting that was basically like this... they gave you an "obscurity" slider which let you set a preference for stuff that was a good match (via the usual "people who like x also like" algo) but removed the stuff that was over a certain threshold of popularity. I found a ton of bands that I really like this way, but at some point they took away the feature, or hid it in some way.

          Long story short, I have found systems like this very useful, and hope to see more of them.

        • Re: (Score:2)

          by tool462 ( 677306 )

          I kind of get this service by listening to XM. There are enough different types of stations, each fairly tightly focused, that I effectively tell it my preferences by which stations I program into my favorites. But then each station is run by DJs who are playing what they like within that format. I have found a TON of new music this way. I'd guess that, outside of my old "standards" (hs/college music) most of the new music I've purchased has been by bands I wouldn't have heard of if I didn't hear them o

          • DJ's are great. I stream The Current [publicradio.org] (89.3 in Mpls/St Paul, streaming link here [publicradio.org]) all of the time because they are a real independent radio station with DJ's who are free to play what they like and no corporately mandated playlist. Sure, there are artists/songs liked by many DJs that are heard frequently but the amount of stuff they pull out of their music collection is astounding. The benefit of having real DJ's with real opinions is that you can find ones who you like better than others and they will do
            • I think you'd like WVBR then... http://wvbr.com/ [wvbr.com] (93.5 in the Ithaca NY area, it's a Nonprofit owned and run mostly by Cornell students but it isn't directly affiliated with the University).
              • And in case it wasn't obvious from the fact that I've recommended them, they play some kickass tunes too :)
              • I'll check it out.

                What I like about the current though is that it decidedly does not have that college radio vibe. It is a straight up professionally managed station (part of the MPR network) with lots of contact with artists for things like in-studio performances and concert engagements and a staff that has spent a long long time in the industry. One of the aforementioned DJ's used to DJ on a college station and I like his sets a lot better in the past 5 years since the current started and the other on

                • Well WVBR gets some of that 'college radio vibe'. Mostly the good parts like the informal atmosphere. They play plenty of current music, as well as classic rock. They also play lots of concerts (though these are more or less all by local bands, plenty of good music just don't expect to hear anyone particularly famous).
                • My own favorite is http://www.kcrw.com/ [kcrw.com] because of their live performances. Some great live performances in their studio in their Morning Becomes Eclectic archives
                  • I'm a regular listener of Radio Paradise http://www.radioparadise.com/ [radioparadise.com], largely because they'll often put songs after each other that sound great in sequence, but are of completely different genres. Independently owned, not what I'd call a college vibe.

        • If you go into a library it's highly unlikely you'll be able to read every book in it, but does that matter? You just want to read the good books about things that interest you.

          I think you could define the ideal recommendation system by adding just one word to your definition above:

          You just want to read the good books about things that will interest you.

        • "But that makes good recommendations more important, not less"

          I disagree, I think most recommendation systems today are "good enough". Think about the time things are released today, most people today go for new stuff and merely want to keep track of recently released stuff, so it's not so much knowing exactly rather then getting X into the awareness of others and there are lots of avenues besides recommendation engines for this.

          I mean lots of interesting things are often times recommended even NOW with cu

      • Re: (Score:3, Interesting)

        by martin-boundary ( 547041 )
        I like to use the [ and ] keys in mplayer to speed up time when watching movies. It's not difficult to keep up with the plot and dialog up to about 1.5x the speed, and this makes the typical 2 hour movie run in 1 hour 20 min, or the typical 40 min series run in 26 min. I'll speed up time more when the movie is crap, and less if I want to really enjoy everything.

        The point is you have more than the binary watch/don't watch choice available to you, if you're willing to assert control over your own time in cr

        • Re: (Score:1)

          by ooshna ( 1654125 )
          If I could find a video player that allowed me to speed up the film say 1.5x but still keep the sound the same pitch I would be very happy

        • Re: (Score:3, Insightful)

          by Sir_Lewk ( 967686 )

          Dude, don't forget stop and smell the roses. Not everything in life is supposed to be a race.

          • Re: (Score:2)

            by bertok ( 226922 )

            Dude, don't forget stop and smell the roses. Not everything in life is supposed to be a race.

            That's almost as bad as my parents. They'll miss the first 5 minutes of the movie, then sit down a little bit, stand up and go get themselves a drink, sit back down, stand up and go to the bathroom or whatever, and so on. They'll end up watching maybe 20% of a movie is small random pieces. I'm not just talking about over-the-air broadcast, but even DVDs, which they could pause, but don't.

            Then the questions start: "who's that guy?"... "why are they shooting at each other?"...

            I just don't show them movies any

            • Mine are like that - I watch movies with them I've already seen. They enjoy the movie - yes, I'd think it was ruined to watch a movie that way but they enjoy it. And I enjoy the time with them, since I'm not paying as much attention to the movie and can answer "Who's that guy? What'd I miss?" without missing the movie.Apparently they re-watch the ones they really like when I'm not around, and those times the movie isn't background to my visit - they settle down more, or pause/rewind the movie. When I have

        • This works great for the olympics. I speed up the between race silliness then drop down to normal speed for the race/event. :)
          But for tv shows I just use } once or twice which is 1.1 or 1.2 times normal speed.
          (and as mentione dbelow you need the af = scaletempo setting in your ~/.mplayer.config so that sounds don't get pitch-distorted at the different speeds.

    • Yep... I even use that occasionally on my little site. Come to me with Windows, and you might see an ad for VMWare's server products. Come to me with a Mac and might see an ad for VMWare Fusion. It's all in reading the user agent.

    • Re: (Score:3, Funny)

      by megamerican ( 1073936 )

      Amazon's recommendations have become so accurate lately that it typically asks me if I'm interested in buying things I've already purchased from them.

      • I know you got the funny mod, but its true, and needs attention.

        I own a Canon DSLR, I've bought a battery charger for the camera from Amazon, and it keeps recommending me more batteries, that's fine. It recommends me new lenses, and filters, that's fine. I add some of the filters and lensea to my "Wish List" and now it wants to sell me a Nikon. That's not Ok.

        I'd like it not to offer me competing/slightly different goods for goods I already own, I have Dragon Age:Origins, Mass Effect, Knights of the Old Repu

    • I'm not so sure - in the case of traditional advertising you get certain groups intentionally targeting users that match a given criteria such as views of a certain type of website or TV program. Even if web advertising then adjusts based on the effectiveness of links between given groups and given adverts, that's still fundamentally driven by a manually selected connection.

      In the case of the system described in the article, to match you with similar uses and then apply a degree of randomness to provide mo

  • How long before it has enough data to recommend we should be destroyed and acts on it?

    • Re: (Score:3, Funny)

      by WrongSizeGlass ( 838941 )

      How long before it has enough data to recommend we should be destroyed and acts on it?

      According to my calculations, about six days. On the seventh day it will rest.

  • So, first we start out with one TV in the house and mass appeal programs. Then, as we get more and more channels, each user watched a specific channel targeted to their demographics. Then we got more specific programs from podcasts, and recommendation systems told us what we'd like before we knew it existed. The problem was, then the content makers didn't know how to sell such small audiences, so we're going to have to muck up the recommendations systems to suit them... sure, good luck with that.

  • Even small percentage increases in per-order purchases can result in huge gains across the board. Netflix, with a comparatively paltry prize amount, has bought themselves an incredibly efficient revenue generating piece of software.

    I'm surprised to see that it still relies on popularity ranking as a cornerstone of the algorithm, but the other areas, especially heat diffusion and random walk are very cool and I'd love to read more about it.

  • Books. I am an avid reader of sci fi and fantasy, and man, most recommendations out there just BLOW.

    • well to be honest... that's because most sci fi and fantasy books blow.

      No, really, I say this as a fan of the genres.

      • Exactly. That's why I'm saying that. I dunno how many books I've started to read then just put down because they are just brutal.

        Of course, one mans junk is another man's gem so whatever. :-)

    • Re: (Score:2)

      by mcgrew ( 92797 ) *

      I've found that most recommendations blow, period. Like movies; if the reviewers pan it, I'm almost certain to like it. The exception, it seems, is recommendations from slasdot commenters; I discovered Cory Doctorow and Terry Pratchett here.

      • Re: (Score:1)

        by maxume ( 22995 )

        Doctorow is awful.

        For movies, the trick is not to read 'the reviews', but to pick a critic and learn his style; then, you get an idea from his review if you will like the movie or not (bonus points for the critic if their likes line up with yours).

        • Re: (Score:2)

          by mcgrew ( 92797 ) *

          Doctorow is awful.

          I'd read some of his magazine articles and had the same opinion, but his books are engrossing; of course, everyone's tastes are different.

    • Re: (Score:1)

      by thms ( 1339227 )

      Yes, these algorithms can finally release you from having to rely on the dictatorship of the masses wrt. recommendations, or investing too much time building your own favourite recommendations-authors.

      Previously you selected a smaller supgroup from the general mass, e.g. move to only reading what the NYT recommended, but you were still in a rather big herd. Now you can pretty much "build your own crowd" as you read and rate what you liked/disliked.

      I wonder if Netflix is really the only company which run t

  • We've been both promised personal recommendations and been threathened with personalized advertisement, yet I hardly ever see any of it.

    Take Youtube I thought there was something fancy behind it but now that it displays _why_ it's recommending a clip, you can tell that it's extremely simple. Being a practictioner in machine learning and AI myself, I must confess that most industry implementations in our field is 10% very simple stuff, with 90% boring database and infrastructure code around it.

    No news websi

    • That's because they are probably lying to you.

      The example I'll use is google but it applies to most companies. Whenever they come out with a press release saying that they're now collecting this or that information it is only to serve you targeted ads, yet every ad I've ever seen while logged in to Google is directly related to my search terms or the e-mail I'm currently reading.

      Here is an article from 2006 [slashdot.org] that states that Google is going to listen in to your microphone and webcam to serve ads. Where are t

      • Re: (Score:1)

        by Kanel ( 1105463 )

        Actually, I think a lot of companies are collecting a lot of data because people _think_ that it will be useful in the future. If the data mining gurus says that a database of consumer behaviour is worth billions, then that's something a company can list as their assets. Drives up the perceived value of the company and the stock price.

  • Tricky Business (Score:3, Interesting)

    by miasmic ( 669645 ) on Friday February 26, 2010 @03:42PM (#31290284)

    Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.

    That, or it suggests really popular things, for example with music always getting a string of well known, popular bands and artists like Radiohead or Pink Floyd suggested as bands I might like - because many people who like similar sorts of music to me like Radiohead, the algorithm thinks I would like Radiohead too - they can't seem to figure that I would already know if I liked Radiohead or not at this point. I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...

    • I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...

      don't buy anything. the first site that correctly recommends something you haven't heard of, but also really like, but it. that is how the invisible hand works.

      • pffffft. BUY it.

      • Re: (Score:2)

        by mdm42 ( 244204 )
        Yup. Did that. Only problem was that the band that caught my attention is on a label that has never been heard of by stores where I live, so the only realistic way to get hold of their music was... BitTorrent. S'ok - the band puts all their releases on the 'net anyway, generally several weeks before the official release. As a direct consequence, I now own CD copies of every album they've ever done. (Well except the latest one which hasn't been officially released yet.)

        The invisible recommendation hand tha

    • Re: (Score:1)

      by Kanel ( 1105463 )

      Most algorithms in machine learning and data mining are either very simple or complicated but very generic.

      In your case the recommendations are based on a similarity metric not on the music itself but on those who like it. Really popular bands are useless for characterising a music lover since the group of people who like pink floyd will be so diverse. Because of that, it won't be able to map from pink floyd to users and back to bands similar to pink floyd :-(

      • Re: (Score:1)

        by miasmic ( 669645 )

        For sure, it's a crappy way to do it.

        The new ideas the article talks about with subsets are a great idea - if you could identify groups of users who might not have heard Pink Floyd, like teens or people who died before 1970, you could recommend it based on tastes among their peers just to them. Though I feel using such metrics is risky - leaving little room for non-conformism, or resting in peace for that matter.

    • Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.

      clearly you haven't used the wreckommender: http://wreckommender.com/ [wreckommender.com]

      • Re: (Score:1)

        by miasmic ( 669645 )
        I just gave it a go with every wish for it to be good, but it didn't seem to recognise my test artists (some mildly obscure drum & bass). I then tried "Metallica" to cover the opposite end of the spectrum, but got a 500 internal server error - an easter egg perhaps? Finally I tried US noise metal band "Lightning Bolt" but it returned an Enya song as a reccomendation. That's when I decided it wasn't built for my tastes! Nice design though.
  • This isn't really that new at all. I've seen several other groups doing something with diversity vs similarity in recommender systems.

  • "However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer.""

    Slashdot: "I see you've subscribed to certain opinions. Here are some more recommendations."

    • Re: (Score:1)

      by Kanel ( 1105463 )

      It's been a while since the viewpoints in Slashdot articles have challenged my opinions. A case of the chicken and egg question? :-o

  • I understand the problem; the direct connection criteria between two different things might be completely indecipherable or insurmountably complex and subtle (let alone indirect relationships i.e. six degrees of Kevin Bacon). That means whatever you build has to account for trends to narrow the band of complexity which leads to the same old problem of only suggesting status quo.

    A tool that can only suggest "obvious" or "random" things leads to undesirable results and at best can only fractionally provide yo

    • A system like this can work in two ways. Either the similarity measure computes a "distance" between the music by analyzing the sound and metadata, or it maps a band to a group of people that like it and then maps from this group back to other bands the group likes.

      If the system is employing the latter, we have a problem. If we select only popular bands or pick bands randomly, there's no hidden wisdom of the crowd for the algorithm to extract. We can't blame the software, only ourselves

    • Re: (Score:2)

      by plover ( 150551 ) *

      I just can't get past the idea that you either tell people what they want (advertising) or let people discover things on their own (interconnection).

      The best advertising is advertising misinterpreted as interconnection.

      Remember the kerfluffle a year or two ago with the web site twittering your friends with your purchases, or some kind of plug in that monitored your purchases? I don't remember exactly which site, nor do I care because the details aren't important, but that was one step away from the goal here. Imagine if all your friends posted all their purchases all the time, and you got tweeted with all those purchases, and all those tweets flowed t

  • I recommend getting out into the Big Blue Room and doing something real, tangible, and unique!

  • ...he always wants to show you something new, and it's always in the back of his van along with the puppies and candy.

  •       They should use IBM's new algorithm, it's faster than the old one.

  • One reason this kind of problem occurs is that many collaborative filtering algorithms are measured based on "root mean squared error", basically the square root of the mean of the differences between what was predicted and what the user actually did.

    The problem with this metric? It doesn't account for a variety of important things, one of which is that most users value diversity. Another is that in most recommendation systems, what is important is the relative relevance of recommendations to each-other

  • ...some other research groups with names that make me giggle like an idiot.

  • Recommendation engines, such as the ones used by Netflix and Amazon.com, already recommend really random choices. Except in the simplest cases (you bought a nail gun, you might want some nails), current recommendation engines stink at figuring out what I want. Trouble is, the recommendations are interesting so few times that I don't even look at them any more. I'll bet these researchers actually used to work for Amazon.com creating their recommendation engine, but got fired. Not knowing what else to do,
  • As someone who neither follows, nor particularly cares about this topic, did anyone have a lul or two when they noticed the acronym used in the first line of the quote? "PNAS"

Slashdot Top Deals

If all else fails, lower your standards.

Close