J.D. Power tablet numbers don’t add up

I saw tonight that J.D. Power gave the 2013 U.S. Tablet Satisfaction Award to Samsung. I thought, well it’s been a good run for Apple, but you can’t always win. Then I looked at the ratings and did a double-take.

Apple scored 5 out of 5 in every category, except price, where it scored 2. Samsung had threes and fours, but still managed to get a 5 in Overall Satisfaction. That doesn’t make sense to me.

jdpower

Here is how J.D. says it rates the categories:

The study measures satisfaction across five key factors (in order of importance): performance (26%); ease of operation (22%); styling and design (19%); features (17%); and cost (16%).

So cost is the lowest percentage of importance. Apple scored higher in every category, except price which is ranked the lowest by percentage, but yet Samsung wins?

Forum poster Ilucide at The Verge brings up a great point in his post pictured below.

Screen Shot 2013-10-31 at 4.51.44 PM

I think J.D. Power has some explaining to do.



  • HankFromage

    From the JD Powers report: “Overall customer satisfaction with tablet devices is 821 (on a 1,000-point scale) … Samsung ranks highest with a score of 835 … Apple ranks second scoring 833.”

    How that works with the individual category ratings I’m not sure.

  • Techpm

    Well, Samsung isn’t just cheating on benchmarks..

    • rattyuk

      They are giving out money left right and center. JD Power the new Consumer Reports; Apple scores highest on all our metrics but we cannot recommend buying the device… Morons.

  • Shameer Mulji

    Either JD Power can’t do math or someone’s being a handsome sum underneath the table.

    • rattyuk

      This.

      Samsung’s marketing budget has to go somewhere. (15 billion bucks and counting…)

  • D

    Think of it a little differently as each category having a value of 0 to 1000 and each dot representing a bin of 200. A score of 500 would equal 3 dots (400 – 600). Multiple each category score times its weight factor and add them up. 5 dots would then mean they scored between 800 and 1000.

    • alextheukrainian

      That’s not how it works. See my comment below in a minute.

      • hoggleboggle

        exactly! the circles or dots have no numerical value

    • Billy Razzle

      Who in the fuck would ever score anything like that?

  • Stephen Washburn

    Also, no matter which column you sort by, Apple is never above Samsung. Even when Apple or other tablets were rated higher, Samsung stays locked to the top of the table.

    • Keith

      Samsung stays at the top even when you click on the category header in order to reverse the sort direction.

  • Saulk Pupét

    They now have this at the bottom of their data site: “Please note that jdpower.com Ratings may not include all information used to determine J.D. Power & Associates awards.”

    • LTMP

      ie: Payments from a certain company.

  • tylernol

    Scamsung.

    • Billy Razzle

      Scamscum

  • tylernol

    also, where are the Nexus 7 tablets in this? Are they classified as Asus(since Asus makes them I believe)?

  • Saulk Pupét

    Actually, this is worse than I thought. The press release page at http://www.jdpower.com/content/press-release/Czb4Aa9/-2013-u-s-tablet-satisfaction-study.htm says “The study measures satisfaction across five key factors (in order of importance): performance (26%); ease of operation (22%); styling and design (19%); features (17%); and cost (16%)”

    But these percentages are clearly not used anywhere: there’s no way Samsung can get an “Overall Satisfaction” 5-star rating on http://www.jdpower.com/consumer-ratings/telecom/ratings/919201690/2013-Tablet+Satisfaction+Study/index.htm —Samsung doesn’t score five stars in any category, so mathematically ANY weighted average can’t result in five stars, no matter what the weights are.

    And the percentages are clearly not used for the “Award Recipient,” because if they were Apple would crush Samsung, since Apple is ahead in every category except price, which is the lightest weight.

    So, the question is to JD Powers is, why do they list those percentages that they didn’t actually use? If you’re an information company, isn’t your job to, like provide RELEVANT information?

  • The Gnome

    So much for J.D. holding any weight… I smell a Shamesung payoff here. Has J.D. gone the way of Consumer Reports? Looks like it…

  • mccldwll

    So cat food beats a Ruth’s Chris filet for diner satisfaction–if you consider cost?

  • alextheukrainian

    Ok, guys, here’s how this works. The stars don’t represent any numerical value whatsoever, they only represent relative relationships. As “D” mentioned below, there are 5 bins. So let’s say Apple scores 180 in performance, and samsung scores 175, and the other 3 score 172, 174, and 177. Then, whoever scored 172 will have 1 star, 174 – 2 stars, 175 – 3 stars, 177 – 4 stars, and 180 – 5 stars. BUT the actual difference is only 5 points out of 200 between Apple and Samsung. The stars literally mean what the say – are you the lowest scorer (eve if just by a tidbit), below average, average, above average, or the best. So, technically 2 tablets could get 4 stars across the board because they’d be both between “average” and “best”, but since the actual underlying values would be slightly different, one of them would win. Whoever wins gets 5 stars. Next up gets 4 stars. Etc.

    So you don’t average out stars or give them weight. The weight only applies to actual scores, which J/D doesn’t provide (I haven’t seen them anyway). J/D only provides the TOTAL score, so scores from all 5 bins added up – and that’s how we get to 800-something or w/e score.

    • Varun Pramanik

      I see your reasoning, but this is simply not the case. The dots aren’t a 1st/2nd/3rd/4th/5th ranking. In several of the categories, companies tie. Look, for instance, at cost: three have four dots, one has five and one has two. If what you say is true, that would imply that Amazon’s in first place; Samsung, Acer and Asus are in second place; and Apple is in fourth place. That simply cannot be the case in a 1st/2nd/3rd/4th/5th ranking method.

      • rattyuk

        The cost of everything…The value of nothing.

        Surely those five stars in every other category count for something…

        Oh wait, they don’t.

        • RoC1909

          “Waaaaaaa! My precious Apple didn’t win the award this year! Waaaa!” LOL!

          • Slurpy2k12

            You must have a really shitty life, to be such a shitty troll. How sad.

          • Sigivald

            Apple losing because people reported not liking the iPad would be fair.

            Apple losing despite winning in 4/5 of the categories – and losing on the lowest-weighted one, that raises questions.

            (When you’re only winning on price, and getting 3/5 stars on usability and performance vs 5/5 on the competition, well… winning the overall doesn’t make a lot of sense.

            I could understand an Apple/Samsung tie, even, on the idea that “if you’re cheap, go Samsung; if you want the best everything and aren’t cheap, go Apple”.

            That makes sense and fits the data and reflects consumer valuations as expressed.

            But the results from JDP make no sense the way JDP’s trying to explain them.)

      • hoggleboggle

        you are missing the point. To stay with the figures used by Alex scoring 180 points warrants 5 circles, of which there was one company, scoring 175 scores you three circles, of which there were 3 companies and scoring 174 points scores you 2 circles which applies to 1 company. This doesn’t mean that a 5 circle company has 2.5x the score of a 2 circle company.

    • lkalliance

      Used to get the same kinds of arguments about the IMDb Top 250, lol. Nice explanation: I think it’s much more likely that the dots are an oversimplification as you’ve described and that JD Power isn’t doing anything untoward, than payouts or incompetence. However, that doesn’t fit a sexy narrative. ;)

      The way they chose to represent it, though, you’re right, is potentially very confusing.

  • Blinx182

    Explanation? Tim Crook doesn’t pay enough.

  • http://deoclicianocgiportfolio.wordpress.com/ ochyming

    This mirrors the mainstream news organization bias toward Samsung, or Samsung is being used to hurt Apple.

    BBC has been doing this, only with more finesse.

    • rattyuk

      More finesse? BBC?

      I just threw up a little in my mouth.

      The BBC hate Apple with a vengeance.

  • Keith

    If you go to the chart that shows the companies and the categories and you click the category headers to sort by that category, you’ll see Samsung sort to the top every time, even though they clearly have fewer dots. So it seems that the underlying data does not match the dot count.

    • Keith

      Oh, Steven already said that…

  • lucascott

    My cousin posted about this earlier. I’m still scratching my head

  • Tom_P

    What the world has come to? Sigh. All greedy and immoral organizations. Really, what I found unbelievable is it took Samsung this long to pay for this price.

  • Bgrant

    Apple beat Samcopy in all categories, except one. Apple got a perfect score in those categories. Those categories are to do with the experience of the device. I’m not even sure how Price factors into this. Only people who believe in fairies wish someone could make a better quality device for a lower price.

    Something pointed out by a MacRumors reader – Asus was only 1.44% behind Apple in the survey, which really doesn’t add up, either.

    JD Powers is owned by a publishing company and has been noted on many occasions, Apple isn’t spending enough on advertising to get favourable reviews by people in the business of selling advertising.

    • RoC1909

      Awwwww, ain’t that original….Samcopy. I bet it took you all night to determine something “witty” to write, and then when the time comes, you come up with that?

      crApple fans….the most pathetic of the bunch!

  • WeaponZero

    I will explain this in as simple manner as I can. Your making the mistake of basing your score on circles(They are not stars but circles, explanation of what circles are is explained on that page link on the bottom). When in reality what is most important is not the circle average but the point average.

    The circles are confusing because you have 6 manufacturers. Circles make a lot better sense when you have way more than 6. At 6 it actually confuses people.

    To explain it best, look at overall Apple at 833 has 5 circles overall. Amazon which has 826 has 3 stars As you can see the difference between the two is 7 points which is less than 1% and yet there is a 2 circle difference.

    Effectively what is going on is this, despite Samsung earning 3-4 circles in other categories and Apple earning 5 circles, the actual point value is most likely very close. To the point where the 2 circles for cost tipped the balance despite cost being the least weighted.

    • RoC1909

      Thank you, Weapon. Finally, someone who has some common sense.

      • mikhailovitch

        Possibly. But no ability to write a comprehensible argument.

  • RoC1909

    LOL @ the butt hurt iFans! I guess Apple didn’t pay J.D. Power enough. /s

    • mikhailovitch

      So, logically, you think that Samsung is, then.

    • mikhailovitch

      Paying them enough, that is.

  • isitjustme

    Reputation is hard-earned and did JD Powers just sacrificed it for a few dollars?

    Is it that desperate?

  • hoggleboggle

    And another person completely missing interpreting the use of the circles. They have no numeral value, they are a graphical representation. They could have just as likely used the values A-E. Without seeing the actual numbers that JD Power have used it is completely pointless to find fault in their calculations. Your graph above is completely pointless.

    • Jemal Cole

      Where by “your graph” you mean either J.D. Powers’ graph or “some guy on the Verge Forums”‘s graph?

  • Space Gorilla

    I noticed this as well. There is no reasonable way to make the numbers work so Samsung wins. Apple is in fact the top rated tablet, if you believe in math.

  • Moeskido

    I have to wonder where all the new baby troll commenters here are coming from. Techpinions? TechCrunch? Outpatient clinic?

  • wubwubwub

    http://www.jdpower.com/about/power-circle-ratings.htm

    The way the stars work is that ONLY the top rated company in each category will get 5 stars. So if Samsung lost to Apple by 1 point in Features and Styling & Design, they would get a 4 and Apple would get a 5.

    This means the Verge scoring may grossly over exaggerate the differences between Samsung and Apple. In fact using the star rankings as points completely misunderstands how JD Power ranks products.

    • Sigivald

      To be fair, it says that “the top 10%” in competitive categories can get five stars…

      I’m sure you’re right in general about the underlying numbers working out, though.

      (Be nice if JDP actually released them – to improve confidence in the JDP brand.

      Because accurate and fair numbers that look unfair at first glance are bad for the JDP brand, compared to explaining why they’re accurate and fair.

      I am perfectly willing to believe that Samsung customers are .25% more “satisfied” than Apple ones, mainly on the grounds of being much more cost-sensitive, which is how I read the data.)

  • Mike

    I took that survey. I was a 3 on price. Weird.