How we scored the Apple Watch

Don't agree with our verdict? Here's the inside story of the Wareable Watch debate
How we scored the Apple Watch

Wearable tech, even more so than smartphones, probably isn't going to produce one device that's right for everyone.

Your boss might have a LG Watch Urbane-shaped hole in his life but your mum could be better off with a Fitbit.

Read this: Apple Watch review

But we still have to put a star rating and a numerical score on our reviews. Admit it, you love that bit! You scroll down to check first, don't you? And so we come to the Apple Watch with its score of 3 1/2 stars or 7/10. Bit underwhelming isn't it?

The Apple Watch reportedly sold more units in a week than any smartwatch has sold so far and still it might not be right for everyone.

Here at Wareable, you no doubt think we speak with one voice, in the manner of a cult worshipping at the altar of a future where wearable tech fits seamlessly into our lives, improving our health, fitness, time management, smartphone addiction, stress levels and overall attractiveness to the opposite sex.

In reality, opposing opinions rage beneath the surface. We've tested every wearable that matters since our launch and there have been some spats along the way.

For one day only we thought we'd give you a peek into how we discussed - to put it politely - the final score on our in-depth Apple Watch review written by senior editor James Stables. Here it is, #nofilter, or whatever those hipper-than-thou tech bloggers say (beware, we are quite a sweary bunch).

Also included in the chat are editor-in-chief Paul Lamkin and (me) contributing editor Sophie Charara. We decided to publish it afterwards - usually James and Sophie are both at least 23% wittier than Paul. Honest.

[30/04/2015 13:18:48] It Begins

Paul Lamkin: score?

James Stables: 3.5

James Stables: torn for 4...

James Stables: think 3.5

sophiecharara: thought the verdict was fair

[30/04/2015 14:38:47] Does it struggle with the basics?

sophiecharara: that bit about Facebook messages not showing up is huge

sophiecharara: that means it can't display iPhone notifications properly

James Stables: whatapp's been a bit flakey

James Stables: need to investigate a bit more

James Stables: i think it's a problem with multiple messages

James Stables: Facebook, you don't see shit

sophiecharara: right ok

sophiecharara: still Android Wear and Pebble do that fine

sophiecharara: just some on Pebble are cut off if it's a long message, on AW it stacks multiple Gmails but can't do that for everything

Paul Lamkin: Facebook app isn't ready,

James Stables: can you send me some whatsapps plz

Paul Lamkin: but you should still get the generic notification, surely?

sophiecharara: yeah I think we shouldn't slate it too much for anything to do with third party apps I suppose

sophiecharara: they will all come, it's Apple

James Stables: you do get a generic notification

Paul Lamkin: whatsapped you

sophiecharara: as did I

James Stables: got Lamkin's number in a photo again

Paul Lamkin: don't publish my phone number again

[30/04/2015 14:39:27] The best (?)

sophiecharara: also one more thing, in verdict we need to say clearly that we think it is the best smartwatch you can buy (if we do)

Paul Lamkin: Do we? We don't if it's a 7

sophiecharara: Stables said yesterday he does

Paul Lamkin: Garmin Vivoactive and Sony SmartWatch 3 got 8s

Paul Lamkin: It doesn't have GPS and the run tracking doesn't work. It's not the best

sophiecharara: I know we would have to bring them down or something

sophiecharara: they got 8 before the Apple Watch existed slash was tested

Paul Lamkin: we shouldn't change other reviews to make the Apple Watch look better

Paul Lamkin: I think the SW3 is better than the Apple Watch

sophiecharara: I'm not saying that, we have now tested the Apple Watch, we need to decide what is best overall and then make sure scores reflect that

James Stables: i think it's a tie between the SW3 and Apple Watch

James Stables: Apple Watch is more wearable and has potential. SW3 is great and has GPS, and the power of Google

sophiecharara: SW3 = best for Android, Apple Watch = best for iPhone

[30/04/2015 14:42:30] The old scores debate

sophiecharara: if we don't change old scores, it's misleading for people who read them now

sophiecharara: if they need to be changed

Paul Lamkin: the Sony SW is still an 8/10 though. I thought that then and I still think that now.

James Stables: this is the eternal reviews debate

Paul Lamkin: If James thinks Apple Watch is 8/10, that's fair enough

sophiecharara: well then if it's a tie the Apple Watch should be 8

James Stables: I'm sticking with 7 for AW

Paul Lamkin: I would

James Stables: the AW could be an 8, if it sorts its shit out.

James Stables: it can't be more than an 8.

[30/04/2015 14:44:00] Price matters

sophiecharara: you think the SmartWatch 3 is better for most people than Apple Watch? I think it's better for fitness but not better for everything

Paul Lamkin: it's also fucking expensive

James Stables: The SW3 doesn't do a lot wrong...

Paul Lamkin: I don't think scores are totally compatible anyway. The SW3 costs £180. The Apple Watch costs, minimum, £300.

Paul Lamkin: The OnePlus One scored massively, not because it was necessarily better than the HTC One M8, but because it offered more for your buck

sophiecharara: also not many people will decide between them, they will have iPhone or Android and probs stick to that

Paul Lamkin: So the Sony is a 8/10 £180 device, the Apple Watch is a 7/10 £300 device.

sophiecharara: but the absolute best/no.1 should be what we think is the best

Paul Lamkin: SW3

Paul Lamkin: I'm dead against changing old review scores though. A review is a review, it shouldn't change because something else comes out.

James Stables: i don't think we should knock off half stars here and there. I think there's value in re-reviewing things. The landscape does change.

sophiecharara: exactly the scores are in relation to everything else

Paul Lamkin: The only way I think you can change a review score is if the product changes due to an upgrade etc. That way, it's a new review

sophiecharara: we gave the SW3 8 because it was better than Moto 360 on 7 etc

sophiecharara: anyway, Stables if AW is 7 and SW3 is 8 and the best, the verdict needs to change

[30/04/2015 14:52:23] This is hard...

James Stables: this is hard, as I have the basic SW3 on right now and I already want to take it off. The screen is horrid and the strap keeps slipping.

Paul Lamkin: It's personal preference though right? I wear a SW3 everyday. And if you are judging things on screens then the Gear S is the best…and it's clearly not.

James Stables: the SW3 Steel is much nicer than the basic

Paul Lamkin: You could say something along the lines of the Watch being, arguably, the best smartwatch but, because of the cost and basic errors, you can't score it as highly as we did the SW3 as the Sony offers much more value for money

Paul Lamkin: Price is important in reviews

sophiecharara: I honestly feel that for most people the Apple Watch would be better - they'd wear it more, use it more, the apps will get better. But with one big caveat that this isn't a fitness watch. The SW3 is best smartwatch for fitness/sports.

sophiecharara: how much is the steel SW3 though?

Paul Lamkin: I'm sick of Apple getting good reviews just because it looks good. Is it actually good? Not really. The apps are fiddly, the digital crown is bollox and the fitness aspects just don't work.

Paul Lamkin: And it's fucking expensive. You could buy an Xbox One or PS4 for the same money.

sophiecharara: I agree but I asked Stables yesterday which he would have (admittedly if someone gave him the money) and he said the Apple Watch

Paul Lamkin: James uses an iPhone out of choice. I never would. It's all down to personal preference.

sophiecharara: squabble squabble

[30/04/2015 15:00:49] That killer feature

Paul Lamkin: I've read a lot and not once have I read about a feature that stands out.

Paul Lamkin: The Apple Watch should have had a killer feature, one to really raise the smartwatch bar. What has it got? Cock doodles.

sophiecharara: OK how about this - would you recommend the Sony to someone who doesn't go running?

Paul Lamkin: Yep.

Paul Lamkin: But I like Android Wear and that's personal preference too. Some people hate it.

sophiecharara: I like Wear but it's not perfect, can be really frustrating, as can Watch OS

Paul Lamkin: If someone wanted a smartwatch, wasn't interested in running - I'd recommend any Android watch over the Apple Watch.

sophiecharara: woah OK

Paul Lamkin: James - is there one single reason you could recommend the Apple Watch? And justify its price-tag?

sophiecharara: this is partly why I think in the review we need a para on who we think it is for

sophiecharara: he's gone

sophiecharara: he's crying into his smartwatches

James Stables: sorry was making a salt beef sandwich

Paul Lamkin: Social Media Manager - can I tweet this?

Paul Lamkin: "Fierce debate at Wareable HQ as we decide how to score the Apple Watch. Review being published in the next couple of hours..."

sophiecharara: sure

Paul Lamkin: let me know when the review is ready to go and I'll take a look.

Paul Lamkin: I'm gagging to read it

Paul Lamkin: GAGGING

[30/04/2015 18:37:24] Verdict time

Paul Lamkin: that first pic

Paul Lamkin: it looks like you've got a broken arm - how have you even bent it that way!

James Stables: new pic in

James Stables: it's done

James Stables: Apple Watch gets 3 1/2 stars, the Sony SW 3 stays on 4 stars. That's taking into account the price and the chance that software updates and more apps will make the AW better.

James Stables: you two pipe down, I'm off to shave my arm

Read this: The full, in-depth Apple Watch review

How we test

Most of our squabbling above is about what smartwatches are for, who they are for, how we should compare them and how important the design, or wearability, is. The nuts and bolts of the review took place over the course of the full week we had the Apple Watch before we published this first review.

We didn't rush our review because we want to be fair, thorough and balanced. Between us we know its competitors inside out - not just Android Wear but Pebble and sports smartwatches such as the Garmin Vivoactive. We tested out the Watch's heart rate monitoring with a chest strap for comparison and we've tried as many third party apps as we've been able to so far.

We will update the Apple Watch review, maybe even increase that score, when new features, apps or accessories become available. It's still very early days for the most important wearable yet.


  • TheWerewolf says:

    Ok. first off - if I ever meet Paul in person, he is so getting beer (or whiskey - not sure why but he strikes me as a whiskey kind of guy... ). :) This was awesome. James is a hoot too. :)

    However... while I agree in principle with Sophie's concern about review 'landscapes' changing, changing reviews after the fact suggests your review process has a flaw: it's reactive and you don't have an objective metric.

    My first suggestion: give up the entire idea of 'best'. Best is one of the most overused and under-defined words in the English language. Is a 12" piece of string better than a 9" one? Well, if your metric is "Length, where longer is better" then yup, it is. Otherwise no (or maybe).

    When you're talking smartwatches, most of the reviews I've read so far have a tone similar to Sophie's: I want the Apple Watch to be 'the best', but I can't figure out any set of metrics that lets it be the best without them being strange. So inevitably, you get 'it's the nicest looking watch', which is highly subjective. Then you get 'it's made from the best materials' - suggesting that the other companies what - use stainless steel recovered from collapsed bridges? Then there's the whole 'metal beats plastic' pretension. 

    There are also flaws in how reviewers see smartwatches. in another comment here, I point out that smartwatches are falling into two broad classes: fitness and fashion. The features of each are different. Fashion watches have to run for long times with no recharge AND be svelt and nice looking. Fitness watches have to be about sensors and GPS and so have to run for a typical workout period and can have larger batteries making them chunky. Oddly, Apple Watch tries to be both, so it's stylish.. AND chunky... and has a lot of sensors - but not all the ones people want in a fitness watch. Oddly - THIS is why the SW3 is such an amazing watch and why in these metrics it beats out the Apple Watch. The ZenWatch on the other hand, has tons of style AND manages to be thinner and yet run much longer than the Apple Watch, which is why it also beats out the Apple Watch.

    Finally, 'it depends on which smartphone you have', while true, is irrelevant. If it's SO dependent on the smartphone that it's user experience derives directly, then it's flawed. It should be possible to say 'if we don't look at the phone while using it, how do these watches compare?' because even in this post, you're actually asking that question.

    • j.stables says:

      Just seen this. Best comment on the site ever.

  • BadDobby says:

    Re: Old scores debate - My old pocket calculator has more computing power than the first mainframe of yestercentury. Yet for their time, one was a game changer, the other a simple tool when I ran out of fingers and toes to count with. So surely a score of 8 today, does not necessarily equal a score of 8 in six months. 

    IMHO it's what that product itself brought to the table upon release relative to public expectation (which changes with advertising hype and competitors' current product offerings). This way the whole Fruit family vs Robot family becomes moot.

    7 = Good with improvement, 8 = Very Good, 9 = Excellent, 10 = Game Changer

     Barring significant software/firmware updates, I agree with keeping the scores set. My 2c.

    BTW, I enjoyed reading this article... "Cock doodles" Lol! Still snorting with laughter on that one. Thanks for sharing.

  • DukeBlue says:

    While I appreciate the transparency of the review process demonstrated here, I think it is more important to highlight the sheer comic genius of cock doodles. I have never heard this phrase and reading it was brought me greater joy than I believe a smartwatch ever could.

    "What has it got? Cock doodles" - best.quote.ever

What do you think?

Connect with Facebook, Twitter, or just enter your email to sign in and comment.