The five biggest questions about Apple’s new facial recognition system

Russell Brandom, The Verge:

Because phone’s all-glass front leaves no room for a home button, Apple is ditching Touch ID in favor of a facial recognition system powered by a new camera array and a specially modified A11 chip.

Not quite sure “leaves no room for a home button” quite captures the motivation for the change. But no matter, the article goes on to ask 5 questions, with thoughts for each.

Here are the questions:

  • Will Face ID make it easier for police to unlock my phone?

  • Could my face leak or get stolen?

  • Will Face ID have a racial bias problem?

  • Can you spoof Face ID with a picture of someone else’s face?

  • Will Apple ever use Face ID for anything other than unlocking phones?

Tempting to just dismiss these questions (the first four with a no, the last with Animoji), but they are interesting topics. Lots of food for thought here.

One particular point:

Soon, millions of people will be enrolled into Face ID, giving Apple control over a powerful facial recognition tool. In the current system, that data stays on phones, but that could always change. The hashing would make it difficult for anyone other than Apple to use the data, but there’s no real limit on what they use it for, particularly if they start to store information outside of specific phones. On Twitter, privacy advocates worried about Face ID data being used for retail surveillance or attention tracking in ads. You could also imagine it as next year’s delightful product breakthrough, integrated into Apple Stores or Apple Cars as a way of carrying over logins no matter who walks in.

Some good thoughts, there. One I’d add: Apple is going to gather a tremendous amount of machine learning data, with incredible value (especially in the phone arms race), if they find a way to bring that data back to their central servers. This is a mighty new frontier.

  • i don’t even pay attention to anything verge writes, they’re an apple troll site. they posted some messed up stuff yesterday already, mocking the steve jobs tribute portion of the event. fuck them, they’re just angry haters.

    • David Stewart

      Against my better judgement I looked at their live coverage of the event as I missed the first part of the presentation. It was awful. All snark, no insight.

    • GS

      Where do you think all those Youtube commenters work?

      • Mo

        Samsung Marketing.

        • GS

          That’s their other job.

    • Sigivald

      Yeah, and “racial bias problem”?

      Makes it sound like Apple would make a product that Doesn’t Like People Based On Race.

      The only plausible frame – which is poorly expressed by those words – is “maybe it’s harder to do the FaceID unlock with people with dark skin because contrast/IR reflectivity”.

      Which is plausible, but not what anyone means by “racial bias” in any other use – and not something Apple wouldn’t have, you know, bothered testing.

      A lot.

      • GS

        How long before someone tries to use their ass to unlock? Day after they are available?

    • Kip Beatty

      The Verge can be tough to swallow at times, but they have some good writers and do create some solid content. However, their listening and comprehension skills could use some work. Question 4 and 5 were addressed directly in the keynote, and given the variety of ethnicities and racial backgrounds shown during the Face ID section of the keynote, you could argue they addressed #3 as well.

      They also went to great lengths to explain how much more secure a face is than a fingerprint, so #2 is the typical fear mongering that always goes along with this stuff.

      Number 1 is a good question. I’m not sure it’s any easier than putting someone’s finger on the phone forcibly, and you could defeat it by keeping your eyes closed or simply shutting down/restarting the device.

  • Heos Phorus

    yeah, easy to answer and dismiss. yes (and you usually don‘t get to press the wake button 5 times when somebody forcibly takes your phone), no (but it doesn’t matter either way), no, no, and probably not for the time being.

    apple‘s detailed faceID data is all about security. optical face, person and object detection aided by machine learning is already good enough to work really well with ordinary security/street/highway/phone cameras, and there‘s more than enough data out there to train any AI. google, facebook and the likes are already running ml-facedetection algorithms on any photo they can get their hands on and are selling the data to anybody who wants it.

    for me, the more important questions would be: – will it be faster and more convenient than touchID (probably not this generation from the looks of it). – will a robber/relative/police be able to unlock my phone if i inadvertently glance at it (seems likely) – will i have to take off opaque sunglasses for it to work? (probably) – how will it react to quick changes ? (beard shaved off, facial injury/swelling) – what will the societal impact be if face detection becomes the norm ?

    • Apple has said no to the sunglasses question, I believe, which is confusing to me. Also, I think I’ve read that beard/no beard won’t impact it.

    • Well, touch ID fails with wet hands, so in some ways Face ID is already better.

      • spazsquatch

        I was meh of FaceID but it’s summer. Remind me again when I need to take a glove off to log into my phone.

    • Sigivald

      Also, the “yes” on the first one depends on what the courts say.

      I mean, the only reason cops can’t hold you still and run your finger over a TouchID sensor now is because they were told “no”.

      I can’t predict what a court would say about “just pointing it at your face” as a “search”, but … neither can anyone else, honestly.

      • I think any biometrics are considered to be fair game for law enforcement, at least that’s what I’ve read. Has the law changed? It’s why a lot of places suggest you deactivate TouchID or Retina/Face scanning if you’re going through a checkpoint since they need a warrant to force you to tell them a password. Doesn’t keep them from confiscating the phone, though.

        Our digital selves may never get the rights we think we always have…

  • Kris

    My only question about Face ID: Can you assign multiple faces to a single device, so, for example, a husband and wife can unlock a phone with their own face?

    • That’s a good question. Easy to do with touch ID, of course, so it’s seems like they should have at least considered it. My guess would be yes, but that’s only a guess.

  • If only there were a source for answers to these fantastic, not-at-all-basic questions!

    (Oh… yes, there is.)

  • David Stewart

    It sounded like Apple was training neural nets on the phone to recognize your face. This would tie the face recognition to the specific inputs the iPhone provides (meaning the angle and sensors of the iPhone are key to the algorithm working). In that case the face recognition isn’t something that could be transferred to other applications (like retail surveillance).

    • brisance

      Actually it has been done, and quite easily at that. It’s called transfer learning; you just chop off the layers you don’t need and load it into a model. Then you tune the hyper—parameters to suit the problem domain.

      • David Stewart

        Of course they could attempt that, but Apple could also just create some more portable structure to store the biometric data if that’s what it wanted. Apple could do anything. My point was just that Apple isn’t just creating a hash that could be reappropriated easily even by them, much less anyone else. It is doing all this in specialized hardware on-device, which is the most secure way this could be handled.

      • Sigivald

        Per Apple this stuff’s also all in the phone’s secure enclave and thus not accessible even to Apple.

        “But in the future they could change that!”, per the Verge’s Brilliant Insightfulness.

        • Sigivald

          (I mean, you said that below, too, but it’s worth repeating in this context.)

  • brisance

    There’s a misconception about how machine learning works. FaceID is pretty likely using a convolutional neural network with many layers that has bottleneck features trained on a large dataset. That is the part Apple alluded to about billions of photos taken during development. These features are then loaded into a model that resides on the iPhone X, which trains the phone to understand what features to look out for on a human face. When you enroll using FaceID, the system learns the user’s unique features and that gets stored on the device. This data is never leaving the phone and is stored in the Secure Enclave. So it’s quite fast because unlike a typical deep learning classification problem, the system only has to answer with a certain probability of whether the face presented indeed belongs to the user.

    • ChristinaMHicks


      blockquote>Boost your earnings on Google & make $99/hour by working from a home computer. on tuesday I bought a great Ford when I got my check for $9355 this past five weeks . this is definitely my favourite-job Ive ever done . I began this 3 months ago and pretty much straight away was bringin home over $99, p/h . see this website !pz10d: ➽➽ ➽➽;➽➽ http://GoogleEasyJobsToolJobNews/get/hourly ★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫:::::!pz10h..,…

    • JoyceJSpeight


      blockquote>Boost your earnings on Google & make $99/hour by working from a home computer. on friday I bought a gorgeous Chrysler when I got my check for $9277 this munth. it’s actualy the most comfortable job Ive ever had . I actually started seven months/ago and almost straight away got over $99, per/hr . check !si156: ➽➽ ➽➽;➽➽ http://GoogleOnlineServiceHomeJobsReports/easy/jobs ★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫★★✫★✫:::::!si156l..,…

  • Sigivald

    “privacy advocates worried about Face ID data being used for retail surveillance”

    Does that really mean, as I read it, “somehow people will be staring at their phones while shopping and faceID handwaving something something about what they’re shopping for”?

    Because while I might use my phone while shopping, it’s literally always when I’m looking at the phone, not products.

    Is there some interpretation there I’m missing?

  • Dan Kelly

    My issue, which seems to have been confirmed in the demo, is with Apple Pay and TouchID.

    At the moment my Pay work flow is: get phone out of pocket, place on NFC reader, with thumb on Home button. NFC activates Apple Pay. Thumb is ready to confirm.

    This is really frictionless and much easier that getting my credit card out of my wallet. It takes no thought at all

    It sounds like Apple Pay will still by activated by touching on a reader. But will FaceID work?

    Most NFC readers are at arms length and flat on a counter. Quit often surrounded by craft such as sweet displays. I really want to know if FaceID can grab my face in those situations.

    • Most things I’ve read says the facial scan catches several angles of your face, so it can find you even when the phone is laying flat or at a low angle at least. Remember it’s not just looking at your face but blasting it thousands of infrared dots to get a 3D map.

      I’m partially speculating; I can’t find the articles I read a while back or the companies Apple bought to do this.

  • JimCracky

    First, the Verge is full of shit. Second, all these questions are trite remakes of questions that followed the intro of touch id.

    So, Touch ID has the SAME limitations as FACE ID. All the other issues and questions are just pandering. The same edition of he Verge has the famous “I’m not buying the new iPhone because…”

    Well, I don’t give a shit what you’e buying. As long s you don’t use my money.

    • Richard Passavant

      ^^^ THIS ^^^

      I recall ALL the same fears about TouchID and I’m busily rolling my eyes at all the “big brother has your face” stories

  • Scott R.

    Doesn’t work while wearing sunglasses! #Fail

    • James Hughes

      Or while wearing a bag over my head. #fail

  • Tom_P

    My first thought: does this Russell guy had questions like this when Samsung or other Android phones has this feature? If he didn’t ask it out loud then, I wouldn’t waste my time on his questions.