Filtering Unwanted Messages with Identity Lookup

Hello. My name is Stewart Montgomery and in this session we'll discuss how your app can use new APIs and iOS 11 to filter out unwanted SMS and MMS messages and help users avoid an increasing nuisance. Before we dive in, let's take a look at what happens when we receive an unwanted SMS message in iOS 10.3.

Here we see what is pretty obviously an unwanted, spam SMS. Messages like this are annoying for users since they play a tone or vibrate just like a normal message and distract from whatever you're doing.

And if I launch messages it's right there at the top, mixed in with my real messages and cluttering up the list. Unfortunately some iPhone users get a lot of these messages and they would love to have a way to filter them out. With iOS 11 we are introducing APIs to allow your app to analyze the sender and content of any SMS or MMS message from an unknown sender and attempt to filter out those which are unsolicited. Let me show you how it works.

Here on my iPhone running iOS 11 I'll launch the New Messages app. And since I've already installed a Message Filter app extension and enabled it in settings I now see a second tab called SMS Junk. And if I get a message which the app believes is junk it'll only appear under that tab. Ah, there's a new message now. Let's see what it is.

This is the same message as before, but now it no longer appears in the regular list with my known contacts and doesn't distract me with a sound or notification.

And if I tap to read it I can see from the label at the bottom that it was marked as junk by an app called Filter It.

Now there are a few reasons why we decided to add this functionality. Unwanted messages, which include any unsolicited or spam messages a user receives, have been an increasing nuisance for users in the recent years. But beyond the annoyance, what's more concerning is that often these messages are phishing attempts and include links which may harm users. So of course we want to prevent these messages from being delivered whenever possible. Now there's an important difference worth noting between iMessage and SMS and MMS messages. For iMessage we offer the report as Junk Service on-device since those messages are encrypted end to end and delivered over the iMessage network. But we don't have the ability to do this for SMS or MMSs since they are delivered directly from a wireless carrier to a user's device. So the filtering of these messages must happen locally rather than on a centralized server, and that's where these new APIs come in. Finally, we've heard that many of you have developed expertise in analyzing messages and detecting which ones are unwanted and we're excited to invite apps to help with this task.

So for the remainder of this session I'll cover a few areas in detail. First I'll walk through the details of what we call Message Filter Extensions and show how they work.

Next I'll talk about some important considerations around privacy since these extensions come with some special rules. Then I'll talk about how an extension can check with a network-backed service, which some apps might find useful and I'll show a couple of demos along the way through creating one of these extensions in Xcode. So let's get started.

The way this works is using something we call Message Filter Extensions. So let's talk about those in-depth. As the name implies, this is a new app extension type, which your app can include. The APIs for it are in a new framework in iOS-11 called Identity Look Up. Now once a user has installed an app with one of these extensions, to begin using it they must first enable the extension in Messages Settings. Only one extension can be enabled at a time, or if the user wants to disable the feature they can choose None. And once it's enabled that extension is invoked every time an SMS or MMS message is received from an unknown sender. And there is some other criteria used when deciding when to send a message to the extension, which I'll discuss in a few minutes. Let's walk through a diagram showing the overall flow. When a message is received by the phone it starts in the Messages app and if it is an SMS or MMS and it's from a sender which isn't in the recipient's contacts, then the extension, which the user selected in Settings will be launched and will be passed the messages sender and body via an object called IL Message Filter Query Request, which is part of the identity look-up framework. When the extension receives this it begins examining the message, looking at the sender or the body of the message, or both, and it might check against a known list of bad phone numbers or it could look for a suspicious looking web link in the body, whatever is appropriate. Ultimately the extension has to form a response using an object called IL Message Filter Query Response, describing whether to allow or filter the message, and it sends this back to the Messages app. And once it receives a response, Messages will either alert the user normally or suppress the notification and move the message thread to the Junk tab.

So that's a basic overview of how it works and before we go on I'd like to touch on a very important topic, and that's user privacy. We thought a lot about how to maintain the strong level of privacy Apple customers expect, but also allow them to enable this if they want to confront this persistent problem of unwanted messages. So there are a few special rules that extensions must comply with when using this API.

The first rule is that a message recipient's phone number is never sent to an extension, only the sender's phone number or email address is included, since that's all that should be required to make a decision about whether to filter a message. Another key rule is that a message filter extension can never export the contents of messages outside its container, and these extensions have some additional restrictions because of this. They cannot write to files shared with their containing app and they cannot perform network operations. The reason for this is that, although some messages may be unwanted junk, others may be legitimate and be sent from someone who's just not on the recipient's contacts yet. So then it's imperative that all the messages be kept private and never exported in any way that reveals more about the recipient than what is contained in the message itself. Although they can't perform networking themselves, it is possible for these extensions to defer a request to their server indirectly, and when an extension requests to defer iOS will make a web request on the extension's behalf in a secure way. We'll see an example of that later. The main thing to keep in mind is that your extension should never export messages outside of its container to maintain user privacy.

Now there is some specific criteria that the Messages app uses to decide whether or not to send a given message to an extension. First, this feature is only used with SMS and MMS messages and never with iMessage. As I mentioned earlier, unwanted iMessages are handled using a different mechanism, so this only applies to SMS and MMS.

As I've mentioned a few times, only unknown senders, or those which are not in the recipient's contacts are actually sent to the extension for analysis. If a sender is in Contacts we assume that that recipient knows the sender and wants to receive messages from them. That also means that if a message is ever miscategorized as junk then the user can add that sender to their contacts to ensure they aren't filtered out in the future. Also, if a user is exchanging messages with someone who is not in their Contacts and they reply to that thread multiple times, then we will stop sending any subsequent messages in that thread to the extension. Or, if the user replies multiple times to a thread, which is already marked as junk, that thread will be restored to the non-junk tab. Responding multiple times is interpreted as a signal from the recipient that they really do want to be communicating with the sender. So all of this criteria doesn't directly affect the API, but it is something you, as a developer of one of these extensions, should be aware of when testing or troubleshooting your app.

So now I'd like to show a demo in Xcode of how you can create a message filter extension. Here we have an app I've made called Filter It and I'd like to add a message filter extension. The first thing I need to do is add a new target.

And I'll choose the new message filter template for iOS and I'll give it a name.

I've got a new file added to my project now called messagefiltereextension.swift. Now let's take a look at that.

The first thing we see is a method called Handle Query Request with Context and this is called on our extension so that it can examine the incoming message and return a response using the completion handler.

Now the template is structured to first attempt an offline check using this method, called Offline Action for Query Request and it returns an action which is either allow, filter, or none.

So for this demo what we need to do is customize this offline action helper method. Let's look at what it does currently.

Right now it always returns none, but I'll replace that with some simple logic to always filter if the message contains the word junk.

In a real extension this could do something more sophisticated, but this works for now and that's all it takes to create a simple, offline only, message filter extension.

Now although some apps may be able to do most or all of their checking offline, other apps may find it useful to check with a network server whether to filter messages or not. So next I'd like to talk about network deferral. The best way to show how network deferral works is with another diagram. As before, when a message is received it starts in the Messages app and is sent to the chosen extension, but this time the extension chooses to defer this request to its network server, whose URL is specified in its info.plist. So it tells Messages to defer and Messages then makes a JSON request to that server URL. The server then examines the Messages contents inside the JSON request and can respond in any format it wants and that response is handed right back to the extension. And from here, the extension reads the response from the server and finally returns an IL message filter query response back to Messages.

There are a few restrictions to be aware of when using network deferral. First, the deferred network requests contain no personally identifiable information about the recipient of the message.

The network URL is hardcoded statically into the extension's info.plist file under a key called IL Message Filter Extension Network URL. So it cannot vary between requests or for individual users.

All URLs must be secure https and the server must be configured such that it doesn't require any app transport security, or ATS, overrides since there's no way to configure them. Also, this feature requires that both your app and server use the associated domains, or Apple App Site Association Feature, which is something you may be familiar with if you've adopted other iOS features like App Links or Shared Web Credentials. And for more information, see the session; Seamless Linking to Your App from WWDC2015. And the last restriction to be aware of is that any cookies that the webserver attempt to set will be ignored to maintain privacy.

The request made to the network server is formatted using JSON and includes the same things which are in the IL Message Filter Query Request Object including the message sender, which is a phone number or email address, and the message body. The request also includes the version of your app, which is the CF Bundle Version Key from the app's info.plist. This may be useful in cases you have -- in case you have released several versions of your app with different capabilities and need to format the response to insure that particular version of the app can understand it. And we include the version of the JSON request format itself, which is currently one. Now, unlike the request format, the response format is entirely up to your app to define and doesn't necessarily have to be JSON. The response body is passed back to your extension to parse, so there are not requirements about its format.

And here's a quick look at the JSON request format, and you can see it includes all of the info I mentioned.

Now, let's go back to my Filter It app and add network deferral capability to the extension.

If we take another look at the Handle Query Request method from earlier we can now see that, after the offline check is performed, if the action it returned was none and it falls into this case of the switch statement, we assume that this query request could not be handled using only an offline check and actually needs to consult our network server to get an answer.

To do this our code calls the defer query requests to network method on our extension context and this causes a network request to be made on our extension's behalf, which will call this completion block asynchronously when it completes. Inside the completion block, if there was a response from the network and there wasn't an error, we use another helper method called Action for Network Response to translate it into an action. Let's jump to that method and see what it does.

Just like with the offline check helper method this method defaults to returning none, but let's customize that to parse the response from our server. I'll assume the server returns JSON, although it doesn't have to, and I'll use the new Foundation Decoding APIs in Swift 4 to decode the response. I'll paste some code I've already written to do this, but let's briefly walk through it.

I first define a struct, which describes the JSON format that my server returns, then I create a JSON decoder instance and I use it to decode the data from the network as an instance of the struct. Finally I return the action, which was decoded, and store it in the struct.

And if there were any errors I handle them below.

And return the default response of none.

And just like that we've added network deferral support to our message filter app extension. So it now supports both offline and network checking of incoming messages.

So that's how your app can help filter unwanted messages using the new message filter extensions and identity look up framework in iOS 11. We've tried to strike a balance between user privacy and solving a very pressing need our users have. And the result is a powerful new API your app can use, but it is subject to some special rules you need to be aware of. So please, download the new STK, check out the new identity look up framework, and try making a message filter extension filter today. For more information see this link for the Sessions page on the official WWDC site.

We do have a few related sessions and this year's conference to mention. For more information about privacy practices on our platforms see the Privacy and Your Apps session happening in the Executive Ballroom on Tuesday at 11:20. And for more info about the new foundation and coding APIs I showed in my demo, see the new -- see the What's New in Foundation session in Hall Two on Wednesday at 11. Also, WWDC2015 had an excellent session called Seamless Linking to Your App, which walked through the associated domains feature which message filter extensions require when using network deferral. So check out that session in the archive for details about how to implement it in your app and on the server. Thanks so much for watching.