Advances in Networking, Part 2

Hello and welcome to Advances in Networking Part 2. If you couldn't join us for Part 1, it will be available for streaming soon on the app and on the web.

I'm Eric Kinnear from Internet Technologies. I'll be joined today by my colleagues Tommy and Stuart. We've got a lot to cover in Part 2. We're going to start by expanding the horizons of what you can do with Bonjour browsing. We'll talk about how you can achieve efficient and easy message transport by building framing protocols. We'll take a look at some new and improved metric collection. And we'll finish off with some status updates and some best practices to help networking in your app become the best it can be.

Before we begin, a brief reminder.

If you're using URLSession and Network.framework, you'll be able to take advantage of everything we're talking about today.

If you're not, these are some additional reasons why you should switch to a more modern networking API. Let's jump right in with Bonjour browsing. Bonjour is how you advertise and discover services on the network. It's used anytime you print with AirPrint, connect to an airplane-enabled device, use HomeKit to automate your home, really anytime you are connecting to something without typing in an IP address or a host name. And as you know, Bonjour is available on all Apple platforms. And it's also available on Linux, Android, Chrome OS, it's how Chromecast does discovery.

What you may not know is that Microsoft also quietly added Bonjour support to Windows 10 back in 2015.

And since then the implementation has matured.

This means that Bonjour is now available on every major platform.

Today, we've got some exciting enhancements to share in this area. Sometimes you're on one network and you'd like to discover services that are available on a different network. Say you've got a device and you'd like to print to a printer that's on a different subnet and multiple hops away. Right now, you would send multicast packets on your local network and you wouldn't hear anything back.

A discovery proxy solves this problem.

You can now send unicast packets over to the discovery proxy. It will send out the multicast packets on the destination subnet, receive the response and proxy the results back to you. Now you can connect directly to your printer and voila, we've got our document. We're excited to announce that the code for this on the client side is included in your developer seed and a server implementation along with instruction for setting it up are available on GitHub. Let's take a look at what this means for your app.

Previously, it's been the recommendation that when browsing you should specifying nil for a domain. And this continues to be the case and the right answer for almost every situation.

In the past you may not have noticed much difference, but now this starts to have a bigger effect, specifying local will explicitly prevent discovery of any remote or proxied services. This is probably not what you want. So double check that your browse calls aren't inadvertently specifying a domain.

And while you're browsing services while building your app, let's take a look at how some new features in Network.framework can make browsing for Bonjour even easier especially in Swift.

Last year, we introduced NWListener and NWConnection along with Network.framework. For example, you can have an NWListener advertising a Bonjour service and you can have an NWConnection connects to a Bonjour service endpoint.

But you had to use one of the other Bonjour browsing APIs to discover the available services. And once you found one, you had to do a bunch of work to convert it into an endpoint that you could use with your connection.

Today, we are thrilled to announce native browsing support in Network.framework via the NWBrowser object.

Browser joints connection and listener to cover the entire workflow from advertise to discover to connect, all using the Network.framework objects that you're already familiar with.

Browser provides native service discovery in Network.framework using a modern dispatch-based API that's optimized to work incredibly well in Swift. It also includes optional support for TXT records. So if your application needs it, you can ensure that a TXT record is requested for every discovered endpoint. Let's take a look at how we use our browser.

We can init it using a Bonjour service type that we'd like to discover and some NWParameters which is how you tell it what you'd like to-- how you'd like to browse, the same way you do all other Network.framework objects. Next, you can set a browseResultsChangedHandler which will be called to deliver the list of available endpoints that have been discovered. And finally, just like other Network.framework objects, you start the browser on a dispatch queue where you'd like to receive those callbacks. Let's take a closer look at the browseResultsChangedHandler.

You have two options. First, you can provide a handler that receives the detailed list of all changes that have happened in that update. This aligns very closely with the lower level APIs and provides you full visibility into everything that changed. Endpoints can be added or removed and they can also have their inner details changed. Those changes are represented by flags.

And in this case, we'll check for interfaces being added or removed as endpoints are discovered over additional interfaces. You can also choose to provide a handler that just looks at the latest list of results that have been discovered.

Be careful if you do this because this handler will be called repeatedly as the list of available endpoints changes. So make sure that you're updating the state and the rest of your application appropriately.

Let's see an example of the NWBrowser in action.

We're going to make an app that provides a service discovery and secure connectivity between two devices.

In our case we're going to make it a tic-tac-toe game. But you could use this for pretty much anything else.

We're going to use an NWListener to advertise games to nearby players, we're going to use NWBrowser to browse for the games that are available nearby, and we're going to take one of those browse results once the user picks a game they'd like to join. And we're going to pass it an NWConnection to connect back to our listener. Let's take a look at this in Xcode.

Here I've got my application and we've already written a bunch of code to handle doing some different views displaying a list of available games to the user, letting them host the game, stuff like that. So, here we can just focus on the browser. I've got a class called PeerBrowser, which I'm going to use to manage my NWBrowser and provide a PeerBrowserDelegate that will pass the list of discovered endpoints too so the UI can display them to the user.

First, I need to add my NWBrowser as an instance property on the PeerBrowser.

Next, we see that when the PeerBrowser is initted, it will immediately call startBrowsing.

We need to fill that in.

First, I'm going to create some NWParameters which is the same objects that you use with other Network.framework objects to describe how you'd like to interact with the network. In this case the default parameters are just fine but we'll set includePeerToPeer to true so that we can discover other available games even if the devices aren't on the same physical network.

Next, we'll create our NWBrowser browsing for a service type of tictactoe.tcp. And we'll make sure to specify nil for the domain.

We'll use the parameters that we created earlier and save it off into our peer browser. Next, we'll set a stateUpdateHandler just like we do with other Network.framework objects to receive updates about the state of our browser if there's any errors, how it's going, things like that, and then we'll set our browseResultsChangedHandler.

Here it's really simple. We take the list of results, we pass them off to our delegate to be displayed in the UI, and we make sure that our delegate is coded in such a way that it refreshes the UI every time this happens so we can always represent the latest list of discovered endpoints.

Finally, we start the browser on the main queue since that's where we'd like to receive our updates. And that's it. With just that code, we're able to bring up an NWBrowser, have it discover nearby games potentially over peer to peer links and display a list of available games to the user so they can choose which one to connect to. We'll talk a little bit later about our listener and connection, but all of this code is available for download as sample code off the website. And there's one thing I want to touch on before we try it out.

We've got our NWParameters that we're going to use with our listeners and connections. And earlier I mentioned that we want to make sure that our connection between these devices is secure so that nobody else can see what moves we're making or worse, modify one of the player's moves.

We're going to do that by defining an extension to NWParameters and creating a convenience initializer.

It takes a passcode as a string.

We're going to display to the host of the game a passcode and ask the person that wants to play with them to type in that passcode and then use that to derive a pre-shared key that we can use to secure the connection with TOS. In order to this, we need to create both TCP and TLS options in our initializer. Let's start with TLS.

Here we defined a function that creates TLS options. We pass at the passcode and for now we start with just the default TLS options.

Next, we use the new CryptoKit framework, which is available this year, to derive an authentication key and code from that passcode. We'll add that pre-shared key to our set protocol options and we'll also make sure to add a TLS cipher suite that supports the use of pre-shared keys. We can return our TLS options. And up here we're ready to go with our TCP options.

We'll use a default TCP options for most things but we'll also enable keepalive.

We then init our NWParameters with the TLS options that we just created down below, and those pretty much default TCP options. Finally, the last thing we want to do is set include peer to peer here so that our connection and our listener will also be able to connect to nearby devices even when they're not on the same network. So that's it, let's try it out. If I go over here, we can see that I've got two devices running the app and we've already got our UI there for hosting a game. And the browser has already started browsing and is displaying searching for games since we haven't found anything yet.

If I come up and type in my name and tap host game, you can see I'm given the passcode and our browseResultsChangedHandler has been called with the list of discovered endpoints, in this case the game that I'm hosting, and we've displayed that for the user. It's that easy. If I tap on join game, I'm presented the opportunity to type in the passcode. And now when I confirm that, we'll see that we've created the pre-shared key, used it to connect back to the listener, and hopefully the game will flip around and ready to play.

There we go.

So far, we've built the beginnings of our app and established connectivity between the two devices. We used an NWListener to advertise a Bonjour service of tictactoe.tcp. We used an NWBrowser to browse for available games and display those to the user. And we were able to take the result that we got back from the browser and pass it directly to our NWConnection to connect back to the listener and establish a secure connection between those two devices.

Of course in order to play the game, the two devices actually have to be able to communicate to share the game state, tell each other about moves that the players making, and things like that. In order to that, I'd like to invite Tommy up to the stage to walk us through building custom framing protocols.

All right. Thank you, Eric.

So today, I'd like to share with you an exciting new way that you can extend your network connections with custom protocol framing code that you write that runs on the same thread as the rest of the protocols in the networking stack. So, in order to finish the game that Eric just started, we need to define a way for the two games to send commands between each other. When one player wants to make a move, he needs to send a message to the other side.

To do this we're going to need a protocol. So here's what our protocol is going to look like. It's a simple type-length-value or TLV protocol.

So we have a 4-byte type which may be make a move. The player wants to put a given character at a given location on our tic-tac-toe board. We have a 4-byte length which indicates the length of the rest of the message. And then after that we have the body of the message. And in this case it could be place the monkey face at row 1 column 2. And then it's going to repeat like that on TLS byte-stream.

So you may have noticed that this protocol, even though we're running on top of a TLS byte-stream which itself is not structured is using structured messages. The application is not thinking in terms of byte-streams but well-delineated pieces of information.

And almost all networking applications do this.

They either have a header and a body or they have a delimiter that goes somewhere to define the boundaries of their messages.

However, traditional transport networking APIs like sockets didn't give you a way to easily read out messages on a connection. You had to do that all yourself in the application.

So, to take a look at how this problem plays out let's look at the relationship of your application to the rest of the networking stack. So up on top you have your application.

And it is communicating with the networking stack via the API.

So in Network.framework, we have running TLS and TCP all within one shared thread inside your application. This is the user space networking stack that we introduced last year.

Let's zoom in to see how your application reads messages when it's on top of a byte-stream. So if you have a protocol like the one that we're using for tic-tac-toe, you may have a fixed-length header. And the simple thing you can do here is to read exactly the length of your header.

Receive 8 bytes.

So this is a fixed length. You know what's going to happen. And the stack will call you back when you have your full header. This allows you to determine the length of the rest of the message. So you can read exactly that and then go back and forth reading body, header, body, header, body. So this is great, but you may have noticed that we have multiple back and forths at least two for every single message. If you have a more complex protocol, maybe you have a variable-length header or you have a delimiter, this can become even more inefficient, even though it's simple for your application to write its logic.

If you care about efficiency a lot, you have another option. You could receive as much as possible at one time. But now you have a bunch of other problems to deal with.

Now, you need to handle the case in which you don't receive a complete message all in one chunk. Or maybe you receive multiple messages in one go, or maybe you receive only part of your header, you only have two or three bytes of your length field. You need to save it off, reconstruct the field and parse it out again.

Getting this fully correct and handling every possible edge case can be really difficult. And it's common to have subtle bugs that only show up when your users actually use your application in the field.

All right. So, it's a bit of a bleak picture at this point. How do we get the best of both worlds? How can we have a way to be both efficient and have simple code that's easily testable and composable? Well, I'm really excited to share that now in iOS 13 and macOS Catalina, you'll be able to write your own protocol code that runs on the same networking thread to handle this problem. So, this is an unprecedented step forward in the amount of flexibility that you have to define your messages within a transport networking API . Thank you. And you get to do this all within NWConnection. And so, to the application on top, it's just as if you're reading and writing datagrams like a basic connection.

So let's look at the world now. Still have your application. It's still sending and receiving but now you have your framing code that's running within the same thread as TLS and TCP. So you can now call receiveMessage. And it will get exactly one callback when you have a complete message that your application can fully process. You can do this over and over again. One call per message and really, really easy to debug and understand what's going on. So, this is great. And if your-- You may be asking right now what can I actually implement as one of these framing protocols? What are the restrictions? So, the good news is, is that pretty much anything that encapsulates or encodes application data to transform it can be written as a framing protocol. You can even send your own messages that do not correspond to application data, if you need to do a handshake or if you need to implement some sort of keepalive on the connection. And the protocols that you're implementing here can be standard IETF official protocols or they could be something custom just for your app like what we're going to be doing for a tic-tac-toe game.

So if you want to build a protocol, you have two steps. First, you implement a reusable piece of code that defines your message framing. This is the protocol.

And then you add that protocol into your connection's protocol stacks so that you can use it for connection establishment as well as sending and receiving message. All right. So let's go on to step 1. We're going to start implementing framing protocols.

What you do here is create a class that conforms to ProtocolFramerImplementation. And there are many things you can do within this class.

But the two most important things to remember are to handleOutput, to send messages; and to handleInput, to parse messages. If you can do these two things, hooray, you are a framer.

Let's take a look at the code. So, here we have my protocol. It's going to conform to ProtocolFramerImplementation.

And the first thing that I recommend you do is create a definition object. And this is a handle to your protocol that you can use throughout the rest of your app. It refers to your protocols you can add it into connections. Next you can handle a lot of the basic callback events. One of the most important ones here is start. Start will get called anytime your protocol is loaded into a connection as being used to bring it up. If you need to do a handshake to exchange something with the other side, you can implement it here.

Or if like our tic-tac-toe game you have a very simple protocol, you don't need to do any setup, just mark the connection ready immediately.

So once you've done this, you now have to handleOutput and handleInput. Let's dive into these.

So here is what handleOutput looks like. You will get called with handleOutput every time the application sends a message. And you'll be given the message metadata with some custom values if you need them, along with the length of the message the application is trying to send. So, if you have a header body protocol, like what we're using, you can first create your header structure and try to serialize some data.

So this can include your type that you get maybe from the message metadata along with the length that was passed to you into handleOutput. You combine these into data and then you call writeOutput. WriteOutput will queue your bytes on to the output stream, but they won't get sent quite yet. Next you need to write the body. And in this case we don't need to transform the application data at all. We can just call writeOutputNoCopy.

This allows us to just take the direct application bytes and then queue them on the stream. When we return from handleOutput, all of the bytes will be sent out to connection. All right.

So let's move on to handling input. Handling input is similar but a little bit more complicated.

You'll be called with handleInput anytime your application has received new bytes on the connection. And if you're doing a header body type protocol, you have two jobs. You need to parse the header and then you need to parse the body. So, let's start with parsing the header. Here we have a fixed-length header for our protocol. We're going to have exactly 8 bytes. And what we do is we call parseInput to start inspecting the stream of bytes that have come into the connection.

And we can call it with a minimum and a maximum of 8 bytes because we want to look exactly at that 8-byte header. If this succeeds, you'll be called in the block and you'll be able to look at the actual buffer bytes, parse out your values, save them off into local variables if you need to. The return value to parse input indicates how many bytes you want to increment the input cursor by. You say, I am done handling these 8 bytes. We don't need to see them again. We don't need to deliver them to the application. Move on. Now you can handle the case in which not all 8 bytes were available yet. In this case, the parseInput function will fail and you can just wait for more bytes to become available. The return value from handleInput indicates the number of bytes that need to be present before you can successfully do more work. So in this case we're telling the connection, make sure there's 8 bytes before you wake me up again. If you are able to successfully read out your header, you can create a message object that you can deliver up to the application along with the data. This allows you to put in any custom values, types or other indicators that you want to send up to your application. And then lastly you call deliverInput or deliverInputNoCopy in this case. This allows you to mark the next certain range of bytes as application data that should be delivered directly up to the application. And this returns Boolean value to indicate whether or not all of the bytes were available and successfully were sent up or if the connection is going to wait. So you can actually deliverInput of a message that's a megabyte long, a gigabyte long if you need to, and will keep on streaming those bytes up as part of that one message and you don't need to wait for all the bytes to be present or handle them yourself.

OK. So, I know it's a lot of code, but I think we're ready to go and implement the protocol for our game tic-tac-toe.

OK. All right.

So this is the same game that Eric began earlier. But I'm now creating a new class which I'm calling game protocol and this is going to conform to ProtocolFramerImplementation. I have defined two different types for my game. I want to have two different commands that we send.

One is to select the character. So the first step of the game is that the player will decide which emoji family they want to be. Do they want to be a monkey or a bird? And then once each character has selected their character, they can start sending moves. And this would be a longer body. It will include the character along with the row and column values.

All right. So, I remember that the first thing I need to do when I'm implementing a protocol is create a definition. And this is a handle based on my object that registers my object with the system and allows me to use this in connections.

Next, I handle all of the basic callbacks. And here again, because I don't need to have my own handshake, when I get called with start, I can return a start result of ready.

Next, let's handle sending and encapsulating my messages.

Here I'm going to define my implementation of handleOutput.

So my header is an 8-byte header that includes a type and a length. So first I need to know what is my type, and this I get from the message that the application sent. And we'll see that later on. I've created a custom extension to the framer message to extract my particular enum type out so that I can know if this is a character selection or a move. So once I have the type, I can instantiate my struct, my game protocol header with the type and the length that I was passed in handleOutput.

I've already written code to encode that data that I got the type and the length as an 8-byte range of bytes. And I call writeOutput to queue that in the output stream. And the last thing I need to do now that I've written my header is write the body. And here I just call writeOutputNoCopy and indicate that the next range bytes are going to be the body of this message.

And so once this returns, those bytes will be sent and I'm ready to handle more messages either in or out. OK. So that was writing. Now on reading, I'm going to handleInput.

So first I want to read out and parse my header. So have a fixed sized header. It's going to be 8 bytes.

I'm going to try to parse out a minimum of eight and a maximum of eight and I'll get called with my buffer whenever those 8 bytes are available. So here I validate that the buffer is valid and then I create my structure to parse out those 8 bytes into the type and length field.

Once I've successfully done that, I indicate that I want to increment my input cursor by 8 bytes to say that I've consumed these bytes. I'm done with them. Now, I do need to also handle the case in which I didn't successfully parse all 8 bytes. Maybe only 5 bytes were available.

And so parse will have failed and I will return from handle input and indicate that I need to wait for 8 bytes to be available before I do more work. But if I did get past this point, I know that I have a valid header that I can use to deliver up the rest of the application data.

So now I'm going to create a message object. I'm going to store within that message object my specific message type.

And lastly, I will call deliverInput with no copying and just tell the connection those next bytes that I parsed out based on the length, those are going to be the application data, and when your application receives a message, they'll receive exactly that chunk.

So that's all I need to do. That's my full protocol. I am ready to learn how to int input that into my game connections.

All right.

So, the good news is that this part is really easy. So all you have to do to add your protocol into your connections and reuse it is take that definition you made earlier and create some protocol options using that definition. So protocol options are the things that protocol stacks are made out of. You have your TCP options, your TLS options, and now you have your own custom protocol options. So when you create your parameters for your connection, let's say you're using TLS as you should to be secure, alongside TLS in your protocol stack, you can add directly into that array of application protocols your own protocol on top. And you can add multiple of these to have multiple layers of framing going on. And this is the same place that if you want to use WebSocket, which is a new system implementation that we have for you this year, you can add it into your connection.

So WebSocket itself is implemented as a protocol framer using exactly the same API that you have available to you now. So it shows you just how powerful a framing protocol can be. But if you don't want to go through the work of writing your own, you can use WebSocket.

So one point I want to make here is that some applications need to use different protocol stacks in different situations.

And framing protocols are a really great way to make the contract between your application and the networking connection the same even when you're using different protocol stacks.

So to give you an example of this, we use this on our system for DNS.

So DNS usually sends datagram messages over UDP.

But occasionally, DNS needs to run on top of the stream like TCP. And when it does this, there's a protocol that just has a very basic length body format to encode DNS over TCP.

So we wrote a framer to define this simple encapsulation so that we can have the same code on top that sends DNS datagrams that doesn't have to care about whether it's going over UDP or TCP and it can just be the same logic. So this is a great way to separate out the concerns and be able to debug the parts of your application separately. So now that we've added our framing protocol onto our connections, we're ready to send and receive messages and we can use custom values as we're going to be using in our game.

So Framer.Message lets you store key value pairs of any object type so that you can add in your own custom values to decorate your send operations and receive that information inside your protocol.

So you can create a message and then when you've set it up how you like, you can add it to the context that you're sending your data on. So every send operation already has content along with context. And so the context describes how you want to send your data. So your Framer.Message is just a new way to send data. Receiving is very similar. When you call receiveMessage, you receive along with your content the context that describes how this data was received. And you can look using the definition of your protocol at the specific message values that your protocol framer delivered to you.

OK. I think we are ready to go finish our tic-tac-toe game. All right.

So we've already finished our game protocol. To add it into our connections, I'm going to go back to the parameters that Eric set up earlier. So he already set up our connections to use TCP and TLS with our passcode.

Now, all I need to do is just add these two lines in. We're going to create some options based on my game protocol definition and just insert them into the array of application protocols I want to use in my connection. Now when the connection starts up between those two devices, it will be ready to start encoding messages over that stream. Now I'm also going to do a couple convenience functions within my connection to make it easier for my application to send and receive my custom message types.

So here I have a connection object that sets up an NWConnection using the parameters that we just defined.

Whenever the connection becomes ready, it's going to start receiving messages from the peer.

So we need to receive next message and implement this.

So what I'm going to do here is take my connection and call receiveMessage on it.

I'm going to get the content along with the context.

So I'm going to take that context and look at the specific metadata for my protocol, the game protocol definition.

And that will give me my message object and allow me to deliver the message type along with the data up to the application.

And then of course once I've successfully received one message, I'm going to call receiveNextMessage to do it all over again.

I'm also going to define some helpers for sending. So whenever the application decides that the player is selecting a character, we can create a message and add the selected character type to it, add that on to our context and send it. I can do the same thing for sending a move, have a convenience here to say that I wanted to send a move, and then just take the application data and send it down the connection.

So that's all we need to do. I think we're ready to play the game and I think we're going to need some help to this. So Eric, come back on stage.

All right.

So here Eric had already started hosting a game but I want to be the one hosting this one.

Here we go. So Eric, how about you type in that passcode, 5176. Don't tell anyone .

All right. So now we have a secure connection set up. It's using TLS. And when Eric selects a character-- he chose birds-- he's sending the select character message across and I call receiveMessage over here, I receive that he selected that, I'm going to choose to be monkeys, why not. At this point, Eric will select a box to play and choose a character.

All right. So he sent the move message over to me. I can receive a message. I get that it's a move. I know where he's placed it and I can decide, all right, there's a monkey up in the top corner, what's he going to do next? Uh-oh, he's looking like he's racing for the win. I don't know. I can't look at this. Tic-tac-toe is hard. It sounds like birds got the day. But as you can see, it's really easy to build this type of game and there's hopefully a lot more nuanced games you can build with this and lots of other applications. So we're really excited to see what you do.

All right.

So before I move on, I want to make one last comment about framing protocols. So, many of you have been asking how you can use techniques like STARTTLS with your NWConnections. So STARTTLS is a technique that comes from the SMTP mail protocol. And it allows you to talk to a legacy server that you don't know whether or not it supports TLS and secure connections and do an initial handshake with it. And then if TLS is supported, you can add it part way through your connection. Now there wasn't a good way to do this before, but framing protocols gives you a great solution.

So, if you create a STARTTLS framing protocol and add it into your connection, when your application starts, you can begin a handshake with the server to determine whether or not it supports TLS. And then we allow you to dynamically add in other protocols onto the stack above your framing protocol before you call ready. This way the application can remain unchanged and not have to worry about adding TLS part way through. It can just happen automatically with your framing protocol. So we think it's a really elegant solution.

All right.

Let's move on. So, we've talked about Bonjour, how we can make better peer-to-peer connections and use wide-area discovery. We talked about framing protocols.

But now I want to take a bit of a step back and look at how you can collect metrics about the connections within your app.

So collecting metrics is really, really critical.

It allows you to validate that when you add new features into your app or onto your server that are supposed to help you get better performance that they're really doing it, like you're having the effects that you want.

But it also helps you identify problems that your users may be encountering out in the real world that you don't notice on your desk. So this year, we have a lot of great new metrics to help you analyze your connections even more. URLSession already has many fantastic metrics but you'll be able to get even more. And for the first time in Network.framework, you will inspect your connections to understand many aspects of their performance.

So in URLSession, you can already get a timing breakdown of DNS, TCP, TLS, and HTTP messages within your app.

Now you'll be able to introspect even more connection properties along with the amount of data that you're sending for individual requests and responses.

And in Network.framework, you'll be able to access a connection establishment report that summarizes everything that happened during your connection bring up along with data transfer reports that allow you to look at the performance of individual periods of time over your connection. And you have multiple of these running at the same time. So let's start with URLSession. As a reminder, all of the metrics in URLSession are available in the didFinishCollectingMetrics delegate call.

So, here are some of the new things that you can access are the endpoints of the connection, the local and remote addresses and ports. You can also check out the security properties. Are you using TLS 1.3, the latest most secure and most performant version of TLS? You can also check the path properties.

So, this tells you things like did your connection use a constrained low data mode network, or did you use an expensive cellular network? The equivalent metrics within Network.framework are in the establishment report. So this is available to you any time after your connection has moved into the ready state. And this gives you a breakdown if your DNS times, your protocol handshakes for TCP and TLS, as well as whether or not you used a proxy. Here's how it looks in code. So you take your connection and you call requestEstablishmentReport. And this takes a queue on which it will deliver the report.

Once you have that you can check the overall time that the connection took. You can check the individual resolution steps. So if you were connecting by a Bonjour name, it may be resolving Bonjour names into host names and host names into addresses, and you can look at the timing breakdown on each of those steps. And you can also look at the individual timings for a TCP, TLS, as well as the round-trip times they observed. One point that I want to highlight that's really important to the overall performance of your connection establishment is the amount of time that it takes to resolve DNS and the source of where your DNS resolution came.

So, many servers have a very short time to live configured on their DNS records. And they do this such that if a server goes down or the server wants to load balance over another IP address, it can quickly change the IP address record and have clients adjust and start using the new address.

The downside, though, is that this really can hurt client performance.

With a short time to live, a client will almost always have to take the round trip to do DNS, to request the address for the host name that you are connecting to. And this can be particularly bad for clients that have a high latency link. This is going to add hundreds of milliseconds or even seconds on to their connection times. And the worst part is most of the time, the server address has not changed at all, and so this is a wasted round trip.

So, optimistic DNS is a solution that we released last year that solves this problem.

Optimistic DNS allows your connection to optimistically connect to the last known good IP address for that host name, in parallel with issuing a new query for the host name's current address.

If nothing has changed, which is almost always the case, the connection will just establish to the old IP address. But if something has changed you'll still get the new IP address and connect to it instead. We have been doing a lot of measurements on this and testing and it is a really great solution. And so this year, it is on by default for connections using Network.framework and URLSession.

When you're looking at your establishment report, you can tell whether or not you used optimistic DNS by looking at the source. And if it says it's from the expired cache, that means we ended up using and benefiting from optimistic DNS. I want to show you a bit how you can use metrics to look at the performance of your connections and the benefit of optimistic DNS and TLS 1.3. OK. So here I have an application that's a very basic app to collect connection metrics. All it does is run a probe to a given website. All right. So here it is. I clicked Run Probe and I connected and it's pretty fast. I'm doing this on a great Wi-Fi network. But if you want to test a more realistic scenario or see the effects of different network conditions, you are now able to within the devices and simulators panel of Xcode access device conditions and simulate different network link conditions. So you can see what it's like potentially for your users in different scenarios . Yes, it's great.

So let's see what it looks like to have a high latency DNS link.

So I clicked Start and you can tell that it's running because I have this gray box up in the upper left hand corner.

So now, let's run that probe again. So it was fast. That's great. But you'll notice that it came from the expired cache. So this means that we ended up using optimistic DNS.

So optimistic DNS is on by default but you-- we let you turn it off if you don't think it's appropriate for your server. So let's run the probe again. You can feel the seconds go by.

So, this is potentially a bit exaggerated. Hopefully your users don't have three seconds of DNS latency but it can make a huge difference. Let's go to a bit more realistic scenario now, something like an average 3G network. I'm going to start this and run the probe one more time. So it wasn't quite as snappy as the first time I ran it. Overall you can see that I have about 600 milliseconds for the connection to establish. And TLS alone took around little bit less than 300 milliseconds, so about half of that time. So our server is configured to support TLS 1.3. Now TLS 1.3 generally only takes one round trip to do the full handshake. It's great improvement.

But if your server doesn't support TLS 1.3, if it only supports TLS 1.2 or if you're using an API on your app that doesn't support TLS 1.3, you may see scenarios more like this in which TLS alone is now taking over 500 milliseconds, taking an extra round trip.

And you can see that the connection time is almost-- it's over three-quarters of a second, almost at a second. And if you have many connections, this can really add up to perceivable user latency. So we encourage you that whenever you're testing your app, do a run through network link conditioner and try out some of these scenarios and validate that your app performs well.

So the other category of metrics, have to do with the data transfer after the connection is established.

So in URLSession, you are now going to be able to access more metrics about the number of bytes that you sent in the header and the body of your requests as well as the number of bytes that your receive in the headers and bodies of your responses.

And this is really important if you are choosing a different URL to download less data in a low data mode network scenario, use this to validate that you're actually saving your user's bytes.

In Network.framework, you can now access a data transfer report that summarizes the performance in terms of bytes and packets and round trip times for a given period of time on your connection. You can have multiple of these running at the same time and they should correspond to your application's activity. So if you send a burst of traffic, have that be within a data transfer report. And it's not as interesting to take reports of idle periods. The way that you do this is that at any point you can call start data transfer report on your connection. This begins gathering data about your connection's performance.

And when you're done sending a bunch of data, you can call collect. This will summarize all of the data and give you a report.

Now, if you're using multipath protocols, this will give you a breakdown of the amounts that were sent over each link that the multipath protocol was using.

But many of you may be interested just in the aggregate path report.

Here you can look at the number of packets that you sent and received, the number of bytes that were transferred, as well as the round-trip time details that you observed.

So this is metrics we were really excited to see people adopt more metrics and help improve the performance of their apps.

And to leave us with some great advice and new updates, I'd like to invite Stuart up to the stage. Thank you, Tommy.

It is my pleasure and privilege to present the wrap-up for what has been two hours of really great networking information from my fellow presenters.

I'm going to start off with iPad apps for Mac.

I know a lot of you are excited about this. When it comes to networking, there are very little differences on Apple platforms. One thing you will want to be aware of is in your Xcode settings, when you check the box for Mac, you will now see some new options. By default, outgoing connections are allowed but if you want incoming connections for your app as well, you have to check that box.

On watchOS, we have new networking capabilities.

Applications that do audio streaming using AVFoundation can now use direct networking, as long as they're using URLSession or Network.framework.

Sockets is not available.

We are also introducing TLS 1.3 which gives you lots of benefits.

TLS 1.3 has better connection performance. TLS 1.2 typically has two round trips to set up a connection. TLS 1.3 almost always does it in one round trip.

TLS 1.2 used cryptographic algorithms that were believed to be good at the time but have since been shown type of weaknesses. And this is not just an academic concern. These have been exploited in practice.

Those have all been removed in TLS 1.3 and all the cryptographic algorithms in TLS 1.3 support authenticated encryption with associated data and forward secrecy.

And finally, you all know that privacy is very important to Apple. TLS 1.3 has much better privacy. Many of their header fields and certificates in TLS 1.2 was sent in the clear. Those are now all encrypted in TLS 1.3. So, the call to action is start using TLS 1.3 on your applications and of course, make sure your servers are updated to support TLS 1.3 too. Now, you all know the importance of privacy to Apple. And one of the things we realized is that accessing Wi-Fi information can be used to infer locations. So starting now, to access that Wi-Fi information, you will need the same kind of privileges that you need to get other location information.

The first step is in Xcode you have to add the capability to access Wi-Fi information to add the entitlement to your project, and then your app must meet one of three other criteria. If the user has given your app location access, then you can access the Wi-Fi network information.

If your app is the currently enabled VPN app on the device, you can access the information. And finally, if your app is in any hotspot configuration app, then it can also access the information but only for the networks that it has configured.

For more information you can also see the Wi-Fi framework. You've heard many times today I'm going to finish with a reminder about the importance of using the network link conditioner.

It's very easy when you're developing your application in the simulator on a Mac with gigabit ethernet in it or talking to a local server on loop-back. When you have a server with zero latency and infinite bandwidth, it's not surprising that it performs well. But if you build your application that way, it can be very misleading and you can find out later when your app is in the hands of real users that it doesn't perform very well. If you get in the habit of going to device conditions and selecting a realistic network link condition, right from the start when you're developing your application and always test and run your application simulating realistic network conditions, then those bugs will not even happen in the first place. Another message we've been giving you for many years is to avoid pre-flight checks. Using constraints such as allow cellular or allow expansive networks gives you much better control. It's much easier to use. Once you start writing your apps this way, you'll wonder why you ever did pre-flight checks. And besides, pre-flight checks can never work reliably because they always have raised conditions.

So to illustrate that, I'm going to use an example. This is an app that I really like. And to illustrate this, I've given them a deliberately exaggerated example of what it might do if it was a badly written app. This is telling the user to make sure that we're on Wi-Fi then click the button. But the user has no way to control what path the network connection will take, what they will typically do is look for the Wi-Fi bars and hope for the best. But as you learned today, knowing in advance how Wi-Fi is going to perform until you try is impossible. And the device may think it's on Wi-Fi but when it tries to use it, it turns out not to work. Now, when Wi-Fi Assist switches you from Wi-Fi to cellular, those Wi-Fi bars will disappear but by then it's too late, your connection has already happened. So, don't make the user guess. Don't just make connections and hope for the best. Let me show you how this application actually works. It makes its connections constrained to not allow cellular access. And starting in iOS 13, it can actually use the allowsExpensiveNetworkAccess control to let the system decide which is an expensive network.

It also sets waitsForConnectivity equals true. That means the application doesn't have to retry repeatedly. The system will just wait patiently for as long as it takes for that connection to succeed.

When the application tries to connect, if there is no Wi-Fi, its taskIsWaitingForConnectivity delegate gets called and that's when it can display UI giving the user the choice either move to somewhere with Wi-Fi or you can press the button if you want to go ahead and use cellular data.

Some news about deprecations.

If any of you are still using PAC files using the file or FTP URL schemes, those are no longer supported.

If any of you are still using SPDY, SPDY was a great experimental protocol, that has now been replaced by HTTP 2, that's what Apple supports and that's where everything should be moving towards. And Secure Transport does not support TLS 1.3 and it will never support TLS 1.3 so another reason to move to URLSession or Network.framework. So to wrap up, this morning we talked about wide-area Bonjour discovery and how to advertise a tic-tac-toe game. Some of you may have wondered if that service type underscore tic-tac-toe was registered with Ayana. The answer is yes it is, you can check on the website .

Tommy talked about building framing protocols and collecting metrics that make it easier for you to write your applications and make it easy for you to measure performance. And this morning, we talked about low data mode that lets you respect your user's wishes about when to conserve data. We talked about combining URLSession which is a great way to chain asynchronous operations together. And we talked about WebSocket. If you have web-based applications that use WebSocket to talk to the server, you can now use that same server with your native iOS apps.

And Christoph Paasch told us all about mobility improvements with multipath TCP and Wi-Fi Assist.

And on that note, many of you will know that ACM SIGCOMM is the world's leading academic conference for network research.

And every year they have the Networking Systems Award that recognizes the work that's had the biggest impact in the area of networking. And this year, today, they announced that this year's award goes to Christoph Paasch and the rest of the team for multipath TCP. We would love to see you all tomorrow in the networking lab. And if any of you are currently writing network kernel extensions, definitely go to tomorrow's session about network extensions for modern macOS. Thank you. [ Applause ]