Continuous Integration and Code Coverage in Xcode

Good morning.

Welcome to Continuous Integration and Code Coverage in Xcode. My name is Matt Moriarity. I am an engineer on the Xcode team. I am really excited to be here today to talk about some tools we have in Xcode to help you get more out of testing and hopefully motivate you to write more tests.

Today we are going to start by talking about what Xcode Server is. Xcode Server is a continuous integration product that we bundle with Xcode.

Then we will step into what's new in Xcode Server and Xcode 7, with a special focus on the new code coverage feature that we introduced this year.

Then we will spend the second half of the session talking about some more advanced features of Xcode Server that will allow you to integrate it into some of your team's unique workflows.

And of course, all throughout, we will have demos showing you how all this great stuff is done.

So let's jump right in.

So Xcode Server is a feature we introduced in Xcode with Xcode 5, and it's all about supporting a process known as continuous integration.

And continuous integration is all about improving collaboration with your team to allow you to build better software.

So what does that mean? It means pulling down all of your code regularly and then building and testing it and surfacing issues like build errors or test failures as soon as possible so you can fix things right away.

Now, there's a lot of ways you can do continuous integration out there, but we think Xcode Server is especially great for app developers like you for two reasons. First of all, it's really easy to set up. Thanks to integration with OS X Server, if you've got OS X Server and Xcode on your Mac, you are minutes away from having a continuous integration server testing your project regularly. And second, Xcode Server has deep integration with Xcode itself. We know a lot about how Xcode projects are built, how we work with devices and things like that, so we can ask you as few questions as possible to get up and running with your code checking out regularly.

So before we go any further, I want to talk about a few concepts that we talk about when we are talking about Xcode Server.

The first is something you are probably familiar with even if you have never used Xcode Server before, and that is a scheme.

Every time you run your project or run tests in Xcode, you are running a scheme. A lot of times they are auto created for you, but you can create your own custom schemes, and they basically form a recipe for building your project, so they tell you what targets to build, what test bundles to run, arguments to pass to your executables, things like that.

Schemes are important for Xcode Server when it comes time to set up a bot.

Now, a bot is -- we like to think about it as having another member of your team, that it's basically taking a particular scheme and building and running it on the schedule you define and doing exactly the actions you tell it to and then reporting those results back to you. And each time that schedule goes off and we run your project, we call that an integration. It's like the act of integrating all the changes from every member of your team together and seeing how everything comes together.

Now that we are all on the same page, let's talk about what's new in Xcode 7 and Xcode Server.

To start, if you've used Xcode Server before in Xcode 6, you know when you go to edit one of your bots, we would take you through the entire workflow for creating a bot again, just with a lot of the values already filled in.

Now, that's a little tedious when you want to make a simple change like adding a trigger, changing your schedule, something like that. Now we have this tabbed interface that allows you to get right in, make the change you want to do, and get out.

A lot of fans of tagged workflows out there.

We've also made improvements to source control in Xcode 7. So whereas previously we would kind of try to automatically handle all your source control and hide the details from you, we now surface more of that to you so you can see exactly which repositories your bot is going to check out, and you can select to not include some of those. And for the repositories you are checking out, you can now see and choose which branch you'll check out instead of just - Thank you. Instead of just hoping that Xcode figured it out correctly. We also made improvements to security in source control, specifically when it comes to SSH fingerprinting and self-signed certificates. Both of these are not automatically secure methods of transportation. They require you to trust the server you are connecting to, so that if the server changes later, you will know that they have a new fingerprint and may be impersonating the server you thought you were connecting to. Previously Xcode would automatically trust the servers and not do any verification. Now we require you to explicitly trust any of these servers and then we store the fingerprint so if it changes in the future, we won't check out from the wrong server. And we've also updated many of the reports that you see in your integrations. So the test report has been cleaned up and compacted.

It's now easier to see any assertion failures that come up in your tests.

And the logs view has gotten drastic performance improvements. Previously we would try to show you a stacked view of all of your logs, but that has some significant performance penalties when you try to expand one of those logs. We now show you one log at a time that you can choose from a pop-up button in the left corner, and now viewing large log files is super fast.

One of the other great things that makes Xcode Server special when it comes to continuous integration is it knows what kinds of issues Xcode projects can produce and knows what a build error looks like, a test failure. It's not just showing you some raw plain text log file and having you go through and figure out what actually happened.

So we do a lot of smart things when it comes to these issues in order to surface that and make that issue useful.

When you run an integration -- this is true in Xcode 6 as well -- we show you this nice report showing you all the issues that came up in your build.

And we surface which ones are new because we can compare them to the previous integration, and that allows us to pinpoint exactly what integration and thus which commits introduce a particular issue, and that makes it much easier to find the cause, track it down, and make a fix. We have added some new stuff in Xcode 7 around issues. Now if you see an issue that comes up and you either know it was your fault and you want to go fix it or you can see just from looking at the issue, like I know what's wrong there, I can go in and fix it real quick and get this cleared up. Then you can claim issues, and that puts your name on it so everyone on your team will see that when they go look at the report, and they will know that they don't have to worry about it; you are on the case. For issues that are either intermittent or know have been alre fixed, you can silence those issues for a certain period of time, and they will disappear from the report.

That allows you to focus on exactly the things that actually need your attention without cluttering it up with things you know are already handled.

But one of the best things about Xcode Server is how well we interact with all the other features that Xcode introduces. And since there are some great new features in Xcode 7, we have taken the special time to integrate those features in Xcode Server as well. I want to look at a few of those.

So user interface testing is new in Xcode 7.

And we've taken special care to make sure that works perfectly in Xcode Server.

When you run Mac tests or iOS simulator tests, we create a full window session in the background on your server. That's where all of your tests run. That means you don't have to worry about is it going to be the right environment for Xcode 2, launch my application, we'll make sure it works fine. If you are using real iOS devices, then you will see the UI test stepping through the application right on the device.

So user interface testing is a great way to test your application at a high level, the same way your users will see it, and test all the different layers interacting together. It's even better when you have a server running those tests across multiple devices at the same time. And on a schedule.

On-demand resources is a new feature in iOS 9 that allows you to make your apps bundle smaller by not storing as many resources in the bundle. Instead the App Store will host them for you when your app is on the Store.

Then your application can download those resources when they are needed and they can be removed from disk when you are done. Now, that's great for when your app is in the App Store, but when you are doing QA internally before you release and you need to test out the builds of your application, what happens to the resources then? The App Store is not going to be hosting them then. You are changing too fast for that to really be practical.

But if you get your QA builds from Xcode Server, if you let your integrations produce IPAS that you install on your devices, then this is handled completely automatically for you. You don't have to check a box, you don't have to do anything. Xcode Server will see you have on-demand resources in your application, it will host them on its own server and will teach your application where to find them.

Finally, I want to dig deeper into another one of the new features in Xcode 7 that goes great with Xcode Server and continuous integration, and that's code coverage. So code coverage is a tool that's all about measuring the value of your tests, specifically, we want to know what code is actually running when we run tests.

Because it's easy to go about building up a big test suite for your application and feel like you are resilient to change and that regressions aren't going to come up that you won't notice.

But how do you really know how many tests is enough tests? Say I've got an application that's got 2,000 unit tests, but only 20% of my application is actually getting run by those tests? Well, that's still not as useful as I might think it is. So code coverage is all about surfacing this information to you so you can make informed decisions.

So it allows you to run your tests and measure exactly which code is getting exercised there and, more importantly, which code is completely untested and, thus, that's code that could have a regression as you go adding new features, and you wouldn't know about it from your tests. So we think code coverage is really important for teams that are serious about testing, and that's why we've integrated code coverage into Xcode.

So like most great Xcode features, code coverage is built off of tight integration with LLVM.

So when you have gathering code coverage enabled in your scheme, the compiler will instrument your code so we can count how often each expression is executed, then we will surface this information to you right in the IDE.

Now, there's two ways we are going to do that.

The first is if you go to the report navigator for your test, you could do this in Xcode 6, but now in 7, there is a new tab will labeled "coverage." If you look at that report, you can see by target and then file and then method exactly how well covered the different parts of your application are.

This is a great way to look at your application at a high level and then drill down and identify which parts need your attention.

But if I get down to finding a method that has 75% coverage, well, that's good to know, but I don't really know what I need to know to fix that. I don't know which branches of my code are tested and which ones aren't.

So if I hit the arrow that shows up when you hover over one of these methods or files, it will jump right to the source code editor where we show inline annotations highlighting exactly which parts of your application are uncovered. For the parts that are covered, we'll tell you how many times that executed in your test.

So code coverage is really great on its own in the Xcode IDE, but I think it gets even better when you put it on Xcode Server where you have a bot running your project over time across multiple devices.

So that's one of the things that's special about integrations and bots is that you can set them up to run on a suite of devices instead of just one at a time, like you do in Xcode.

So when you do that, we'll show you the coverage data for all of your devices together, and we'll highlight in orange any methods or targets or files that had different coverage across your devices, so you can identify where things are different and decide if that's a bug or maybe that's expected behavior.

It could be fairly common for you to have code coverage that's different on different devices, especially in user interface code, where you might be doing something different on, say, an iPad versus an iPhone. One of the things that Xcode Server provides you as well is a history that's stored and tracked along the life of your project. This allows us to, when you are looking at the code coverage for an integration, highlight when changes happen and exactly which methods and files had those changes in coverage so you can pinpoint that down to a specific set of commits.

This history also allows us to show trends, so in Xcode 6, we had build history and test history charts for your bot. That shows as you're adding more tests, you can see the chart grow, and you can see how well your issues have been going and how stable your bot has been. But now we also have a new code coverage graph that shows the overall coverage of your project trending over time.

Now this is great to know if, for instance, if it's improving, then you know as you add features, you are either adding tests for those features or you're adding tests for features that previously had no coverage.

Or maybe it's trending downward because you are moving a little too fast and not testing the features you are adding now. This can help you make informed decisions going forward about what you want to do and how you want to spend your development time. And of course, if you are using big screen in your workspace to show your overall bot status, right below the number of tests in your project we'll also show you your overall coverage percentage, so you can keep an eye on that.

All right. So now I'd like to bring up my colleague, Eric, to demonstrate some of the code coverage features in Xcode and Xcode Server. ERIC DUDIAK: Thank you, Matt. I am going to show you an application that we've built for internal use that we use to track coffee owed from one person to another. You may have seen this application last year, we've made a few enhancements to it since then.

Basically, the rule is you owe someone coffee if they fix a bug for you or otherwise owe you a favor. So we have a whole application to track that. And like any good application, we have unit testing to make sure as we are adding features to it, we are not impacting existing working code. So having you test those is really only half the battle. I don't know how many unit tests I am comfortable with in terms of when this application is actually fully tested, and I want to look at coverage and see what we have here.

Because maybe we don't have that many tests. So let's take a look at it. I actually ran the test earlier before I got up here. We can just take a look on the device. We only have about seven tests. That's pretty bad.

But it's a small application. Maybe that's enough to cover everything. They are all passing at least, so that's a good start. Let's look at coverage. So if we look at the coverage report here, we see the application is broken down into two targets. We have a UI-level application, coffeeboard.app, which doesn't have much coverage.

But that's okay. It's a UI application. I should write some UI tests for that. I am more concerned about this foundation level framework. Here we see only about 50% coverage. Now, that's pretty sad because this could be 100% covered if we tried hard. And it looks like the class that's really falling behind is this transaction class.

And if we look at the transaction class, we see we have a bunch of methods in here that aren't getting called at all in our unit tests. We are creating a few transactions, and we can see that in the initializers being called, but we are never actually doing anything with them. So I should explain a little bit about our app. It has a really neat feature, which is using a proprietary algorithm that we have. If I owe Matt two coffees and he owes me one coffee, we actually merge that down to just a single coffee that I owe Matt.

Really secret algorithm that I want to make sure is very well tested, and also I am not that good at math, so I have to make sure.

So let's see and look at the actual source file for this, and just like Matt said, I am going to use the arrow that shows up on hover and jump straight into my source editor.

Here we see a whole lot of these dark, highlighted sections. Anything that's using the default background of my source editor -- white in this case -- is code that's already covered, and I don't really need to worry about it too much. I am really more concerned about all of this code that's showing up with a gray background. I can confirm it's uncovered because on the right side here, we see a bunch of zeros indicating that this code was never passed over inside any of my unit tests.

So that's not good.

Now let's navigate over to the unit test and take a look at this.

If I go to the transaction tests, I see that I just don't have any tests. That's kind of a problem. So let's fix that right now. I am going to go ahead and create a little blank space here and write in a little bit of Swift.

And of course, I assume you all have magic macros that add all your unit tests for you. That's how we all develop, right? I am going to go ahead and run the test on the device. It's going to take a second. As Matt said, we are using LLVM to go ahead and instrument your code while it's running so we know exactly which expressions are being run.

When that runs on my device, I am going to look at the scheme. One thing to keep in mind, since it is a feature of LLVM, it's optional in Xcode. The way I turned it on is went into the scheme editor and selected the test action. I then made sure that check box for "gather data coverage" is selected.

That ensures I get coverage data.

Perfect timing. All my tests succeeded. Great, they always do when I build them with magic macros.

If I look at the test report now, I see I have more unit tests. But that's not really the whole story. Let's look at that coverage report again. Here we see a much prettier picture. If I zoom in on that, we will see I am now at 76% covered, not all the way there but doing a lot better than we were before. I'm going to go ahead and disclose the transaction class again, and here we see a lot more of these methods are getting covered. I am testing that merge code that I was worried about earlier and was causing me to lose a lot of sleep. Now one interesting thing, if we go back to the source editor, if we look at the is equal method, if you noticed before it was only partially covered. We can see here exactly why that is. If we look at the is equal, we don't actually compare our transaction class to something that isn't a transaction class, so this return false is never getting called in any of our unit tests.

So being able to see this coverage as you go through different branches is very useful if you have a lot of branching logic in your code that you know has edge cases. This way you can ensure that every edge case is covered when you are writing your unit tests.

We can also see a lot of this code is getting covered multiple times, and that's really important to make sure if you know there are multiple cases that hit the same path in your code but ultimately are different, you are getting coverage on all of them. Now, everything I just did was a bit tedious, and I wish I could just have someone run these unit tests all day not have to worry about it, not to compare them one to the other. I was told there wasn't money in the budget for an intern to do that.

Let's look at Xcode Server. I already set up a bot for this and it's on a different branch that already has the commit, so let's look at the bot. Here just like Matt was showing in the slides earlier, we see a high-level overview of my project as it's been running for the past 24 hours and over the past amount of time I have been running this bot. At the top we surface high-level statistics, and you can show the statistics for any period of time, week, hour, month, year, or since the beginning of your bot.

Then we see build history. This is where we surface errors, warnings, analysis issues over time. In this case, I had a few warnings earlier, but I've gone ahead and fixed those, so we are currently showing no issues.

The next two graphs are probably most important for continuous integrations, assuming your project is building cleanly with no warnings or errors. And that is your tests. In this case we see I had a few test failures earlier but I've been steadily adding tests.

That's great, but what we really want to see is a steady increase in tests all passing, but also a steady increase in code coverage with those tests. If you are adding tests but not adding coverage, you are not really adding as much value as you might think you are.

If I look at the coverage, the last integration -- this is that commit I just made you didn't see, but I promise I made it -- we can actually go straight into the coverage report.

So this looks a lot like what we just saw in Xcode when I ran my tests locally.

Just like in Xcode, we break it down by target and across different classes. And I can expand that transaction class and see the exact same methods getting the same level of coverage. But there are two big differences. So in this case, I can see change over time right in the report. So instead of having to go through two different reports to see whether I actually increased the coverage, especially if it's a less drastic change than the one I just made, we can surface that right here. So my coverage is now 22% better in CB foundation.framework than it was before.

And in particular, that transaction class increased 48% more covered.

Now we highlight down here in orange an interesting thing, which is difference between devices. And actually, in Xcode Server's reports, I can click this checkbox and show the device differences and highlight that immediately. Now, in this case, it looks like this detail view controller isn't showing up at all on my iPhone. That's actually to be expected. In this case, our application uses the split view, and that second view controller doesn't show up on the iPhone unless someone actually were to tap it. And our unit tests don't exercise that code. This isn't particularly unusual, but it's important to make sure that every time you see something like this, that you expect it to happen. That's why we make it very easy in Xcode Server to see differences between different classes of devices.

It makes it very easy to determine when code that should be covered isn't. That's continuous integration showing code coverage. With that, I am going to let Matt talk about some of the more advanced features of Xcode Server.

MATT MORIARITY: Thank you, Eric.

So like Eric said, I want to talk about some more advanced features in Xcode Server.

I know that many of you developers are very interested in extending Xcode Server and integrating it to parts of your team's workflows. We know Xcode Server is not the only tools you use to get things done, so we want to provide ways for Xcode Server to work with everything you use and be a perfect fit for your tetam. So we have two ways I am going to talk about today in terms of integrating Xcode into everything else you are using. The first one is triggers.

So triggers were introduced in Xcode 6, and they are all about providing custom actions that integrate in the lifecycle of your bots and your integrations.

So triggers can either be email notifications that provide details and configurable information about your integrations and how they ran, what issues they produced, committers, things like that. Or more interestingly, they can be arbitrary scripts written in the programming language of your choice.

So by default, we will run these scripts using Bash so you can type any old shell command into your triggers and that should work. But if you include a hash bang at the top of your script like you would if you were writing a command line tool, then we will use that, and you can use any interpreter you like. You can even write your triggers in Swift if you so choose.

Eric will demonstrate that later. A trigger can run either before or after your integration runs, and each of those phases provides a unique opportunity to do something cool with your bots.

Triggers that run before integration run after your source code checks out. That's important because it means you have full access to your project and you can make any automated changes you want to do before the build actually happens. And triggers that run after integration can be gated on the result of that integration, so they can run perhaps only on success or only when test failures happen. And they also have access to a lot of information about what happened in your integration. And one of the ways we provide access to that information is through environment variables.

This is just a sample of some of the variables that are defined when your scripts run. Any scripting language worth using is going to give you an easy way to get at those environment variables and use them.

So as an example of what you can do with these, last year we demonstrated a trigger that posted a message to a jabber chat room every time an integration completed. We used the bot name, integration number, and result to do that.

It was really easy, very quick to set up.

There's two things up here that I think I want to call your attention to, because they seem a little weird and out of place.

So we have a bot ID and integration ID for the integration running and its bot. That's a little weird if this is all we give you. You can't really do much with just an ID. It's an arbitrary string of characters.

What good is that? No user wants to see that.

So they don't do a lot on their own but are very interesting when combined with the Xcode Server API.

So the Xcode Server API forms the underpinnings of how the Xcode ID and Xcode Server communicate. You can also harness this API for your own use and do some pretty interesting things with it.

So like most web services APIs, it's built off of open, well-established standards.

We use HTTPS to do secure communication back and forth between client and server. And we use basic authentication over that encrypted channel for authentication. Our API follows a REST pattern, so it's all about interacting with resources, like bots and integrations and using the standard HTTP verbs like get, post, patch, and delete, to perform operations on those resources.

And of course, we communicate data back and forth using JSON. It's easy to use, it's easy to parse, it's the lingua franca of Web Services. Everybody is using it.

This is great news for you, the aspiring extender of Xcode Server because it means no matter your preferred programming language or environment, you surely have a way to speak HTTPS and JSON, which means you can make almost anything talk to Xcode Server. Let's walk through how we might use this API. So what's the simplest question we could ask our server? Let's start with what bots are on my server? And this is very easy to do. We make a get request to the bots resource.

If you look at the URL there, you will see that we communicate over port 20,343, and all our API requests have a /api prefix.

It's not important here, but I want you to remember that when you try out the API yourself on your own server. When we make that request, we get a JSON object that tells us the count of the results we have and gives us an array of the results themselves.

Each of these JSON objects represents one of the bots that's set up and configured on our server.

Like most resources, bots have an ID, they have a name like you gave it in Xcode when you created it, and they have all the configuration parameters that you set up when you are setting up your bot.

All right. So what's something else we can do? Now that we have a bot, let's see what integrations have run for this bot.

So now we access the integration subresource for that bot.

And we do that, we get a very similar result as what we saw before, except now the results are -- represent integrations, not bots.

But integrations are resources as well, they also have IDs. One interesting thing is that we also keep a snapshot of the bot on the integration, and that's important because as your -- your project changes over time or maybe you adopt new Xcode features, things like that, the bot configuration might change when you edit it, so we store a snapshot on the integration so that you know exactly how your bot was configured when that integration ran. So we also keep track of the step in the build process that your integration is at. If it's completed, you will see a result there, and also a breakdown of all the different issue types that your integration can produce and the changes from the previous integration. So these are both get requests I've shown you so far. That's great for collecting information, but if you are using the API, you probably want to do something with it.

So what's something we could do to actually have an impact on our server? Well, we could trigger an integration manually. Maybe we have some kind of other automated process that's not the built-in scheduler in Xcode Server, and we want to use that to trigger integrations for certain bots.

That's easy to do. We can use the same URL and change our HTTP method to a post.

When we do that, we've gone from saying I want to list the integrations for this bot to I want to create a new integration for this bot. If we make that request, we get an object representing an integration, much like you saw before, but a lot lighter this time. There's a lot of properties that get set on the integration during the process of building, but this integration is only in a pending state until the builder picks it up in the queue and actually starts running with it. Now, most of the post end points in our API actually require you to put some JSON data in the body that says this is the properties of this resource, the attributes I want set on the thing I am creating.

Integrations are a bit of a special case here because bots are essentially already that. They are already the template for each new integration, and so they know everything they need to know to create a new integration.

Or at least they know everything they need in the general case. You can have a bit of an impact on your integrations. For instance, if you want to have an integration that specifically runs cleanly with no leftover build artifacts, then you can do that by passing some JSON in the body to tell that this integration should clean before it runs. All right.

So that's an example of some of the things you can do with our API. Now I'd like to bring Eric back up to do a demonstration of some of these advanced Xcode features.

ERIC DUDIAK: Thank you again, Matt.

So like Matt said, we are going to show some of the advanced features of Xcode Server with this project.

Now, like many of you, we developed this app both internally, but we also distribute it, and I want to be able to easily differentiate my internal builds from my external builds.

Now, one common strategy for that is if we look at our Asset Catalog, we see two different images. In the first case, we see the standard app icon, which is this white coffee cup. In the case of our internal builds, we want to always show this icon as an internal flag and a black coffee cup. This way when we are running internal builds on our device, we know that they are internal and not the ones we might have shipped through the App Store or somehow distributed through our normal channels.

It's very useful so we know when we find a bug, we are hopefully the only ones seeing it. Now, the other way we want to differentiate our internal app is that when we're running internally, we normally have a settings bundle, and in that settings bundle, we define, like many applications, a version, in this case version 2.0, since we've made some recent major changes.

Now, 2.0 is perfectly sufficient for our external users to see, we'll bump this version every time we submit to the App Store.

But for internal use, we probably want a little bit more fine-grain information. More specifically, we want some unique identifier for the build.

In this case, I think I want to use Xcode Server to actually distinguish which integration the build came from on my server. I am going to go ahead and go to the bot. And this is the same bot I was using before. I am going to go into the edit workflow. And just like Matt said, this is now completely nonlinear, so since I am most interested in the triggers, I can jump straight there. So let's take our first -- our first step is to get the icon changed. I am going to open up a trigger script I have already written. In this case, it's a simple batch script because all we are going to do is simple file operations. We are going to go ahead and use the XES source directory environment variable to figure out the path to that app icon, and we are going to go ahead and delete it. Then we are going to take our server version and move it into its place.

So this will be a before integration trigger because we want this to run after our source code is checked out and ready to build but before we actually do build. So I am going to go ahead and copy that as a run script trigger.

Now, setting up the settings bundle is going to be a little trickier. I could do that with a batch script, but that's editing a lot of batch script code manually, or letting a lot of plist code in a batch script would be painful. So I am going to go ahead and pull up a Swift trigger.

In this case, just like the bash trigger, I go ahead and set it up using the hash bang for user bin swift.

I haven't finished writing it, so bear with me while I do that. I am going to get import foundation. This is where I am going to get all the extra power I want from Swift.

With the foundation imported, I am going to go ahead and use NSProcess info to get those environment variables that I was using in the other script through bash, in this case, the XCS source directory. After that I am going to find the settings bundle I had. This is where things get more interesting with Swift. I can load the plist as a dictionary right in Swift. I don't have to do any manual editing of the plist file myself. That gets particularly useful when I want to add something to it, I simply just create a new dictionary literal in swift and go ahead and shove that into my dictionary.

Here we grab the environment variable for the integration number to set as this build number title. Finally, I am going to go ahead and write that out to disk before the script finishes so that when we actually go to build, that will be picked up in my settings bundle. And to do that, I am just going to add another trigger below. All right.

So I now have two triggers. Now, this is where I could push the integrate now button and get a new integration, but I want to use some of the API Matt was talking about on the server to do that. I am going to go ahead and pull up another application I have. And if we look here, we see an application that actually is connected to a serial device plugged into my computer.

And it's using some of the I/O kit sample code, and we are just going to read that buffer, and whenever we see something on it, we are going to go ahead and run through this block of code. So I am going to open up an NSURL session, and I am going to call the API for getting all of the bots.

This is just local host. The port Matt mentioned earlier, API/bots, and it's going to be the get method. I go ahead and call that, and using the NS JSON serialization API, go ahead and read in each bot and find the ID for coffee board bot. That's the bot I want to integrate.

With that ID, I am going to build up a new URL like Matt was showing earlier with the bots/my UUID/integrations and switch that request to be a post request, so this is going to create a new integration. For the purpose of this, we don't really care what the integration is going to do.

Now, in order to trigger this, I managed to sneak past security a fun little button, and this button, I am going to plug it into my Mac.

And I am going to run this application. Let's try that one more time.

Make sure it's fully connected before I run. Don't want to get ahead of myself. All right. Good. So that application is now running. We have the modem picked up. So I am just going to zoom in here on my bot and step away for a second, and just with the button, create a new integration.

So that was entirely with the button in our API. I am not going to wait for that integration to finish just to show you what happened, here we see the integration, I can install that on my local device I was using earlier to test.

Install. And if I open up QuickTime to see the -- so you guys can see what I see -- all right.

There's my device.

You see that we have the internal icon on the device, even though I never switched it locally. We downloaded it from the server, and now our app, we know, is an internal build. So that's some of the more advanced features of Xcode Server, and I am going to let Matt talk a little bit more about where you can find us in the labs and things like that.

Matt? MATT MORIARITY: All right. Thank you, Eric. That was great. I love pressing big red buttons to make integrations. All right. So here are some more of the endpoints we have available in our API, at least to you guys, developers out there. There's a few more that are available, but they're locked down so that they are internal use. We want to be really secure and make sure that integrations don't get messed with too much while they are running, especially with the client server interaction where you might have your server exposed in more places than just your network.

But I encourage you to reference this slide later when we've uploaded the videos and the slides and try out some of these endpoints yourself on your own servers and see what you can come up with and what's interesting for your team. All right.

So today we looked at Xcode Server improvements we've made in Xcode 7, both around quality and adopting new testing features that have been added in Xcode 7, especially the new code coverage feature that makes it really great for measuring how much of your applications code is actually getting used by your tests. And we, of course, showed you two great new ways to extend Xcode Server and integrate it better with your team with triggers and the Xcode Server API. So for more information, we have a -- there's a Continuous Integration Guide on the developer library, and of course, you can come post any questions to the developer forums. There's a few related sessions. Both of them have already happened, some more recently than others.

UI Testing in Xcode was yesterday, but I encourage you to go look at that video. We touched on UI testing but we didn't get to demonstrate how it works in Server. It is pretty automatic, but the UI testing feature is great and I encourage you to try it out. If you want to know more about getting started with Xcode Server, last year we demonstrated setting up your server from scratch and getting your bots set up initially in the Continuous Integration with Xcode 6 session.

Thank you, everyone. Have a great rest of your WWDC. [Applause]