Getting to Know Swift Package Manager

Hi, welcome to Getting to Know Swift Package Manager. I'm Rick Ballard. With me today is Boris Buegling. And we're very excited to tell you all about Swift's Package Manager, or as we call it sometimes SwiftPM. We're going to focus today on the open source project. And not on Apple's other developer tools. But we've got plenty to tell you about here today.

The Swift Package Manager makes it easier to develop and distribute source code in the Swift ecosystem. Today we're going to talk about its goals, some of its design, and a little bit about where we can go from here.

We'll start by telling you a little bit about why we decided to create a new Package Manager as part of the Swift project. We'll show you little bit about how to use it. And then dive into its design and features.

We'll tell you a little bit about where we can go in the future and close out by describing SwiftPM's open source process and how you can get involved if you're interested.

I'm sure most of you are familiar with package managers they're a great way to share and reuse code. But why did we decide to create a new one for Swift? First of all, Swift is a cross-platform language. So we wanted a great cross-platform tool for building your Swift code. This makes it easy to configure your code in a consistent way and run it on all of Swift's supported platforms. SwiftPM includes its own complete build system, allowing you to configure your software, build it, test it, and even run it from one tool.

We also wanted to make it as easy as possible for you to share your Swift libraries with anyone wherever they are by providing a canonical package manager in the Swift project, we hope to define a common standard for the way that you can distribute your libraries. This makes it easy to grow the Swift ecosystem and make Swift better for everyone. Many of you may have great ideas for features that you'd like to add. But we'd like to be careful about what is added to the core libraries so we can maintain a careful and curated API. A great package manager makes it easy to distribute these ideas as packages instead of having to put them into the core library. The best ideas can organically gain traction with the community over time and become increasingly standardized.

Finally, by building a package manager alongside Swift, we're able to take advantage of Swift's power and philosophy. SwiftPM is itself written in Swift and it's even a Swift package. Beyond that, we have the opportunity to work closely with the Swift language and core library projects to build great package manager features that will help your Swift code sing.

SwiftPM is part of the Swift open source project, and has its own presence on Swift.org, and on GitHub. The Swift Package Manager section of Swift.org is a great place to go to get started.

When you're ready to try it out you can find it included in every Swift toolchain, also downloadable from Swift.org. And of course, it's included in every release of Xcode alongside the Swift tools.

So, to start telling you a little bit about how to use it, I'd like to invite Boris Buegling up to show you the basics. Thanks, Rick. Let's take a look at how to use SwiftPM.

SwiftPM consists of four command line tools, and at the top level Swift Command. Swift Build, to build your package. Swift Run to run its executable products. Swift Test to run tests. And Swift Package to run various non-build operations on the package. Packages are stored in git repositories. And diversions are represented by git tags.

Next, I'm going to show you a demo of how easy it is to create your own first Swift Package.

We start in terminal, and we create a new directory, called helloworld. This will also be the name of our package. Switch to that directory and we will run Swift Package init with the type executable. With this, SwiftPM creates a basic package and structure for us. Let's open finder to look at it a little bit more closely.

We have the Package.swift manifest file, which describes the structure of the package. We get a basic README. You have the Sources directory with a subfolder for our helloworld target. And the main.swift file for our executable. You also get a test directory, where we could later put some unit tests. Let's switch back to terminal. And we will type swift run to build and run this package. This compiles the package, links executable, and we see helloworld is the output. Next, I'm going to switch to another terminal window, where I've prepared a more complex package. We will use this in the following to discuss the basic concepts of SwiftPM. But, first, let's also just run it to see what it does. So, you can see, it outputs randomly generated playing cards to the terminal.

Now, we can switch back to the slides to talk about SwiftPM's basic concepts.

A package consists of three major parts; dependencies, targets and products. And we'll look into each of these in more detail in the following.

Dependencies are those Swift packages that you can use when developing your features. Each dependency provides one or more products such as libraries that your package can use. Let's take a look at how dependencies look in the package of Swift manifest file. Each dependency has a source location and it is versioned.

Targets are the basic building blocks of packages.

A target describes how to build a set of source files into either a module or a test suite.

Targets can depend on other targets of the same package and on products exported from other packages, declared as dependencies.

Products are executable to libraries and products are assembled from the build artifacts of one or more target.

Packages provide libraries for other packages by defining products. By default, you do not have to declare the type of library explicitly, but SwiftPM will choose it for you based on its use. If needed, you can explicitly declare a library a static or dynamic. Let's take a look how our targets are configured in the manifest. In our example, we have three targets.

The first is called libdealer, and it contains the implementation of our main functionality. And it has one dependency, the deck of playing cards product which comes from the dependency we declared earlier.

Second target, dealer depends on that to provide the command line tool that we just run earlier. And finally, we also have a test target that depends on the two other targets and this is where we can unit test our functionality.

In our example package, we have also configured two products.

The first is a library product corresponding to the libdealer target. And this provides our implementation as a library for external consumption. Second, we have an executable target depending on the dealer target which provides an executable for command line use.

To complete this section, I will show you how we can use a package to add a new feature to the example.

This, we switch to a new terminal window and we open up the package.swift manifest file. We want to add a new dependency. In this case, this is actually SwiftPM itself. As Rick told you, it is itself, its own Swift Package. It doesn't provide a stable API, though, that is why we're depending on an exact version number.

We also want to depend on one of its products in the libdealer target. It is called utility. And among other things, it has a class called terminal controller, which provides us with the possibility to color the terminal output. Note however, that this is no official Apple API, we're just using it for the demo. Let's switch back to the terminal.

I already changed the code to make use of the new dependency before this demo. So, we can just run it to see the result. And as you can see, we have the same output, but now it's a little bit more fun with some colors.

I want to show you one last demo which is how SwiftPM can run tests. For this, we're using the Swift Neo package. A networking library that Apple open source earlier in the spring. We will run Swift Test with a parallel option.

This allows us to run tests in parallel. So, you get your test results faster. And we also pass the filter option.

This allows you to run a subset of tests to you can iterate on a single future.

This will now again compile our package and run the tests in just a few seconds.

And as you can see, we get a nice progress bar and the tests finish really fast, because we were running them in parallel.

Let's switch back to the slides again.

Next, I'm going to talk to you about the design of the Swift Package Manager.

SwiftPM follows Swift's philosophy. It is fast, safe, and expressive. It is safe due to its isolated build environment and the fact the builds cannot run arbitrary commands. It is fast due to using a build engine that is scalable to large dependency graphs. And it's expressive due to using the Swift language for the package manifest. And this also allows you to use a programming language you're already familiar with. For the rest of the section, I will take you on a journey through the different steps you will encounter when creating your own Swift packages. We will start with configuration.

As we saw earlier, SwiftPM's manifest is based on Swift. Using Swift makes it easy to understand because there is no new language to learn. We follow Swift's API design guidelines to make it even more familiar. And, it allows us to take advantage of existing tooling written for Swift, but when writing your own manifest, you should still prefer declarative syntax and avoid side effects.

Because SwiftPM makes no guarantees about when or how often your source code is being evaluated. On the left-hand side here, you see an example that is not really declarative. We cannot see the name that is being generated and it's used in a couple of times across the package. In contrast, on the right-hand side, we have a fully declarative manifest by using string constants. It is easy to understand and see what the targets are.

So, as you can see, not using declarative syntax also makes your manifest harder to understand for you and your users.

Source files organized on disks in folders named after each target in the package. This makes it easy to get started and allows packages to adopt a common structure, so that you can navigate them quickly.

Package Managers and other build tools often have attention between what is explicitly configured by the user and the conventions that are imposed by the Package Manager.

As I told you earlier, the source file is automatically picked up from convention base locations on disk, so that you can very easily add or remove source files without having to edit the package manifest.

Products and targets, however, are worth explicitly configuring to make it easier to understand the package and what it defines without needing to cross reference with the layout on disk.

It also makes it easy for clients to see what a package provides just by looking at the manifest.

SwiftPM also supports building source code for other program manager languages, such as C, C++, and Objective-C. This allows integration with existing code.

Note however that we do not support mixing those languages with Swift in the same target. Next, we're going to look at dependencies and versioning.

To make sure your packages can benefit from bug fixes without constant churn, Swift packages should adhere to semantic versioning.

This is a commonly used standard which assign specific semantic meaning to each of a version number's components.

The major version signifies breaking changes which required clients to update their code.

Examples for the changes could be deleting an existing type, deleting a message, or changing its signature. But they also include backwards incompatible bug fixes, or major changes to the behavior of existing API. The minor version should be updated if functionality is added in a backwards compatible manner. Examples for this is adding a new method or type. And finally, the patch version should be increased when you're making backwards compatible bug fixes.

This allows clients to benefit from bug fixes without risking breaking the source code. SwiftPM needs to determine the exact versions of all packages in the package graph before it is ready to build. We do this with a process called dependency resolution. As part of this, SwiftPM looks at all the requirements specified for packages and finds the latest version that is compatible with all of them. Let's take a closer look at what SwiftPM is doing in the process using the demo I showed you before.

The dealer package from the demo has two direct dependencies. One, is SwiftPM itself, and the other one is deck of playing cards. SwiftPM will resolve the versions of these direct dependencies. For the first one, this is straightforward because we specified an exact version, beginning exactly that tag. For the second one, we're using the from syntax. That means we're getting updates to the minor or patch components. In this case, we're ending up with a tag 3.1.4. The whole process is recursive. So next SwiftPM will look at the transitive dependencies of all diary points. So, PM has no further dependencies, so there's nothing to do there. But a deck of playing cards depends on the Fisher-Yates and playing card packages. Next, SwiftPM has to resolve the versions of these packages again. For the Fisher-Yates package, this works the same way as before because we are also using the from syntax. In this case, we're ending up with a tag, 2.2.5. For the playing card package, we're using up to the next minor syntax. This means, we're getting only updates to the patch component.

You might want to use that syntax if you want to be more conservative as a dependency and only take bug fixes. In this case, we're ending up with a tag 3.0.2. Finally, when looking at a target, SwiftPM has to match up its required products with the packages that we resolved. For this, we're looking at the dealer target from the demo, and as you can see, the utility product is provided by the SwiftPM package.

And the rest of the packages provide the other products. After dependency resolution, the resolves are recorded in a file called package.resolved. the purpose of this file is so that you can share the resolve versions with other members of your team, or your continuous integration infrastructure, so that you get dependable build results, and you can deliberately choose when you want to update a dependency. You do so by running Swift Package Update, when you're ready to update. Note also, that this is your top-level package that contains package.resolved. If one of the transitive dependencies contains a package.resolve file, it will be ignored for dependency resolution.

Next, let's take a look at building your package. SwiftPM uses llbuild as its underlying build engine. llbuild is a set of libraries for building build systems. And it's built around a general purpose and reusable build engine. This provides us with ability to do fast as well as correct incremental builds. And is also used by Xcode's new build system. It is also the part of the Swift open source project.

Developing software in isolation with all dependencies explicitly declared ensures that even packages with complex requirements can be reliably built and used in different environments.

Instead of installing packages globally into the system, SwiftPM only allows you to use packages that you explicitly depend on.

We also leverage build sandboxing so that nothing can write to arbitrary locations on the file system during the build.

SwiftPM does not allow executing arbitrary commands, or shell scripts, as part of the build.

This allows us to fully understand your build graph and all of its inputs and outputs to do fast, as well as correct, incremental builds. Because we have a view of all your dependencies. As I showed you in the demo before, SwiftPM also supports testing. This is based on the XCTest framework that you're already familiar with. We support parallel testing, so that you can get your test results faster.

And we support test filtering so that you can run a subset of tests and iterate on a single feature. As we're evolving SwiftPM, we're thinking about workflow features, especially so that you can do all of your development on the command line. One such feature is edit mode, which allows overwriting all transitive occurrences of a specific package, with a locally checked out copy so that temporary edits can be made, and changes to transitive dependencies can be tested without having to forward all packages in the graph upfront.

Branch dependencies allow depending on packages without strict versioning requirements. This is useful when you're developing multiple packages together.

This is a development only feature, so you have to change to certain version dependencies before you publish a tag. Local packages allow you to use packages directly from the file system, instead of from a git repository. This is useful to make it possible to bring up multiple packages during the initial creation.

So, last topic, I want to talk to you about adopting new versions of SwiftPM and the Swift language. Each new version of Swift can bring a new version of the package.swift manifest API. The previous API is still available, so that you can take advantage of new source tools without having to update your package or losing access to existing packages.

New API can be adopted independently of changing to a new Swift language version for your packages' source code. To specify which version of the API is being used, we're using the Swift Tools Version command at the top of the package.swift manifest file. This specifies the minimum required version of the Swift tools that is needed to process the given manifest. Each package can also declare which versions of the Swift language it uses for compiling its source code. This is a list, so you can choose to support multiple versions of Swift with the same version of your package, by using compiler directives. A package graph can be a mix-and-match of packages with different language versions.

I told you a lot about how SwiftPM works today, next I'd like to invite Rick back up to the stage to tell you where we can go from here. Thanks, Boris.

So, Boris has shown you what you can do today, but there's a lot more potential out there. SwiftPM is still a young project with lots of room to grow.

Swift uses an open evolution process which means that anyone, including you, can contribute your ideas. If you're looking for some inspiration, we'd like to share some of our ideas, but none of this is a plan of record. We're sharing these ideas so that you can see the potential of the Swift Package Manager. And we welcome you to provide your feedback, comments, and your own ideas as we evolve is this product.

The ideas I'm going to cover today break down into four different themes.

Letting the Swift Package Manager integrate with other tools that may want to sit on top of it. Helping you publish new versions of your package and deploy their products. Supporting more complex packages than SwiftPM is able to build today. And finally, some forward looking thoughts on package discovery and trust. While the SwiftPM command line experience is important, we want to make sure that SwiftPM can integrate with other tools, such as development environments, automation, and more.

We've already laid the groundwork for this with SwiftPM's library-based architecture. SwiftPM doesn't have a stable API today, but for tools that are willing to keep up with the SwiftPM changes, it's available for adoption and additions today.

If you're looking to build support for SwiftPM into your developer tools, we welcome your contributions and discussion. We want to make SwiftPM part of a thriving ecosystem of developer tools. One thing we've seen requested recently on the Swift forums is a way for people to edit their package.swift manifest from automated tools, instead of making their users always edit the source code directly. We think that it's possible for SwiftPM to support this, probably by using libSyntax. libSyntax is a library being developed in the Swift open source project that makes it easier for you to understand and manipulate Swift syntax from other tools.

Boris told you earlier that you should prefer declarative syntax for your package.swift manifest, and this is another reason why. That will make it much easier for SwiftPM to understand your manifest, so that it can make automatic changes, such as adding new dependencies or targets as shown here. So, there's a lot of room for SwiftPM also to add new functionality to help you publish new versions of your packages and deploy their products.

Today, when you want to publish a new version of your package, you tag it manually with git. And if you want to inspect your published tags, you use git directly for that as well.

We could add new functionality to automate this process and perform additional housekeeping, validation, and other auxiliary tasks you might want as part of a streamlined publishing workflow.

One especially useful feature we could add here would be assistance with maintaining correct semantic versioning. We could have SwiftPM analyze the API differences in the new version of your package and detect when you've made a change that is not compatible at compile time, so that it can suggest updating the major version of your package.

Another thing we could do is make it easier to deploy the products of your packages from SwiftPM. You may want to customize the linkage with libraries, or the product layout for your specific deployment environment, whether local or on a server. Or, maybe you want to include version information about what packages were built into the product. Or, you otherwise want to use the context that SwiftPM has about your packages somewhere in your product. SwiftPM could add new commands to support all of these needs.

There's a lot that you can build with SwiftPM today, but we also want to be able to support more complex packages with more sophisticated needs.

The biggest gap we have right now, is probably support for resources. If you have any images, data files, or other assets, SwiftPM currently provides no way to bundle these up you're your products.

The foundation core library, actually just added API this spring for doing resources in a cross-platform manner so SwiftPM could adopt this API if we want to build this feature.

We know that some users also want support for specifying compiler flags, linker flags, and other properties that SwiftPM doesn't support today. It would be really great for us to add a robust build settings model, potentially including things like conditional settings, or fine-grain control over what parts of the package get which setting values.

Boris also talked to you earlier about SwiftPM build isolation, and why it's important.

We don't let you run arbitrary shell scripts. But many users may want some level of customization for their build, either because they want to support custom languages, or processors, they want to run their own documentation generator implementor, or they have other steps that they need to bring to the build process. We think that SwiftPM could support this safely, possibly even through real tools packages that bring new tools into your build process. The important thing we need to make sure here, if we do such a feature, is that any new tool that's brought into the build process have to correctly declare their input and output dependencies, so SwiftPM can continue to maintain correct incremental, and parallelizable builds.

Finally, I want to talk a little bit about some forward-looking thoughts on package discovery, trust, and management. Git itself supports, the protocols that get supports, provides security mechanisms like TLS to make sure that you're actually talking to the remote repository that you think you are. But a malicious actor could still compromise remote repository and put malicious content in. This is actually something anytime you're using third-party code, you should be aware of these sorts of risks. But the Swift Package Manager provides a great opportunity for us to build security features to make sure that you're actually getting the package content that you expected.

SwiftPM also prevents your package.swift manifest evaluation in your build from escaping and writing things out into your file system or accessing the network. We're using macOS' sandboxing technology for this today. And it's great. But we'd like to bring this kind of security to other platforms as well.

Many users may want to be able to fork their packages easily, either because they want to make a private customization to one of the packages in their graph. Or, even because they just want to override the origin URL of where they're getting each of those packages from, so that they can point at a private mirror that they control and not depend on the original package always being there. Ultimately, I'd like some day for us to have a real index for Swift packages. In addition to providing a standardized namespace and making it easier to discover new packages, we could even support things like quality metrics for a package. Like what is its automated test coverage? Or ways for you to evaluate the trustworthiness of a new package that you're considering adopting. So, I've gone over a lot here. But these are just some of the possibilities. Ultimately for those of you, who are interested, we're interested in your feedback, ideas, and contributions to help make Swift Package Manager the best tool that it can be for the developer community. So, to talk about how you can do that if you'd like to, I'd like to talk about Swift's open source process.

As I said earlier, the Package Manager is part of the Swift open source project. And Swift.org is also a great place to go if you want to learn about the community and the process.

SwiftPM uses the Swift language evolution process which means anyone can submit a proposal for major new features or changes to the Swift Package Manager.

Before you go off and draft a whole formal proposal though, I recommend that you swing by the Package Manager section of the Swift forums and socialize your idea with the community. You may get a lot of feedback that helps make your idea even better. If you're interested in dipping your toe in the water with a smaller contribution the Swift bug tracker at bugs.swift.org has plenty of ideas. In particular, you may want to look for bugs tagged with this starter bug tag. And since, as I said, SwiftPM is written in Swift, you may find it's actually pretty easy to dive in and take a look. Of course, if you find bugs when you're using SwiftPM, we encourage you to go file them on bugs.swift.org as well, where you can track how we handle them. SwiftPM gets to take advantage of the same great continuous integration infrastructure that the Swift project has. Which means that poll requests can be automatically built and had their tests run before their merged. Because the SwiftPM code base itself has great test coverage, we found that this infrastructure is really useful for us.

When you're ready to try out the latest changes, you can download the Trunk Snapshot Toolchains that are updated on a regular basis available on Swift.org.

We've been really happy with the growth of the SwiftPM community so far. We've had over 180 people contribute, either with bug features or new features. And the Swift Package ecosystem is growing at a healthy rate as well, often with cross-platform packages, and many public packages available on GitHub. What this means is that you can focus on what makes your product special and let package dependencies handle the rest. There are a couple things that I recommend you try SwiftPM out for today, even though it has a lot of room to grow in the future. Those two things are command line utilities and libraries and for developing with Swift on the server.

The server-side Swift community has been making extensive use of the Swift package manager. And server-side Swift has been growing well itself with many frameworks now available for doing web and backend development. If you would like to take a look at this approach, I think you'll find that Swift is a great language to do this kind of cross-platform development on.

But you could also go ahead and use SwiftPM for creating command-line utilities and libraries today, whatever makes sense for you. Ultimately getting started is as easy as opening a terminal window and running Swift package init. So, the next time you're thinking about trying something new, I encourage you to give it a try. And if you're interested in contributing, swing by the Swift forums and start a conversation. If you'd like to come chat with us, you can find us in the labs tomorrow at 3 p.m. Ultimately, I'm really excited about where we can go from here and what this developer community can do together. Your contributions will help us design a package manager that will be great for the whole Swift community. Thank you. Enjoy the rest of WWDC. [ Applause ]