SwiftUI Accessibility: Beyond the basics

♪ Bass music playing ♪ ♪ Nathan Tannar: Hello, and welcome to WWDC! My name is Nathan, and I'm an engineer on the Accessibility team.

Today we'll be going beyond the basics to learn how to deliver exceptional and accessible SwiftUI apps.

This year marks a huge leap forward for accessibility in SwiftUI.

At Apple, accessibility is one of our core values.

Our assistive technologies across all our platforms make sure that anyone can use your app regardless of any physical, visual, audible, or motor impairment.

My team and I work to make sure most of your app is accessible by default, but there is always more you can do to enrich the experience.

Today I'll show you the new tools and APIs for SwiftUI that make enriching this experience easy.

Let's begin with the tooling improvements in Xcode.

SwiftUI Previews have changed how many of us develop apps.

It dramatically improves your ability to iterate upon your views across multiple environments, all without the need of running your app.

To make the most important accessibility modifiers just a click away, we've added a curated list of accessibility editors.

I hope this encourages everyone to always make their views accessible.

But there's more.

Since accessibility modifiers don't have visual changes in the previews, a new tool was developed that allows you to inspect the accessibility of your view without leaving Xcode.

Let's switch from the editor's panel to the new accessibility panel.

Shipping in Xcode 13, SwiftUI Previews will now also feature an Accessibility Preview.

With it, you can inspect the accessibility elements for a preview in real time.

And this is a game changer.

Accessibility Preview will help you make accessible apps even if you don't have a deep understanding of each assistive technology.

Let's take a closer look to see how changes are reflected in the Accessibility Preview.

This is a simplified version of the view we just saw.

After running the preview, I can select the VStack and the Accessibility Preview will update to display the elements in their sorted order.

Notice that each element will always display its basic properties -- such as a label and traits.

For example, the Text view will create an accessibility element with the string as its label.

It also gains the .isStaticText trait.

The preview will update for any accessibility changes we make, such as adding the .isHeader trait.

You will also be able to see some of the automatic accessibility that takes place behind the scenes, such as automatic SF Symbol labels.

For example, the checkmark.seal.fill symbol is labeled "Verified" by default.

If you're relying on a symbol's default label, it's important to check that it accurately describes your interface.

We'll continue to use the Accessibility Preview as we step through five importance areas that'll help you deliver an exceptional experience to all of your app's users.

We'll start by examining the best way to make custom controls accessible.

Next, we'll discuss how to make a view accessible with children.

I'll highlight how to audit your app for common navigation problems and how the Accessibility Preview can assist with this.

Then I'll show you how to supercharge your app's navigation with VoiceOver rotors.

And finally, we'll look at focus and how it relates to assistive technologies.

I've been working to prototype a new finance app I call Wallet Pal.

It's still in the early stages, but so far, I'm pretty happy with the initial design.

I was pleased to hear that early beta testers are also loving it.

Now that I've polished the UI, I should spend some time to polish the accessibility interface.

The accessibility interface compliments the visual experience to make sure that it’s usable by everyone.

I've asked some VoiceOver users to help test Wallet Pal and I've heard that the app is hard to navigate and not fully accessible.

If parts are not accessible to VoiceOver, then they are also not accessible to other assistive technologies.

It's important that anyone be able to use Wallet Pal, so let's investigate to see where we can improve the experience.

We'll begin with reports of users not being able to edit their budgets, a critical feature in Wallet Pal.

This is the Budget Planner view which allows users to edit their food, entertainment, and savings budgets.

To fit the design of Wallet Pal, I ended up needing to create custom sliders.

I accomplished this by creating a budget slider out of shapes and used a drag gesture for interaction.

Shapes in SwiftUI make it easy to create stunning and unique views, but they are not accessible by default, and so neither is my budget slider.

This must be why some users are not able to edit their budgets.

We can confirm it's not accessible by running a SwiftUI preview and selecting the SliderShape.

The Accessibility Preview shows that there are no elements, so this control is not accessible.

Ideally, we should have a single labeled accessibility element with a value that can be changed.

In contrast, standard controls are accessible by default, meaning little to no extra effort is required to provide a great experience.

SwiftUI uses the view type and its initialization parameters to automatically derive the accessibility element's label, value, traits, and actions.

So what's the best way to make our custom budget slider accessible? Since the standard slider is accessible by default, my team and I thought it would be great to have an API that allows the accessibility of one view to be represented by another.

This is what led us to us to create accessibilityRepresentation(representation:).

It's an API that allows the accessibility of one view to be defined by another.

Since I want this to be perceived by assistive technologies as a slider, that's what I'll use for the accessibility representation.

With accessibilityRepresentation(representation:), we're now well on the way to making this budget slider accessible.

The only change needed to improve the experience of the slider is to describe the value in dollars.

And great, just like that, now our budget slider is perfectly accessible.

On macOS, the type of control will also be spoken to VoiceOver users.

So by using the slider view in our budget slider's representation, it will be announced as a slider.

It's recommended to use accessibilityRepresentation(representation:) when possible, to make custom controls accessible.

This is just one way to use accessibilityRepresentation(representation:).

Its flexibility allows for many more creative uses.

Let's see if we can use it to fix other accessibility bugs in our app.

While we have now fixed the functional ability to edit budgets, some users reported that they couldn't discover how to navigate to the Budget Planner view.

When designing Wallet Pal, we decided to use SF Symbols for all of our buttons.

While SF Symbols may have great default accessibility labels, they may not always fit the intended use case.

Let's investigate if the problem is a poorly labeled button.

The NavigationBarView contains the Edit Budgets button.

Since we rely on SF Symbols for so many of our buttons, I created a custom buttonStyle called SymbolButtonStyle.

But it appears that despite initializing the button with the label "Edit Budgets", the label for the button is slider.vertical.3.

So the accessibility label is being derived from the SF Symbol.

Let's take a closer look at the SymbolButtonStyle.

The SymbolButtonStyle's makeBody(configuration:) method returns an Image view.

The Edit Budgets string we initialize the button with would create a Text view as the configuration's label.

But this style completely ignores the configuration's label.

This is why the button is not labeled "Edit Budgets".

While we want an SF Symbol to appear visually, we want the accessibility of the button to be represented by the configuration's label.

And so this seems like a great use case for accessibilityRepresentation(representation:).

Using accessibilityRepresentation(representation:), I can substitute the accessibility of the Image view with another view, in this case, the configuration's label.

This way, we can preserve the label we used to create the button.

accessibilityRepresentation(representation:) is not just the ideal and recommended way to make custom controls accessible, it also opens up new creative possibilities to make views accessible.

Next, let's discuss the relationship between children and accessibility containers.

As you may recall, accessibility elements can be wrapped together in a group as children of an accessibility container.

This is done with the accessibilityElement(children:) modifier and the contain child behavior.

In case you're unfamiliar, the contain child behavior can be used to create an accessibility container, which wraps existing accessibility elements as children.

But what if instead you have an accessibility element and you want to set its children? I'm running into this case when trying to make the Budget History graph accessible.

Users reported the graph is completely inaccessible.

And so this means when assistive technologies are focused on the Budget History header and try to navigate to the next element, they end up at the Alerts header.

VoiceOver users would not even know that this graph exists.

I built the graph using the new Canvas view.

Canvas makes it easier to draw a collection of shapes.

For more on Canvas, please check out Jacob's presentation on how it can be used to add rich graphics to your app.

The most important takeaway for accessibility is that Canvas draws a collection of shapes.

And just like we saw with BudgetSlider, shapes are not accessible by default.

All users need to be able to view their budget history, so let's make this accessible for everyone.

Starting with the basics, let's give our Canvas a label.

This will automatically create a new accessibility element for the Canvas and assign its label.

Now, I'd like each bar in the graph to be represented by their own accessibility element.

This is a use case where we have an accessibility element and want to provide its children.

To do this, I'll use the new accessibilityChildren(children:) modifier.

This will transform the accessibility element into an accessibility container, preserving other accessibility properties, such as the label.

The modifier takes a ViewBuilder, which allows us to set new views as children of the accessibility container.

Recall that our Budget History graph is drawing a horizontal bar graph, so we'll use an HStack and return a view for each budget.

I'll use a Rectangle for each accessibility element so that the frame will fill all available vertical space.

This will make each accessibility element's frame bigger than what's displayed visually, and that's OK.

Having a large but consistent frame will make it easier to navigate on iOS when VoiceOver users drag their finger across the screen to scan for accessibility elements.

If I select the HStack from within the accessibilityChildren(children:) modifier, the Accessibility Preview confirms that an element has been created for each bar in the graph.

These will all be accessible as children of the Canvas accessibility container.

With these changes, the Budget History graph is now completely accessible, and assistive technologies can navigate through each bar in the graph.

Notice that the frame for each element is the same, which is ideal.

For more complex charts, we have other ways of making them accessible.

For that, please check out Preston's presentation on bringing accessibility to charts.

But with accessibility children, the accessibility elements of a view can differ from what's presented visually, allowing you to tailor a great experience.

But accessibility children can also be used to compose the accessibility with the help of the combine behavior.

As a quick refresh, the combine child behavior will merge properties from multiple accessibility elements into a new or existing accessibility element.

But with the addition of the accessibility children API, it can now also be used to compose accessibility in a generic way.

With accessibility representation, the original accessibility is completely replaced.

This means no composition can take place; whereas accessibility children is additive.

This means you could later combine the children to merge their properties into the original element.

This is a more advanced use case of the accessibilityChildren(children:) modifier, but it's a functionality I wanted to highlight.

Examples are featured in this presentation's Accessibility Catalog Sample Project.

I hope to see many of you explore what's possible with this kind of composition.

accessibilityChildren(children:) gives you control over the children of a container.

And with that, complex graphs drawn with Canvas can be made accessible with modifiers and views you're already familiar with.

And with the combine child behavior, the accessibility of a single view can be composed of many.

Now that we've learned how to make individual components of our app accessible, we can start putting things together to refine the navigation experience.

I've heard that navigating Wallet Pal with VoiceOver is confusing and difficult.

So we have some more work to do if we want to deliver a great accessible app.

Let's take a closer look at the Friends carousel at the top.

While I don't have this feature built yet, I plan to add some kind of gamification to Wallet Pal.

So I've added a challenge button at the top left of each friend view.

We already fixed the SymbolButtonStyle, so the challenge button will be properly labeled now.

But since users are reporting that navigation is confusing, are there other issues? To answer that, we must first understand how assistive technologies will navigate Wallet Pal.

By default, accessibility elements are sorted based off their geometric position in relation to other elements, from top left to bottom right.

This means that without accessibility containers to differentiate the content, VoiceOver would navigate through each of the challenge buttons, then the Image and Add Friend button, and finally, the text with the user's name.

A great feature of the Accessibility Preview is that it shows accessibility elements in their sorted order.

This makes it easy to visualize how assistive technologies will navigate right from within Xcode.

As expected, the sorted order matches what we previously saw.

And this order would certainly be confusing to navigate.

Now there are multiple ways we could fix the sort order of the accessibility elements.

One thing we could do is to introduce accessibility containers.

I'll add the accessibilityElement(children:) modifier with the contain behavior.

This will wrap the accessibility elements of each FriendCellView in an accessibility container.

This fixes the navigation order, because children of an accessibility container are navigated before moving to the next accessibility element.

With this change, VoiceOver will navigate through the children of the accessibility container before moving to the next accessibility element.

And so we achieve a much more desirable navigation order.

But could we do better to improve this experience? One problem is that the challenge button is navigated to before knowing who the user is.

A VoiceOver user would want to know the name of the user before sending a challenge, so this button should really be sorted last.

We can use the accessibilitySortPriority(_:) modifier to fix this.

accessibilitySortPriority(_:) can be used to change the order of elements within an accessibility container.

An element with a higher priority will be sorted first, whereas a lower priority will be sorted last.

Elements with equal priorities are then sorted based off their geometric positioning.

I'll add the accessibilitySortPriority(_:) modifier to the challenge button to improve the navigation order.

I'll use a priority of -1 since the default is 0, to force the sort order of the challenge button to be last.

Now the challenge button will be navigated to last.

This is a good improvement, since there is less chance a VoiceOver user would be confused as to who they are sending a challenge to, but it's still not great.

Rather than wrapping each FriendCellView in an accessibility container, we could combine the children into a single element.

Recall that combine merges properties from children into an existing or new accessibility element.

The combine behavior also curates which properties are merged for the best default result.

For example, the challenge button has become an action named "Send Challenge".

This too fixes the navigation order and reduces the number of accessibility elements.

Now there's a single element for each user, and each element has a Send Challenge action.

It is often ideal to combine the accessibility elements of views that are represented in a ForEach.

As you may have picked up on by now, combine is a super useful child behavior.

Instead of children being individually navigable, it merges their properties into a single, navigable element.

For when you need a single element but do not want it to inherit properties from the children, use the ignore behavior.

And lastly, the contain child behavior wraps the children in an accessibility container and should be used to express a groups of views that are related.

This not only improves the default sort order but has other advantages to assistive technologies.

With the new Accessibility Preview, discovering these navigation problems is easy.

And with a few small changes, you can dramatically improve the experience.

But to deliver a truly exceptional navigation experience to VoiceOver users, you'll will want to consider rotors.

Now if you are new to accessibility, you might be thinking, "What are rotors?" In short, rotors are a powerful navigation tool.

They can be thought of as bookmarks that allow users to quickly navigate between them.

System rotors, such as the headings and containers rotor, provide the foundation for this supercharged form of navigation.

For example, users can quickly navigate through sections using the headings rotor.

This is because the Section view automatically adds the isHeader trait to the header view If you are not using the Section view, you can add the isHeader trait to your view with accessibilityAddTraits(_:).

Similarly, accessibility containers are added to the container's rotor, which as we saw earlier, are created with the accessibilityElement(children:) modifier.

As you can see, supporting the headings rotor is incredibly easy, and the container's rotor further adds to the benefit of grouping your accessibility elements with the contain child behavior.

Now that the basic navigation experience of Wallet Pal has been refined, let's take it a step further with rotors.

In Wallet Pal, alerts help keep users' spending habits in check by providing motivation messages and warnings when budgets near or exceed their limit.

We've incorporated SF Symbols for the different types of alerts.

These allow visual users to quickly scan through the list.

But this iconography will not help VoiceOver users.

They would instead have to navigate through every alert just to know if they have any warnings.

To replicate a similar experience that sighted users have, VoiceOver users need to be able to exclusively navigate warnings, that way, in an instant, they can navigate to the next warning alert.

To do this, we can use rotors.

For more examples of custom rotors and why they're so important to efficiency, please check out our "VoiceOver Efficiency for Custom Rotors" presentation from 2020.

To add a warnings rotor to my AlertsView, the first thing I'll do is to make sure I'm adding my rotor to an accessibility container.

Some views in SwiftUI are accessibility containers by default -- such as List or LazyVStack -- but VStacks and HStacks are not, so I'll add the accessibilityElement(children:) modifier with the contain child behavior.

Next, I'll create my rotor with the accessibilityRotor(_:entries:) modifier, giving it the name "Warnings".

And lastly, I'll declare which of my alerts I want included in the warnings rotor, in this case all of the warning alerts.

And that's it! It's this easy to supercharge your app's navigation.

Part of what makes rotors so easy for these simple use cases is that SwiftUI can automatically match the rotor entry to the accessibility element based off the ID.

This is because the ID of our rotor entry will match the ID given to the AlertCellView by a ForEach.

The identity of a view is important to understand for accessibility rotors.

So if view identity is unclear or you'd like a refresher, I strongly encourage you to check out the "Demystifying SwiftUI" presentation to learn more.

Now you might be thinking, "Hold on, my view isn't this simple.

What about views that are not within a ForEach?" Don't worry, the accessibility rotor API can scale from simple to complex views.

This simple case works elegantly because there is a single accessibility element -- due to the combine -- for the AlertCellView.

And the AlertCellView is identified by the ID of the alert.

What if for every alert there was also an Actions view? In this case, the VStack is now the root view of our ForEach, and it's the VStack which will be given the identity of the alert.

So to include the AlertCellView in our warnings rotor, we will need to explicitly mark it as a rotor entry.

We can do that with the accessibilityRotorEntry modifier.

This modifier requires a namespace and an ID, which could be anything as long as they match the ID and namespace used to create the AccessibilityRotorEntry.

And lastly, we'll need to include this namespace for each rotor entry.

The ability to refer to an explicit namespace is what enables the accessibility rotor API to scale from simple to complex use cases, allowing accessibility elements that span across multiple views to be included in the same rotor.

Accessibility rotors can also be added to enrich text navigation.

To do that, use a different variant of the accessibilityRotor modifier, which allows you provide an array of text ranges.

This modifier is great for giving VoiceOver users quick and easy access to specific strings within a TextEditor, such as emails, links, or phone numbers.

Accessibility rotors makes complex navigation easy for VoiceOver users.

And with the new SwiftUI rotors API, it's never been easier to deliver this exceptional navigation experience.

The last topic for today is focus.

The concept of element of focus is something you may already be familiar with, but what you may not know is that many of our assistive technologies have their own focus state.

This is what we call "accessibility focus".

It's the focused view you have been seeing change when I refer to assistive technologies navigating.

The position of an assistive technology's cursor is critical to the user experience.

When focus is changed in VoiceOver, the cursor moves to match the focused element's path in addition to speaking a description of the element.

So when does focus change? Focus can but may not always change when one of three events occur.

The first and most common use case is when a user drives a focus change when navigating to a different accessibility element.

The second case is when the UI changes, and the previously focused view is no longer on screen or is covered by a modal view.

When this happens, focus is often reset to the first sorted accessibility element.

This covers many use cases automatically, such as moving to the first accessibility element of a newly presented view.

But that may not be the best behavior in your app, which is why focus can also be changed upon programmatic request.

However, this should be handled with caution, because moving a VoiceOver user's focus can be very disruptive.

On that note, let's focus -- no pun intended -- on the last case.

This year we have a new API that will allow you to both request an assistive technology to move its focus, in addition to reading where an assistive technology is currently focused.

Here we have a simple view which tracks a notification, and if it exists, overlays a custom NotificationBanner.

We want to use this view in Wallet Pal to show alerts for push notifications received while the app is in the foreground.

When the NotificationBanner is added, assistive technologies will not automatically focus on it.

But we can request they do so with the new AccessibilityFocusState.

AccessibilityFocusState is a property wrapper that provides both a way of reading where an assistive technology is focused and requesting a programmatic focus change.

I'll add one to my view and bind it to the NotificationBanner with the accessibilityFocused(_:) modifier.

Next, I'll use the onChange(of:perform:) modifier to track when a new notification is received.

Only when a high-priority notification is received do we request accessibility focus to change.

It's especially important to note that moving focus programmatically can be very disruptive if no user interaction occurred.

It takes the user out of whatever context they are currently focused on, so it must be handled with care.

So for lower priority notifications, I'll post an accessibility notification for VoiceOver to announce.

This way, VoiceOver users still know a new notification has appeared and can navigate to it if desired.

Now let's step inside the NotificationBanner view.

When the notification appears, a timer is started.

Upon it completing, the notification is set to nil.

Recall from before that when the notification is nil, the NotificationBanner will be hidden.

This means if a VoiceOver user was focused on the notification banner and the timer expired, their focus would be reset, since the view is removed; and this isn't a great user experience.

To fix this, let's read if our NotificationBanner is focused by an assistive technology, and if it is, delay the notification dismissal.

This is the ideal solution.

A VoiceOver user's focus will no longer be reset, since the view is no longer removed while focused.

Additionally, we have given assistive technology users an unlimited number of time to digest the content and interact if desired; two things that may take assistive technology users much longer to do so.

AccessibilityFocusState is the last piece you need to deliver exceptional and accessible SwiftUI apps this year and beyond.

With it, you can read and direct the focus of assistive technologies to create smooth transitions between views.

Wow, we've covered a lot today for SwiftUI accessibility.

We were introduced to the new Accessibility Preview, which enhances the way you can develop and debug SwiftUI accessibility; we covered how to make custom controls and complex graphs accessible; and we took the time to learn how the navigation experience can be improved with grouping, rotors, and focus.

Put everything together, and it truly has been a huge leap for accessibility this year.

For some great simple and complex examples of our Accessibility APIs, please checkout the Accessibility Catalog Sample Project.

It features more examples that we couldn't cover today and some of our recommended best practices.

Thank you so much for joining me.

We look forward to seeing how you make your app accessible to everyone.

♪