What's New in UICollectionView in iOS 10

Good morning.

My name is Steve Breen. And my name is Peter Hajas, and we're both engineers on the UIKit Frameworks Team.

We're really excited to share with what we're working on in CollectionView. So let's get right into it.

We've got three big topics to talk about this morning. Our first topic is going to be smooth scrolling.

Now, every iOS app expects to have great scrolling performance, and we got some great additions to CollectionView to help your apps scroll better than ever, and the best part about this is many of these additions require little to no work in your applications. Next, we're going to talk about improvements to self-sizing cells. Now this API was introduced in iOS 8, and we're bringing some great improvements to it in iOS X to make it easier to adopt.

And, finally, we're going to go over interactive reordering. This API was introduced last year in iOS 9, and we've got some great enhancements that we're going to go over for iOS X. Let's start off with smooth scrolling. A hallmark of the iOS device experience is responding immediately to the user's touch, and a big part of this responsive user experience is making sure that as I scroll my finger across the screen, the objects onscreen move as if they're moving in the real world.

This is really important to keep your users immersed in their applications.

We're going to talk through smooth scrolling and some enhancements we've made to UI CollectionView by talking through a demo app. As you'll see, this application starts off not scrolling that great. We wanted to scroll like butter, but right now, it's scrolling a lot more like chunky peanut butter. So let's go to the iPad.

I'm going to switch over to the iPad.

Just to clarify, blue squares are usually pretty cheap, but we've intentionally made these slow. So imagine they're more complicated. For example, maybe they have two colors in them.

Anyways, we've got this CollectionView, and we can see that scrolling already loaded content is nice and fast, but check this out. Steve, watch closely. When I scroll a little bit more.

Ooh, ouch. One star, do not buy. I wouldn't buy it either. This type of user experience is something we really want to avoid. So what's going on here? Well, we said before that these are some really expensive blue squares to simulate to really expensive complicated cell in your app, and that's running up against CollectionView and the fact that it loads cells exactly when they're required. Let me show you what I mean.

We'll reload our data here, and I'm going to turn on a view that's going to show all the cells that are loaded in the CollectionView.

So you can see those bottom cells are just peaking up beyond the visible bounds. Now watch what happens when I scroll.

We're bringing in an entire row of cells at once. This is what's leading to that stuttery scrolling performance.

In performance terminology, we would say that the app is dropping frames. Let's switch back to slides to hear a little bit more. So what exactly do we mean when we talk about dropping frames. Now, in your applications, the users expect a smooth scrolling performance, which means your app wants to hit that magical number of 60 frames per second.

Now if we do the math, that means that for every time we refresh the display we need to hit that window in 16 milliseconds.

OK. So let's look at a couple frames. Here we have a diagram that shows three distinct frames. In our first frame, we got very little work to do. Now as Peter mentioned before, we're moving existing content up and down the display. It's a highly optimized situation on iOS. So it's superfast. Not much work to hit, and we've got this great five-star scrolling performance. That's a good frame.

However, in the demo, it wasn't always so great.

Occasionally, we'd have this situation where we have a lot of work to do. Not only do we fill up our current frame, we went into the next frame.

This is our ad frame. We drop a frame, one star. Let's look at this in a little bit different light. Now on this graph, we can see we've got two distinct regions. We have the region on the top, which we're going to call the red zone. This is the area where we miss a frame, and we're up above that magical 16 millisecond line.

Let's check out the labels that we have on the axes. So on our y-axis, we're going to graph the CPU time on the main thread, and on the x, this is the display fresh events as the scrolling's occurring. OK. So let's look at a graph.

Alright. So in this graph, this shows what Peter was demonstrating a second ago where we have these visits up into that red zone where we're dropping frames, but most importantly, look at these quiet times where the CollectionView's doing very little work, but then we have this other visit to the red zone at the end. So what if we could change this behavior a little bit and smooth out the amount of work we're doing as the user's scrolling. OK, cool. Look at this. So now instead of these visits up into that red zone and those periods of quiescence, we've got a nice even amount of work, and we're spreading the work out across time.

To help talk through how we're going to flatten these peaks and bring up this valley to create this nice, consistent line of work, I'd like to talk through the life cycle of a cell as it existed on iOS 9. We're going to go through the whole circle of life of a cell. It's beautiful.

Let's bring our CollectionView cells, and we're scrolling along, and let's say we need to bring in a new cell.

We'll take it out of the reuse queue, and we'll call prepareForReuse on it.

This gives the cell an opportunity to reset itself to its default state, ready to receive new data from your app.

Next, we'll continue calling the rest of cellForItemAtIndexPath.

This is where you should be doing the majority of your work populating the cell. You'll access your data models, set them on the cell, and return them back to the system.

Now, right before that cell's about to go on screen, we'll call willDisplayCell.

This will give your app a chance to do any other last-minute work it needs to do before that cell goes on screen.

And for any outgoing cells, we'll call didEndDisplayingCell. OK. So that's the life cycle of the cell before iOS X.

Now let's check out what this looks like in iOS X.

So here we have the same type of layout that Peter just talked about. The single column. It's simple for demonstration purposes.

So now as the user scrolls up, notice here that well before this cell is needed to be displayed on screen, we're going to bring it in from the reuse queue. And then following that familiar pattern Peter was talking about, we're going to send it a prepareForReuse, and then construct the rest of the content in your cell with cellForItemAtIndexPath. Now as the user continues to scroll, here's what's different.

Now we didn't call willDisplayCell right when we created this cell. We have a hesitant, hesitation, and then right when it displays, we're going to call willDisplayCell.

OK. So now the user continues to scroll.

We're going to fade away those other cells to focus on the lifetime of this cell, and now the cell is about to exit the visible bounds from the CollectionView. So we'll send it the expected didEndDisplayingCell. Now what Peter was talking about iOS 9, at this point, the cell entered the reuse queue, and we'd be done with it. To display the data in this particular cell again, we'd have to go through the beginning of the life cycle to cell and call cellForItemAtIndexPath.

But in iOS X, we're going to hold onto that cell just a little bit longer.

So if the user scrolls up a little bit, and then says, "Oh, wait a minute. That was a picture of my sister's new kid. I'm going to scroll back down," we're going to hang onto that cell, and send it a willDisplayCell again. Now the content will continue on in the cell. So notice as Steve's showing you here, this also applies to multicolumn layouts.

We're going to be bringing in the cells one at a time instead of all at once to get better scrolling performance. That's right, Peter, and notice here, these cells aren't actually displayed on screen quite yet. They're still off the screen. And now that we sent the second cell that we DQ'd, the cellForItemIndexPath, and scroll up both cells, now we're going to send the willDisplayCellMessage to both right before they appear on screen.

While this may seem like a subtle change, it's actually a lot bigger than that. By adopting this new life cycle in iOS X, we get automatic faster scrolling performance. Let's go back to iPad. So I'm going to switch back to the iPad, and here you see that same CollectionView we were just looking at in iOS 9. Remember, scrolling existing content is nice and fast, but when we have to bring in more cells, that's when things get choppy. Now, while preparing this CollectionView, we also prepared one in the oven cooking under iOS X. So I'm going to switch over to that, and here we've got the exact same CollectionView with those same really expensive blue squares running on iOS X.

We can see that scrolling existing content is still nice, fast, and fluid, but watch closely, Steve. Alright. When I scroll a little bit more.

Oh, that's great. Five stars. That's exactly right. This is because we're using this new cell life cycle. The application hasn't changed at all. I'm going to turn on that same view, which is going to show you all the cells that CollectionView's loaded to help highlight this difference.

So let me turn that on.

So here you can see that bottom row of cells is just peaking above, but watch what happens when I scroll faster. Instead of bringing in a row of cells at a time, we're now spreading that work out during the scroll. This is what leads to that much smoother scrolling performance. Isn't that great? To learn more about this, let's go back to slides.

So this is pretty great. So today we're very pleased to announce UI CollectionView Cell Pre-Fetching. Now this is enabled by default when you compile your apps on iOS X.

There's no step one. Now for any reason you need to use the old life cycle behavior pre-iOS X, opting out is easy. Just set the new property on UICollectionView to isPrefetchingEnabled defaults. Now we've got some best practices for adopting this new technology.

The first thing we want to talk about is we want to do all the heavy lifting in cellForItemAtIndexPath. All the, all the content creation for your cell. Everything should be centered in cellForItemAtIndexPath.

Additionally, we want to make sure we do minimal work willDisplayCell in didEndDisplayCell.

And, finally, it's important to note that cellForItemaAtIndexPath may prepare a cell that's never actually displayed. The user may scroll away before the cell has an opportunity to be displayed.

So this is great. By just recompiling in iOS X and doing what you're likely already doing with the majority of your work in cellForItemAtIndexPath, you automatically get better scrolling performance for free.

But we wanted to go a step further. We understand that there's a class of application which has a simple question for preparing their CollectionViews. What do I do about my expensive data models? The fact of the matter is that many CollectionView cells require expensive data model access to create. I'm talking about things like decoding images, talking to your database, or loading things out of your core data store. And we understand that for this class of application, we don't want to show things like template cells while we do an asynchronous network request.

To help solve this problem, we're introducing new API in iOS X to help inform how your data model loads content.

Since its introduction, UI CollectionView has always had two companion objects: The data source and the delegate. And new in iOS X we have this third companion object. It's optional, and it's called the prefetchDataSource.

There's just one required method. It's really easy to implement.

ColletionView prefetchItemsAt indexPaths.

This will be called on your prefetch data source when it's time for you to preload content from your asynchronous model.

That argument index paths is an ordered array of index paths so earlier items in that array are coming up sooner. You can use this to influence your asynchronous model reads.

There's a second optional method in this protocol. CollectionView cancelPrefetcingForItemsAt indexPaths.

This will be called on your prefetch data source when we determine that we're no longer scrolling towards a set of index paths.

You can use this to cancel or lower the priority of any pending loads. Now there's something really important about this API that I want to highlight.

This is not a replacement for the data model that you've already written. Instead, it works alongside your existing asynchronous solution that you've built for loading data in your app.

What you'll do is use this as an additional hint for when to load content in your CollectionView.

So let's tie all this together with a demo that shows all of the concepts we've talked about so far, and we're going to introduce a little science, too. Check this out. Steve's about to do some real magic.

Science, man.

Alright. We're going to switch back to our demo app, which we showed before. OK. So here's the demo app we've been looking at so far, but we've hidden this really great feature we call the science panel. Alright, so I'm going to turn that on now.

OK. So we have these two distinct regions, right. We've got that red zone. That's the one star bad zone, and then we've got that nice fat green zone at the bottom where we get the super smooth scrolling performance.

So what I'm going to do now is I'm going to run through with the iOS 9 version of the app, and I'm going to play back a scrolling session that I recorded earlier to show you what this looks like and do the science.

OK. So here we go.

Scrolling along, doing our science.

Chop, chop. OK. What do we have here? So check out this graph. We got eight visits up into that red zone. Eight dropped frames.

But also we can see that we've got these long periods of quiescence like we talked about earlier. So the area on the graph is this big spiky thing, and then going back down to the valley. Let's see what this looks like in iOS X. OK. I'm going to switch back to that iOS X mode that Peter was talking about earlier, and now I'm going to playback that same scrolling session with . Check it out, Peter. It's a lot smoother, Steve. Yeah, that definitely looks better. See what we get. Hey, look at that. No missed frames.

Pretty great.

Now check out what we've got under the curve on this particular graph. Instead of those peaks way up in the air and the quiet periods, we've blended those together, and now we have this smooth amount of work that makes your app much more responsive on the main thread. OK. You ready for this, Peter? I'm set. Let's do it.

OK. So now we're going to look at the iOS X version, but we're going to have the API, pre-fetch API's that Peter talked about earlier. We're going to adopt those in this application. Alright. So I'm all set in the demo app. I'm going to play back that same scrolling session.

Whoa, Peter, look at that. Now that's butter, Steve. That's looking pretty good.

Five stars. And the science agrees. OK. So, but, Peter, something's different here, right. The area on the curve between this main queue activity and the prior version don't match. This is a lot lower. What's going on here? So if we're adopting the pre-fetch API correctly, we're probably moving that data model read onto a background queue to free up the main queue.

That's exactly right.

That's what's going on. So now we've moved all that work onto a background queue, and we no longer have to clutter up the main thread. Let's switch back the slides.

So next I'd like to talk a little bit about some pre-fetching API tips to make your apps adopt this API in the best possible way.

The first, when you get the pre-fetch call, you want to make sure you do all the work on the background right away. Now we've got two great technologies for this: GCD and NSOperationQueue. Now it's also important to remember that pre-fetch is an adaptive technology. Now what do I mean by adaptive technology? Well, remember when we talked about those quiet periods and how pre-fetch takes advantage of those by doing additional work.

There are times in an application where the user's scrolling so fast that there's no quiet times. During these times when we have to update the display very often, we will not do pre-fetching.

And, finally, use the cancelPrefetchingAPI to adapt to your user's shifting focus. Now it's possible that the user is moving up in their CollectionView, and they're scrolling along with the content, but then they change their mind and move in another direction. We will notify you of this event with a cancel message so you can deprioritize those prior ones and focus on the new content that the views are scrolling to. So this is really great for CollectionView. By doing no work at all, you get better scrolling performance, and by doing just a little bit of work and using the classes you've already written, you get even better scrolling performance. And if you're using UITableView, don't worry. You're not chopped liver. We've brought the exact same API to TableView, too.

It's got a very similar optional pre-fetch data source companion object with a similar required method, just one.

TableView prefetchRowsAt indexPaths. Again, index paths is an array ordered by their proximity to the table's visible area. So earlier index paths are coming up sooner.

Just like with the CollectionView API, you can use this to inform your asynchronous data model reads. And it's got that same second optional delegate method.

TableView cancelPrefetchingForRowsAt indexPaths. As Steve just mentioned to you earlier, you can use this to cancel or deprioritize any pending data model loads.

And this is what's really great. Just like with the CollectionView API, this works alongside your existing asynchronous model solution. You don't have to throw anything away. Instead, use this to inform the loads that you're already doing. So that's it for cell pre-fetching in iOS X. So we're really excited about bringing this technology, guys. I mean, we can't wait to see the actual build with the smoother scrolling experience. So next up, we're going to talk a little bit about improvements we've made this year to self-sizing cells. This API was introduced in iOS 8 and we're bringing some additional enhancements this year to make it easier to adopt in your apps.

Now, before we get into this, I want to go over the existing API, chat a little bit about it. So we shift a concrete layout class in CollectionView called UICollectionViewFlowLayout, and we have full support for self-sizing cells with this class.

To enable this, we just need to set estimated item size to some non-zero CG size, and this tells Collection that you want to, you want to calculate the layout dynamically as content is displayed. Now as far as getting the actual sizes of your cells, there's three different methods for doing this.

The first method is using auto layout. If you can fully constrain the content view of your hierarchy to the content view of the CollectionView cell, we'll ask the auto layout system how large that cell should be and use that value. Now if you don't use auto layout, or you need more manual control, you can override sizeThatFits. And, finally, for the ultimate control, you can override preferredLayoutAttributes FittingAttributes, and provide not only the size information but also you can tweak the attributes like alpha and transform.

So specifying the cell size using one of these three mechanisms, it's pretty easy. Most apps will use auto layout, but for those that don't, they can take on more manual control with the second two mechanisms there.

But we found that for types of layouts, picking a good estimated item size turns out to be really challenging because sometimes it's just hard to guess.

I mean, sometimes what do you use? 50 by 50, 100 by 100, even more. It's hard to say.

For these types of layouts, what would be really cool is if flow layout could adapt its estimate and do the math on your behalf and instead compute that estimated size from actual sizes of content that we've sized.

So in iOS X, we've got new API on flow layout to do that.

All you have to do is set your flow layout.EstimatedItemSize to a new constant, UICollectionViewFlowLayout AutomaticSize.

By setting your estimated item size to automatic size, you'll indicate to the CollectionViewFlowLayout that it should do the math for you. It'll keep a running tally of all the cells that it's already sized, and use that to influence its future sizing estimates. As we'll see in a minute, this makes the flow layout much more accurate while it's sizing your CollectionView cells leading to better performance and a more accurate layout while we're sizing. So we're going to do a demo to show you the benefits of UICollectionViewFlowLayout automatic size.

So I'm going to switch back to the iPad.

Now here we've got a CollectionView using a flow layout. Each cell represents a word in a run of text.

Just to clarify, we're not recommending that you build a text viewer or editor using UICollectionView, but this does make a really good demo.

I'm going to put the app into a mode where we can watch the flow layout as it sizes each individual cell. No user would ever see this, but it's really great for visualizing the advantages of this new API.

Let's start out with fixed estimates like we'd use on iOS 9. So I'm going to turn on the simulation.

Here, you can see that we've seeded each cell with that initial estimated item size where we just guessed. And I'm going to size the first word in this run of text, Lorem, and watch what happens.

So we size Lorem, and now it's the correct size, but notice the CollectionView is really no closer to what its destination layout size will be. We didn't use that estimate to influence anything else, and everything else is still this large initial estimate that we passed in. Now this continues as we size each cell individually in this CollectionView.

Notice how we're not actually adopting, adapting any of the other cell sizes to account for the sizes that we've already computed.

This is especially obvious when we size the last word in this run of text, Fusce, and as we go to size this word, you'll notice that we slide up all the other cells, invalidating their y-positions, and we haven't used the information from that first line of text to influence the sizes of any of the other cells.

So now I'm going to turn the device into iOS X mode using UICollectionViewFlowLayout automatic size. So let's switch over.

OK, great. So we've seeded this with the same initial size to help you visualize the difference. I'm going to size the first word, Lorem, and watch what happens.

Wow. So we size that first cell, and use that cell size as a running estimate for all the other CollectionView cells. Now the layout's not totally accurate yet, but notice how it's a lot closer to what it will eventually be. In fact, the heights in y-positions are pretty much spot on.

And notice that as I size this run of text, we're adapting that estimated size, making it more and more accurate for your CollectionView cells holistically.

And notice when we size the last word in this run of text, we're no longer invalidating the y-positions of all the other cells. This is really great and can help out a lot when you're doing things like scrollToItemAt indexPath. Now due to the nature of this API, you'll get the most bang for your buck using automatic size if your cells have similar widths or heights. So that's it for self-sizing improvements in iOS X. Let's go back to slides. So that wraps up our self-sizing cell segment, and now we're going to talk a little bit about an API we introduced last year, interactive reordering.

Now, this is a familiar user experience to users of TableView cells. They like to, users might grab a piece of content and move it and rearrange it vertically in your TableView. So we brought this technology to CollectionView last year with a new interactive reordering API. Let's run back over to the iPad and see what this might look like in the demo.

Switch back over to the iPad. Alright. Here we are. OK. So we've got this really pretty looking custom layout. Oh, man, look at that scroll, Peter. We probably adapted pre-fetching, which is really easy. That's probably it. OK. So check it out. So let's say the user loves this content, but they want to rearrange it. Might drag their finger around and move the content smoothly around the CollectionView, and notice how it reflows automatically. And not only that, if we change to an item that has a different size, let's you be able to do the right thing. Now when the user's done reordering, they might let go. Smoothly animates into place, and that's it.

Let's switch back to slides, and let Peter go through that API. So that's interactive reordering as it existed on iOS 9, and this API is really simple.

To start an interactive movement, you're going to call CollectionViews beginInteractive MovementForItemat indexPath where index path represents the index path of the cell that we'd like to move around.

If you're doing this in response to a gesture, you can hit test the CollectionView using index path for item at location.

Next, each time the gesture updates, we're going to want to update the cell's position in response to our finger. For this, we're going to call updateInteractiveMovement TargetPosition. Passing in the location of the gesture in the CollectionView's coordinate space. Next, when we'd like to end the interactive movement and confirm the reorder, we're going to call endInteractiveMovement on the CollectionView.

CollectionView will put the cell down, handling all the layout attributes correctly, and then message back to your app's data source so that you could do the actual rearrange in your model.

Now, if the gesture cancels, or if you'd like to not allow reordering at this time, you can call CollectionView's cancelInteractiveMovement.

Here, we'll put everything back to where it was before, and we won't call your data source.

Now for those of you using UICollectionViewController, this is even easier for you to use. You just have to set the install standard gesture for interactive movement property to true, and CollectionViewController will add a gesture that will call these methods on your behalf. All you have to do is implement the data source parts. So that's the API that we shipped last year in iOS 9.

This year we're very proud to announce we're going to add paging support to this. Now there's no new API, which is the best kind of API, and since CollectionView is derived from UIScrollView, all you need to do is set the isPagingEnabled property of the ScrollView's inheritive property to true, and that'll enable the support. Let's switch back to the iPad and check this out in a demo.

And here, we've got a horizontal scrolling CollectionView.

This is currently not using paging. It's just continuous, and look at that smooth scrolling performance. Hey, hey. So in reordering when we're scrolling continuously, we pick up the CollectionView cell and move it to the edge of the screen, and then we get this really nice auto scrolling behavior. This is familiar to users of many CollectionView based applications. Now I'm going to turn on paging support, and we'll see that CollectionView now advances in page size increments. This is really natural for some types of CollectionViews. Now here's what's new.

We're calling the same reordering API, but new in iOS X, that reordering works alongside paging. So as I move my cell to the edge of the screen, we're going to automatically scroll in page size increments.

This can give your app a really home screen style reordering experience. I'm going to let go, and that's it. Let's go back to slides.

So they're a little . So while we were developing iOS X in the early phases, Pete would often come to my office very enthusiastic.

"Steve, I'm working on this great feature. Developers are going to flip." He's like that. And I agree, it's a great feature. I can't wait to show it to developers everywhere.

So during the run up to WWDC, now he's looking for a place to land this particular slide so we can talk about it and share it to people.

So even a couple days ago, he made the pitch again, "We've got to do it." So I said, "OK, fine." Two slides, ninety seconds, Peter. Can you get it done? Thanks, Steve. It'll just take a minute. Another feature that I'm really excited about that's new in CollectionView in iOS X is UIRefreshControl! That's right! UIRefreshControl is now directly supported inside of CollectionView.

But not only that, it's also directly supported inside of UITableView without using UITableViewController! And not only that, it's also supported inside of UIScrollView because RefreshControl is now just a ScrollView property! It's really easy to use, too. All you have to use, do is create the RefreshControl, add yourself as a target to it with an action, and set it on your CollectionView, and you'll be pulling to refresh in no time. Thanks a lot, Steve. It means a lot to me. Hey, great job. Great job. It's like the summarizer we talked about today. We went over the brand new UICollectionView cell pre-fetching stuff, and we can't wait to see what you guys are going to do in your apps with this. And then we covered the new pre-fetch data source API for Collection and TableView.

Next, we covered the improvements to self-sizing cells with the new automatic size constant. And then we chatted a little bit about the iOS 9 introduced API interactive reordering and the new paging support we have in iOS X. You want to check out the sample code and other resources for this app, for this session, we can see the address there on the developer website. We got some great related sessions with these technologies . We want to thank you very much for coming out. Have a great -- >> Thank you so much.