pouët.net

Detail wanking: Scrolling lists on touch screens.

category: code [glöplog]
So, me and psonice started having a conversation about drag momentum on touch screens, and we decided to bring it to pouët because it's more fun.

This is the initial mail I got :
Quote:
from BB Image psonice

Hi, psonice here, about that scrolling :D

I'm hopefully going to get to the scrolling part later today, so it could be good to share some ideas / code.

What I'm writing is basically a photo library viewer, where it shows a grid of thumbnails (that can be scrolled), and tapping an image will zoom it in to fullscreen. The whole thing is done in openGL (on iOS), and I'm basically writing it from the ground up. Which is frustrating as hell, since iOS has a decent photo library viewer, and I've written my own using the system APIs already, but this needs to do some 'special' stuff :/ So far the thumbnail screen is running and the tap-selection/zoom part is working (just getting the interpolation right, which is tricky for zoom + move somehow!)

Next up will be scrolling, although it'll be complicated because I have to load / unload thumbnails in on the fly, otherwise it's going to be slow + crashy if I pre-load all the (possibly thousands) of thumbnails at once :(

At the moment I'm thinking straight linear deceleration should be enough. Something like this:

1. When the finger leaves the screen, get the distance between the last 2 sampled positions (more than 2 + smoothing might be better), divide by time between samples to get velocity.
2. Update velocity per frame using "v -= friction;" where friction is a constant.
3. Move display position per frame by v to scroll.

There's probably a massive flaw in my logic, there usually is :D What do you reckon?

Oh, these might be helpful, a bunch of 'missing features' for animation:

// clamps X in the range A..B
#define CLAMP(X,A,B) MAX(A, MIN(X, B))
// Interpolates between A and B, position set by X
#define MIX(A,B,X) A*(1.-X)+B*X
// Smooth in/out transition. X should be in 0..1 range.
#define SMOOTHSTEP(X) X*X*(3.-2.*X)
// Accelerate in transition. X should be in 0..1 range.
#define ACCELERATE(X) X*X
// Decelerating transition. X should be in 0..1 range.
#define DECELERATE(X) 1.-(1.-X)*(1.-X)
To which I responded:

Quote:
from BB Image Graga

Cool about the photo viewer, I hope it has a nice twist to it!

So, I'm writing my list in Unity3D because I want to make my app to be cross platform from its birth. Exactly what is does is a little more secret, but right now I have implemented nothing but lists anyway, so no harm in showing you ;-)

I went around and looked at the scrolling behavior of my Android phones. I sadly don't have an iPhone to test with, but I knew I liked the way that my phones scroll already, so I'll just stick with that.

From their lists, I have observed that there seems to be 3 "phases" in the scrolling:
Unemcumbered movement (for a split second)
Decelerating movement
Full stop when the velocity reaches a certain threshold.
At least it feels like that. If you look at your phone, maybe you can find somewhere that behaves similarly ("Weee!" - "Slowing down" - "Full stop!").

From my own own failed implementation attempts, this it what I have to say:
Physically based models are not the way to go. Deceleration is either wrong or takes way too long.
You really should let the list come to rest quite quickly (see the last phase of Android's behavior that I described ago).

The problem about linear deceleration is, that while it mimics friction, it has an exponential relationship between the initial velocity of the object and the total distance travelled.

I got tired of typing characters into my e-mail window at this point (I am afraid it becomes hard to understand) and went on to create a graph (see attachment).

I also attached a little test for you using the theory that I just talked about. It requires that you install the Unity3D webplayer, but it is a very unintrusive piece of software.

Here is my code-snippet:

Code: float touchDrag_releaseTime = // Time when the list was released //; listPosition = Vector3.MoveTowards(listPosition , target, Mathf.Abs(touchDrag_velocity) * Time.deltaTime * 0.9f); touchDrag_velocity *= Mathf.Pow(Mathf.Max(0.1f, 1f - (Time.time - touchDrag_releaseTime) * 4f), Time.deltaTime);


Let me know what you think / if that was useful.


Attachments #1: Image

Attachment #2: Exe

Quote:
from BB Image psonice
Cool, I'm going to take a good look through this in a minute (I've literally just got scrolling working, so momentum is next :)

Quick thought though: if you do a fast swipe, you want to travel further down the list. So momentum must be linked to initial velocity, and the deceleration time can't be constant.

Also, how are you planning to handle the end of the list? I know this is different between iPhone and some (not all) android phones. The 'easy way' is just to stop, but it's much more natural if there is a 'rubber band' effect, where it scrolls past the end and bounces back.


Quote:
from BB Image psonice

Ah, ok. Your unity thing scrolls as expected (linear relationship between velocity + scroll distance, I should pay more attention :)

I'm going to put your code into grapher, see what it looks like, but it 'feels' good.


Quote:
from BB Image psonice

Interesting - your scrolling seems to have a rapid initial falloff, then a gradual falloff (assuming I've got the maths right there).

I've just compared it with what apple do on the iPhone too. It seems apple have 2 different scrolling methods, for different situations (possibly 'short list' and 'long list'?):

1. Rapid (possibly linear) deceleration. This seems to happen in emails, where you don't normally scroll very fast.
2. Gradual deceleration (feels a lot like my 2nd graph, but might be a higher power). This happens on the main email list, which is long and generally scrolled through faster.

I'm going to try that 2nd method first, see how it goes.
Quote:


After much fucking around with powers and curves, this works beautifully for me:
Code: scrollVelocity *= 1. - frameDuration * 4.; if( fabsf( scrollVelocity ) < 0.01) <end scrolling>

Set the scroll velocity to the distance moved per frame initially. I'm using signed velocity, it seems easier than tracking speed and direction separately. The 4 on the end is deceleration speed.

Once again, the old rule "if it's difficult, you're probably over thinking it" is true :D

Chris
Quote:
from BB Image Graga

Velocity Curves
Great, thanks for the graphcs. I spent some time doing my own plotting after I saw yours.

I forgot to tell you that when my velocity reaches a certain threshold, it stops (like the 3 phases in android)

In my program, initial speeds range between 1,000 and 10,000. When a velocity falls under 500, I stop it dead in order to avoid slug-like movement.

If you look at my plots, the fall-off is actually close to linear... so I suppose you can use a linear fall-off, as long as you scale it to the initial velocity! Are you gonna try that? :)

BB Image

Rubberbanding / End of lists
About the end of lists, I'm considering between rubber-banding and "heat" overlays like in this demo: http://www.youtube.com/watch?v=sZaW29Rh5u0

Averaging velocity over several frames
One thing that can be a little irritating about my scrolling is that you can some times get insane speeds if you just sample your velocity over one frame. So I might consider experimenting with something like:

Code:touch_velocity = 0.5 * (this frames raw velocity + last frames raw velocity)


[quote] from BB Image Graga

I really like how your scrolling feels at low speeds, but at high speeds I feel that velocity falls too quickly.

I changed my approach to include a linear component. It isn't simple, but it works... I simulated it at different initial speeds, and it is quite stable with regards to FPS!

Code: *** 100 FPS *** >>> 10000 initial speed gets to 4906 5000 initial speed gets to 2071 2500 initial speed gets to 787 >>> *** 10 FPS *** >>> 10000 initial speed gets to 4758 5000 initial speed gets to 1972 2500 initial speed gets to 726 >>> *** 5 FPS *** >>> 10000 initial speed gets to 4819 5000 initial speed gets to 1954 2500 initial speed gets to 680 >>>



Here is your approach ( v *= 1.0 - deltaTime * 4):

Code:>>> 100.0 FPS 10000 initial speed gets to 2399 5000 initial speed gets to 1199 2500 initial speed gets to 599 >>> *** Remote Interpreter Reinitialized *** >>> 10.0 FPS 10000 initial speed gets to 1499 5000 initial speed gets to 749 2500 initial speed gets to 374 >>> *** Remote Interpreter Reinitialized *** >>> 5.0 FPS 10000 initial speed gets to 499 5000 initial speed gets to 249 2500 initial speed gets to 124 >>>


You have some issues with regards to framerate, it seems.
So how does everyone else approach scrolling momentum and rubberband-effect in their apps? How to do it the most intuitive, robust, responsive, and best way?
This is why I hate Windows Phone and Android btw.
added on the 2013-06-04 14:06:49 by gloom gloom

basically as long as the finger is on the screen the scrolling should follow it but if you want the list to go faster you can "accelerate" by repeated swiping and releasing.

what i like about my twitter timeline (and probably many other touchscrolled lists on my android phone) is that it immediately full stops when i tap it while it scrolls.

the important thing about touch navigation is imho the felt responsiveness. if i have to wait too long for the scrollng to start or stop or until my gesture has an effect on the device it feels clumsy and annoying.
added on the 2013-06-04 14:22:13 by wysiwtf wysiwtf
as a user or a developer?
catch up time, here's my final (unsent since it was unfinished when the email->pouet move came :) reply.

Quote:
I really like how your scrolling feels at low speeds, but at high speeds I feel that velocity falls too quickly.


Yeah, I just compared it with the iOS standard scrolling, falloff was too fast. However, after I changed it to:

Code:scrollVelocity *= 1. - frameDuration * 2.;


It now matches the iOS default scrolling behaviour pretty much exactly. I think this is probably the exact method apple use :D I don't see much point in experimenting more, this works great and the code is fast.

Quote:
Velocity Curves
Great, thanks for the graphcs. I spent some time doing my own plotting after I saw yours.

I forgot to tell you that when my velocity reaches a certain threshold, it stops (like the 3 phases in android)


Yes, this is wise (in theory it shouldn't matter, but in practice it's probably going to move 1 pixel some seconds after it looks like it's stopped, which is horrible..)

Quote:
In my program, initial speeds range between 1,000 and 10,000. When a velocity falls under 500, I stop it dead in order to avoid slug-like movement.


Haha, mine are always <1 :D I'm matching velocity against view coordinates though, and the thumbnails are always 2*2 units in size. Because i have to zoom and scroll, sane coordinates are critical - I wasted a ton of time getting very confused before I figured that out :(

Quote:
If you look at my plots (attached), the fall-off is actually close to linear... so I suppose you can use a linear fall-off, as long as you scale it to the initial velocity! Are you gonna try that? :)


That's pretty much how my setup works. You take the initial velocity, and just multiply it by a number less than 1 each frame.

Quote:
Rubberbanding / End of lists
About the end of lists, I'm considering between rubber-banding and "heat" overlays like in this demo: http://www.youtube.com/watch?v=sZaW29Rh5u0


The 'heat' overlays were only used because of patent disputes (apple patented rubber banding like this). It's not as good, but probably a lot easier to implement. (Not sure what happened with that patent, but I have a feeling it might have been invalidated - anyway, I'm pretty sure it's safe to use rubber banding in an app.)

Quote:
Averaging velocity over several frames
One thing that can be a little irritating about my scrolling is that you can some times get insane speeds if you just sample your velocity over one frame. So I might consider experimenting with something like:

touch_velocity = 0.5 * (this frames raw velocity + last frames raw velocity)


Yeah, I guess worst case some background process triggers, a few frames get skipped and you get a large distance? I'm dividing distance by frame duration, so hopefully that doesn't happen, but still it'll be down to finger tracking accuracy (normally good, but there are some low end android devices out there that might not be).

One thing though: what about very rapid finger movements? The speed of the very last frame might be a lot bigger than the previous frame, and in some cases that might be accurate, so smoothing would make it worse. Only thing I can think to fix that, is to smooth over multiple frames for slow movement, and fewer frames for faster movement (down to no smoothing for very rapid gestures). Guess this has to come from testing, maybe it's not needed.

I'm not sure it's necessary on iOS. I'm using 'UIPanGestureRecognizer' to track scrolling, and without any smoothing at all it seems to track my finger perfectly and momentum scrolling always seems to follow finger speed. So I guess either the touch tracking is near enough perfect that it doesn't matter, or that object has built-in smoothing.
added on the 2013-06-04 14:36:17 by psonice psonice
graga: as a user of course
added on the 2013-06-04 15:38:55 by wysiwtf wysiwtf
added on the 2013-06-04 14:22:13 by wysiwtf
added on the 2013-06-04 14:22:19 by Lord Graga
I don't think he meant you. ;)
shouldnt this be an OS problem? well, it is deffo an OS problem when there's no standards for it :D but roger-roger, gloom :)
maali: normally, yeah. In fact it's trivial to just put a list of stuff in a scrollview and forget about it on iOS, I guess there's some similar feature on android. But there are times where the OS-level stuff just isn't suitable.

In my case I'm doing a custom UI in opengl, and there's really not much in the system I can use for scrolling. I could subclass the iOS view rendering system and replace the drawing parts, but then I have to handle things like scrolling at different zoom levels and some pretty unusual animation setups that the OS stuff just isn't well suited to.
added on the 2013-06-04 16:12:03 by psonice psonice
A bit out of sync, but this bit was missing the graph that might help make it a bit clearer:

Quote:
from psonice

Interesting - your scrolling seems to have a rapid initial falloff, then a gradual falloff (assuming I've got the maths right there).

BB Image

I've just compared it with what apple do on the iPhone too. It seems apple have 2 different scrolling methods, for different situations (possibly 'short list' and 'long list'?):

1. Rapid (possibly linear) deceleration. This seems to happen in emails, where you don't normally scroll very fast.
2. Gradual deceleration (feels a lot like my 2nd graph, but might be a higher power). This happens on the main email list, which is long and generally scrolled through faster.

I'm going to try that 2nd method first, see how it goes.
added on the 2013-06-04 16:33:17 by psonice psonice
First order variable-framerate iir with resonance. Actual position (real part of the filter value) follows finger exactly when pressed and the filter takes care of inertia when released. Work out the non-resonating position when clamping and you get the rubberband-effect.
added on the 2013-06-04 16:33:45 by 216 216
I coded some scroller like this and fought a lot with all of this back in the PocketPC era... not that I remember shit (this was MANY years ago). I'll try to find those sources but I wouldn't count on it. If I find them I'll post them around.
added on the 2013-06-04 19:33:54 by Jcl Jcl
216: nice considerations!
Other aspect: distinguishing between:
. Tap to select
. Tap to stop
. Tap+drag (threshold) to scroll

Overall not totally trivial and there's not much out there... some call it "kinetic scrolling".
added on the 2013-06-05 00:12:30 by bdk bdk
While implementing a slide/scroll control that have this "analog feel", I went for a physics-based model:
* The widget to slide have a mass, an elastic friction
* The user input is translated into forces applied on the widget to slide
* The boundaries (where the widget is supposed to stop) are also translated into forces
* I used Verlet integrator to update the physical model

Essentially, it boils down to : Aw(t) = Ai(t) - Ab(t) - k * Vw(t)
* Aw(t), Vw(t) is acceleration and velocity of the widget
* Ai(t) is the force from the user input. In my case, 0 if key not pressed, 1 if key pressed. With a mouse, you can use the averaged difference between the N last mouse moves
* k is the elastic friction
* Ab(t) is the acceleration from the boundaries, it was non-trivial, forgot the details :p

When the widget is at rest, it accelerate : moves slowly initially, eventually get faster, with the maximum speed controlled by the amount of elastic friction. When the widget meets a boundary, it slows down and halt just at the border. The units of the physical model where millimeter/milliseconds, to be screen-resolution independent. It took a few minutes to find satisfying parameters. The relationship friction/maximum speed have a simple analytical form (exponential something)
(woooops, correction : in the formula, multiply the whole right side of the formula by the mass of the widget)
Lord Graga: as a user. The scrolling and flipping just doesn't feel right at all.
added on the 2013-06-05 09:46:06 by gloom gloom
bdk: yeah. Tracking state is critical, as is determining whether a touch is a tap, pan or swipe. Luckily iOS handles that well for me, but there's still the issue of determining whether a tap should select something or stop scrolling, and whether a pan should have momentum scroll on release (I use a simple "if(velocity > someValue) <do a momentum scroll>" to prevent annoying tiny movements).

marmakoide: what situation did you use that in? I can imagine it working great in some situations, but i think scrolling a list view or something the weighted acceleration would feel somehow wrong. Weighted movement from a 'flick' gesture works well, but if you're just panning up and down with your finger on the screen I can see that feeling laggy.

And the boundary handling, do you mean the scroll slows automatically as it approaches the boundary? I think apple's way of scrolling past the boundary but springing back seems more natural there, but I'll try this both ways.
added on the 2013-06-05 10:55:12 by psonice psonice

login