Sunday, May 07, 2006

Frame rates in the Flash Player

There have been numerous posts about frame rate issues in Flash over the years, sometimes with quite inconsistent tips and workarounds. As we are approaching the final phase of development for Flash Player 9 at Adobe our bug database is filling up with duplicate bugs concerning old known issues. What is frustrating to designers is that they perceive the Flash Player changing its behavior over the past few releases, although it has not. Well, that might actually be the problem.

Flash uses a relative timing model, meaning it does not really care about a global frame rate, but will instead try to enforce frame intervals as best as it can. Say you have f.ex. set your frame rate to 30 frames/sec. That means that the Flash Player will try to wait for 33 milliseconds before trying to display the next frame (excluding the time it takes to render the frame). This loose timing causes all kinds of problems. First, the Flash Player depends on high level OS events to deliver timing messages. In the worst case this means the use of WM_TIMER, dependence on the NetScape plugin API or in the best case we use multimedia timers provided by a special Internet Explorer API. Second, we round frame intervals to milliseconds since Windows and MacOS can't support fractional time intervals. Third, the OS, the browser and the Flash Player will add overhead to the code executed on each frame, meaning that in the end the actual frame rate will sway between -10 to +5 frames/sec from the actual selected frame rate, depending in what environment you play it in. In Flash Player 8 and Flash Player 9 new overhead is originating mostly from the GC, something for which there is no workaround. As I said we do not calculate frame rates on a global basis so we can't correct it actively.

Lets talk about maximum frame rates. In Internet Explorer this is 100 frames/sec. Why? Because the minimum time slice Windows timers can provide is 10 milliseconds. What about FireFox? FireFox does not use special timers and made a decision to limit the maximum frame rate for plugins. Why? The thinking is that users constantly complain that plugins take too much CPU time. A valid complaint I think and every designer who puts online ads out there at higher than 8-12 frames/sec and more that 2 or 3% CPU usage should be ashamed. While a single ad will not be a problem, most pages easily serve 2 or 3 ads on a single page.

The Mozilla team also decided that plugins would get no time when they are on a hidden tab so it would not render the browser unresponsive or less responsive by adding new tabs. So do not be surprised if your SWFs and FLVs do no play on hidden tabs. Apple went even a step further in Safari: If the browser is not active, plugins will only get about 4 frames/sec, mainly to save battery and avoid the dreaded noise of the fans. Try it, go to Google Video, play a video and then switch to another application. The frame rate will drop to about 4 frames/sec. While we could drive our own background thread and work around this, there is a reason they decided to take these steps. We would be ill advised to just hack around it.

What does this mean? Well, the frame rate you select does not really mean too much and you should not depend on it in a way to be accurate to the millisecond. This especially goes for ANY sort of synchronization. If you need synchronization your only choice is placing code in ActionScript which will 'correct' your timing or workarounds like placing a streaming sound on your main time line (In which case we use the audio device to report time correctly to the nanosecond. Due to bugs this does not work correctly on Linux right now, which is the reason audio and video are out of sync, even for FLVs :-( )

What does the future hold? As I explained in a earlier post we will likely add synchronization primitives into the player to allow SMILE like global timing at some point. But there is also a good chance we'll limit what users can actually do when it comes to frame rates and overall CPU usage. There are various ways we could enforce low CPU usage. SWF files originating from a different domain (speaking advertisement) could get a lower priority and have a frame rate cap which would be user selectable. Secondly, with the advent of GPU support in the OS there will be a time when we finally add VBL wait, meaning tearing free drawing. In most cases this means the maximum frame rate will be 60 frames/sec. On high CPU load we might actually cut this into half, e.g. 30 frames/sec. OS X already does this in certain conditions.

50 Comments:

Anonymous Brooks Andrus said...

Great insights Tinic--I definately appreciate your blog. However, I'm a little unclear on what exactly this means for Flash Player 9? Are there any improvements with regard to frame rate and audio synchronization in store for Flash Player 9, or are we waiting for Player 10 and beyond? I'm sure you guys are pretty much feature complete, but I'm hearing more and more feedback from the edu market requesting variable speed playback of Flash files. There's a strong push in these markets to use wmv and quicktime because of their support for 1.5x, 2x and 3x speed playback. Any chance that this feature might be supported at some point?

Sunday, May 07, 2006 8:02:00 PM  
Anonymous Metah said...

Now Frame rates are clear for me !
A bit more complex that I thought :)

Is there anyway to have a CPU feed-back in Flash ? A sort of System.capabilites of the CPU usage ?

Cheers,
Ahmet.

Monday, May 08, 2006 2:58:00 AM  
Anonymous Tink said...

You have variable framerate in F9 using Stage.frameRate.

http://livedocs.macromedia.com/labs/1/flex/langref/flash/display/Stage.html#frameRate

Monday, May 08, 2006 3:03:00 AM  
Anonymous Andre Michelle said...

As my previous speaker said: Thanks for mention this problem finally from the flashplayer-developer site. It is nice to see, why thing are so chunky, but are they solvable ?

Can't you contact mozilla for updating the firefox providing for power to the plugin ? I tried to adress the problem via their board, but got no answer. I could imagine, that a person from adobe has more influence about this politics.

related blog entry

Monday, May 08, 2006 3:16:00 AM  
Anonymous Metah said...

Great :)

Monday, May 08, 2006 3:20:00 AM  
Anonymous Mike said...

In that case would it be possible to build in something to the ide that lets us test as per the different environments? A setting in the publich preferences maybe?

Test as IE
Test as Firefox
Test as Opera
Test as Standalone

Monday, May 08, 2006 3:31:00 AM  
Anonymous peter said...

FPS always was a problem , but why always guilty CPU or internet viewers, why not use another POWER ? GRAPHIC CPU and jump this problem ?, all graphic cards have acceletators ... all OS may use OPENGL, why not suport this power ? where is the problem ? please tell us, thx

Monday, May 08, 2006 3:50:00 AM  
Anonymous Brooks Andrus said...

"http://livedocs.macromedia.com/labs/1/flex/langref/flash/display/Stage.html#frameRate"

When using this new api, the question is whether changing the frame rate of a swf file that has streaming sound would have any effect at all. Currently streaming sound will dictate the "frame rate" which Tinic mentions as one possible hack for synchronization.

The edu market wants swf / flv files with audio to playback at accelerated rates (turn a one hour lecture into a 40 minute lecture).

Monday, May 08, 2006 8:42:00 AM  
Anonymous Anonymous said...

maybe Flash 17 will use opengl...maybe

Monday, May 08, 2006 8:44:00 AM  
Anonymous Anonymous said...

I think I understand, however if adding a streaming sound is one way to 'hack' a more reliable framerate then why should it even be 'hack'? Surely if you can make things easier with a streaming sound you could make things easier without a stremaing sound too - if it's possible it shouldn't need hacks, if it's not possible then it needs fixing - Adobe has real muscle in the digital world so why not flex it a little, hand over a big donation to mozilla and maybe buy opera :D and just get it sorted - yes it may take a while, but it almost sounds ludicrous to have a frame-based animation app that can't actually produce reliable framerates.

I'm not angry, just frustrated, this is an issue that whilst I'm happy it is being looked into and I appreciate it's not the former macromedia's fault exactly, it still should have been fixed.

Monday, May 08, 2006 12:18:00 PM  
Blogger Tinic Uro said...

Brooks,

there will be no changes for Flash Player 9 concerning any of this. As you might have noticed the focus of Flash Player 9 was really ActionScript 3.0.

As for speeding up video playback, this is definitly something we could look at for the next version

Monday, May 08, 2006 12:25:00 PM  
Blogger Tinic Uro said...

metah,

There is currently no way to access the CPU frequency in ActionScript. As we think about features like these we always have balance this with privacy concerns.

Although it might not make much sense to you, leaking any local information to a server is regarded with great suspiscion by the security and privacy community. We have to be extremely careful what we choose to expose.

Monday, May 08, 2006 12:30:00 PM  
Blogger Tinic Uro said...

andre,

As I said, the limitations in FireFox were a concious choice, so convincing them would be pretty pointless. In the end we have to balance the complaints by users about hight CPU usage and unresponsiveness of the browser against designers who want to have the highest performance.

Since neither users or designers are happy, we might have found the perfect compromise. ;-)

Monday, May 08, 2006 12:32:00 PM  
Blogger Tinic Uro said...

Anonymous,

the streaming sound hack will not create a more consistent frame rate. It will only improve global timing by dropping frames, allowing to better synchronize content (although it still has some problems in this area due to hard limitations). It does not change the average or maximum frame rates.

Monday, May 08, 2006 12:36:00 PM  
Anonymous mray said...

The streaming sound is a great trick, problem is if you need to use it in multiple timelines it can use up the available sound channels.

What if we could create one such 'kicker' clip and bind the framerates of other clips to its update interval. That way we can ensure timelines will proceed in lockstep and drop frames identically.

Even if I can't get timing down to the nanosecond, as long as animation can degrade easily, consistently, and gracefully, I would be a very very happy developer!

Monday, May 08, 2006 4:36:00 PM  
Anonymous RonB said...

I’ve tested a 999 fps Director shockwave file in a browser (IE and FireFox) where the frame rate wasn’t reduced – it actually ran at 999 fps inside a browser. How is it that shockwave can maintain such a high target frame rate, but Flash can’t get much more than 30 fps?

I’ve also tested an empty swf file inside Director (with the latest 10.1 Flash Xtra), and a 120 fps swf will only run at 30 fps with the default settings. Interestingly, placing 12 copies of the swf on the stage on the same frame will still show a frame rate of 30 fps for all 12 swf sprites. Either this is a bug, or there’s some kind of fps limiter in the Flash Xtra.

Tuesday, May 09, 2006 11:07:00 AM  
Blogger Tinic Uro said...

RonB,

I thought I was pretty clear:

"FireFox does not use special timers and made a decision to limit the maximum frame rate for plugins. Why? The thinking is that users constantly complain that plugins take too much CPU time. A valid complaint I think and every designer who puts online ads out there at higher than 8-12 frames/sec and more that 2 or 3% CPU usage should be ashamed. While a single ad will not be a problem, most pages easily serve 2 or 3 ads on a single page."

and

"While we could drive our own background thread and work around this, there is a reason they decided to take these steps. We would be ill advised to just hack around it."

Shockwave, QuickTime and Windows media use their own timing threads. The downside is that it's fairly easy to take over the CPU, therefore waste battery and make browsers unresponsive. Users already constantly complain to us about the fact that a lot of Flash content uses too much CPU time. There is no reason we should make it worse by working around measures browser have already taken to lessen this issue.

Tuesday, May 09, 2006 11:32:00 AM  
Anonymous Anonymous said...

Interesting reading!

So why not make the player less cpu-intensive and make use of the graphics card for rendering, like someone mentioned? As I understood it player8 for osX uses opengl in some way, or are there any problems regarding that?

Or couldn´t you make it so the player works like it does now when the swf is out of focus. But when the swf has focus from the user it moves over to it´s own thread and are allowed to use more cpu and so on..?

It´s kind of frustrating knowing this issues is partly because "..most pages easily serve 2 or 3 ads on a single page." ;)

/klas

Thursday, May 11, 2006 4:22:00 AM  
Anonymous Anonymous said...

Where does setInterval() to the frame rate debate?

Friday, May 12, 2006 8:01:00 AM  
Anonymous Anonymous said...

you want consistant motion without worrying about frame rate.

check out moocks lecture at flash forward 2001:
programatic motion based on time not frame rates.

http://moock.org/lectures/ff2001sfMotion/

Tuesday, May 23, 2006 9:46:00 AM  
Anonymous Cameron said...

The problem with using time corrected animation techniques is that the timer (in flash) isn't all that accurate. I've built some physics simulations this way and anything above 40fps seems to introduce timing spikes that send the simulation crashing out of control as I'm using the time to derive various equations of motion.

This usually only happens when you move/resize the window or activate the conxext menu, but with systems that rely on time, it's a killer.

Tinic, how about a background thread for flash content playing from the same domain or the ability for the end user to indvidually grant a movie rights to such things?

Director is on it's last legs, we need flash to step up to the plate and play ball already.

Thursday, May 25, 2006 5:09:00 AM  
Anonymous Anonymous said...

Java plugin in that same browsers works more more faster. Is ALL problems with web browsers ?

Monday, May 29, 2006 12:55:00 PM  
Anonymous Anonymous said...

Re: Cameron and Time-based programming

Any physics-based simulation should always have a fixed time-step. As you noted, feeding the change in time between frames to a physics engine will result in non-deterministic and inaccurate results, because this time-step will vary wildly! :) Preferably, a physics engine should calculate states at fixed time intervals, and the display should interpolate between these states in the meantime.

-Mike Welsh

Tuesday, June 06, 2006 3:11:00 PM  
Anonymous Anonymous said...

As I said, the limitations in FireFox were a concious choice, so convincing them would be pretty pointless. In the end we have to balance the complaints by users about hight CPU usage and unresponsiveness of the browser against designers who want to have the highest performance.

Since neither users or designers are happy, we might have found the perfect compromise. ;-)


I really can't see why this would solve anything? For example using the new bitmap capabilities in FP8, you really could hog your CPU even when using 12FPS.

If you really would like to compromize you would need to control how much power the FP can get from the CPU.

Wednesday, June 07, 2006 12:53:00 AM  
Anonymous Anonymous said...

Good article! Is it possible to use JavaScript to adjust the frame per second of a Flash movie that is currently playing?

The idea is that my Flash movies play at 12 fps by default, but this may be too slow for some users. Ideally, there would be an HTML jump menu with options of 12, 14, and 16 that would adjust the playback speed of the .swf on the fly. Is this possible and can you point me in the right direction? Also, will it be possible for .flv files too?

Thank you

Friday, November 24, 2006 4:45:00 PM  
Anonymous Anonymous said...

Great read, very interesting.

My only concern regarding framerates is really the inconsistency of said rates between the various browsers. You can easily have an 80fps movie run smooth in IE, but in FireFox you will not only run around 50fps but also those 50 frames are not running smooth in the browser, constantly jumping between 30 and 50.

Personally I'm not that keen on being provided with high framerates because I can see the reasoning behind the limitations in FF. I also do all my programming based on timing so sync is not really an issue to me, but the inconsistent performance is.

To me it would be a great favour if we could at least have the FF-plugin run in a way that keeps a defined maximum framerate smooth. Because as I said it is easy to adjust for 30/50 fps, but it becomes a problem when those rates are jumping around like Kangaroos, making animations look sloppy.

What I really want is clear guidelines regarding how far I can push the plugin without running into problems. For example, it doesn't make sense to make any extensive use of the Bitmap classes when they can easily trash the performance in FF (while running smooth in IE... at least *one* thing they did right ;)

Regards,
Tango

Monday, January 08, 2007 3:05:00 AM  
Anonymous DrNeroCF said...

So basically you're saying that Flash isn't meant for anything powerful whatsoever? I guess that would explain why my games run fullspeed in the standalone and lags in the browser while only using 1% of my CPU. *sigh* what about Flash programs that the user is completely focusing on? You know, Flash games? People still make and play those, but I think you guys are ignoring us Flash game designers completely.

I don't even see why you guys have a brush tool in Flash anymore, Flash is obviously not supposed to be an easy to use self contained 'Flash game' creation program anymore. Focus, guys!

Plenty of sites out there need Flash to at least have the option for opening up another thread for more CPU usage! ESPECIALLY now that so many computers out there are duel core.

Flash created simplistic online gaming, they're not called 'Flash games' for nothing, why are you guys ignoring this market now?

Wednesday, January 31, 2007 2:58:00 PM  
Anonymous Jesper said...

What I am really missing in Flash is a fullscreen capability.

Now that almost all videos on the internet is played by flash, it is REALLY needed to be able to go fullscreen.

Best the videosites can do now, is open it in a maximized browser, wich is not really good enough at all.

It would also be good for many flash games.

Besides that. Please find a way to give us the exact framerate that we chose. Atleast the flash should always be played at the desired framerate, as long as not violating any issues of taking too much processor time or whatever limitations the user wants.

Tuesday, February 27, 2007 6:48:00 AM  
Anonymous gry said...

Very good and great site with very good look and perfect information...i like it.

Saturday, April 14, 2007 9:59:00 AM  
Blogger BadSeed said...

Hi,
I am looking for a way to access the flashplayers output. I need to capture a flv video without screen capturing. Is there a way? And i could not understand how flash renders to the screen. Is it using opengl or directx. If it is its own renderer is there a way to access the frame buffer?
Regards.

Wednesday, April 18, 2007 8:43:00 PM  
Anonymous Anonymous said...

Well the problem IS that it isn't using OpenGL or DirectDraw to render, so rendering eats up your CPU (if you have a lot of vector graphic) and this is the reason why the browsers limit it. I'm a game developer and I find this a killer, you limit the plugin and don't leave the user the choice to set these things? It's like saying well you will use 640x480 resolution from know on, just so that everything runs smoother and you won't be allowed to change that even if you wanted to. The other thing is that thanks to the software render that the player uses you don't even get the 25 frames per seconds in rendering if you set it to 25, try any desktop game at 25fps and then a flash game and you will notice what I mean. Don't get me wrong Adobe made a move in the right direction with AS3, but we are only half way there, a hardware renderer is a must!

Wednesday, May 09, 2007 9:07:00 AM  
Anonymous strony internetowe świdnica said...

Cool site! Your website is a powerful tool for the visitors!

Monday, May 21, 2007 1:02:00 AM  
Anonymous Anonymous said...

tinic : you said

--
If you need synchronization your only choice is placing code in ActionScript which will 'correct' your timing or workarounds like placing a streaming sound on your main time line (In which case we use the audio device to report time correctly to the nanosecond.)
--

How does one go about including streaming sound in the main time line using AS? Does this refer to a small mp3 or something else? I am finding (much) higher variability in FF (compared to IE) in brief displays using setinterval (even at high frame rates which does increase its resolution a bit) and would appreciate any ways on using the "audio" timer for FF users on windows.

Monday, May 21, 2007 6:24:00 PM  
Blogger shiretu said...

Another issue that I've detected in flash player is the time to access USB webcams. 'till now, I was unable to make a webcam work in 640x480@30fps in flash player. I'm not an expert, but here is what I've discovered (may be wrong):

1. AMCAP manages to put them in 640x480@30fps (so the camera supports the setting) but flash plugin doesn't. To grab a video packet out of a regular USB webcam, flash is spending between 30 and 60 milliseconds. Add to this the time to grab an audio packet which is between 30 and 40 and you get a value around 100 milliseconds spent by flash to get a hold of a complete audio/video frame (I suspect that is not doing audio/video grabbing in separate threads). That is between 9 and 15 (the best case scenario) fps. Crappy! Without sound, the things are a little bit brighter. Also linux is a little bit faster at grabbimg packtes out of the audio/video sources. This conclusion was made after testing with 6 different webcams in windows and 2 on linux. I really don't know what the flash player is doing in that 30 to 60 milliseconds interval in which he is trying to grab the video packet: intentionally sleeping or waiting for the packet to arrive from the USB.

2. I suspect that FME is working at another level with USB webcams (lower level, more close to drivers). It manages to obtain a very fast access between 0(I've seen that, maybe is something wrong with the header sent by the FME) and 20 milliseconds. Verry good. That is why FME is capable to achieve high fps rates at high resolutions

Friday, July 20, 2007 1:25:00 PM  
Anonymous <a href="http://drugscenterhere.com">ShopMan</a> said...

I like articles like this. Thanks!

Saturday, August 25, 2007 11:28:00 PM  
Anonymous <a href="http://m1.aol.com/phentermine4">Phentermine</a> said...

Great Article! Thank You!

Tuesday, August 28, 2007 5:50:00 PM  
Anonymous <a href="http://m1.aol.com/phentermine4">Buy Phentermine</a> said...

Thanks to author! I like articles like this, very interesting.

Wednesday, August 29, 2007 5:16:00 AM  
Anonymous <a href="http://free-metro-pcs-ringtones.blogspot.com">Free Ringtones</a> said...

nice blog!

Sunday, September 02, 2007 3:10:00 PM  
Anonymous <a href="http://buy-viagra2007.blogspot.com">buy viagra</a> said...

nice blog!Nice information

Monday, September 03, 2007 6:21:00 PM  
Anonymous <a href="http://buy-levitra--ooz.blogspot.com">Levitra</a> said...

:-) ochen\' zaebatyj blog!

Tuesday, September 04, 2007 7:00:00 AM  
Anonymous <a href="http://courses.cvcc.vccs.edu/ENG112_GROSS/_Chat_Room/000008fd.htm">Anonimous</a> said...

Well done. Keep up the great work. Best regards!

Sunday, September 09, 2007 10:06:00 AM  
Anonymous <a href="http://search.cnn.com/search?query=site:cialis-online-2007.blogspot.com?cialis_online.html">Anonymous</a> said...

Keep up the great work. It very impressive. Enjoyed the visit!

Sunday, September 09, 2007 4:43:00 PM  
Anonymous <a href="http://buy4soma.eamped.com">Anonimous</a> said...

I like it a lot! Nice site, I will bookmark!

Monday, September 10, 2007 6:10:00 PM  
Blogger Hai said...

I'm facing a problem, I'm recording voice to accompany a music video, when I playback the voice, trying to synchronize it with the music video, I face multiple problems.

1. I don't have fine-grained control on the timing of playback of the FLVPlayback component. I try to time the voice stream based on the Play event fired by the FLVPlayback component, it is inconsistent. Also the timing is limit by the SWF frame rate.

2. As the music video proceeds, it starts lagging relative to the voice recording. I think it lags because of the video rendering.

My question is, if you were to do what I describe, how would you do it? Is it possible to perform synchronization on that level within the limitations of the flash plugin?

Tuesday, October 09, 2007 10:57:00 PM  
Anonymous Anonymous said...

"A valid complaint I think and every designer who puts online ads out there at higher than 8-12 frames/sec and more that 2 or 3% CPU usage should be ashamed. "

8 fps? Ashamed? That is too funny. Have you ever seen an ad run at 8 fps? Looks like a flipbook, a glorified animated gif. Might as well go back to using DeBabelizer too. After 10+ years now I understand why Flash has NEVER performed well and still has not taken advantage of the computing power available today. The goals/aspirations of Macro/Adobe are so meager.

Thursday, October 11, 2007 10:47:00 AM  
Anonymous Jim H said...

Meager indeed..

Flash in their 9th release runs like like it was written in VB4 running on a 386.. Using higher resolutions makes the problem worse even though no more information is being displayed from the video. Most media players have no problem rendering much higher resolution content but with Flash forget it, IMO it's garbage--using the term "frame rate" alone over states its current performance potential.

Saturday, November 24, 2007 1:04:00 PM  
Blogger Alex123 said...

8-12 FPS - ashamed? Running tweens at this rate is a far worse experience than watching your CPU meter jump 10% or 20%. Big deal. That's what CPUs are for. A frame-rate that low is ridiculous and a non-starter. This is what makes Flash animations look like feeble stuttering 3rd-rate cartoons.

I'm going the other way here (and have been for over 4 years).

I run all of our ads at 120 fps. The designs take into account the 'frame-choke' with heavy tweens and over saturating the display area with movement. Just don't do it. Plan your choreography to accommodate these concerns.

Learning to concept designs that utilize high-impact tweens (i.e. high frame-rate) is the challenge. Typing in a lower number into the frame-rate field is not, and should not, be the typical strategy. Users watching and engaging well executed high impact (rich-media) creative will appreciate it and respond positively.

Tuesday, April 01, 2008 3:44:00 PM  
Blogger Anil Philip said...

I am trying to find the duration of a clip, and trying to set the play position.

I am really stuck because I don't know how to find the frame rate - can you please reply? also copy to my email account.

http://www.adobe.com/cfusion/webforums/forum/messageview.cfm?forumid=44&catid=184&threadid=1367557&enterthread=y

thanks,
Anil

Tuesday, June 03, 2008 1:34:00 PM  
Blogger Anil Philip said...

how do you get the frame rate?
I haven't been able to find the answer.
I need to calculate clip duration = FlashPlayer.TotalFrames() / frame rate

Please email me your reply
anil AT juwo DOT com

thanks, :)

Wednesday, June 04, 2008 8:24:00 AM  

Post a Comment

<< Home