kaourantin.net

Adobe Open Screen Project

noreply@blogger.com (Tinic Uro) — Fri, 02 May 2008 20:07:00 +0000

Unless you have been living under a rock you probably saw the announcement Adobe made this week:

http://www.adobe.com/openscreenproject/

This is a significant move by Adobe to bring the Adobe® Flash® Player to a much broader audience than ever before. By doing this we are responding to the requests from customers throughout the different industries which want to leverage the Flash platform.

So, yes I haven't been blogging in a while, that has been due to my busy schedule (I'll be more specific soon). But since I've worked on fixing up the FLV and SWF specs a little bit for this project I figured I should add my own personal thoughts.

As the site above mentions this project essentially consists of the following changes:

Removing restrictions on use of the SWF and FLV/F4V specifications
Publishing the device porting layer APIs for Adobe Flash Player
Publishing the Adobe Flash® Cast™ protocol and the AMF protocol for robust data services
Removing licensing fees – making next major releases of Adobe Flash Player and Adobe AIR for devices free

All of these are very significant for a lot of customers although individual parties might think that it is one particular item which matters most to them. I assure you that all of them are extremely important.

Bringing a platform like the Adobe Flash Player to devices has its challenges. One of them revolves around documentation. By making the file format and API documentation open, the entry barrier for a lot of developers becomes much lower. In the past it could have been a struggle to get the necessary documentation to the teams which need them. This problem is now gone.

In many circumstances technologies Adobe provided were not compatible with how a particular device or infrastructure was set up. Given the old licensing restrictions it could prove difficult to find technical solutions. By making the specifications open and without restrictions a particular vendor might instead choose a clean rewrite of some parts if they see fit with the result that it integrates better into their eco-system.

Now there have been questions if there are any gotchas in what Adobe has announced concerning the SWF and FLV specifications. There IS in fact a license, it is on page two of the specifications. You might be shocked however by how small it is. It's essentially BSD license style damages disclaimer plus some additional information about trademarks and such. Read it before you use it.

The specifications are available here as PDF files:

http://www.adobe.com/devnet/swf/
http://www.adobe.com/devnet/flv/

Does that mean you can implement your own clone which implements what is in these specifications without fear of Adobe coming after you? Yes. Notice that I am careful how I ask this question though. Adobe still holds all the trademarks around this technology, you can not use any of those in your clone or anything related to it. Unless we give the OK to do so obviously.

If you find inaccuracies or missing information in these specifications let us know, we can integrate the feedback right away so they can appear in the next revision of these documents.

Adobe Flash Player 9 Update 3 released

noreply@blogger.com (Tinic Uro) — Tue, 04 Dec 2007 05:09:00 +0000

We made it! We just shipped Adobe Flash Player 9 Update 3, the final version number is 9.0.115.0. As usual a number of different flavors of the web plugin are available, the main install page is here. And yes, we also shipped the Linux x86-32 version. We have done this for several versions now and we will not stop. The only exception is the Solaris version which is still in testing. I can't tell you though when a new player for Flash CS3 will be available since I do not know.

This version is a significant milestone, and I expect it will be adopted fairly quickly once content using H.264 and AAC starts appearing. Emmy Huang has some additional information and links you should look at.

Here is a condensed list of bugs which have been fixed since the last release candidate (9.0.64.0). As you can see these have been mostly fixes for crashers and backwards compatibility issues. It's a pretty long list which explains why 2 months have passed since the last release candidate. Maybe you'll spot a few you were affected by:

212379 audio playback in swf off sync.
212294 on (release) events not firing on sprites
212224 Heavy FMS Seeking in Vista can cause bad audio and crash
211921 Crash when paging through a Flex app and closing the player
211898 FileReference IO errors when uploading multiple files
211840 playlist does not advance after end of commercial
211836 [MP4] If you seek in the NetStream before a movie that has the moov atom at the end has completely downloaded, you can crash.
211813 BitmapData draw ColorTransform not affecting all text on all platforms
211759 inputted simplified Chinese characters are not correct inside of Flash Player
211727 Japanese/RHEL4/Firefox 1.0 -- hang when selecting print dialog
211725 Player crashes when Premiere Express is connected to a server that times out
211700 Add support for Opera XEmbed
211672 the pop-up print panel layout is incorrect when R-click on the red button and choose ‘print…’
211625 Flash player crashes while playing servierside playlists containing MP4 files
211593 Crash in NavigateToURL when using local relative URL
211584 can't input anything to flashplayer for Simplified Chinese Linux OS
211582 New AS3 NetStream.receiveVideoFps() should be NetStream.receiveVideoFPS()
211558 Flash Player movies soft-freeze
211504 getURL loading with UTF8 Japanese text data crashes IE6 on WinXP J
211498 crash if flash unloads while in a JS function called by ExternalInterface
211493 Audio locks up and starts looping on Linux, requiring the page with the player to be shutdown
211491 AS3/2 keyDown event (Code Keys) not firing for NumPad on Linux
211485 Flash previewer hangs the browser
211403 Found an audio sync issue after a pause and seek
211367 Memory leak when subscribing live stream with buffer time 0 published by Flash Media Encoder 2 in MP3 audio format
211362 One more time: An A/V Sync issue
211346 extra lines appear when a radial gradient is on top of a radial gradient
211343 Video does not resume after pause and seek
211298 Accessibility Plugin reads text from dynamic text field after tf.text has been set to ""
211264 Fms seeking while paused won't always generate an image
211263 FMS Video streams when seeking can go out of A/V sync
211186 H264 files : Loss of audio sync during the playback after seek
211171 site no longer automatically advances to next song
211120 new file icons for F4V, F4P, and F4A for the standalone player to register
211070 Intermittent (8/10) Crash on site
211067 NetStream.Seek.InvalidTime NetStatus event coming in different order from other events, breaking legacy content
211051 File Upload broken on leopard. FileReference throws cancel event when a file is selected for upload in the file-browsing dialog
211017 Scrubbing streaming flv doesn't work (possible injection)
210983 Incorrect sprite size and position on several platforms
210962 Player crashes when finish streaming a mp4 movie.
210957 Bitmap.draw() producing erratic behavior when subsequently drawn in bitmapfill
210915 Blur filter behavior uses black on edges instead of nearest pixel
210909 Fullscreen not painting over screen
210907 perform execstack and strip commands on executables
210901 [MP4] Handbrake videos made with certain settings play but halt batch at end of playback
210871 Smart Buffering: Few issues when streaming on2 files from Flash Media Server 2.0.4.
210854 scrolling images show stuttered or jumpy or jerky movement on playback in Vista
210846 [MP4] Errors in the 'ilst' parser
210844 Audio stutters
210809 widget hangs the browser
210808 Player crashes by scrubbing the playhead when streaming a mp4 file under rtmps, rtmpt and rtmpe connections.
210780 All the strings for the linux context menu leak [Memory Leak]
210769 Repeated use of shared objects in an event handler causes slow script dialog and hosed Settings UI to appear
210767 [LNX RHEL4] Crashes interacting with site
210746 When a release swf loads a debug swf, flash player doesn't look for the debugger
210720 Loader acts strangely in FullScreen mode
210717 Keyboard non functioning with the Netscape
210716 full screen does not respond to mouse events or display context menu
210715 Selecting printer dropdown cause hang with plugin on linux
210679 Certain videos look bad when played on Mac
210667 Context menu in full screen does not display, intermittently hangs
210645 streaming checks may unnecessarily reject valid socket policy files
210642 -root option sends sometimes a gtk warning, and sometimes a segfault
210641 Null Objects when iterating through children
210636 non-default locations not rejected for missing Content-Type
210593 Cannot open Settings manager page after clicking advance settings.
210574 Full episode videos don't stay in FullScreen Mode in Firefox
210573 Smart buffering: Artifacts with episodes
210562 dragging performs much poorer than 9r48
210561 Smart Buffering: NetStream.Play does not function after NetStream.Pause is called in this app.
210545 Netstream.receiveVideo parameter change is breaking existing content
210538 [MP4] Pauses due to buffer running out cause loss of sound sync
210532 Animation is skipped, causing audio and swf playback off sync.
210501 Enhanced Seeking driving audio/video out of sync on lower bandwidth like DSL
210485 Pause/Unpause on MP3 Files doesn't play from the paused location when setBufferTime is used.
210476 extraneous reload of content after close of sub-pane
210451 Player Crashing while playing a video through a serverside stream
210434 Player crashes when streaming this bad flv under an AS3 streaming application.
210433 massive performance degradation after being in fullscreen for a long time
210380 Refreshing the browser when AS3 content is loaded will cause the browser to crash.
210335 Flash content does not display
210333 Can't input local words with IME on linux
210331 [MP4] Lots of screen artifacts on certain trailers. PPC Only
210292 fscommand ("fullscreen", "false") crashes the Projector on Linux
210245 Can't input any words with IME
210209 [MP4]: When unpausing after seek while paused, video is black for one frame
210170 [MP4] AVC Profile 110 videos all appear purple on MacIntel machines
210097 Multiple SWFs hang page
210039 Leaking file descriptors to Flash resources
209833 Intermittent Flash Player 9.x crashes in Internet Explorer 6 and 7
209513 Changing IME conversion mode does not work on Mac X.5
208369 Consecutive shared object flush calls causes slow script error when used repeatedly in non-mouse/keyboard events
208212 Crash when loading external swf file in AS3
208125 page with 9 Flash video instances generates "R6025 - pure virtual function call" and crashes IE
208039 Flash plugin logs keyboard in Safari on Mac OS X
207349 Linux stalls on random test cases
206369 ByteArray.readMultiByte returns garbled characters from a binary file containing Win-1252 encoded characters.
147787 removeMovieClip should not fail for objects > kDisplayClonedEnd in depth

New File Extensions and MIME Types

noreply@blogger.com (Tinic Uro) — Wed, 31 Oct 2007 16:23:00 +0000

There have been a few questions around file types and mime types which should be used for the new video container format. To summarize this post, we will promote new file extensions and will stick with standard MIME types.

The new file extensions and MIME types will be the following:

File Extension	FTYP	MIME Type	Description
.f4v	'F4V '	video/mp4	Video for Adobe Flash Player
.f4p	'F4P '	video/mp4	Protected Media for Adobe Flash Player
.f4a	'F4A '	audio/mp4	Audio for Adobe Flash Player
.f4b	'F4B '	audio/mp4	Audio Book for Adobe Flash Player

These are pretty much in sync with what Apple does offer for their downloads. Why new file extensions? It will be an easy way to distinguish files which can be played back by the Flash Player. There are simply too many instances where .mov and .mp4 files can not be played back, or vise versa, a file compatible with the Flash Player might not play back in QuickTime, an iPod or other video device. I will also be working on a technical document to exactly outline what the Flash Player does support and what it does not support although my previous posts already pointed out that we are very close to full support of the H.264 standard with the exception of Extended profile and FRExt.

Why didn't we stick with .FLV? Technical reasons, mostly. Various technologies, including our own products expect that .FLV files have a certain file structure. In case of Flash CS3 a file which was renamed from .MP4 to .FLV would stall the application at import time. Not a good experience. In case of FMS file type plug ins can only handle unique file extensions. In this context it made it impossible to have files with the new container format using the .FLV file extension. We expect other tools to have similar issues.

It's not yet clear when our video tools will start using these extensions, although I expect it to happen sooner than later. The first one to be used will clearly be .f4v, the other ones are in 'reserved' state for now and not destined for a particular product right now. Why do I talk about this now in this case? Well, as we ready the Flash Player for release we did add new icons and file extension registration of these file types into the standalone Flash Player. So it's better to make this announcement now as you will discover this on your own very soon anyway.

If you ask what the FTYP column is, it is the ftyp atom within ISO 14496-12 files. If you have a custom encoder and are targeting the Flash Player you should add one of the ftyp major brand 4CCs mentioned above. This will make it much easier for servers to handle and recognize these files. And to prevent you from panicking now: the Flash Player will not even look at this atom. This atom is nothing more than a hint for tools handling ISO14496-12 files. A list of known ftyp major brand 4CCs can be found here. We'll hopefully make this list also at some point.

It might be a good time to update your IIS mime-type entries to include the new file extensions, otherwise IIS will refuse to play serve up these files. This is the same process as for flv files described in Technote 19439, but using the file extensions and mime types mentioned above.

Update, since I am apparently not clear enough and have to repeat myself: the Flash Player will not look at the file extension when loading files. It just means that if you are targeting the Flash Player or AIR we suggest to use these file extensions.

The obligatory post on Hydra

noreply@blogger.com (Tinic Uro) — Fri, 05 Oct 2007 04:26:00 +0000

Now that MAX 2007 in Chicago is over and the Flash Player team is somewhat settling for the final landing of Flash Player 9 Update 3 I've got to talk a little about Adobe Image Foundation, code named Hydra.

I have not been involved in this project as much as I would have liked to, with Bob Archer and Kevin Goldsmith driving most of the specifications and communication so far, nevertheless some technical questions in the context of Flash have been asked I should be able to answer. But let me make this post a little more approachable and explain the basics.

What is Hydra? If I would have to explain it in one sentence I would say: It is a shader language specifically tuned for 2D pixel graphics.

What is a shader language? Most of you will not know the answer to this. Let me try to put it into terms you might understand, with the assumption that you already have played with BitmapData objects in ActionScript 2.

Scenario: You have a bitmap and you want to exchange the red channel with the green channel. Easy enough, in Flash 8 or newer you can actually use the ColorMatrixFilter to do this. Lets say you do not have that filter, how would you do it? Well, this is what I would write in ActionScript 2:

import flash.display.*;

// create a green bitmap
var bitmap:BitmapData = new BitmapData(550,440,false,0x00FF00); 

// go through the whole bitmap pixel by pixel
for(var y:Number=0; y<bitmap.height; y++) {
    for(var x:Number=0; x<bitmap.width; x++) {

        // get a single pixel from our bitmap
        var pixel:Number = bitmap.getPixel(x, y);

        var red:Number   = (pixel>>16)&0xFF; // extract red
        var green:Number = (pixel>> 8)&0xFF; // extract green
        var blue:Number  = (pixel    )&0xFF; // extract blue

        // create a new pixel with the red and green channels flipped
        var newpixel:Number = (green<<16)|
                              (red  << 8)|
                              (blue     );

        // replace the ol' one with the new one
        bitmap.setPixel(x, y, newpixel);
    }
}
// now the bitmap is pure red

// show it
this.createEmptyMovieClip("bitmap_mc",this.getNextHighestDepth());
bitmap_mc.attachBitmap(bitmap, 2, "auto", true);

That sure looks ugly and slow. Now lets express the same as above using Hydra, and believe me, it will do exactly the same as the above AS2 code:


kernel FlipRedGreen
{
    void evaluatePixel(in image3 source, out pixel3 result)
    {
        result.grb = sample(source, outCoord());
    }
}

There are several things you will notice:

There are no for loops to go over the pixels. That's right, shader languages only express what it inside the loop and assume that this is the operation you want to perform on every pixel.

There are a bunch of strange types here! Not really: image3 stands for a bitmap with RGB data and pixel3 stands for an individual RGB pixel. If you would want to include alpha you could say image4 and pixel4.

What is sample()? It is the same as BitmapData.getPixel() and returns a pixel, in this case a pixel3 type.

What is outCoord()? It is the same as the x and y positions in the AS2 code but it returns a single value containing x and y without the need to specify them seperately.

How does the actual flip work? Now that is some of the beauty which comes with using a shader language. You can perform all kinds of operations directly with the component identifiers.

In this case we say 'result.grb =' which will flip the red and green channel. I could write 'result =' or 'result.rgb =' to just copy the pixel without a change. Or I could write any other combination like .bgr, .gbr to flip the different components.

If you really want to get fancy you can also write something like this:
'result = sample(source, outCoord()).rrr;' or
'result = sample(source, outCoord()).bbr;'.
In this case we make the component choice on the other side of the assigment.

In essence, no more bit fiddling anymore, you have a high level access to the actual component data.

This is one of the most simple examples I can think of, Hydra is much more powerful and expressive than that. So I invite you to take a look and get comfortable with it, this is something which will stay with us for a long time to come. And possibly not only for graphics, but let me talk about that in a future post. :-) I also promise that I will have another post specifically talking about the technical aspects of the Flash Player implementation. I can think of those and more:

How will I be able to access this in my ActionScript code?

How much faster will it really be than ActionScript code?

Where will I be able to use Hydra? Just filters and blend modes, or more?

Will Flash Authoring support Hydra?

Will the Flash Player take advantage of hardware acceleration?

Will this work in the Flash Player even if there is no hardware acceleration available?

I know you are clamoring for answers, though for most questions we have not figured them out ourselves just yet.

There is obviously one more question: "Instead of working on new features, when do you get your act together and ship a 64-bit version of the player?" :-) I always have a good laugh when we get this one, especially since it has been answered many times already.

Getting ready for showtime

noreply@blogger.com (Tinic Uro) — Mon, 01 Oct 2007 07:31:00 +0000

MAX is now underway and to celebrate this we just released another prerelease of Flash Player 9 Update 3 code named 'Moviestar', the version is 9.0.64.0.There have been numerous changes and fixes since the last beta, especially in the area of MP4, H.264 and AAC support. I figured I should summarize the bugs we fixed. In this area only as I was not able to keep track of the other stuff. As a warning, this is highly technical stuff, but I know that some wanted this information.

A frame ordering issue when using more than 1 reference frame has been fixed. There should be no limit anymore on how many reference frames can be used.

One multi core systems H.264 streams using the loop filter showed artifacts along the slice boundaries. This has been fixed by switching from sliced based multi threaded decoding to picture based multi threaded decoding.

Overlaying flash content would sometimes cause black artifacts during a seek because the underlying YUV buffer was invalid.

AAC streams would sometimes not play back if it contained SBR information. This usually happened when switching between SBR and non-SBR streams.

The H.264 codec would crash on octo (8) core machines.

AAC streams with sample rates higher than 44.1Khz could crash the Flash Player.

Corrupt or fuzzed AAC streams could crash the Flash Player.

The H.264 codec now supports FMO decoding, PicAFF and MBAFF decoding should be improved.

H.264 decoding should generally be about 10-15% faster because of the removal of a full frame copy. Note that this is only the case if a square pixel ratio is used, the video stream is not interlaced or cropped on the top, left or right.

The PPC version of the H.264 codec would call kill() during certain situations which resulted in a user experience similar to a crash. The reason behind this was the incorrect behavior of assert() in release builds under CodeWarrior.

The H.264 codec now has AltiVec versions for all the functions which are accelerated by MMX and SSE2 SIMD instructions on x86. Note that due to the extremely low available memory bandwidth of G4 and G5 PowerPC processors the overall performance will never match x86 CPUs. Refer to our system requirements page which should be up once the final version of MovieStar is released.

The mp4 parser will now parse variable chunk sizes correctly. Usually this bug manifested itself in bad or accelerated AAC audio playback.

The mp4 parser should now be much more robust when encountering corrupt or fuzzed files. Two new NetStream events have been added to capture invalid or unsupported files: NetStream.Play.FileStructureInvalid and NetStream.Play.NoSupportedTrackFound.

The H.264 codec could crash when trying to decode unsupported profiles like Extended, High 4:2:2 and High 4:4:4.

Past onImageData and onTextData events are now resent when seeking. This f.ex. allows previewing of cover art when scrubbing pod casts and will assure you will get the current timed text for any particular time. Note that also means that you will receive a lot of these events during a seek.

supported AAC aot types are now correctly parsed and validated, meaning we allow aot types 1, 2 and 5.

Some encoders (like Handbrake) embed aspect ratio information into the matrix of the track header box atom, which is inconsistent with ISO 14496-12. Despite this the Flash Player will now apply this matrix on the reported width and height in the onMetaData object. This does not affect the actual video stream size which can be different from what is reported in onMetaData and in this case usually points to a bug in the encoder.

Artifacts on the right and the bottom when mixing cropping and aspect ratios in H.264 streams should be gone.

Along with times, the seek points array will now also contain a byte offset which allows to determine if a particular seek point is reachable during a progressive download.

What is still outstanding before the final release?

There is an injection concerning scrubbing H.264 files. Currently no images are shown during scrubbing. You have to resume playback before the video is updated. There is a one line fix I have which did not make it in time for the 9.0.64.0 build.

When playing back video using FMS you can end up in a situation where you get 1 black frame when seeking. Another minimal low risk change will address this for the final build.

AAC streams using SBR do not play back in certain situations if the original sample rate is higher than 24Khz.

What are the known issues we will not address?

We still do not support Extended, High 4:2:2 or High 4:4:4 profiles. Unfortunately the most commonly non-professional codec out there which is x264 does not adhere to standard terminologies when specifying encoding options. This means you will likely run into a case were you encode using options which are part of the profiles we do not support. The most common option you will want to use is 4:4:4 support to obtain screen casts without YUV artifacts. This is not a bug, although I am sure we will continue to get reports on that. I strongly suggest you stick with Adobe's AME encoder which uses standardized terminologies defined in the H.264 specifications.

On the more controversial side we decided to postpone support for H.264 color profiles as the risk of adding tons of new SIMD code was too high at this point in time. Like MPEG-2 default H.264 streams use the color profile standardized under ITU-R BT.709 and in addition allows the selection of 6 more less frequently used color profiles. The streams will be displayed under the assumption that the content is ITU-R BT.601 meaning it will be a little brighter and slightly color shifted if the encoder did use ITU-R BT.709. Video buffs will frown upon this, but most end users will probably not notice it. We'll try to address this in one of the next versions of the Flash Players. I realize that it is important. For designers this also means that color matching with other Flash content will be more tricky for now.

A more sophisticated way of displaying field coded streams which we had prototyped but were not able to fully test will not make it into the final release. So field coded streams will simply be frame blended.

Corrupt AAC or H.264 streams or audio dropouts caused by the underlying OS can cause bad audio/video sync. This is due to an architectural limitation of the Flash Player which we will try to remedy for the next major version of the Flash Player, code named Astro.

What just happened to video on the web?

noreply@blogger.com (Tinic Uro) — Tue, 21 Aug 2007 04:09:00 +0000

That's a question you should ask with the announcement we made tonight. I think a lot will change. This is probably one of my longest and information packed posts ever, but I think it is important we put down all cards on the table. Lets summarize what new functionality Flash Player 9 Update 3 Beta 2 contains (for the impatient: It will be available on labs.adobe.com this afternoon):

An file format parser implementing parts of ISO 14496-12. In terms you might understand this means a very limited sub set of MPEG-4, 3GP and QuickTime movie support.

Support for the 3GPP timed text specification 3GPP TS 26.245. Essentially this is a standardized subtitle format within 3GP files.

Partial parsing support for the 'ilst' atom which is the ID3 equivalent iTunes uses to store meta data. This really more a de-facto standard which came through the ubiquity of iTunes, there is no official documentation on the format. Look here for an incomplete list of supported tags iTunes does use.

A software based H.264 codec with the ability to decode Base, Mainline and High profiles. This is also an ISO standard with the identifier being ISO 14496-10.

An AAC decoder supporting AAC Main, AAC LC and SBR (also known as HE-AAC). The corresponding ISO specification is ISO 14496-3.

That's pretty much what we say publicly. Truth is that these specifications are so complex that no one supports 100% of it. I realize that it will be important for Adobe to communicate exactly what is and what is not supported. We are working on this and will be trying to help novices and experts alike. For those who scream murder and accuse us of going with incomplete standards support let me tell you that ISO 14496-12 specifically allows for the definition of sub sets. 3GP is one of those. We did not extend or add proprietary extensions whatsoever to the mentioned standards above, it is a pure sub set.

Why now? Short answer: Because you wanted it. Long answer: We've been working on this for a while and this was planned to be part of the next major revision of the Flash Player. What was unexpected was how impatient a lot of our customers are :-) It seems many are trying to make choices when it comes to video technologies right now. We wanted to make sure that we would offer the best possible choices to them and set a signal that we are willing to embrace industry standards. No one believed that we would make this happen.

Unfortunately, and we realized while working on this: along with adopting industry standards also comes completely new terminology which seems designed to confuse non-insiders. This makes it difficult to pin down exactly what it is what we did and how you might benefit from it. It took me several months to just understand the basics in the ISO specifications. By now I might have lost the ability to boil it down into simple terms everyone can understand. But I'll try anyway. :-)

Lets talk about actual functionality you can leverage in the Flash Player. Now I am getting really technical:

You can load and play .mp4,.m4v,.m4a,.mov and .3gp files using the same NetStream API you use to load FLV files now. We did not add any sort of new API in the Flash Player. All your existing video playback front ends will work as they are. As long as they do not look at the file extension that is, though renaming the files to use the .flv file extension might help your component. The Flash Player itself does not care about file extensions, you can feed it .txt files for all it matters. The Flash Player always looks inside the file to determine what type of file it is.

A new version of FMS is upcoming and will support the new file format. This is powerful stuff. Simply drop video files you might have encoded using one of the countless tools out there onto the server and it'll stream. Even if the moov atom is at the end of the file. Ah, that is something I have to mention as you are 100% likely to fall into this trap:

If you use progressive download instead of FMS make sure that the moov atom (which is the index information in MPEG-4 files) is at the beginning of the file. Otherwise you have to wait until the file is completely downloaded before it is played back. You can use tools like qt-faststart.c written by our own Mike Melanson to fix your files so that the index is at the beginning of the file. Unfortunately our tools (Premiere and AfterEffects etc.) currently place the index at the end of the file so this tool might become essential for you, at least for now. We are working hard to fix this in our video tools. There is nothing we can do in the Flash Player and iTunes/QuickTime does behave the same way.

The Flash Player will display the first supported video and audio track it finds in a file. Subsequent audio and video tracks are ignored and not selectable right now. This covers the majority of files out there on the web, only in rare instances do you have additional audio tracks f.ex. But I believe that for the web you would rather create several versions of a file anyway to save bandwidth. One of next major revisions of the Flash Player will add new APIs to enhance this most likely. Our goal was not to add any new APIs for this release.

Video needs to be in H.264 format only. MPEG-4 Part 2 (Xvid, DivX etc.) video is not supported, H.263 video is not supported, Sorenson Video is not supported. Keep in mind that a lot of pod casts are still using MPEG-4 Part 2. So do not be surprised if you do not see any video. We should be close to 100% compliant to the H.264 standard, all Base, Main, High and High 10 bit streams should play. Extended, High 4:2:2 and High 4:4:4 profiles are not officially supported at this time. They might or might not work depending on what features are used. We have no artificial lower limit on B-frames or any problems with B-pyramids like other players do. We also decode field coded streams, although this beta displays the images progressively using the weave method. The final release will be blending the two fields. There are still a couple of bugs with frame ordering/timing I need to fix in the Flash Player itself for the final release. And there is also a problem with files using the loop filter on dual core machines which causes horizontal artifacts along slice boundaries, which is my bad. The fix for this did not make it into this beta. Overall though and leaving out the bugs I listed here which are my fault, the H.264 decoder is a remarkable piece of engineering, it is provided to us by MainConcept. It weights in at less than 100KB of compressed code which is quite an achievement for such a complicated standard.

Audio can be either AAC Main, AAC LC or SBR, corresponding to audio object types 0, 1 and 2. We also support the '.mp3' sample type meaning tracks with mp3 audio. MP3inMP4 which intends to do multi-channel mp3 playback within mp4 files is not supported. Also, the old QuickTime specific style of embedding AAC and MP3 data is not supported. It is unlikely though that you will run into these kind of files.

3gp timed text tracks. Any number of text tracks are supported and all the information, including esoteric stuff like karaoke meta data is dumped in 'onMetaData' and a new 'onTextData' NetStream callback. Language information in the individual tracks is also reported. That means you can have sub titles in several languages. Study the 3GPP TS 26.245 specification to see what information is available. Note that you have to take care of the formatting and placement of the text yourself, the Flash Player will do nothing here. Time for you to start working on one of those components which do that. You can use MP4Box to inject text data into existing files.

Meta data stored in the 'ilst' atom. This is usually present in iTunes files. It contains ID3 like information and is reported in the onMetaData callback as key/value pairs in a mixed array with the name 'tags'. ID3V2 is not supported right now. An incomplete list and link to tools which can edit these tags is available here.

Since these files contain an index unlike old FLV files, we can provide a list of save seek points, e.g. times you can seek to without having the play head jump around. You'll get this information through the onMetaData callback in an array with the name 'seekpoints'. On the downside, some files are missing this information which also means that these files are not seekable at all! This is very different from the traditional FLV file format which is rather based on the notion of key frames to determine the seek points.

Unencrypted audio book files contain chapter information. We expose this in the onMetaData callback as an array of objects with name 'chapters'.

Image tracks encoded in JPEG, GIF and PNG are accessible. Unfortunately only in AS3 as I pass this information as a byte arrays through a new callback 'onImageData'. You can simply take that byte array and use the Loader class to display the images. Most often these images represent cover artwork for audio files. TIFF image tracks are not supported, you might come across files using this. Also note that we support the 'covr' meta data stored in iTunes files, these are also accessible as byte arrays.

Will it be possible to place H.264 streams into the traditional FLV file structure? It will, but we strongly encourage everyone to embrace the new standard file format. There are functional limits with the FLV structure when streaming H.264 which we could not overcome without a redesign of the file format. This is one reason we are moving away from the traditional FLV file structure. Specifically dealing with sequence headers and enders is tricky with FLV streams.

Will it be possible to place AAC streams into an FLV file structure? Yes, though the same limitations as for H.264 apply.

Will the Flash Player play back multi channel AAC files? It will play them, though the sound is mixed down to two channels and resampled to 44.1Khz. We are targeting multi channel playback for one of the next major revisions of the Flash Player. This requires complete redesign of the sound engine in the Flash Player which dates from circa 1996 and has not been improved since.

Will the Flash Player be limited to 11Khz, 22Khz and 44.1Khz sampling rates like for MP3? No, we support all sampling rates from 8Khz to 96Khz. I implemented a 32 tap Kaiser Bessel based FIR filter which resamples the sound to 44.1Khz, retaining high quality. The most common sample rate combinations have a hard coded number of phases. In case of a 48000 to 44100 Hz conversion the filter has 147 phases f.ex. Even better: Flash Player Update 3 Beta 2 now can play back any MP3 sampling rate leveraging the same code I implemented for AAC. No more chipmunks. Ever. Err, this is actually kind of major as I have seen complaints about this bug for years :-) I fixed this problem in the AS3 Sound class, though it was using very low quality resampling. This change I made this time will fix it even for AS2 and sound in FLV files while retaining excellent quality.

Will it be possible to place On2 VP6 streams into the new file format? Not right now, we are still trying to figure out if it is possible for us to support this.

Can you play files protected by FairPlay? No.

Do we support MPEG-4 BIFS or other esoteric stuff (scripting, VRML etc.) from the MPEG-4 Systems specification? No. Whatever is not listed above we do not support.

Do we support SMIL? No. You can easily write your own SMIL parser in ActionScript though.

Can you use the Sound class to play back AAC/.mp4a files? No, you have to use the NetStream class. We are now getting into a situation where there is not much difference between audio and video files anymore. They are the same essentially. Hence we figured we should not further add confusion and allow to do things ten different ways which would also increase the Flash Player binary size. My guess is that we will enhance the Sound class in the future but it might go into a different direction and will not be dedicated to pure playback of static files anymore.

Here is a list of data which is reported in onMetaData:

duration - Obvious. But unlike for FLV files this field will always be present.

videocodecid - For H.264 we report 'avc1'.

audiocodecid - For AAC we report 'mp4a', for MP3 we report '.mp3'.

avcprofile - 66, 77, 88, 100, 110, 122 or 144 which corresponds to the H.264 profiles.

avclevel - A number between 10 and 51. Consult this list to find out more.

aottype - Either 0, 1 or 2. This corresponds to AAC Main, AAC LC and SBR audio types.

moovposition - The offset in bytes of the moov atom in a file.

trackinfo - An array of objects containing various infomation about all the tracks in a file.

chapters - As mentioned above information about chapters in audiobooks.

seekpoints - As mentioned above times you can directly feed into NetStream.seek();

videoframerate - The frame rate of the video if a monotone frame rate is used. Most videos will have a monotone frame rate.

audiosamplerate - The original sampling rate of the audio track.

audiochannels - The original number of channels of the audio track.

tags - As mentioned above ID3 like tag information.

Here are some good links to get an understanding of what MPEG-4, H.264 and AAC are:

http://forum.doom9.org/showthread.php?s&amp;threadid=62723
http://forum.doom9.org/showthread.php?t=96059
http://en.wikipedia.org/wiki/H264
http://en.wikipedia.org/wiki/Advanced_Audio_Coding
http://daringfireball.net/2007/04/some_facts_about_aac

Let's put together some thought up scenarios I would imagine are important:

You created a pod cast for iTunes and happily distribute over this channel. Now you want to add value to it and easily make it accessible over the web without special plug-ins, reaching an audience which does not have QuickTime installed. Well, this new feature will allow you to do this. You can take your existing podcast in .m4a format and present it on any web page through the Flash Player. Add more value by adding interactivity and branding if you want to. The possibilities are endless.

Your media company has made or is about to make a significant investment into web video or video archiving. You are wondering what format you should choose. Video for Flash reaches everyone now, but the format is not an 'industry standard' so you have the fear that content you will create will become obsolete and unsupported at some point. Flash Player 9 Update 3 comes to the rescue: MPEG-4 is an extremely well documented ISO standard and completely vendor independent. And by using the Flash Player now you get instant gratification for viewers.

You want to get best the possible quality out of your video and do not want to be tied to a particular encoding solution. You also like open source software to do all of the work you need to do to encode video. A combination of libfaad, x264 and MP4Box which are all licensed under the GPL will do exactly that, albeit with little usability and requiring lots of expertise. But it will now play just fine through the most distributed run time in the world, the Adobe Flash Player.

Those are immediate benefits, there are plenty more when we look ahead. Let me mention a few of them:

H.264 will be supported natively by most new graphics cards. NVidia, ATI and Intel have made a commitments to have full support for it. This means better than HD video on your PC will become possible in the not so distant future.

There are hardware based H.264 encoders which encode at better than real time. This is important if you need to be quick to market like f.ex news organisations.

Digital TV, especially in Europe is quickly adopting H.264. The interoperability with the web will open new doors for a lot of media companies.

AAC SBR offers demonstrable advantages over plain MP3, think 5.1 channel surround sound f.ex. While the Flash Player does only support 2 channels output at this time, there is opportunity to go beyond that.

And last but not least here are some things I will not give a complete answer to since they are begging for controversy:

Comparing H.264 against other video codecs, might it be performance or quality. I've looked at the comparisons out there, they are at best subjective, most of the times outright marketing bull and almost always completely biased. My take is: Take a good and well accepted encoder and compare the results yourself. Your mileage will vary. And that is fine. Quality is not the main reason Flash Player 9 Update 3 has H.264 support.

Tell you if On2 VP6 is better or worse than H.264. Truth is that they have different strengths, not only performance and quality wise. It totally depends on your individual situation of what fits best. The Adobe Flash Player now offers more choice which is more important than anything else.

I am not in a position able to explain to you why we will not allow 3rd party streaming servers to stream H.264 video or AAC audio into the Flash Player. What I can tell you is that we do not allow this without proper licensing. Refer to Adobe's friendly Flash Media Server sales staff for more information.

I can also not help you with anything regarding broadcast fees for commerical use of H.264 and AAC streams. Please refer to the FAQ Adobe provides which usually point to contacts at MPEG-LA and Via Licensing. A summary of licencing terms for H.264 is available here.

Multi core support

noreply@blogger.com (Tinic Uro) — Tue, 12 Jun 2007 17:57:00 +0000

As I mentioned in Flash Player Update 3 we finally realized that multi-core CPUs are here to stay. So why not follow the times and take advantage of it? As most of you hard core Flash developers know, rendering is a huge bottleneck. I've seen a couple of blog post complaining that their second core/CPU is not doing anything when they run the Flash Player. Well people, this is about to change in this update.

Flash Player Update 3 takes advantage of multiple CPUs in many different areas. The first one is obvious: The vector rasterizer. The Flash Player will split the rasterization workload by dividing the stage into horizontal stripes. In case you have to 2 cores/CPUs, the top half is rendered by CPU 1, the bottom half by CPU 2.

How much performance improvements will you see? That depends, unlike with a 3D ray tracer the workload is not totally independent and there is some additional overhead to handle multiple threads. As a rule of thumb you could say that you get an additional 25-33% performance improvement per CPU. That does not sound like a lot, but keep in mind that the Flash Player still does some stuff which can not be multi-threaded.

The second area we focused on a are bitmap filters. If you deal with large movie clips and use lots of filters on them you will see a nice performance jump. Again the workload is split into horizontal stripes. The following filters are supported:

DropShadowFilter

GlowFilter

GradientGlowFilter

BevelFilter

GradientBevelFilter

BlurFilter

ConvolutionFilter

ColorMatrixFilter

DisplacementMapFilter

Third, compositing and color transformations of bitmap cached object are also accelerated. The BitmapData APIs are NOT multi-threaded yet, hopefully we get around doing this at some point.

Fourth, VP6 video decoding now happens in a separate thread, independent of rendering and post processing of the video. In most cases that means a dramatic improvement in performance, especially if you did reach a limit performance wise with your current content despite the fact that you had a dual core machine. On my Athlon X2 4800 I can now play a true 1920x1080 pixels video at 30 frames/sec without a single dropped frame. Really cool if you ask me. It was a reason for me to buy a new toy: a Canon HV20 which can do 1080p at 24 frames/sec (actual pixel resolution is 1440x1080). Perfect for the web. Now if these video files would not clog up my drive, they are really huge...

Why is the Sorenson Spark codec not multi-threaded? Because of its design: It does not have a dual YUV buffer setup which allows us to separate blitting and video decoding.

Since I am talking about VP6 already let me mention some of the more generic improvements in VP6:

I added a new gaussian noise filter which is turned on when post processing is active. It lessens the effect of blockiness and makes videos more look like movies. If you are turned off by this change, please let me know as soon as possible. In most of the cases it is seen as a improvement, but you designers should tell us if things still look okay.

We fixed a long standing issue with VP6's automatic post processing selection on Athlon X2 class CPUs. It never worked correctly on these machines. This is due to a broken CPU driver on stock installs of Windows which affects the QueryPerformanceCounter Win32 call. Incidentally the new version of QuickTime and the Real Media Player seem to have problems related to this API, playback stutters heavily on my machine with some content because of it. I usually fix this problem by setting the process affinity to a single CPU in the Task Manager. That's what you get for not testing on AMD CPUs :-)

Entropy decoding and some parts of the decoder should be faster, you should see a generic 10% improvement in CPU usage.

Mip map what?

noreply@blogger.com (Tinic Uro) — Mon, 11 Jun 2007 17:00:00 +0000

True to my reputation here at Adobe I added a miniature feature into Flash Player 9 Update 3 (9.0.60.120). It is Mip mapping. The Wikipedia entry is nice, but there is an in depth article I highly recommend to anyone interested in the subject: Part 1 and Part 2. As described in the article we use the fast and simple box filtering as apposed to the more complex methods which are not really suitable for real-time rendering. So no Kaiser in the Flash Player. Neither do we have trilinear or anisotropic filtering, but then again they make more sense with real 3D graphics. And since we are still stuck in software only land for bitmap rendering I was not interested in a performance loss either.

Why do I call it a miniature feature? Because the whole feature occupies less than 1Kb of compressed code in the binary. This is the kind of changes we engineers in the Flash Player love. Extremely small in code size, large in impact.

If you feel kind of overwhelmed with the descriptions in these articles let me summarize that the feature does in more simple words: This feature improves the quality and performance of downscaled bitmaps. Not slightly downscaled, but anything which is scaled down by more than 50%. Mip maps are nothing more than precomputed smaller and higher quality versions of an original bitmap. They are used instead of the original bitmap when something is downscaled a lot.

A few words about the Flash Player implementation and its limitation might be worth talking about here:

Mip maps are only created for 'static' bitmaps, e.g. anything like a JPEG, GIF or PNG you display via loadMovie(), a bitmap in the library or a BitmapData object. They do not apply to filtered objects or bitmap cached movie clips.

In case of video things are more tricky. If smoothing is turned on the mip mapping applies since this is overall faster than rendering the original bitmap. For non-smoothed video mip mapping is not used since it would make things much slower.

Mip maps level generation stops when it encounters an odd width or height. What does that mean? In the most extreme case it means that if you have a bitmap with an odd width or height pixel value to begin with, no mip map can be generated at all and therefore you will not see any benefit. This is unfortunately hard technical limitation. That also means that 'perfect' mip maps are generated from bitmaps which have a width and height which are 2^n, e.g. 256x256, 512x512, 1024x1024 etc. On average though it is enough to have bitmaps sizes which are dividable by 8, meaning you get at least 8 levels, f.ex. 640x128 -> 320x64 -> 160x32 -> 80x16 -> 40x8 -> 20x4 -> 10x2 -> 5x1.

The quality improvements are more visible when smoothing in turned on for a particular bitmap.

The most immediate effect of this feature can be seen with Papervision3D. The Rhino demo f.ex. shows much less aliasing in the textures when you rotate. Also, a couple of frames per second more are displayed with this demo.

Good and well, but how does 'normal' Flash content benefit? Any time you create something like an image gallery with small thumbnails which are based on larger bitmaps you will see better bitmap quality and slightly higher performance.

Will this feature ever make things look worse? Not unless you do something really dumb like downscaling heavily aliased images and still expect it to look pixelized.

I made up a quick and dirty demo which shows the reduced aliasing effect: http://www.kaourantin.net/swf/mipmap.html Make sure you have Flash Player Update 3 Beta 1 installed, otherwise the two samples will look exactly the same.

What about control on this feature? Can you select the threshold or select your own mip map bitmaps? No, right now everything is automatic. That might change in a next major revision of the Flash Player. And the threshold value, well, let me simply tell you that we use the standard OpenGL value which is <= 0.5. If you want to know how this actually works, here is I would express it in pseudo ActionScript 3, you graphic experts will get it right away:


var bitmapData:Array; // contains the mip map bitmaps
var m:Matrix; // the affine matrix to used for display

var i:int = 0;
while ( Math.sqrt(m.a*m.a+m.b*m.b) <= 0.5 &&
        (bitmapData[i].width % 1) == 0 &&
        (bitmapData[i].height % 1) == 0) {

   // switch to next mip map level
   i++;
   m.a /= 2.0;
   m.b /= 2.0;
   m.c /= 2.0;
   m.d /= 2.0;
   m.tx /= 2.0;
   m.ty /= 2.0;

}
mc.draw(bitmapData[i],m);

At last I should mention that mip maps apply to any SWF version 9 or newer content. SWF version 8 or lower will not use mip maps since we feared that this would impact on backwards compatibility.

Flash Player Update 3 Beta 1

noreply@blogger.com (Tinic Uro) — Mon, 11 Jun 2007 04:04:00 +0000

Flash Player Update 3 Beta 1 (build 9.0.60.120) is now ready for download here. This is probably the first time we have done a dot release as big as this one. We've been extremely busy over the past few months since Flash Player Update 2 (build 9.0.r45), so now it is finally time to talk about the improvements we have made. There are tons of them, so I'll use this post to summarize those I know something about and I have been working on. Stay tuned for more detailed posts of mine explaining the technical details behind these and how you will be able to leverage them:

Mip map support for all bitmaps for Flash 9 or newer content. This improves the quality and rendering performance of downscaled bitmaps. Perfect for thumbnails and such. Even better, Papervision 3D content now automatically looks better and should be slightly faster when large textures are used.
Multi-threaded vector renderer. Now we take advantage of up to 4 Cores/CPUs for vector rendering.
Multi-threaded bitmap filters. Same as above but this applies to bitmap functionality specifically instead of the core vector rasterizer only.
Multi-threaded video decoding. The VP6 video codec will now run in a separate thread if a multi-core system is detected which leaves the main thread to do rendering and post processing of the video. With this true 1080p video is now possible on most modern dual core machines. Also, the responsiveness is improved with this change. The Sorenson codec on the other hand did not get this change for technical reasons.
Full-screen mode with hardware scaling. Probably the biggest new feature in the Flash Player Update 3. This leverages DirectX on Windows and OpenGL on OSX. There is an new API to control the behavior which was required since we could not change current behavior and we wanted to give the maximum flexibility possible. I know you are probably eager to use this feature, we will post more information on this on labs.adobe.com soon (Update: Link to labs page is active). I'll also will give you much more technical details in an upcoming blog post.
Less tearing in the new full screen mode. We now have some code which will try to do VBL syncing. It's still a work in progress but we hope we can fix the remaining issues.
Going into full screen mode has a zoom transition effect. The beta does not work perfectly right now, but we want to get feedback if this is acceptable to end users. We will not expose an API to access/control this, either we'll leave it in and fix the remaining bugs or it is out. Also you might notice that this even affects the current full screen mode, something we will remove in the final release.
The Linux plugin now uses the XEmbed protocol. This is work in progress. The downside is that konqueror and Opera do not support this right now, so the Flash plugin will not work until these vendors update their plugin support. Also we are seeing decreased performance because GTK lacks somewhat in the the basic graphics API department. I'll explain in a later post.
Tons of performance tweaks and bug fixes. Looking the the current bug database statistic we fixed 371 bugs since 9.0.r45. Fixed really means fixed, it does not include duplicates, unreproducible bugs etc.
Much more.

A word of warning, this is a beta version! Do not use this version in a production environment. There are several known issues with the new features and while they might work on your machine they will not on others. We are obviously interested in the others and are looking for any issues you might encounter.

The limits of software rendering

noreply@blogger.com (Tinic Uro) — Fri, 02 Feb 2007 00:46:00 +0000

I have been following the progress of Papervision3D recently, the 3D engine for Flash. Here at Adobe we are always interested in applications which challenge the Flash Player performance wise. We spent some time to profile some demonstrations to see if there is anything we can do performance wise. The short answer is: not really.

What is the problem? Well, there is a limit what software can do on current CPUs. The limit can be squarely attributed to memory bandwidth and the size of L1 and L2 caches. I thought I'd explain the point by showing some actual profiling data I collected with the demonstration mentioned above. You can reproduce the same results including seeing the actual assembly instructions with any 3rd party profiler tool out there.

For testing I used a pretty beefy system, an AMD Athlon 64 X2 4800+. CodeAnalyst was my profiler of choice since this is what AMD provides for their CPUs, Intels VTune does the job just as well though if you have an Intel CPU.

Here are the top three loops which show up in the profiler and together take about 40% of overall CPU time:

The first loop does the affine bitmap blitting, meaning it handles rotation and scaling. This loop takes about 20-25% of overall time. Its characteristic is that memory access is almost always non sequential, meaning you get lots of L1 and L2 cache misses.

The second loop does a simple copy from one buffer to another. Some will recognize this code, this was adopted from the AMD Athlon Processor x86 Code Optimization Guide. This loop takes about 10-15% of overall time.

The third does a blur for a radius which is power of two, I have presented this code in a blog post before. This loop takes another 10-15% of overall time.

So what can you determine from the above? Look at the CPU clock column. This is the number of total cycles spent on an individual instruction. Any memory access, especially random access like the affine bitmap rendering is the most expensive operation. It really does not matter at that point, without GPU hardware support there is pretty much nothing we can do anymore on modern CPUs when it comes to graphics performance.

Obviously there are other areas we can improve, the other 60% of overall time have maybe another 10% overall performance bump in store if we fundamentally change some algorithms and fix bottlenecks in Tamarin. Although in this example ActionScript execution really does not account for much, it is really all related to graphics code. In the end the conclusion remains: It is memory access time which kills us.

What is even more surprising though for some is that even multi core systems will not help dramatically. In most cases the memory bus and even the L2 cache are shared among the cores.

58th Annual Emmy Awards Technology & Engineering

noreply@blogger.com (Tinic Uro) — Thu, 18 Jan 2007 22:29:00 +0000

On January 8th I had the privilege to attend the 58th Annual Emmy Awards for Technology & Engineering in Las Vegas and receive, together with other co-workers, an award. In case you did not know, Adobe Inc., among other well known companies in this industry got an award for advancing streaming technology, specifically video streaming over the internet.

The actual team at Adobe working on Flash Video technology has remained small over the years. So it was fairly easy to get the most important people from our engineering teams to be on stage to receive the award, something Microsoft and Apple were apparently not able to do:

(Engineers in suits! I am the second guy from the left, low resolution is on purpose to protect the innocent.)

And here is a close-up of the award:

Why did we get the award? Well, some will understand and some will not. I will dwell into my personal opinion at some point, right now I do not have the energy to defend myself in that regard :-) It is interesting though that all the major players did receive the award, which is more of a sign that it is really about the acknowledgement that video on web has finally become mainstream. This has been the dream of many for decades, but it really started taking off 2 or 3 years ago.

I am also happy for another award winner this year: John Carmack. He received two awards, one for Doom and one for Quake. These two games were definitely ground breaking and help shape the gaming industry to what it is now.

A non-beta Flash Player 9 for Linux release

noreply@blogger.com (Tinic Uro) — Wed, 17 Jan 2007 05:45:00 +0000

We just released another version of the Flash Player 9 for Linux, the version number is 9.0.31.0. What is different this time is that we do no call it a beta anymore and that it is available on our main download site (Click on the Linux x86 link). For the impatient ones, the direct download of the tarball is here, and the .rpm file is here:

Alright, so far so good... That does not mean that this is a release which I consider feature complete. We do however consider this version to be vastly better than any previous Flash Player for Linux. Since customers were starting to ask for a final version we decided to feature freeze what we have. So two months ago we moved the Flash Player 9 for Linux code into a separate code branch for stabilization. QE (Quality Engineering) has been hammering on this branch for all this time. Every bug fix had to be approved by management. The build you can download now (9.0.31.0) represents that branch.

The internal unstable branch is currently open for all developers and for the first time we build the Windows, Mac and Linux versions on a daily basis from the same code branch. There are various large work items which still need to be tackled and we are in full swing getting this ready as soon as possible. These changes are disruptive for the stability of the build and will be for a while.

Here is a list of some of the Linux specific items we are working on right now, it is far from complete and each of the items might or might not make it into the next version:

XEmbed support (specifically GtkPlug). We have tons of bugs which can only be fixed this way.

WMODE support. This will be tricky since we need to coordinate this feature with Mozilla as the browsers on Linux have no support for this right now. Also note that XEmbed support is a prerequisite for this feature to work at all.

Support for Opera. Hopefully we will get it into a state where it runs stable for more than 5 minutes. Opera 9.10 solved a lot of issues, but not all.

Rewrite the braindead/non-standard http stack in the standalone player. Right now this bug can make the standalone player timeout and block for a long time if you need to open any external URL (local files are fine). I'll spare you the details, but this code is really embarassing in my mind.

Try to fix other functional bugs which require larger infrastructural changes.

Finish the flashsupport add-on library. Adding camera, microphone support etc. Move the project to sourceforge.net or similar site. BTW, it's my fault that this has not seen too many changes lately since I own this piece. :-)

Add full screen support.

64bit work is ongoing. You can follow some of the daily progress in the Tamarin CVS tree. (Alright, not too much happened end of December. Forgive us, the holidays are always a little slow in a company like Adobe.)

So what changed between Flash Player 9.0.21.78 beta and 9.0.31.0? Here is a partial list of Linux specific bugs we fixed:


192642 Crash during "Browse multi"   
191932 After plug-in player installed, firefox cannot launch (crash while launching)  
191298 Firefox 1.5 crashes during tests execution (automated section) 
191514 Non Functioning rtmps not functioning 
192615 Crash hang navigating away from camera/mic  
192747 LocalConnection not functioning between standalone players 
192653 change standalone name from gflashplayer to flashplayer 
192399 Crash - GrowableBuffer and/or Generated Code 
191741 crash when "Allow" is clicked in camera settings UI popup when no camera driver is installed on SUSE 9 
192006 Clicking on URL failed to launch websites 
191749 mouseDown count advancing more than once per click - stand alone 
187060 Rotating the image moves image out of the panel 
177699 Bug in detection kit in combination with Linux player 
191427 rtmpt not functioning 
192036 Incorrectly Functioning File->Close is still selectable in the projector 
191843 website crashes with a segmentation fault.  
191267 Unable to copy/paste text from text fields using context menu. 
191857 remove all printfs from release builds of the plugin and standalone

(Update: I started this entry last week to start listing the fixed bugs, but actually posted the news January 16th. Darn. So some you you saw January 11th as the post date... I am really sorry for the confusion...)

Flash Player 9 for Linux Beta 2

noreply@blogger.com (Tinic Uro) — Tue, 21 Nov 2006 01:44:00 +0000

We have just refreshed the beta of Adobe Flash Player 9 for Linux, the current version is now 9.0.21.78. This version fixes several critical issues, especially some of the sound problems should be a thing of the past. So we encourage you to update.

Here is a partial list of bugs which were fixed (GNU/Linux specific, general bugs are mentioned in the labs release notes):

188992 - ALSA 2-second stopper problem
189976 - Leaking X11 connections
183202 - Flash crashes with X COMPOSITE
187715 - Asian characters are not displaying as expceted
187409 - Artifacts when initially playing + video stream freeze when selecting second video clip
189437 - Device text rendering performance is less than optimal
187850 - OS freeze right clicking flash movie after clicking another active link
184706 - browser crashes when trying to run use getters or setters from a different scope
188398 - in all the System.codePage 'FileReference', when user attempts to browse, a CRASH is generated
184549 - Mandriva 2006. Installing libflashplayer.so crashes browser
181584 - GUI not rendering correctly. Green out edges on inner frames.
189254 - wipe effect doesn't look right on components if they have no descenders
187885 - Some part of the image is shown out of panel
188349 - Clipping and Masking is not working correctly for Flex Containers when you scroll Text
187689 - Text can become garbled when window redraws.
188709 - Browser hangs
187490 - Crash accessing flash game
188805 - ALT key does not function in Linux player. (Ubuntu, others?)
188089 - RichTextEditor's formatting are not preserved during cut/copy/paste only in linux player
188997 - Selection copy won't paste into Flash.
186982 - activate, deactivate, mouseLeave events don't get fired when clicking inside/outside the player instance
187170 - caps lock and num lock are detected when flash player instance, or even the browser, does not have focus
187741 - change event not triggered when entering text in text field

If you are using flashsupport.c you should update it since I have removed the ICU dependency. I am currently revising this internally since I have received some great feedback on it. You should see these changes in the next version. Hopefully I'll also be able to finally check this into our public SVN repository at that time, since it really belongs there, not on a random web server, without a correct installation script or working versioning system.

Another update: Please note that this version does NOT work with Opera (nor did 9.0.21.55 btw). We know about it and are working with Opera to resolve it. Some sites might work, but most will eventually display a gray or pink rectangle, meaning that the Flash Player did crash. We know it is a problem with the way X11 is used (if you switch X11 to use synchronous mode it should work).

Also, we do NOT support GTK1 browsers and probably never will. That means any FireFox version older than 1.5 will probably not work and older Mozilla/SeaMonkey versions will also fail. The final version will have a safety check for this and the Flash Player will not try to run at all in this case. Right now you will most likely crash.

SpiderMonkey's relative Tamarin joins the family

noreply@blogger.com (Tinic Uro) — Tue, 07 Nov 2006 01:29:00 +0000

Today Adobe announced that the ECMAScript 4 compatible virtual machine in the Adobe Flash Player has been contributed to the Mozilla project under the name Tamarin. It is the single largest contribution to the Mozilla foundation since its inception and consist of about 135.000 lines of source code. The engine is fully open source using the standard Mozilla license, with the Mozilla foundation retaining full ownership.

What does this mean?

I anticipate that Tamarin will become the reference implementation for ECMAScript 4 VMs. Most of you have probably not realized yet, but ActionScript 3 IS an ECMAScript 4 compatible VM (or at least the closest one as one of my coworkers wants me to point out). Yes, it is the implementation of a standard and not something Adobe has made up. Yes, I know ECMAScript 4 was and still is preliminary, but the changes which will be adopted before it goes official will probably be extremely minor (depending on which further additions get approved though).

Adobe does NOT open source the Flash Player by doing this. Just the Verifier, JIT, core frameworks and the garbage collection engine are now open source. Also important to note: The java based compiler which converts ECMAScript 4 to the byte code understood by Tamarin is not included in this agreement. Instead the focus will be an ECMAScript 4 based compiler, which will be required for eval() to work anyway since we don't want to pollute Mozilla with too much Java code. Adobe Flash Player 9 does not support eval() to keep the binary size of the Flash Player to a minimum. Our focus on the Java compiler was required to support Flex Builder 2.

The plan is that Mozilla and the Adobe Flash Player will share the exact same code base now and in the future. There is no plan to fork the virtual machine at any point. Changes which go into the Mozilla source tree will be directly adopted by Adobe and vice versa.

Mozilla might adopt the conservative garbage collector even for SpiderMonkey. Again, what most people do not realize is that we made the switch from a simple reference counting system to a more advanced mechanism for ActionScript 1&2 while maintaining backwards compatibility. This was an incredible challenge. In case of the Adobe Flash Player this resulted in about half the memory requirements in many situations. This would be a huge boon for AJAX based applications, in addition to resolving circular reference issues if they still exist in SpiderMoney.

If you study the source code you'll realize that a 64bit port is NOT a recompile away. We are actively working on the 64bit port, the source code right now is still 32bit until the changes required are stabilized.

Flash/XComposite crasher in X11

noreply@blogger.com (Tinic Uro) — Mon, 30 Oct 2006 20:37:00 +0000

This one has been around for a while: When you enable the XComposite extension in X11 and run your desktop using a 15 or 16bit screen depth, the Flash Player will always crash with this error message:

The program 'firefox' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadMatch (invalid parameter attributes)'.
(Details: serial 115 error_code 8 request_code 143 minor_code 3)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
Segmentation fault (core dumped)

There are two workarounds for this:

Set the screen depth to 24bit. If you have enough VRAM available this is a good move anyway since you'll get much better performance and quality in general when using Flash. Other applications probably also benefit, there should be less color banding in general.

Set the XLIB_SKIP_ARGB_VISUALS environment variable to 1 in one of your init scripts before X11 is started.

So why have we not fixed this in the past? Well, I've not written the code in question and IMO it's even questionable if it is a bug in Flash, depending on what XMatchVisualInfo API is really supposed to do. The docs really do not provide too much information here. The core of the problem is this code in the Flash Player:

int depth = 0;
int mode = 0;
XVisualInfo vi;
static struct
{
  int depth, mode;
} formats[] = {
  { 24, TrueColor},
  { 32, TrueColor},
  { 16, TrueColor},
  { 15, TrueColor}, 
  { 8,  PseudoColor},
  { 0,  0}
};
for (int i=0;formats[i].depth; i++ )
{
  if(XMatchVisualInfo(display,    
    DefaultScreen(display),
    formats[i].depth, 
    formats[i].mode, 
    &vi)) {
      depth = formats[i].depth;
      mode = formats[i].mode;
      break;
  }
}

If XComposite is turned on and your screen depth is 16bit this loop will break when the depth is 32. That depth is used to create the XImage the Flash Player will use for its backbuffer. The actual crash occurs when XPutImage or XShmPutImage is called. At this point X11 will bail out when given an XImage with a depth of 32bit.

The fix? It's as simple as prepending this code to the above loop:

XWindowAttributes attribs;
if(XGetWindowAttributes(display,window,&attribs)) {
  if(XMatchVisualInfo(display, 
    DefaultScreen(display), 
    attribs.depth, 
    TrueColor, 
    &vi) {
      depth = attribs.depth;
      mode = TrueColor;
  }
} 
if(!depth){
  ...
}

Pretty much every distribution has complained about this issue over the past few years, so I am happy that we finally resolved this for good. Is this fix in the current beta of the Flash Player (9.0.21.55)? No, I just changed this Friday, so you'll have to wait for a newer build. It also needs to go through QE before this change gets the final approval.

Extending the reach of the Flash Player on Linux

noreply@blogger.com (Tinic Uro) — Tue, 24 Oct 2006 23:08:00 +0000

If you are a Linux user you have probably downloaded the Flash Player beta for Linux by now and played around with it. And if you are running on a system without ALSA you have no sound right now. Or you might have other sound playback issues. Bummer. You really want OSS don't you? And you probably think that you can write better code than we can, don't you? ;-)

We want to partially address this for the upcoming release. The truth is that it is rather difficult for us to support all the different APIs available on Linux. Things change frequently. Overall this is good in since it keeps innovation going. On the other hand we want to concentrate on the APIs which are used the most and ideally drop any legacy and bleeding edge support from the player to keep things lite, predictable and testable by our QE staff. There is still a lot of work left to do on our side, but for now have a look at this page:

Additional Interface Support for Linux

flashsupport.c currently has sample code for OpenSSL, GnuTLS (which is not working), ICU, ALSA and OSS interfaces. The Video4Linux1 code in flashsupport.c is currently not used at all by the Flash Player but I hope to get this up and running for the final release although I can't make any promises here. For now most of you will probably want to play around with the sound support.

As said on the Wiki this is all work in progress, highly experimental and completely untested (apart from me trying it on my lone Ubuntu box). I am sure I will change a bunch of stuff in the not too distant future and hopefully QE will also be able to give this a spin. But if you have anything to say about the approach we are taking, now is the time to speak up and give feedback. And no, we are NOT making the Flash Player open source with this, we are simply trying to give the community a chance to make the player work on as many environments as possible.

(Update: I fixed the line endings in flashsupport.c, OSS should hopefully not have the AV sync problem anymore and I missed to copy&paste a sem_init in the ALSA code)

Flash Player 9 for Linux Beta 1

noreply@blogger.com (Tinic Uro) — Wed, 18 Oct 2006 01:58:00 +0000

To quote someone well known, 'hell froze over' and we finally released a beta of the GNU/Linux version of the Adobe Flash Player 9 (look for the "Linux version" download link). It did take more to get to this point than you might expect. And no, Mike is not the only engineer working on it. Currently I count 6 engineers working on the GNU/Linux platform, even more have GNU/Linux boxen and/or VMWare images now. And there is still a lot of work left to do before I consider this penguin version not a beta anymore.

Now is this beta version bug free? By no means. Will it install and fully work your specific distro? There is a good chance it will not, although I have tried a good dozen of distributions. Do not walk away frustrated, instead submit as many bugs reports as you can. And no, the feedback page is not a black hole. We collect all feedback, enter it in our bug database and then prioritize it.

There are various things I am not quite happy about in the current version. We'll address these things over time. On my plate for research right now (and not in the beta 1 version) is Xembed support which should yield much better integration into the desktop environment and numerous other advantages.

What I am happy about is performance. This is the fastest GNU/Linux version we ever released. And we haven't reached the maximum yet. While annoying on my part since I had to deal with some rather mechanical conversion from Intel to AT&T assembly notation, it was really worth it. Even better, we will use the same code on OS X to get a completely relocatable binary, something Apple does not offer if you use MS style inline assembly, although this was a big new feature promoted by them to allow easy porting. In some cases though the GNU/Linux version is still way behind the Windows 32bit version (device text rendering performance which is using FreeType f.ex.), but in some cases the GNU/Linux version is up to 20% faster on the same hardware compared to the Windows 32bit version. It will be interesting to see what kind of benchmarks people will come up with. I will try to get this as good as it can get for the final release and so will everyone else on the team.

What about 64bit? There is no Windows 64bit or OS X 64bit version either right now. As I said before it is not a question of 'recompiling' the source code, there is lots of generic non platform specific work which needs to be finished first. We will ship a 64bit version for Windows, OS X Leopard and GNU/Linux. It will happen. When? ... When it is ready.

linux lite and fonts

noreply@blogger.com (Tinic Uro) — Fri, 15 Sep 2006 17:32:00 +0000

Mike has talked about some of this before, but this week I went over our Linux code and tried to make it more robust in some areas. Some parts have not been touched in a while, so house cleaning is a good thing to do. Part of this for me is to test different kind of scenarios, some of them might look a little weird. Well, let me provide a screenshot and you'll see what I mean:

We are still discussing if we should actually require GTK/GDK/glib to run at all as this would allow us to do many things we can't otherwise. Some of them are more essential than others. Something non-critial and certainly optional would be the ability to access GNOME specific font settings (which are usually slightly different than the ones specified in the global and user fonts.conf file). The biggest problem with requiring GTK/GDK/glib is that right now using any sort of GTK/GDK functions within nspluginwrapper simply does not work and we haven't spent any time addressing this. That leaves konqueror pretty crippeled, even though it is not crashing anymore. Simple things like copy&paste spit out errors like this:

(process:22016): Gtk-CRITICAL **: 
gtk_clipboard_get_for_display: assertion 
`GDK_IS_DISPLAY (display)' failed

Yes I know, we could add support for X11 selections, but this is besides the point. KISS (keep it simple stupid) is our coding method of choice for this release, otherwise we will never be done.

Talking about fonts: I recently took the liberty of revamping device text support in the Flash Player, the braindead font fallback system we had in the past is now the last resort, only invoked if fontconfig is not available. That means that if you are on a simplified Chinese locale you'll get the default font configured for the 'ch-Hans' locale on your system. Finally. I am still in shock we called this list of pre-baked font names in the player a solution.

gcc challenges

noreply@blogger.com (Tinic Uro) — Mon, 11 Sep 2006 02:55:00 +0000

Dealing with compilers can be quite a challenge sometimes. Recently Mike stumbled over a pretty bad crash on one of his older machines. It looked innocent enough. To enable SSE1/SSE2 intrinsics the only option in gcc is to use '-msse -msse2'. While it did not use SSE to do floating point it does emit cvttss2si instructions to do general floating point conversions if this option is selected. Since his machine was an using an Athlon Model 4 an illegal instruction error killed the browser. SSE1 was introduced with the Athlon XP and Pentium III generations only. This behavior makes it completely impossible to safely do runtime detection of SSE1/SSE2 when using gcc unless you split out the various architectures (x86, MMX, SSE1, SSE2 and SSE3) into separate files and customize compile options which is also the suggested hack in the gcc man pages:

"These options will enable GCC to use these extended instructions in generated code, even without -mfpmath=sse. Applications which perform runtime CPU detection must compile separate files for each supported architecture, using the appropriate flags. In particular, the file containing the CPU detection code should be compiled without these options."

There is also an old bug about this which was essentially rejected. Why is there no option to simply turn on intrinsics? This would make things so much easier for developers and guarantee better portability of source code.

I see two practical options right now: Either we totally disable any SSE1/SSE2 optimizations which will severely cripple performance (yes, some of the rendering code will see a 30-60% slowdown which will affect most users), or, we use the Intel compiler to compile these files. Using ICC would also allow us to compile some of the inline assembly for which I have not created intrinsics yet. The real challenge for me now is to modify the autoconf scripts to use two compilers, something which it does not seem to support by default. Google and various forums I looked on are not of much help either on this subject.

Why not using separate files as suggested by the gcc man pages? Well, it would add another few weeks of refactoring and stabilizing code to make this happen. I fear that I will be tasked with this eventually though.

We also discovered recently that '-O2' can generate bad code and had to switch to '-O1' for the time being. It triggered badness in our support math routines for the JIT and funny enough some rather simple C code (our MMX code works great here):


for ( ; n ; n-- ) {
    uint32 srcP = src[0];
    uint32 src0 = ((srcP&0x0000FF00)<< 8)|
                  ((srcP&0x000000FF)<< 0)|
    uint32 src1 = ((srcP&0xFF000000)>> 8)|
                  ((srcP&0x00FF0000)>>16);

    uint32 dst0 = dst[0];
    uint32 dst1 = dst[1];

    dst0 = ((dst0*(srcP>>24))>>8) + src0;
    dst1 = ((dst1*(srcP>>24))>>8) + src1;

    dst[0] = dst0;
    dst[1] = dst1;

    src+=1;
    dst+=2;
}

This is not reproducible in a standalone test app, I've tried it. Why would gcc make it easy on us anyway? :-) We are trying figure out which option is the issue and disable it. Also, the same code compiles fine using Apple's gcc build 5431 vs gcc 4.0.3 which I am using on Ubuntu right now. Even if it were to fail in a future revision of Apple's version of gcc this code would never be triggered as SSE1 and SSE2 are always guaranteed to be available on MacIntel machines.

compiz

noreply@blogger.com (Tinic Uro) — Fri, 18 Aug 2006 20:35:00 +0000

As I said in the previous blog post I am using compiz on my MacBook. It's kind of funky in a good way, even though I prefer the less glizzy OS X window manager which is also based on OpenGL.

While playing around with the Flash Player I noticed that Xorg-air was using quite a bit of CPU, along with FireFox itself. Now this is not completly unexpected, but switching to metacity confirmed my fears: The X11 shm extension might not work well or at all with AIGLX/compiz, meaning it probably spends a lot of time copying data to the GPU on each frame we pass to X11. Here are the two different tops testing the same content (and no, changing the active windows makes no change):

I'll try to debug that further once the player is more stable, for now we have to live with this. Hopefully there is an option to fix it up in the AIGLX/compiz code in a way that it uses DMA transfers to the GPU and only uploads parts of the image (usually by using glTexSubImage2D).

Don't get the idea that we can use OpenGL in the plugin for Linux, last time I checked the Netscape plugin API had no access to an OpenGL context for Unix targets. If anyone wants to do the work for Unix they are welcome, we would certainly use it and save a lot of CPU time with it if it's correctly done. It will soon be possible on OS X, Josh Aas from the Mozilla foundation is working on it and Tim Omernick from Apple has already started extending the WebKit to support CoreGraphics and OpenGL (look for his changes in WebKit/JavaScriptCore/bindings/npapi.h). This support will be required for 64bit targets and Leopard since QuickDraw is going away for good.

pthreads and signals

noreply@blogger.com (Tinic Uro) — Sat, 12 Aug 2006 18:21:00 +0000

Mike, the others on the Linux team and now also some core player developers are still working to get the Linux version into good shape. Now that I am done with the MacIntel version I could not resist to install Ubuntu on my MacBook in all its glory including compiz. We have tried a bunch of machines already so I wanted to make sure that the Flash Player would work on it.

Sure enough, the snd-hda-intel driver seems to have bugs. First I had to patch the module to get any sound output at all as they person who merged in the MacBook kernel patches did remove the custom Apple pin configs required for alsamixer to work correctly.

The second problem I had was that snd_pcm_avail_update() sometimes would return values like 0x3ffffee5 instead of the actual number of frames. It looks kind of similar to a negative value, meaning the buffer is overrun with data. But why are the 6 high bits not set then? Anyhow, this was causing havoc with sound playback and I had to add a hack to get around this temporarily until we figured out why we were shoving in too much data, which we did soon after.

Still, for some reason the Flash Player would hang after a few minutes. Mike showed me the backtrace, it was hanging in a pthread mutex lock in the progressive FLV playback code. As I had written the code for progressive FLV playback way back in Flash Player 7 and still feel responsible for it I took a peek. Progressive FLV playback was designed to be able to read from CD-Roms asynchronously so it's running in its own thread to avoid blocking the main playback thread. Hence primitives like queues need to be protected with a mutex.

Good lord, it turns out that mixing pthread mutexes and signals (semaphores are probably affected the same way) is a bad idea. Right now we are using ALSAs callback method to notify us when we can add more data to the buffer using snd_pcm_writei(). When looking at the backtrace it is obvious that the callback is initiated from a signal. The problem is that if a pthread mutex lock is contested and therefore blocking it can become the actual thread to initiate another callback. So we are now in the ALSA callback trying to acquire the same mutex it was trying to lock in the first place. A nice deadlock ensues. And while we could probably detect this situation, more bad things are bound to happen with hacks. I spent an afternoon trying to find a good solution until I decided we would use the ALSA callback only to kick off another thread (using a semaphore) which would then do the actual sound mixing. Sure enough, videos now play for hours at end on my machine without crashing/hanging. For those interested, this is how the backtrace looked like in gdb (I changed the method names so things make more sense):


#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7e5b2ae in __lll_mutex_lock_wait ()
 from /lib/tls/i686/cmov/libpthread.so.0
#2  0xb7e57fc7 in _L_mutex_lock_159 ()
 from /lib/tls/i686/cmov/libpthread.so.0
...
#12 0xb06e9b53 in PlatformMutex
 (this=0xbfbf48ec, pMutex=0xaf58b57c)
#13 0xb0998d1d in NetStreamQueue::PopPacket
 (this=0xaf58b530)
...
#18 0xb071e527 in PlatformSoundMixer::FillAlsaBuffer
 (this=0xaf583008)
#19 0xb071e78a in PlatformSoundMixer::AlsaCallback
 (ahandler=0x8b32178)
#20 0xb5ab3aba in snd_output_buffer_open ()
 from /usr/lib/libasound.so.2
#21 <signal handler called>
#22 0xb7e57f7b in pthread_mutex_lock ()
 from /lib/tls/i686/cmov/libpthread.so.0
..
#25 0xb06e9b53 in PlatformMutex
 (this=0xbfbf5164, pMutex=0xaf58b57c)
#26 0xb099692e in NetStreamQueue::HandleOnStatus
 (this=0xaf58b530)
#27 0xb099c21a in NetStream::Receive (this=0xaf58b430)
#28 0xb09901bc in NetSocket::Receive (this=0xaf5ad140)
#29 0xb09765fa in FlashPlayer::Idle (this=0xaf57c008)
#30 0xb0700ebb in PlatformFlashPlayer::OnTimer
 (this=0xaf57c008)
#31 0xb06df318 in gtkTimerCallback (data=0xaf57c008)
#32 0xb784a4a8 in g_main_context_is_owner ()
 from /usr/lib/libglib-2.0.so.0
#33 0xb78488d6 in g_main_context_dispatch ()
 from /usr/lib/libglib-2.0.so.0
#34 0xb784b996 in g_main_context_check ()
 from /usr/lib/libglib-2.0.so.0
#35 0xb784bcb8 in g_main_loop_run ()
 from /usr/lib/libglib-2.0.so.0
#36 0xb7c96765 in gtk_main ()
 from /usr/lib/libgtk-x11-2.0.so.0
#37 0xb691590a in nsAppShell::Run (this=0x8147828)
 at nsAppShell.cpp:139
#38 0xb68433d2 in nsAppStartup::Run (this=0x8151f70)
 at nsAppStartup.cpp:150
#39 0x0804f321 in XRE_main (argc=3, argv=0xbfbf59b4,
 aAppData=0x80595e0) at nsAppRunner.cpp:2374
#40 0x0804abe4 in main (argc=0, argv=0x0)
 at nsBrowserApp.cpp:61
#41 0xb75deea2 in __libc_start_main ()
 from /lib/tls/i686/cmov/libc.so.6
#42 0x0804ab31 in _start ()
 at ../sysdeps/i386/elf/start.S:119

Now I am sure someone could explain me why this is expected behavior and by design. My first reaction was that it had to be a bug in the pthreads implementation. And why ALSA is using signals for callbacks is also a mystery to me. I am sure there is a perfectly acceptable explanation for this also. Anyway, my goal is to make it work, the rest does not really matter...

Builds, builds and more builds

noreply@blogger.com (Tinic Uro) — Thu, 20 Jul 2006 19:04:00 +0000

Following up some requests to see more what happens behind scenes here at Adobe I thought I talk a little bit about our daily build process. There is not too much terribly exciting as we follow common industry practices, but some numbers may be surprising for some.

First and foremost everything here centers around SCM (Source code management). Without it nothing would get done. Every kind code and documentation we create is manipulated with it. The tool of choice for us is Perforce and after the merger between Macromedia and Adobe we were happy to find out that Adobe was using it all across the company. It might not have the fancy features other newer solutions have, but it's always been rock solid, fast and has excellent client tools. A long time ago (Flash 3&4) we were using CVS which proved to be too primitive in the end though (and yes I know, everyone is migrating to subversion at this point).

Every time we make a change to the Flash Player we face the challenge that every single project and target still needs to compile and work (for those who know, we compile with the '-Werror' option). Normally it takes a few hours to rebuild all of our projects, but we employ a few tools to help engineers find build issues. One such tool is IncrediBuild which allows to check all Windows builds within a few minutes. Also, every time an engineer checks in a change, a server will automatically start builds of all targets and send out an email if a build failed including the offending change list number. On OS X we use distcc which is part of XCode to achieve the same. A luxury we do not have with CodeWarrior (typically a full build of one target in CodeWarrior takes 20-30 minutes). We do not have a distcc setup for Linux yet, but Mike is working hard on making this happen right now. That's the reason he ordered a few Gentoo servers recently.

What is a target? Every project has 4 targets: "Release", "Release Debugger", "Debug" (update: left this one out originally) and "Debug Debugger". The "Debugger" targets contain code for debugging ActionScript whereas the "Debug" targets are non-optimized binaries with a full set of symbols for debugging C++ code. Every project we support has these targets and are build daily. Our set of standard projects we build every day are:

Flash.ocx (Win32 x86)

NPSWF32.dll (Win32 x86)

authplay.dll (Win32 x86)

standalone.exe (Win32 x86)

FlashPlayer.plugin (PPC 32-bit 10.1 or newer)

FlashPlayer.plugin (Universal 32-bit 10.4 or newer)

AuthPlay.bundle (Universal 32-bit 10.4 or newer)

Standalone.app (Universal 32-bit 10.4 or newer)

libflashplayer.so (Linux x86 32-bit)

gflashplayer (Linux x86 32-bit)

That makes 40 binaries which are build overnight plus a wide variation of installation packages for different deployment scenarios. Quite a lot of stuff and that only represents our mainline builds. We keep separate branches for all older versions and special ongoing projects (which I can't talk about ;-). So in the worst case we could end up with a couple of hundred binaries and installer packages on a given day. Why do we do this? Typically QE (quality engineering) needs to confirm fixed bugs in every binary. If injections are found it is valuable to be able to quickly find the build a bug was injected by testing older builds.

Talking about bugs, we use like almost everyone in the industry a bug database which not only holds information on bugs, but also engineering tasks, ECRs (Enhancement/feature requests, usually from users) and other technical information which needs to be tracked and followed up on, f.ex. in tech notes. Currently the system consists of a SQL database, ColdFusion with an easy to use forms based HTML front-end. Nothing really special. But this is probably the one tool everyone in the team uses, including our beta testers, although they see a somewhat different interface.

Something which might not be well know, but every time you submit a bug report or feature request through the wish form we have a human verify the submission and add a new entry to our bug database. It's not automatic, as you can imagine we get a lot of spam and inappropriate stuff through this medium. Although sometimes feedback can be outright amusing...

Universal binary update, sound woes...

noreply@blogger.com (Tinic Uro) — Mon, 17 Jul 2006 20:32:00 +0000

I am still busily working on the universal binary beta. So far the beta feedback has been very sparse (probably because penetration of Intel machines is still low), with half the complaints being around sound playback. The good news is that we can fix it for the Intel binary, the PowerPC binary will still fail in certain situations as we are uncomfortable changing the PowerPC binary too much at this point.

What situations? Well, until now the Flash Player was using the Carbon Sound Manager to output sound. It turns out if you change some system settings which some audio application like to do it breaks that API. It seems that the QuickTime player is also using the Sound Manager, so you do not get any sound playback there either when playing back a .mov.

To reproduce the bug in the universal binary beta (9.0r18 or earlier) go to Applications->Utilities->Audio MIDI Setup and change the properties for the 'Build-in Output', changing the format from 44100.0 Hz to f.ex. 96000.0 Hz. I used OS X 10.4.7 with QuickTime 7.1.1. Now, you'll notice that there is suddenly no audio anymore with any Flash content.

A user reported that Audacity is a source of this issue and indeed, if you just launch Audacity it always changes the system wide setting to 96000.0 Hz without reverting it to the old value when you quit. So when using older applications, use Audacity and suddenly have no sound output make sure to double check this setting.

What's the fix? Well, the easy way for me would be to fault the Sound Manager, it should support any sampling rate or have Audacity not change this setting unless asked for. On the other hand I know that the Sound Manager is some extremely old API which is basically there to support legacy applications only. Therefore I switched the whole sound output to use Core Audio which is probably the Right Thing to do at this point. Although as I said, we are uncomfortable with drastic changes like this, so we will not touch the PowerPC binary for now.

Udpate: QE has informed me that is is apparently not an issue with native PowerPC G4 and G5 machines as their sound hardware does not support anything higher than 48000.0 Hz with which Sound Manager has an issue. This seems to be the reason this only happens on Intel hardware. Well, except if you have a special sound card which supports 96000.0 Hz.