December 19, 2005
Faking Transparency in Flash Video
By Elliot Mebane, Roguish.com
In September 2005 Macromedia released Macromedia Studio 8, including Flash 8 and a new video codec from On2 that gives better video quality than earlier versions of Flash video at the same file size, and also offers native alpha-channel video transparency. Before this release of Flash, the Flash video codec (made by Sorenson), did not support transparency the new codec allows Flash developers to composite video with alpha-channel transparency live at the time of playback (including editable background text) in Flash applications.
Whiskas wanted to release a virtual cat that would walk around on top of their website. They wanted users to be able to interact with it and it had to be real video of a cat Oh, and they wanted it in the first quarter of 2005, 6 months before the release of Flash 8 with native video transparency. So, Roguish and 2 other agencies worked together to create a method of faking transparent video.
A Solution... Abandoned...
The easy, highest-quality solution to the problem would have been to shoot footage of the cat against greenscreen, key out the background, then render out each clip as a series of .png files with transparency. Flash supports the 24-bit transparent .png format, so the .pngs could be imported into Flash and placed on the timeline as a sequence of individual frames in a Flash movie. Testing showed that this method results in file sizes about 7-times greater than the file sizes of the movies we created with our final solution.
The Real Fake Solution
While Flash 7 Video doesn't natively support transparency, Video objects are treated just like all other objects in Flash and can be masked by vector shapes. Do you see where I'm going with this? If there were a way to create vector shapes around the foreground subject in each frame of video (the cat), those vector shapes could be used as masks to reveal the foreground subject and hide the background, frame-by-frame. Flash 7 didn,t support partial-transparency when applying masks, so with this solution the foreground subject would have a solid edge. The tradeoff is file size vs. natural feathered transitions from foreground opacity to background transparency that can be achieved with an 8-bit alpha channel.
From On-Set through Post-Production
It took a professional cat wrangler with his 3 nearly-identical cats and 2-days of greenscreen capture to get 1200 clips to select from. We culled 50 usable clips from the bunch. Careful placement and sequencing of the final clips allows the cat to move through a series of pre-defined motions, and the clips can also be dynamically combined at runtime to allow users to interact with the cat - feeding it, giving it a ball to play with and dangling a string toy over its head.
Once the clips were selected, we used Keylight (bundled with AE 6.5 Pro) to extract the foreground subject from the background. Next, we created a new Solid layer and used the Auto-Trace feature (also part of AE) to automatically create a new mask for each frame in each clip based on its alpha channel. Auto-Trace was critical to making this project feasible within our timeframe and budget - without it, we would have had to resort to painstaking manual rotoscoping.
After some final hand-manipulation of the masks, 2 elements were exported from AE to be re-composited later in Flash: The masks were exported as vectors, and the videos were rendered as Quicktime movies for conversion to the Flash .flv video format using a third-party encoder.
More about the Vector Mask Exports
AE includes some basic exporting to the .swf format (Flash), exporting vector elements such as masks as vector shapes, and most other content as .jpgs. We wanted to include only the masks in the .swf export, so before exporting the masks we disabled the video layer. AE has a peculiarity when exporting masks to the .swf format; The vector mask shapes are exported from AE, but they are placed in the .swf as vector shapes that are masked by a rectangular mask the size of the AE stage. This presented some difficulty when the .swfs were brought into Flash to be used as masks in the final composition (more details about this problem and the solution in the Breeze recording tutorial linked below).
More about the Quicktime Exports and FLV Compression
At the time the project was being completed there were 3 commonly used software packages for converting video to the .flv video filetype that Flash uses: Sorenson Squeeze, Wildform Flix, and Macromedia's built-in encoder for Flash MX 2004 Pro. The third party plug-ins created much higher quality than the built-in encoder, and for this project we used the Wildform tool. FYI - The story is a little different today; The encoder that now ships with Flash Professional 8 encodes the newfangled Flash 8 video (still a .flv filename extension) and is quite good - The 2 third-party encoders for the new format (On2 and Sorenson) differentiate themselves from the built-in Macromedia encoder with 2-pass VBR encoding for an incremental quality improvement over the 1-pass CBR encoder that ships with Flash Professional. For a look at the differences between codec manufacturers and the new Flash 8 vs. old Flash 7 video formats, visit FlashVideoFAQ.com. The site allows you to compare Flash 7 and Flash 8 video encoded with the 3 major .flv-encoding software packages at a variety of bit rates.
Bringing all the Pieces together in Flash
After exporting the video layer from AE (without masking - just premultiplied over black) and converting it to .flv format using Wildform Flix, the .flv video was imported into a new Flash document The associated .swf that was exported from AE including the vector masks was also imported and placed on the Flash timeline in the layer above the video. After some manipulation of the imported .swf due to the quirky way that AE exports masks to the .swf format, the vector mask layer was set to mask the underlying video layer, and voila! Every frame of walking, jumping, sitting, playing, eating, climbing and scratching cat was perfectly matched with a vector outline, revealing only the cat and hiding the background.
What is this Good For Today?
We would surely leverage the native alpha transparency in the new Flash 8 video codec if we could deploy the Whiskas Virtual Cat today, but that doesn,t mean that the process described here isn,t useful in a variety of ways. Macromedia estimates that it will take approximately 9 months after its release for the Flash Player 8 (necessary to play the new Flash 8 video format) to reach near-ubiquitous market penetration. This leaves a good 6 month window after the writing of this article for Flash developers to use this process to deploy Flash video with transparency without requiring their website visitors to download the latest Flash Player plug-in. Additionally, the process can be used for a variety of other purposes - vector shapes that trace the contour of an object in video can be used as interactive and design elements in Flash.
To view the Whiskas Virtual Cat, please visit the Whiskas USA website here, and select the USA version of the site:
The Breeze recording below is an audio/video tutorial of the procedure described above, detailing the entire process of faking transparency in Flash 7 video from the process of using Keylight to key-out the greenscreen through the process of using AE-generated masks in Flash: http://mmusergroup.breezecentral.com/p14271579/
Elliot Mebane is President of Roguish, a multimedia production company in Los Angeles, CA with clients including Apple, Yahoo!, Mars(Whiskas) TBWA Chiat\Day, ClearChannel, Hilton Hotels, Toyota and McGraw-Hill.
Other resources mentioned in this tutorial:
© 2000 -2005 Apple
Computer, Inc. All rights reserved. Apple, the Apple logo, Final
Cut Pro, Macintosh and Power Mac
are either registered trademarks or trademarks of Apple. Other company and product names may be trademarks of their respective owners.
All screen captures, images, and textual references are the property and trademark of their creators/owners/publishers.