An Uphill Stream
By Carlo Longino, Tue Nov 19 10:00:00 GMT 2002

MPEG-4 has been made the de facto mobile streaming standard. But is it the best choice?


The holy grail of 3G has been to put video in everyone's pocket. But the delay in the build-out of third-generation networks and unforeseen emerging technologies may bring mobile streaming video to consumers much earlier than previously thought. Of course, whether they'll want to pay for it is a different matter completely.

Streaming media is already being incorporated into new mobile devices and networks - NTT DoCoMo has gained a lot of attention (although not many users) for its 3G network featuring V-Live, a streaming video service that offers movie trailers, music videos, Hello Kitty family stories, golf lessons, and even archived ping-pong matches. Audible.com has put its software into the new Palm Tungsten handhelds and the Orange SPV smartphone, allowing users to access and download audio files from its library of books-on-tape, newspaper summaries, and other programming. The SPV also features PacketVideo's pvPlayer application for streaming media. RealNetworks' RealOne Player is included with Nokia's 9200 series Communicators, and is also available for the PocketPC platform. And of course Microsoft includes its Windows Media Player software with all PocketPC devices, including mobile models like the O2 XDA and Audiovox Thera.

So although in its infancy, streaming media has arrived. But what technologies best enable it?

Square MPEG in a Round Hole


Industry bodies have been hard at work to define standards for streaming media in the interest of interoperability. The clear choice for multimedia has been the MPEG-4 standard. Developed by the Moving Picture Experts Group, the same group within the International Standards Organization that came up with the successful MPEG-1 and MPEG-2 standards, MPEG-4 was based on Apple Computer's QuickTime technology to take advantage of its extensibility - the ability to combine various media (not only audio and video, but also 3D objects, sprites, text, and so on) - and scalability, the ability transport a media at any bit rate.

For audio, the MPEG-4 standard uses Advanced Audio Coding, or AAC, developed by many of the same companies behind the popular MP3 and AC3 Dolby Digital codecs. Think of AAC as a souped-up version of MP3: the bottom line is that it offers improved compression that results in better sound quality with smaller file sizes, but also importantly for mobile devices, allows for more efficient decoding, requiring less processing power.

MPEG-4 offers a number of benefits when used in a mobile application, the least of which are the available "profiles" which segment the standard for different applications. Most appropriate for mobile streaming are the simple and simple scalable profiles, which offer efficient, error-resistant encoding.

Any mobile streaming technology needs to be extremely error-resilient, thanks to propagation problems, interference, and handoffs, not to mention high latency. Errors can occur at the physical or link layers, as well as in the network layer, and there may also be packet loss due to network buffer overflows.

Non-real-time data such as e-mail can make a trade-off of delay for error control. Network protocols are used to guarantee the accuracy of the data, but often involve delays in doing so. Streaming media and other applications like videoconferencing, however, cannot accept this, and although network protocols can help the stream along, errors must be dealt with by the player application.

The MPEG-4 standard offers several features to help with this. Errors can be constrained in the data stream by using resynchronization markers placed in the video by the video encoder. This breaks the stream up into "video packets" which can help contain and conceal errors. It also uses motion-compensated concealment techniques, which separate motion and texture information. When there are errors in the texture information, the image can be decoded and created using only the data associated with motion vectors; should there be errors in the motion data, estimation can be used to recover the lost data, and then the recovered data is used to conceal the error.

Another problem in streaming on mobile networks are the variable bit rates of user connections. Packet delay often occurs, having a direct impact on user throughput. MPEG-4 deals with this limitation by scaling itself to the throughput, adding enhancement information via additional encoded frames to a base layer of video. It also utilizes spatial scalability, in the simple scalable profile, by encoding information to enhance images within a frame.

MPEG-4 production and player applications are already being developed and delivered. RealNetworks recently announced its Helix Mobile Producer, powered by an encoder from Envivio, which gives content creators a single encoder that will output 3GPP and 3GPP2 as well as RealAudio and RealVideo content. It also has live and on-demand encoding designed for the small screens of mobile devices, and supports MPEG-4 as well as other formats like AAC, N-AMR, and H.263.

Take 'Em Down an MPEG or -4


But as has been the case in the tech world time and time again, the biggest standard, or the one with the most support, may not necessarily be the best. Mobile streaming media is no different, and several companies have developed codecs and algorithms they say outdo MPEG-4.

The Nancy code, developed by Japan's Office Noa, has already been incorporated into Japanese models of the Sharp Zaurus PDA, and is used by J-Phone for its Video Sha-Mail video messaging service. Nancy's main strength is its compactness. Office Noa says its algorithm requires significantly less CPU power than MPEG-4; in fact, less than 1%. An MPEG-4 chip that could decode 15 frames per second at 174x144 would require about 20 million transistors; at 200,000 transistors, Nancy can decode 30 frames per second at 320x240.

The codec uses a proprietary algorithm called "Structured Meta Scale Polygon," which divides images into blocks of different shapes and sizes for compression, ranging from 1 pixel up to 32x32, depending on the images' complexity. MPEG-4 uses motion-estimation and a discrete-cosine-transform algorithm; Nancy uses only the four basic arithmetic operations and comparison and bit-shift operation. Accordingly, its codec only requires 30 to 40k of memory and operates at a tenth of the power of a comparable MPEG-4 operation, the company says.

Pulsent, a US-based company, has developed a compression technique that eschews the block-based methods used in MPEG standards and RealVideo and Windows Media formats in favor of an object-based scheme. Whereas block-based method simply segment an image into blocks, then look for matches from frame to frame, Pulsent's method breaks an image down into objects and tracks and predicts their movement from frame to frame. The company says this cuts down on errors and needed bandwidth when compared to standard block-based methods.

Vector quantization is another method gaining ground. This technology was used by Scottish company Essential Viewing in Orange's HSCSD videophone product of a few years back, and is the method favored by Oplayo, which recently launched a streaming content portal with Finnish marketing firm Soprano Communications. Oplayo's technology is extremely attractive in the mobile space because it is plug-in-free, meaning that it's displayed through a Java applet that can be sent to the receiving device along with the stream.

Vector quantization works by creating a "codebook" of tiny image patches. The video stream then tells the decoder which patch to call up and where to place it for each frame, creating a mosaic of the patches to approximate the original image. Oplayo says its motion vector unitization-based system can stream at ten times faster than MPEG-4.

And of course the big dog's not standing still, either. Microsoft has inked deals with PacketVideo and Emblaze to integrate its Windows Media technology into their players, meaning users with devices running operating systems apart from Microsoft's will be able to access Windows Media-encoded material.

While MPEG-4 may have a lot of traction, some of these other technologies seem like they'll have an impact, although another high-tech lesson has been that the best method or product isn't necessarily the most successful. That may be the case in streaming technology, certainly when digital-rights-management-hungry content producers exert their influence.

Itís Media Week on TheFeature! Check back daily for in-depth articles and analysis on the developing world of mobile media. And share your thoughts with TheFeatureís global mobile community through The Podium.

Carlo Longino is a freelance writer based in Austin, Texas. His previous experience includes work for The Wall Street Journal, Dow Jones Newswires, and Hoover's Online.