There is a new playback engine implemented by Google comes with Android 2.0 (i.e, Stagefright), which seems to be quite simple and straightforward compared with the OpenCORE solution.
- MediaExtractor is responsible for retrieving track data and the corresponding meta data from the underlying file system or http stream;
- Leveraging OMX for decoding: there are two OMX plugins currently, adapting to PV's software codec and vendor's hardware implementation respectively. And there is a local implementation of software codecs which encapsulates PV's decoder APIs directly;
- AudioPlayer is responsible for rendering audio, it also provides the timebase for timing and A/V synchronization whenever audio track is present;
- Depending on which codec is picked, a local or remote render will be created for video rendering; and system clock is used as the timebase for video only playback;
- AwesomePlayer works as the engine to coordinate the above modules, and is finally connected into android media framework through the adapter of StagefrightPlayer.
Will Stagefright finally allow Android to play M4B files? If so, will it support bookmarking and embedded graphics?
ReplyDeleteThank you very much for posting class diagram of stage fright player. Could you please let us know whether stagefright architecture is better than OpenCore?? As per my understanding, this code is just a R&D work. Very basic player with lot of bugs, not flexible enough to add new features....Dave spark says, it's going to replace OpenCore in next release, my gut feeling is that it will take couple of years for google to match openCore ...
ReplyDeleteAs Google decided to push Stagefright as the default engine for local&http playback, you could expect many bug fixes and enhancements have already been integrated into Froyo.
ReplyDeleteI agree OpenCORE is functionally richer currently, however I would prefer the simplicity of Stagefright on the perspective of architecture, from which it would be easier to extend and evolve.
And sorry, I don't work at Google, so I have no idea whether it will support bookmarking finally:)
At the moment, it seems that stagefright is unable to follow HTTP redirects on HTTP data sources, which is quite annoying.
ReplyDeleteThe diagram really helps to understand stagefright. But which component is resposable for A/V sync in video playback?
ReplyDeleteThanks for the overview.
ReplyDeleteCan you give an example of how to leverage this framework, like for adding a new codec?
As I understand, one of the targets was to build a generic and dynamic enough framework. However I find it hard to understand how add components to it - both on the implementation and administration (deployment) levels.
To add a new codec support, you'll need to add code for retrieving the track from the corresponding MediaExtractor, and change various places in OMXCodec.cpp, e.g. the codec/component role map, the code to negotiate parameters with the OMX component, and probably new quirks specific to your codec implementation, etc.
ReplyDeleteIt's not as generic or dynamic as other multimedia frameworks. It's just simple.
Thanks.
ReplyDeleteI understand now that I can't do it when installing a new app - only upon a new android image.
I want add my own codec while compiling android source.. Please can you explain what all the steps I need to take.
ReplyDeleteMy client will be giving his own codec to me I need to add that to android.
is this architectural diagram generated using reverse engineering tool...? Can you pls tell which tool u have used for this
ReplyDeleteHi
ReplyDeleteI am trying to trace down the code flow for mp4 decoding and it seems its calling avc decoder directly rather than calling openMAX API(OMXCodec.cpp). I have got my own openMAX compliant codec which i would like to add to stagefright.
Regards,
Parag
have anyone tried NMM framework from motama ?
ReplyDeleteMediaPlayer(24881): error (1, -1004)
ReplyDeleteAny insight on this would be helpful. Thanks.
*-1004 means ERROR_IO which is defined in MediaErrors.h
ReplyDeleteThere are many cases where ERROR_IO is returned if you do a search in Stagefright. So you probably need to add some logs to track down the issue.
*AVCDecoder is a software codec, so the code in OMXCodec.cpp will not be invoked after it's created;
*The diagram was drawn by me manually. You can buy Rational Rose or use UMLet(http://www.umlet.com/) for free.
I am working on a project to capture H264/AVC stream from Stagefright/AwesomePlayer.cpp file.
ReplyDeleteTo do that, within function onVideoEvent,
in for loop:
I replaced mVideoSource->read(&mVideoBuffer, &options);
with mVideoTrack->read(&mVideoBuffer, &options);
I was expecting mVideoBuffer->range_length() returns the size of the a trunk of H264 stream, which is bigger than 100 bytes. In fact, it always returns 12 bytes while playing back a video clip.
Anyone can give me an insight?
from a native app , is simultaneous encode + decode possible in StageFright ?
ReplyDeleteFirst of all Thank You very much for giving us the details of Stagefrgiht. This document and the explanation is really helpful for beginners like me.
ReplyDeleteI would like to develop a VideoCall application for Android 2.3 so is learning library of Stagefright is useful if so then what is the starting point for me.
Secondly, if not then what is the part in Android that I should be looking at.
Appreciate your reply.
Thanks
Can you please explain the AV sync from AwesomePlayer perspective. Which module is responsible to do that?
ReplyDeleteProblem with stagefright mp3 streaming
ReplyDelete====================
When trying to stream the following links through the Stagefrightplayer,
Player is taking around 60- 70 sec to play the audio
http://pubint.ic.llnwd.net/stream/pubint_wbcx.
The same url is working fine with Open Core PV player
pv player is able to stream the audio in 2-4 seconds
LOGCAT LOG ::
D/MediaPlayer( 3271): android.media.MediaPlayer 736 Couldn't open file on client side, trying server side
01-21 13:20:52.789: W/NuHTTPDataSource(1470): Server did not give us the content length!
=======================
Can anybody help to resolve the issue?
I have a lot of individual files on my webpage. In my logs, Stagefright is downloading all 100+ MP3 files. That might be one reason that the Android Toshiba Thrive locked up. I see some iPads doing the same thing with Safari.
ReplyDeletehttp://dialajoke.us/
Hey Hi,
ReplyDeleteI have enabled libstagefright_h264 with ffmpeg-0.10. But when i am trying to decode the h264 video i am getting errors as follows.
# ./ffmpeg -i /mnt/sdcard/Video/stream.h264 /mnt/sdcard/Video/out.yuv
ffmpeg version 0.10.2 Copyright (c) 2000-2012 the FFmpeg developers
built on May 4 2012 11:01:41 with gcc 4.4.3
configuration: --target-os=linux --cross-prefix=arm-linux-androideabi- --arch=arm --cpu=armv7-a --sysroot=/home/anilv/projects/android-ndk-r7c/plat0
libavcodec 53. 61.100 / 53. 61.100
libavformat 53. 32.100 / 53. 32.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100
[h264 @ 0xd00a10] max_analyze_duration 5000000 reached at 5040000
[h264 @ 0xd00a10] decoding for stream 0 failed
[h264 @ 0xd00a10] Could not find codec parameters (Video: h264)
[h264 @ 0xd00a10] Estimating duration from bitrate, this may be inaccurate
/mnt/sdcard/Video/stream.h264: could not find codec parameters
@ Batteryman
ReplyDeleteI just finished porting ffmpeg to android with android-ndk-r7. It works fine after I found out the gnu-libstdc++ has serious bugs in std:string implementation, it creates memory leak that leads to network malfunction. The results are I don't get UDP packets on Android. After I threw out everything using string, it works perfectly. But this ffmpeg h264 decoder is too slow to decode HD video with 1280x720, therefore, I am now thinking of using hardware decoder shipped within an android device. The only way to access this codec is through openmax IL. But android-ndk doesn't provide OMXIL (Integration Layer), just OMXAL (Application Layer). In OMXAL I could only get the player interface, that is useless to me, I just need the decoder, because I have already fully control via my own rtsp client to handle video streaming.
It is also useful to you and me to know how to get access to openmax integration layer.
Steven
Please help me to compile ffmpeg for android on mac ..detail steps
ReplyDeleteNice work Freepine :P
ReplyDeleteGaurav
Well written by blog owner & i really appreciated your blog. Thanks for the post..
ReplyDeletebest android tv box