使用MediaPlayer播放音视频时,最终会创建AudioTrack对象用于播放音频数据。

MediaPlayer创建AudioTrack分为三个步骤,分别是:

1. 创建NuPlayer对象

2. 创建AudioOutput对象,传递给AudioSink

3.在MediaPlayer::start()中创建AudioTrack对象

1. 创建NuPlayerDriver对象

(1) MediaPlayerFactory::createPlayer

使用MediaPlayer播放时,需要先设置数据源,调用mplayer.serDataSource(),该方法会调用到MediaPlayerService::Client::setDataSource_pre(),该方法中调用createPlayer返回的指针p就是NuPlayerDriver对象。

(2)MediaPlayerFactory::createPlayer

在该方法中,会继续调用factory->createPlayer,其中IFactory类是MediaPlayerFactory的内部类,而NuPlayerFactory类又继承自MediaPlayerFactory::IFactory

(3)MediaPlayerFactory::IFactory的createPlayer

    virtual sp<MediaPlayerBase> createPlayer(pid_t pid) {
        ALOGV(" create NuPlayer");
        return new NuPlayerDriver(pid);
    }

最终,返回一个NuPlayerDriver对象。

在初始化列表中,实例化NuPlayer(mPlayer->init(this)),并将new出来的对象赋值给mPlayer,调用 mPlayer->init做一些初始化工作。NuPlayerDriver初始化列表,获取NuPlayer类的实例化对象mPlayer,这样在NuPlayerDriver就可以调用NuPlayer类的成员函数和成员变量了。

2. 设置AudioOutput

创建完NuPlayerDriver对象后,setDataSource()方法中会继续创建AudioOutput对象,并通过setAudioSink设置到NuPlayerDriver中。

    if (!p->hardwareOutput()) {
        mAudioOutput = new AudioOutput(mAudioSessionId, IPCThreadState::self()->getCallingUid(),
                mPid, mAudioAttributes, mAudioDeviceUpdatedListener);
        static_cast<MediaPlayerInterface*>(p.get())->setAudioSink(mAudioOutput);
    }

此时p就是第一步创建的NuPlayerDriver,继续根据代码分析:

void NuPlayerDriver::setAudioSink(const sp<AudioSink> &audioSink) {
    mPlayer->setAudioSink(audioSink);
    mAudioSink = audioSink;
}

此时的mPlayer就是NuPlayer

void NuPlayer::setAudioSink(const sp<MediaPlayerBase::AudioSink> &sink) {
    sp<AMessage> msg = new AMessage(kWhatSetAudioSink, this);
    msg->setObject("sink", sink);
    msg->post();
}

在此发送消息kWhatSetAudioSink,再来看接收消息的地方:

        case kWhatSetAudioSink:
        {
            ALOGV("kWhatSetAudioSink");

            sp<RefBase> obj;
            CHECK(msg->findObject("sink", &obj));

            mAudioSink = static_cast<MediaPlayerBase::AudioSink *>(obj.get());
            break;
        }

最终将获取的MediaPlayerBase::AudioSink指针对象赋值给了NuPlayer类的成员变量mAudioSink,也就是把AudioOutput赋值给mAudioSink。

3.创建AudioTrack对象

开始播放时,调用mplayer.start(),通过binder机制调用到MediaPlayerService::Client::start()

status_t MediaPlayerService::Client::start()
{
    ALOGV("[%d] start", mConnId);
    sp<MediaPlayerBase> p = getPlayer();
    if (p == 0) return UNKNOWN_ERROR;
    p->setLooping(mLoop);
    return p->start();
}

此时的p就是第一步创建的NuPlayerDriver对象

status_t NuPlayerDriver::start() {
    ALOGV("start(%p), state is %d, eos is %d", this, mState, mAtEOS);
    Mutex::Autolock autoLock(mLock);
    return start_l();
}

start_l()中调用 mPlayer->start();,mPlayer就是NuPlayer,发送消息kWhartStart

void NuPlayer::start() {
    (new AMessage(kWhatStart, this))->post();
}

找到接收kWhartStart消息的地方

        case kWhatStart:
        {
            ALOGV("kWhatStart");
            if (mStarted) {
                // do not resume yet if the source is still buffering
                if (!mPausedForBuffering) {
                    onResume();
                }
            } else {
                onStart();
            }
            mPausedByClient = false;
            break;
        }

onStart中,我们重点看两个动作,第一是创建Render对象,在实例化Renderer时,mAudioSink成员变量传给Renderer类的成员函数mAudioSink, 而mAudioSink是第二步骤中setAudioSink时传入的AudioOutput对象

mRenderer = new Renderer(mAudioSink, mMediaClock, notify, flags);

第二是调用了postScanSources(),发送kWhatScanSources消息

void NuPlayer::postScanSources() {
    if (mScanSourcesPending) {
        return;
    }

    sp<AMessage> msg = new AMessage(kWhatScanSources, this);
    msg->setInt32("generation", mScanSourcesGeneration);
    msg->post();

    mScanSourcesPending = true;
}

找到接收kWhatScanSources消息的地方,最重要的处理是初始化解码器:

            if (mSurface != NULL) {
                if (instantiateDecoder(false, &mVideoDecoder) == -EWOULDBLOCK) {
                    rescan = true;
                }
            }

最终调用到mRenderer->openAudioSink方法

void NuPlayer::tryOpenAudioSinkForOffload(
        const sp<AMessage> &format, const sp<MetaData> &audioMeta, bool hasVideo) {
    // Note: This is called early in NuPlayer to determine whether offloading
    // is possible; otherwise the decoders call the renderer openAudioSink directly.

    status_t err = mRenderer->openAudioSink(
            format, true /* offloadOnly */, hasVideo,
            AUDIO_OUTPUT_FLAG_NONE, &mOffloadAudio, mSource->isStreaming());
    if (err != OK) {
        // Any failure we turn off mOffloadAudio.
        mOffloadAudio = false;
    } else if (mOffloadAudio) {
        sendMetaDataToHal(mAudioSink, audioMeta);
    }
}

在openAudioSink()中,发送消息kWhatOpenAudioSink

   sp<AMessage> msg = new AMessage(kWhatOpenAudioSink, this);

找到接收kWhatOpenAudioSink的地方,

        case kWhatOpenAudioSink:
        {

            status_t err = onOpenAudioSink(format, offloadOnly, hasVideo, flags, isStreaming);

在onOpenAudioSink()中,调用mAudioSink的open方法,即调用AudioOutput的open方法

            err = mAudioSink->open(
                    sampleRate,
                    numChannels,
                    (audio_channel_mask_t)channelMask,
                    audioFormat,
                    0 /* bufferCount - unused */,
                    &NuPlayer::Renderer::AudioSinkCallback,
                    this,
                    (audio_output_flags_t)offloadFlags,
                    &offloadInfo);

在AudioOutput的open方法中,创建了AudioTrack对象

status_t MediaPlayerService::AudioOutput::open(
        uint32_t sampleRate, int channelCount, audio_channel_mask_t channelMask,
        audio_format_t format, int bufferCount,
        AudioCallback cb, void *cookie,
        audio_output_flags_t flags,
        const audio_offload_info_t *offloadInfo,
        bool doNotReconnect,
        uint32_t suggestedFrameCount)
{
  ...
 sp<AudioTrack> t;
 if (!(reuse && bothOffloaded)) {
        ALOGV("creating new AudioTrack");

        if (mCallback != NULL) {
            newcbd = new CallbackData(this);
            t = new AudioTrack(
                    mStreamType,
                    sampleRate,
                    format,
                    channelMask,
                    frameCount,
                    flags,
                    CallbackWrapper,
                    newcbd,
                    0,  // notification frames
                    mSessionId,
                    AudioTrack::TRANSFER_CALLBACK,
                    offloadInfo,
                    mUid,
                    mPid,
                    mAttributes,
                    doNotReconnect,
                    1.0f,  // default value for maxRequiredSpeed
                    mSelectedDeviceId);
        } 

整理调用流程如下:

注:通过学习盼雨落,等风起的博文,自己添了些总结。

Logo

魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。

更多推荐