美文网首页
Surfaceflinger 合成流程

Surfaceflinger 合成流程

作者: 程序员Android1 | 来源:发表于2021-08-10 09:25 被阅读0次

    和你一起终身学习,这里是程序员 Android

    经典好文推荐,通过阅读本文,您将收获以下知识点:

    一、Consumer端的FrameListener
    二、消息队列MessageQueue
    三、显示设备DispalyDevice

    通过前面的简单介绍,我们对HWC合成有大致的了解。下面我们根据实际代码进行讲解。前面章节,我们已经说过,Layer的创建,和BufferQueue,那么Buffer进入到BufferQueue队列中后,怎么进行合成显示的呢?我们继续来看。

    一、Consumer端的FrameListener

    你还记得Producer的frameAvailableListener吗?Buffer放入队列BufferQueue后,是不是通过frameAvailableListener->onFrameAvailable通知Consumer?大家可以再回望一下BufferQueueProducer::queueBuffer

    frameAvailableListener是哪里来的?

    我们先来看一下Consumer中Listener间的相互关系

    image
    • BufferLayer有专门的Consumer,BufferLayerConsumer;BufferLayerConsumer继承ConsumerBase。ConsumerBase通过IGraphicBufferConsumer和BufferQueue进行通信的。
    • BufferQueue中的frameAvailableListener,是一个IConsumerListener的接口,对应的Binder的Bn端实现为ProxyConsumerListener。
    • BufferLayer实现了ContentsChangedListener,ContentsChangedListener继承FrameAvailableListener。BufferLayer的Listener实现,被传给了ConsumerBase。
    • ConsumerBase实现ConsumerListener接口,构建ConsumerBase时,会创建ProxyConsumerListener,将ConsumerBase实现的Listener接口传给ProxyConsumerListener。
    • BufferQueue中Listener回调时,会回调到ConsumerBase中。ConsumerBase中再通过BufferLayer实现的,传下来的Listener回调到BufferLayer中。

    层层回调,别弄混淆了。

    关键代码:
    BufferQueueProducer中通过frameAvailableListener->onFrameAvailable回调到ProxyConsumerListener中:

    * frameworks/native/libs/gui/BufferQueue.cpp
    
    void BufferQueue::ProxyConsumerListener::onFrameAvailable(
            const BufferItem& item) {
        sp<ConsumerListener> listener(mConsumerListener.promote());
        if (listener != NULL) {
            listener->onFrameAvailable(item);
        }
    }
    
    

    ProxyConsumerListener中的mConsumerListener是ConsumerBase中的实现。这里的listener->onFrameAvailable将回调到ConsumerBase中。

    * frameworks/native/libs/gui/ConsumerBase.cpp
    
    void ConsumerBase::onFrameAvailable(const BufferItem& item) {
        CB_LOGV("onFrameAvailable");
    
        sp<FrameAvailableListener> listener;
        { // scope for the lock
            Mutex::Autolock lock(mFrameAvailableMutex);
            listener = mFrameAvailableListener.promote();
        }
    
        if (listener != NULL) {
            CB_LOGV("actually calling onFrameAvailable");
            listener->onFrameAvailable(item);
        }
    }
    
    

    ConsumerBase中的mFrameAvailableListener是BufferLayer中的实现:

    * frameworks/native/services/surfaceflinger/BufferLayer.cpp
    
    void BufferLayer::onFrameAvailable(const BufferItem& item) {
        // Add this buffer from our internal queue tracker
        { // Autolock scope
            Mutex::Autolock lock(mQueueItemLock);
            mFlinger->mInterceptor.saveBufferUpdate(this, item.mGraphicBuffer->getWidth(),
                                                    item.mGraphicBuffer->getHeight(),
                                                    item.mFrameNumber);
            // Reset the frame number tracker when we receive the first buffer after
            // a frame number reset
            if (item.mFrameNumber == 1) {
                mLastFrameNumberReceived = 0;
            }
    
            // Ensure that callbacks are handled in order
            while (item.mFrameNumber != mLastFrameNumberReceived + 1) {
                status_t result = mQueueItemCondition.waitRelative(mQueueItemLock,
                                                                   ms2ns(500));
                if (result != NO_ERROR) {
                    ALOGE("[%s] Timed out waiting on callback", mName.string());
                }
            }
    
            mQueueItems.push_back(item);
            android_atomic_inc(&mQueuedFrames);
    
            // Wake up any pending callbacks
            mLastFrameNumberReceived = item.mFrameNumber;
            mQueueItemCondition.broadcast();
        }
    
        mFlinger->signalLayerUpdate();
    }
    
    

    BufferLayer中调用onFrameAvailable,去通知SurfaceFlinger进行合成。

    到这里,应用端(Producer)生产完Buffer这件事,就通知到了SurfaceFlinger中了。

    SurfaceFlinger的signalLayerUpdate,是通过MessageQueue来处理的,我们先来看看MessageQueue。

    二、消息队列MessageQueue

    MessageQueue是SurfaceFlinger中的消息队列,为什么需要消息队列?我们应用有一个主线程,专门进行UI的处理。SurfaceFlinger同样的,也有一个主线程,SurfaceFlinger的主线程主要进行显示数据的处理,也就是合成。

    SurfaceFlinger中,mEventQueue是MessageQueue的一个栈对象,采用mutable修饰。SurfaceFlinger在初次引用时,会对mEventQueue进行初始化。

    * frameworks/native/services/surfaceflinger/MessageQueue.cpp
    
    void MessageQueue::init(const sp<SurfaceFlinger>& flinger)
    {
        mFlinger = flinger;
        mLooper = new Looper(true);
        mHandler = new Handler(*this);
    }
    
    

    MessageQueue初始化时,创建了一个Looper和一个Handler。

    此外,在SurfaceFlinger初始化时,创建了一个EventThread,并传给了MessageQueue。

    void SurfaceFlinger::init() {
        ... ...
        sp<VSyncSource> sfVsyncSrc =
                new DispSyncSource(&mPrimaryDispSync, SurfaceFlinger::sfVsyncPhaseOffsetNs, true, "sf");
        mSFEventThread = new EventThread(sfVsyncSrc, *this, true);
        mEventQueue.setEventThread(mSFEventThread);
    
    

    MessageQueue的setEventThread函数如下:

    void MessageQueue::setEventThread(const sp<EventThread>& eventThread)
    {
        if (mEventThread == eventThread) {
            return;
        }
    
        if (mEventTube.getFd() >= 0) {
            mLooper->removeFd(mEventTube.getFd());
        }
    
        mEventThread = eventThread;
        mEvents = eventThread->createEventConnection();
        mEvents->stealReceiveChannel(&mEventTube);
        mLooper->addFd(mEventTube.getFd(), 0, Looper::EVENT_INPUT,
                MessageQueue::cb_eventReceiver, this);
    }
    
    

    MessageQueue在setEventThread时,主要做以下几件事:

    • 创建一个BitTube对象mEventTube

    • 创建一个EventConnection

    sp<EventThread::Connection> EventThread::createEventConnection() const {
        return new Connection(const_cast<EventThread*>(this));
    }
    
    

    Connection在第一次引用时,将会被注册到mEventThread中。

    void EventThread::Connection::onFirstRef() {
        // NOTE: mEventThread doesn't hold a strong reference on us
        mEventThread->registerDisplayEventConnection(this);
    }
    
    

    在注册时,Connection将会被添加到mDisplayEventConnections 中。

    status_t EventThread::registerDisplayEventConnection(
            const sp<EventThread::Connection>& connection) {
        Mutex::Autolock _l(mLock);
        mDisplayEventConnections.add(connection);
        mCondition.broadcast();
        return NO_ERROR;
    }
    
    

    mDisplayEventConnections是一个已经注册的Connection的集合。

    • 将mEventTube和EventConnection关联
    status_t EventThread::Connection::stealReceiveChannel(gui::BitTube* outChannel) {
        outChannel->setReceiveFd(mChannel.moveReceiveFd());
        return NO_ERROR;
    }
    
    

    Connection创建时,将默认创建一个4k的BitTube,BitTube封装的是一对socket,一个发送,一个接收,可传输的Buffer大小为4K。

    void BitTube::init(size_t rcvbuf, size_t sndbuf) {
        int sockets[2];
        if (socketpair(AF_UNIX, SOCK_SEQPACKET, 0, sockets) == 0) {
            size_t size = DEFAULT_SOCKET_BUFFER_SIZE;
            setsockopt(sockets[0], SOL_SOCKET, SO_RCVBUF, &rcvbuf, sizeof(rcvbuf));
            setsockopt(sockets[1], SOL_SOCKET, SO_SNDBUF, &sndbuf, sizeof(sndbuf));
            // since we don't use the "return channel", we keep it small...
            setsockopt(sockets[0], SOL_SOCKET, SO_SNDBUF, &size, sizeof(size));
            setsockopt(sockets[1], SOL_SOCKET, SO_RCVBUF, &size, sizeof(size));
            fcntl(sockets[0], F_SETFL, O_NONBLOCK);
            fcntl(sockets[1], F_SETFL, O_NONBLOCK);
            mReceiveFd.reset(sockets[0]);
            mSendFd.reset(sockets[1]);
        } else {
            mReceiveFd.reset();
            ALOGE("BitTube: pipe creation failed (%s)", strerror(errno));
        }
    }
    
    

    MessageQueue中的mEventTube,和mReceiveFd关联。

    • addFd函数,将fd添加到MessageQueue的Looper中。
      注意,Looper的callback为MessageQueue::cb_eventReceiver,data为MessageQueue本身。
    int MessageQueue::cb_eventReceiver(int fd, int events, void* data) {
        MessageQueue* queue = reinterpret_cast<MessageQueue *>(data);
        return queue->eventReceiver(fd, events);
    }
    
    

    回到SurfaceFlinger的signalLayerUpdate函数:

    void SurfaceFlinger::signalLayerUpdate() {
        mEventQueue.invalidate();
    }
    
    

    signalLayerUpdate中,调用MQ的invalidate。

    void MessageQueue::invalidate() {
        mEvents->requestNextVsync();
    }
    
    

    MQ的invalidate的函数,将请求下一个Vsync。Vsync是一种同步机制,垂直同步,我们可以理解为SurfaceFlinger的工作节拍。

    void EventThread::requestNextVsync(
            const sp<EventThread::Connection>& connection) {
        Mutex::Autolock _l(mLock);
    
        mFlinger.resyncWithRateLimit();
    
        if (connection->count < 0) {
            connection->count = 0;
            mCondition.broadcast();
        }
    }
    
    

    注意这里的count >= 0。

    EventThread就是一个事件分发的线程,第一次引用时,线程启动。

    void EventThread::onFirstRef() {
        run("EventThread", PRIORITY_URGENT_DISPLAY + PRIORITY_MORE_FAVORABLE);
    }
    
    

    EventThread的threadLoop函数体如下:

    bool EventThread::threadLoop() {
        DisplayEventReceiver::Event event;
        Vector< sp<EventThread::Connection> > signalConnections;
        signalConnections = waitForEvent(&event);
    
        // dispatch events to listeners...
        const size_t count = signalConnections.size();
        for (size_t i=0 ; i<count ; i++) {
            const sp<Connection>& conn(signalConnections[i]);
            // now see if we still need to report this event
            status_t err = conn->postEvent(event);
            if (err == -EAGAIN || err == -EWOULDBLOCK) {
                ... ...
                ALOGW("EventThread: dropping event (%08x) for connection %p",
                        event.header.type, conn.get());
            } else if (err < 0) {
                removeDisplayEventConnection(signalConnections[i]);
            }
        }
        return true;
    }
    
    

    waitForEvent,等待事件Event。看看哪些Connection是被触发的,对于被触发的Connection,signalConnections,通过postEvent将事件Event分发出去。

    Event这边的控制逻辑,基本都在waitForEvent中。waitForEvent中,采用while循环,条件是signalConnections为空。EventThread中主要控制两事件,Vsync事件和显示屏的HotPlug热插拔事件

        enum {
            DISPLAY_EVENT_VSYNC = fourcc('v', 's', 'y', 'n'),
            DISPLAY_EVENT_HOTPLUG = fourcc('p', 'l', 'u', 'g'),
        };
    
    

    我们分段来看:

    Vector< sp<EventThread::Connection> > EventThread::waitForEvent(
            DisplayEventReceiver::Event* event)
    {
        Mutex::Autolock _l(mLock);
        Vector< sp<EventThread::Connection> > signalConnections;
    
        do {
            bool eventPending = false;
            bool waitForVSync = false;
    
            size_t vsyncCount = 0;
            nsecs_t timestamp = 0;
            for (int32_t i=0 ; i<DisplayDevice::NUM_BUILTIN_DISPLAY_TYPES ; i++) {
                timestamp = mVSyncEvent[i].header.timestamp;
                if (timestamp) {
                    // 有一个Vsync事件要分发
                    if (mInterceptVSyncs) {
                        mFlinger.mInterceptor.saveVSyncEvent(timestamp);
                    }
                    *event = mVSyncEvent[i];
                    mVSyncEvent[i].header.timestamp = 0;
                    vsyncCount = mVSyncEvent[i].vsync.count;
                    break;
                }
            }
    
    

    看看有没有Vsync事件要分发,timestamp不为0,表示有Vync事件要分发。

            if (!timestamp) {
                // no vsync event, see if there are some other event
                eventPending = !mPendingEvents.isEmpty();
                if (eventPending) {
                    // we have some other event to dispatch
                    *event = mPendingEvents[0];
                    mPendingEvents.removeAt(0);
                }
            }
    
    

    mPendingEvents,这里主要是是HotPlug事件。

    找出在等待事件的Connection:

            size_t count = mDisplayEventConnections.size();
            for (size_t i=0 ; i<count ; ) {
                sp<Connection> connection(mDisplayEventConnections[i].promote());
                if (connection != NULL) {
                    bool added = false;
                    if (connection->count >= 0) {
                        // 需要Vsync,至少有一个Connection的count >= 0。
                        waitForVSync = true;
                        if (timestamp) {
                            // we consume the event only if it's time
                            // (ie: we received a vsync event)
                            if (connection->count == 0) {
                                // fired this time around
                                connection->count = -1;
                                signalConnections.add(connection);
                                added = true;
                            } else if (connection->count == 1 ||
                                    (vsyncCount % connection->count) == 0) {
                                // continuous event, and time to report it
                                signalConnections.add(connection);
                                added = true;
                            }
                        }
                    }
    
                    if (eventPending && !timestamp && !added) {
                        // 没有Vsync事件要处理,但是有其他的事件要处理
                        signalConnections.add(connection);
                    }
                    ++i;
                } else {
                    // Connection不存在了
                    mDisplayEventConnections.removeAt(i);
                    --count;
                }
            }
    
    

    Connection的count >= 0表示需要Vsync。eventPending && !timestamp && !added表示没有Vsync事件要处理,但是有其他的事件要处理。

            if (timestamp && !waitForVSync) {
                disableVSyncLocked();
            } else if (!timestamp && waitForVSync) {
                enableVSyncLocked();
            }
    
    

    timestamp && !waitForVSync如果有Vsync事件要分发,但是又没有Connection需要Vsync事件时,把Vsync给关掉。相反,如果有Connection需要Vsync,而此时又没有Vsync事件时,需要将Vsync打开。

            if (!timestamp && !eventPending) {
                if (waitForVSync) {
                    bool softwareSync = mUseSoftwareVSync;
                    nsecs_t timeout = softwareSync ? ms2ns(16) : ms2ns(1000);
                    if (mCondition.waitRelative(mLock, timeout) == TIMED_OUT) {
                        if (!softwareSync) {
                            ALOGW("Timed out waiting for hw vsync; faking it");
                        }
                        // FIXME: how do we decide which display id the fake
                        // vsync came from ?
                        mVSyncEvent[0].header.type = DisplayEventReceiver::DISPLAY_EVENT_VSYNC;
                        mVSyncEvent[0].header.id = DisplayDevice::DISPLAY_PRIMARY;
                        mVSyncEvent[0].header.timestamp = systemTime(SYSTEM_TIME_MONOTONIC);
                        mVSyncEvent[0].vsync.count++;
                    }
                } else {
                    mCondition.wait(mLock);
                }
            }
        } while (signalConnections.isEmpty());
    
        return signalConnections;
    }
    
    

    如果此时没有Vsync事件,或其他的Event事件,那就处于等待中。如果是等待Vsync,那么通过mCondition.waitRelative进行等待,如果是硬件Vsync还不能用或者出现问题时,设置一个超时时间,进行屏幕的唤醒。
    如果Connection需要Vsync,那么就进程sleep。

    Vsync事件到来时,将回调到onVSyncEvent:

    void EventThread::onVSyncEvent(nsecs_t timestamp) {
        Mutex::Autolock _l(mLock);
        mVSyncEvent[0].header.type = DisplayEventReceiver::DISPLAY_EVENT_VSYNC;
        mVSyncEvent[0].header.id = 0;
        mVSyncEvent[0].header.timestamp = timestamp;
        mVSyncEvent[0].vsync.count++;
        mCondition.broadcast();
    }
    
    

    Hotplug事件到来时,将回调到onHotplugReceived:

    void EventThread::onHotplugReceived(int type, bool connected) {
        ALOGE_IF(type >= DisplayDevice::NUM_BUILTIN_DISPLAY_TYPES,
                "received hotplug event for an invalid display (id=%d)", type);
    
        Mutex::Autolock _l(mLock);
        if (type < DisplayDevice::NUM_BUILTIN_DISPLAY_TYPES) {
            DisplayEventReceiver::Event event;
            event.header.type = DisplayEventReceiver::DISPLAY_EVENT_HOTPLUG;
            event.header.id = type;
            event.header.timestamp = systemTime();
            event.hotplug.connected = connected;
            mPendingEvents.add(event);
            mCondition.broadcast();
        }
    }
    
    

    注意,Event事件哪儿回调回来的我们先不管,我们先记住这里的逻辑。

    Vsync事件是跟屏幕的刷新频率有关,比如60Hz的屏幕,两个Vsync事件间的时间为1/60s,也就是16.67ms左右。SurfaceFlinger每隔16.67ms进行一次合成,显示。

    另外,需要注意的是,SurfaceFlinger和App的EventThread是分开的,不是同一个。

    void SurfaceFlinger::init() {
        ... ...
    
        // start the EventThread
        sp<VSyncSource> vsyncSrc =
                new DispSyncSource(&mPrimaryDispSync, SurfaceFlinger::vsyncPhaseOffsetNs, true, "app");
        mEventThread = new EventThread(vsyncSrc, *this, false);
        sp<VSyncSource> sfVsyncSrc =
                new DispSyncSource(&mPrimaryDispSync, SurfaceFlinger::sfVsyncPhaseOffsetNs, true, "sf");
        mSFEventThread = new EventThread(sfVsyncSrc, *this, true);
        mEventQueue.setEventThread(mSFEventThread);
    
    

    回到MessageQueue,Connection通过postEvent将Event抛出来后,通过sendEvents将事件发出去。

    status_t EventThread::Connection::postEvent(
            const DisplayEventReceiver::Event& event) {
        ssize_t size = DisplayEventReceiver::sendEvents(&mChannel, &event, 1);
        return size < 0 ? status_t(size) : status_t(NO_ERROR);
    }
    
    

    DisplayEventReceiver中是通过BitTube将事件发出去,sendObjects注意这里的参数。

    ssize_t DisplayEventReceiver::sendEvents(gui::BitTube* dataChannel,
            Event const* events, size_t count)
    {
        return gui::BitTube::sendObjects(dataChannel, events, count);
    }
    
    

    数据是什么地方接受的呢?回到SurfaceFlinger

    SurfaceFlinger线程run时,启动一个死循环,循环等待事件。

    void SurfaceFlinger::run() {
        do {
            waitForEvent();
        } while (true);
    }
    
    

    waitForEvent中,调用MessageQueue的waitMessage函数:

    void MessageQueue::waitMessage() {
        do {
            IPCThreadState::self()->flushCommands();
            int32_t ret = mLooper->pollOnce(-1);
            switch (ret) {
                case Looper::POLL_WAKE:
                case Looper::POLL_CALLBACK:
                    continue;
                case Looper::POLL_ERROR:
                    ALOGE("Looper::POLL_ERROR");
                    continue;
                case Looper::POLL_TIMEOUT:
                    // timeout (should not happen)
                    continue;
                default:
                    // should not happen
                    ALOGE("Looper::pollOnce() returned unknown status %d", ret);
                    continue;
            }
        } while (true);
    }
    
    

    waitMessage,通过采用一个死循环,处理Looper的pollOnce。Looper内部的逻辑就不看了,主要是采用epoll_wait对fd进行监听,BitTube发送Event对象后,epoll_wait结束,调用callback,处理事件

    int callbackResult = response.request.callback->handleEvent(fd, events, data);
    
    

    MessageQueue对应的callback为cb_eventReceiver:

    int MessageQueue::cb_eventReceiver(int fd, int events, void* data) {
        MessageQueue* queue = reinterpret_cast<MessageQueue *>(data);
        return queue->eventReceiver(fd, events);
    }
    
    

    eventReceiver,处理事件:

    int MessageQueue::eventReceiver(int /*fd*/, int /*events*/) {
        ssize_t n;
        DisplayEventReceiver::Event buffer[8];
        while ((n = DisplayEventReceiver::getEvents(&mEventTube, buffer, 8)) > 0) {
            for (int i=0 ; i<n ; i++) {
                if (buffer[i].header.type == DisplayEventReceiver::DISPLAY_EVENT_VSYNC) {
                    mHandler->dispatchInvalidate();
                    break;
                }
            }
        }
        return 1;
    }
    
    

    dispatchInvalidate,封装为MessageQueue::INVALIDATE

    void MessageQueue::Handler::dispatchInvalidate() {
        if ((android_atomic_or(eventMaskInvalidate, &mEventMask) & eventMaskInvalidate) == 0) {
            mQueue.mLooper->sendMessage(this, Message(MessageQueue::INVALIDATE));
        }
    }
    
    

    MessageQueue中,两种Message,INVALIDATEREFRESH

        enum {
            INVALIDATE  = 0,
            REFRESH     = 1,
        };
    
    

    Message的分发略过,Handler对Message的处理如下:

    void MessageQueue::Handler::handleMessage(const Message& message) {
        switch (message.what) {
            case INVALIDATE:
                android_atomic_and(~eventMaskInvalidate, &mEventMask);
                mQueue.mFlinger->onMessageReceived(message.what);
                break;
            case REFRESH:
                android_atomic_and(~eventMaskRefresh, &mEventMask);
                mQueue.mFlinger->onMessageReceived(message.what);
                break;
        }
    }
    
    

    收到消息后,再调回SurfaceFlinger在onMessageReceived中处理。

    再看SurfaceFlinger的处理之前,我们先稍微整理一下MessageQueue,MessageQueue的类图如下:

    image

    三、显示设备DispalyDevice

    SurfaceFlinger中,每个显示屏我们用DisplayDevice进行描述,它除了描述了Display的信息,还包括很多和合成相关的逻辑。相比于native层,Display信息是在Android的Framework层管理的,提供了专门的服务DisplayManagerService(DMS),DMS后续再介绍。

    3.1 Display接口服务

    。从Android 8.0开始,Vsync和hotplug的接收接口IDisplayEventReceiver作为一个单独的库从SurfaceFlinger中独立出来,设计为3层模式,JAVA层,Native层和HAL层。编译为libdisplayservicehidl库,代码在如下位置:

    frameworks/native/services/displayservice
    
    

    Display 接口 Android.bp如下:

    cc_library_shared {
        name: "libdisplayservicehidl",
    
        srcs: [
            "DisplayService.cpp",
            "DisplayEventReceiver.cpp",
        ],
    
        shared_libs: [
            "libbase",
            "liblog",
            "libgui",
            "libhidlbase",
            "libhidltransport",
            "libutils",
            "android.frameworks.displayservice@1.0",
        ],
    
        export_include_dirs: ["include"],
        export_shared_lib_headers: [
            "android.frameworks.displayservice@1.0",
            "libgui",
            "libutils",
        ],
    
        cflags: [
            "-Werror",
            "-Wall",
        ]
    }
    
    

    hal层也抽象出Display的单独模块displayservice。代码为在:

    frameworks/hardware/interfaces/displayservice
    
    

    displayservice的Android.bp如下:

    hidl_interface {
        name: "android.frameworks.displayservice@1.0",
        root: "android.frameworks",
        vndk: {
            enabled: true,
        },
        srcs: [
            "types.hal",
            "IDisplayEventReceiver.hal",
            "IDisplayService.hal",
            "IEventCallback.hal",
        ],
        interfaces: [
            "android.hidl.base@1.0",
        ],
        types: [
            "Status",
        ],
        gen_java: true,
    }
    
    

    displayservice还比较简单,没有太多接口:

    • types中只定义了一个状态
    * frameworks/hardware/interfaces/displayservice/1.0/types.hal
    
    package android.frameworks.displayservice@1.0;
    
    enum Status : uint32_t {
        SUCCESS,
        BAD_VALUE,
        UNKNOWN,
    };
    
    
    • IDisplayEventReceiver.hal中定义了Receiver的接口
    package android.frameworks.displayservice@1.0;
    
    import IEventCallback;
    
    interface IDisplayEventReceiver {
        /**
         * 添加callback,开始接收Events事件,热插拔是默认打开的,Vysnc需要通过setVsyncRate打开
         */
        init(IEventCallback callback) generates (Status status);
    
        /**
         * 开始或停止发送callback
         */
        setVsyncRate(int32_t count) generates (Status status);
    
        /**
         * 请求一个Vsync,如果setVsyncRate是0,这不起作用
         */
        requestNextVsync() generates (Status status);
    
        /**
         * Server端丢弃所以的callback,停止发送
         */
        close() generates (Status status);
    };
    
    
    • IDisplayService.hal
    package android.frameworks.displayservice@1.0;
    
    import IDisplayEventReceiver;
    
    interface IDisplayService {
        /**
         * 创建新的receiver.
         */
        getEventReceiver() generates(IDisplayEventReceiver receiver);
    };
    
    
    • IEventCallback.hal
    package android.frameworks.displayservice@1.0;
    
    interface IEventCallback {
        /**
         * Vsync事件
         */
        oneway onVsync(uint64_t timestamp, uint32_t count);
    
        /**
         * hotplug事件
         */
        oneway onHotplug(uint64_t timestamp, bool connected);
    };
    
    

    displayservice的接口,主要是提供给Vendor的HAL使用,让Vendor的HAL也能够接收Vsync数据。libdisplayservicehidl中也主要是DisplayEventReceiver。所以,这里的IDisplayEventReceiver接口 这么设计的 主要是提供给Vendor用。

    3.2 显示屏的类型

    显示屏幕什么时候创建?各类型的显示屏不一样,Android支持3中类型的显示屏:主显外显虚显

    * frameworks/native/services/surfaceflinger/DisplayDevice.h
    
        enum DisplayType {
            DISPLAY_ID_INVALID = -1,
            DISPLAY_PRIMARY     = HWC_DISPLAY_PRIMARY,
            DISPLAY_EXTERNAL    = HWC_DISPLAY_EXTERNAL,
            DISPLAY_VIRTUAL     = HWC_DISPLAY_VIRTUAL,
            NUM_BUILTIN_DISPLAY_TYPES = HWC_NUM_PHYSICAL_DISPLAY_TYPES,
        };
    
    

    主显示屏幕和外显,都采用热插拔的形式,连接断开时从底层驱动上报热插拔事件,这是在EventThread中处理的。

    • 主显示屏 DISPLAY_PRIMARY
      主显示屏幕默认是必现支持的,也就是说,开机时就应该上报 * 连接* 事件,知道屏幕关闭时,才断开。这里说的屏幕关闭是真正的关闭,休眠,锁屏等状态屏幕还是开着的。基本也就是关机的时候。

    • 外显示屏 DISPLAY_EXTERNAL
      外显示屏幕,一般是有线连接的屏幕,比如HDMI,MHL或者是其他连接标准连接的屏幕,外显一般经常进行热插拔。

    • 虚拟显示屏 DISPLAY_VIRTUAL
      虚拟显示屏,也就是说这个显示屏是不存在物理设备的,是个虚拟的。

    我们先看看DisplayDevice相关类之间的关系:

    image

    我们可以这么来理解:

    • Android支持多个显示屏幕,每一个显示屏幕用DisplayDevice进行描述,SurfaceFlinger中有这些显示屏的信息即mDisplay
    • 每个显示屏幕,都一个相关联的Buffer,这个Buffer用 DisplaySurface进行描述。
    • 每种类型的显示屏,具体采用的Buffer不尽相同,主屏和外显采用FramebufferSurface,而虚拟显示屏采用VirtualDisplaySurface。
    • DisplaySurface有自己的BufferQueue,都继承ConsumerBase,所以这里DisplaySurface比较特殊,都是消费者。VirtualDisplaySurface更猛,它还继承BnGraphicBufferProducer,也是生产者。
    • RE_Surface是RE命名空间的Surface,和BufferQueue中的Surface同名,但是作用不一样。主要是给RenderEngine用,RenderEngine是一个抽象类,渲染引擎,用于Client端的合成,一般用OpenGL进程合成。

    3.3 DisplayDevice的热插拔处理

    SurfaceFlinger在初始化的时候,注册Callback接口后。显示屏幕插上和断开时,将通过HAL回调回来。HAL回调的过程先不关注,从SurfaceFlinger中开始看。

    void SurfaceFlinger::onHotplugReceived(int32_t sequenceId,
            hwc2_display_t display, HWC2::Connection connection,
            bool primaryDisplay) {
        ALOGV("onHotplugReceived(%d, %" PRIu64 ", %s, %s)",
              sequenceId, display,
              connection == HWC2::Connection::Connected ?
                      "connected" : "disconnected",
              primaryDisplay ? "primary" : "external");
    
        ConditionalLock lock(mStateLock,
                std::this_thread::get_id() != mMainThreadId);
    
        if (primaryDisplay) {
            getBE().mHwc->onHotplug(display, connection);
            if (!mBuiltinDisplays[DisplayDevice::DISPLAY_PRIMARY].get()) {
                createBuiltinDisplayLocked(DisplayDevice::DISPLAY_PRIMARY);
            }
            createDefaultDisplayDevice();
        } else {
            if (sequenceId != getBE().mComposerSequenceId) {
                return;
            }
            if (getBE().mHwc->isUsingVrComposer()) {
                ALOGE("External displays are not supported by the vr hardware composer.");
                return;
            }
            getBE().mHwc->onHotplug(display, connection);
            auto type = DisplayDevice::DISPLAY_EXTERNAL;
            if (connection == HWC2::Connection::Connected) {
                createBuiltinDisplayLocked(type);
            } else {
                mCurrentState.displays.removeItem(mBuiltinDisplays[type]);
                mBuiltinDisplays[type].clear();
            }
            setTransactionFlags(eDisplayTransactionNeeded);
        }
    }
    
    

    接收到屏幕插拔事件后,主要做了如下的处理:

    • 通知HWC onHotplug

    通过onHotplug通知HWC;如果是连接,HWC将去获取新添加Display的config信息,如果是断开,将HWC中的Display同步断开。

    • 创建Display的Token
      createBuiltinDisplayLocked如果是连接状态,都将通过createBuiltinDisplayLocked创建Display的Token。添加到mBuiltinDisplays中,mBuiltinDisplays它只是Display的IBinder列表,我们也称之为Token列表。
    sp<IBinder> mBuiltinDisplays[DisplayDevice::NUM_BUILTIN_DISPLAY_TYPES];
    
    

    这个时候还没有创建DisplayDevice,只是创建了一个Token而已。Display的Token通知添加到mCurrentState的displays中。

    • 创建主屏幕对应的DisplayDevice

    如果是主屏,还会通过createDefaultDisplayDevice创建默认的DisplayDevice。

    void SurfaceFlinger::createDefaultDisplayDevice() {
        const DisplayDevice::DisplayType type = DisplayDevice::DISPLAY_PRIMARY;
        wp<IBinder> token = mBuiltinDisplays[type];
    
        // All non-virtual displays are currently considered secure.
        const bool isSecure = true;
    
        sp<IGraphicBufferProducer> producer;
        sp<IGraphicBufferConsumer> consumer;
        BufferQueue::createBufferQueue(&producer, &consumer);
    
        sp<FramebufferSurface> fbs = new FramebufferSurface(*getBE().mHwc, type, consumer);
    
        bool hasWideColorModes = false;
        std::vector<android_color_mode_t> modes = getHwComposer().getColorModes(type);
        for (android_color_mode_t colorMode : modes) {
            switch (colorMode) {
                case HAL_COLOR_MODE_DISPLAY_P3:
                case HAL_COLOR_MODE_ADOBE_RGB:
                case HAL_COLOR_MODE_DCI_P3:
                    hasWideColorModes = true;
                    break;
                default:
                    break;
            }
        }
        bool useWideColorMode = hasWideColorModes && hasWideColorDisplay && !mForceNativeColorMode;
        sp<DisplayDevice> hw = new DisplayDevice(this, DisplayDevice::DISPLAY_PRIMARY, type, isSecure,
                                                 token, fbs, producer, useWideColorMode);
        mDisplays.add(token, hw);
        android_color_mode defaultColorMode = HAL_COLOR_MODE_NATIVE;
        if (useWideColorMode) {
            defaultColorMode = HAL_COLOR_MODE_SRGB;
        }
        setActiveColorModeInternal(hw, defaultColorMode);
        hw->setCompositionDataSpace(HAL_DATASPACE_UNKNOWN);
    
        // Add the primary display token to mDrawingState so we don't try to
        // recreate the DisplayDevice for the primary display.
        mDrawingState.displays.add(token, DisplayDeviceState(type, true));
    
        // make the GLContext current so that we can create textures when creating
        // Layers (which may happens before we render something)
        hw->makeCurrent();
    }
    
    
    • 看到没有,主Display的BufferQueue是就是此时创建的。并且创建了前面所说的FramebufferSurface,这就是FBTarget。
    • DisplayDevice对应的类型为DISPLAY_PRIMARY,创建了DisplayDevice后,添加到mDisplays中。而Display的Token被添加到mDrawingState的displays中,注意,是mDrawingState。
    • 这里还有颜色模式Colormode和数据空间DataSpace的设置,这两个概念后续介绍。
    • 最后,调用makeCurrent,表示该DisplayDevice进入可用状态
    bool DisplayDevice::makeCurrent() const {
        bool success = mFlinger->getRenderEngine().setCurrentSurface(mSurface);
        setViewportAndProjection();
        return success;
    }
    
    

    makeCurrent主要做了两件事:其一,设置RenderEngine的Surface,这个Surface封装了前面BufferQueue中创建的Producer,以及对应的NativeWindow;其二,设置Display的Viewport和Projection,做过OpengGL开发的,对这个应该没有什么难题,视窗大小和投影矩阵。也是设置到RenderEngine中,GPU合成用。

    • 非主屏删除处理
      非主屏时删除时,将Display的Token从mBuiltinDisplays中删掉,且Token也从mCurrentState中删掉。注意,这里的是mCurrentState。主屏一般只会添加一次,没有断开操纵,断开时系统已经关了。

    • Transaction处理
      连接或断开显示屏,也算是一种Transaction。通过setTransactionFlags,设置处理的flag eDisplayTransactionNeeded。

    到此,连接时,主显的Token是添加到mDrawingState中的,已经创建对应的DisplayDevice,且没有断开处理。而非主显只创建了Display的Token,添加到这里的是mCurrentState中,还没有创建对应的DisplayDevice,断开的屏幕,Token从mCurrentState删除。简单点来说,mDrawingState中的Token都创建了DisplayDevice;在mCurrentState中的不再mDrawingState中的,都是添加的;在mDrawingState中的,不在mCurrentState中的都是断开的。

    3.4 创建DisplayDevice

    上面设置的setTransactionFlags什么时候处理?Vsync来的时候,Vsync来后,通过INVALIDATE消息,又回到SurfaceFlinger处理。中间的过程稍后介绍,我们直接看对这里设置的eDisplayTransactionNeeded的处理流程。

    处理eDisplayTransactionNeeded时,其实就是同步 mDrawingState 和 mCurrentState 中displays。

    • 屏幕断开时的处理
      先处理删除的Display,屏幕的Token在mDrawingState中的,不在mCurrentState中的都是断开的。逻辑如下:
    void SurfaceFlinger::handleTransactionLocked(uint32_t transactionFlags)
    {
                ``` ```
        if (transactionFlags & eDisplayTransactionNeeded) {
            const KeyedVector<  wp<IBinder>, DisplayDeviceState>& curr(mCurrentState.displays);
            const KeyedVector<  wp<IBinder>, DisplayDeviceState>& draw(mDrawingState.displays);
            if (!curr.isIdenticalTo(draw)) {
                mVisibleRegionsDirty = true;
                const size_t cc = curr.size();
                      size_t dc = draw.size();
    
                for (size_t i=0 ; i<dc ;) {
                    const ssize_t j = curr.indexOfKey(draw.keyAt(i));
                    if (j < 0) {
                        if (!draw[i].isMainDisplay()) {
                            const sp<const DisplayDevice> defaultDisplay(getDefaultDisplayDeviceLocked());
                            defaultDisplay->makeCurrent();
                            sp<DisplayDevice> hw(getDisplayDeviceLocked(draw.keyAt(i)));
                            if (hw != NULL)
                                hw->disconnect(getHwComposer());
                            if (draw[i].type < DisplayDevice::NUM_BUILTIN_DISPLAY_TYPES)
                                mEventThread->onHotplugReceived(draw[i].type, false);
                            mDisplays.removeItem(draw.keyAt(i));
                        } else {
                            ALOGW("trying to remove the main display");
                        }
                    } else {
                        // this display is in both lists. see if something changed.
                        const DisplayDeviceState& state(curr[j]);
                        const wp<IBinder>& display(curr.keyAt(j));
                        const sp<IBinder> state_binder = IInterface::asBinder(state.surface);
                        const sp<IBinder> draw_binder = IInterface::asBinder(draw[i].surface);
                        if (state_binder != draw_binder) {
                            sp<DisplayDevice> hw(getDisplayDeviceLocked(display));
                            if (hw != NULL)
                                hw->disconnect(getHwComposer());
                            mDisplays.removeItem(display);
                            mDrawingState.displays.removeItemsAt(i);
                            dc--;
                            // at this point we must loop to the next item
                            continue;
                        }
    
                        const sp<DisplayDevice> disp(getDisplayDeviceLocked(display));
                        if (disp != NULL) {
                            if (state.layerStack != draw[i].layerStack) {
                                disp->setLayerStack(state.layerStack);
                            }
                            if ((state.orientation != draw[i].orientation)
                                    || (state.viewport != draw[i].viewport)
                                    || (state.frame != draw[i].frame))
                            {
                                disp->setProjection(state.orientation,
                                        state.viewport, state.frame);
                            }
                            if (state.width != draw[i].width || state.height != draw[i].height) {
                                disp->setDisplaySize(state.width, state.height);
                            }
                        }
                    }
                    ++i;
                }
    
    

    如果Token在mDrawingState中,而没有在mCurrentState中,说明这个屏已经被断开了,需要删掉DisplayDevice。如果Token在两个状态中都存在,有修改,暂时将Token中mDrawingState删掉。注意两个状态啊,要不然理解不对。

    断开时,调用DisplayDevice的disconnect,这其中将调用HWC中,HWC中创建的对应的Device也将被删除。同时将DisplayDevice从mDisplays中删除。

    • 处理添加的Display
      屏幕的Token在mCurrentState中的不再mDrawingState中的,都是添加的;处理逻辑如下。
    void SurfaceFlinger::handleTransactionLocked(uint32_t transactionFlags)
    {
                ``` ```
                for (size_t i=0 ; i<cc ; i++) {
                    if (draw.indexOfKey(curr.keyAt(i)) < 0) {
                        const DisplayDeviceState& state(curr[i]);
    
                        sp<DisplaySurface> dispSurface;
                        sp<IGraphicBufferProducer> producer;
                        sp<IGraphicBufferProducer> bqProducer;
                        sp<IGraphicBufferConsumer> bqConsumer;
                        BufferQueue::createBufferQueue(&bqProducer, &bqConsumer);
    
                        int32_t hwcId = -1;
                        if (state.isVirtualDisplay()) {
                            // Virtual displays without a surface are dormant:
                            // they have external state (layer stack, projection,
                            // etc.) but no internal state (i.e. a DisplayDevice).
                            if (state.surface != NULL) {
    
                                // 虚拟显示用硬件
                                ... ...
    
                                sp<VirtualDisplaySurface> vds =
                                        new VirtualDisplaySurface(*getBE().mHwc,
                                                hwcId, state.surface, bqProducer,
                                                bqConsumer, state.displayName);
    
                                dispSurface = vds;
                                producer = vds;
                            }
                        } else {
                            ALOGE_IF(state.surface!=NULL,
                                    "adding a supported display, but rendering "
                                    "surface is provided (%p), ignoring it",
                                    state.surface.get());
    
                            hwcId = state.type;
                            dispSurface = new FramebufferSurface(*getBE().mHwc, hwcId, bqConsumer);
                            producer = bqProducer;
                        }
    
                        const wp<IBinder>& display(curr.keyAt(i));
                        if (dispSurface != NULL) {
                            sp<DisplayDevice> hw =
                                    new DisplayDevice(this, state.type, hwcId, state.isSecure, display,
                                                      dispSurface, producer, hasWideColorDisplay);
                            hw->setLayerStack(state.layerStack);
                            hw->setProjection(state.orientation,
                                    state.viewport, state.frame);
                            hw->setDisplayName(state.displayName);
                            mDisplays.add(display, hw);
                            if (!state.isVirtualDisplay()) {
                                mEventThread->onHotplugReceived(state.type, true);
                            }
                        }
                    }
                }
            }
        }
                ... ...
    
    

    添加屏幕时,根据前面已经创建的BufferQueue,创建对应的DisplaySurface,外显和虚显的不一样。创建DisplaySurface后,再创建DisplayDevice;设置DisplayDevice的stack,投影矩阵Projection;将创建的DisplayDevice添加到mDisplays中;最后,对外显,调用EventThread的onHotplugReceived。

    EventThread的onHotplugReceived函数中,将封装一个hotplug的Event事件DISPLAY_EVENT_HOTPLUG,EventThread再将事件分发出去。这里对hotplug感兴趣的主要就是框架层了,回调给DisplayManagerService。

    3.5 hotplug的流程

    这里主要将了Display相关的逻辑,主要是热插拔的处理。下面是整个Android系统添加屏幕时的处理流程。

    image

    Display相关的就介绍到这里,后续讲合成时,还会有很多相关的流程。

    作者:夕月风
    链接:https://www.jianshu.com/p/fa115146949f

    至此,本篇已结束。转载网络的文章,小编觉得很优秀,欢迎点击阅读原文,支持原创作者,如有侵权,恳请联系小编删除,欢迎您的建议与指正。同时期待您的关注,感谢您的阅读,谢谢!

    相关文章

      网友评论

          本文标题:Surfaceflinger 合成流程

          本文链接:https://www.haomeiwen.com/subject/ddqmvltx.html