车载技术——Window Display之surface的绘制过程与原理

一、Surface 概述

OpenGL ES/Skia定义了一组绘制接口的规范，为什么能够跨平台？ 本质上需要与对应平台上的本地窗口建立连接。也就是说OpenGL ES负责输入了绘制的命令，但是需要一个 “画布” 来承载输出结果，最终展示到屏幕。这个画布就是本地窗口。

因此，每个平台的有着不一样的本地窗口的实现。Android平台上是 ANativeWindow。

疑问：

• 那么如何将OpenGL本地化？ 通过 EGL来对OpenGL ES来进行配置。关键点就是提供本地化窗口。
• 本地化窗口的作用是什么？ 本地窗口是OpenGL ES和 物理屏幕之间的桥梁。

1.1 Android本地窗口简述

Android图形系统提供的本地窗口，可以分为两类：

• FrameBufferNativeWindow

面对SF(SurfaceFlinger)。它通过HAL层的Gralloc系统调用(alloc/free)来分配内核中的FrameBuffer帧缓冲区。 这个帧缓冲区就代表了物理屏幕(fb驱动节点，表示屏幕数。如fb0主屏幕、fb1等)。 FrameBuffer的数量一般情况下是2，也就是双缓冲。当然还有三倍缓冲。

• Surface

面向应用程序。对应的是内存中一块缓冲区，称为：GraphicBuffer。是由SF来进行分配。app从SF中获取一块GraphicBuffer， 通过OpenGL/Skia将图形数据绘制(软件/硬件)到GraphicBuffer上。最终SF会把各个应用的GraphicBuffer数据进行合成，最终 通过 FrameBufferNativeWindow 输出到屏幕上。

有了一个整体的概念，接下来就好理解很多。

二、引出SurfaceSession

2.1 从WindowManagerImpl的addView()说起

app：
WindowManagerImpl.addView()WindowManagerGlobal.addView()ViewRootImpl的setView()IWindowSession.addToDisplay()
​
WMS:new WindowStateWindowState.attach()session.windowAddedLocked()new SurfaceSession()

复制代码

view添加到window的过程中， 从WindowManagerImpl 的 addView()，到WindowManagerGlobal(构造方法中会在system server 进程中创建一个Session对象)的addView()。最后会调用 ViewRootImpl的setView()方法。 内部会调用 IWindowSession 的addToDisplay() 方法。IWindowSession是WMS提供的一个binder服务(实现类就是Session)。

2.2 IWindowSession.windowAddedLocked()

内部会创建一个WindowState 对象。 调用 WindowState的 attach()方法。最终调到Session中的windowAddedLocked()，会创建 一个SurfaceSession对象。这就是我们要找的的跟SurfaceFlinger建立联系的地方。

 SurfaceSession mSurfaceSession;
void windowAddedLocked(String packageName) {mPackageName = packageName;mRelayoutTag = "relayoutWindow: " + mPackageName;if (mSurfaceSession == null) {// 一个进程只有一个session，因此也只创建一次 SurfaceSession 对象
​// 创建 SurfaceSession 对象mSurfaceSession = new SurfaceSession();
​// 每个session 都存入WMS中的mService.mSessions.add(this);
​if (mLastReportedAnimatorScale != mService.getCurrentAnimatorScale()) {mService.dispatchNewAnimatorScaleLocked(this);}}mNumWindow++; // 进程中所有窗口的数量+1
}

复制代码

一个应用进程对应一个Session对象，一个Session对象对应一个SurfaceSession。 WMS会把 这个Session 存储起来。也就是说WMS 会把所有跟SurfaceFlinger保持连接状态的应用Session存储起来。

2.3 SurfaceSession 创建过程

这个类的实例代表了和SurfaceFlinger的一个连接。我们可以通过它 创建一个或多个 Surface 对象。

2.3.1 构造方法

> SurfaceSession.java
private long mNativeClient; // SurfaceComposerClient*
​
public SurfaceSession() {//native 方法mNativeClient = nativeCreate();
}
​
> frameworks/base/core/jni/android_view_SurfaceSession.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz) {// 新建一个 SurfaceComposerClient 对象SurfaceComposerClient* client = new SurfaceComposerClient();client->incStrong((void*)nativeCreate);//返回SurfaceComposerClient对象的引用到java层。return reinterpret_cast<jlong>(client);
}

复制代码

SurfaceComposerClient 是什么呢？

2.3.2 SurfaceComposerClient

在 SurfaceComposerClient第一次被引用的时候会走onFirstRef()方法。

> frameworks/native/libs/gui/SurfaceComposerClient.cpp
void SurfaceComposerClient::onFirstRef() {//创建sf代理binder对象sf，类型为 ISurfaceComposersp<ISurfaceComposer> sf(ComposerService::getComposerService());if (sf != nullptr && mStatus == NO_INIT) {sp<ISurfaceComposerClient> conn;//创建一个 ISurfaceComposerClient 对象，用来跨进程调用conn = sf->createConnection();if (conn != nullptr) {mClient = conn;mStatus = NO_ERROR;}}
}

复制代码

• ISurfaceComposer 实现类就是 SurfaceFlinger对象。在server进程的代理对象是 ComposerService。This class defines the Binder IPC interface for accessing various SurfaceFlinger features.
• 通过SF.createConnection(),创建一个 ISurfaceComposerClient 对象 mClient，用来跨进程调用。

那么 ISurfaceComposerClient的实现类是哪个呢？ 继续看看 SF.createConnection()。

2.3.3 SurfaceFlinger.createConnection()

注意，此时是在SF进程。
> frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {// new client对象。return initClient(new Client(this));
}
static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) {status_t err = client->initCheck();if (err == NO_ERROR) {// 返回该对象return client;}return nullptr;
}
> frameworks/native/services/surfaceflinger/Client.h
class Client : public BnSurfaceComposerClient{...class BnSurfaceComposerClient : public SafeBnInterface<ISurfaceComposerClient> {...

复制代码

原来，ISurfaceComposerClient的实现类就是 SF中定义的 Client。也是一个binder服务。 我们回到 SurfaceComposerClient 类，它持有 ISurfaceComposerClient的binder引用 mClient。通过 mClient实现与SF通信。

2.3 小结

• Session 类中，创建了一个 SurfaceSession 对象，内部引用c++层的 SurfaceComposerClient 对象。
• SurfaceComposerClient 对象是通过SF创建的另一个binder服务。减轻SF的工作量。
• SurfaceComposerClient 对象则通过 mClient成员(ISurfaceComposerClient)代理binder，后续用来创建 Surface。

Surface绘制原理

Surface的Buffer是从哪里来的？

源码：frameworks/base/core/java/android/view/ViewRootImpl.java View触发绘制是通过requestLayout()函数或者setLayoutParms()函数：

performTravsersals()函数实现：

private void performTraversals() {……performMeasure(childWidthMeasureSpec, childHeightMeasureSpec);performLayout(lp, mWidth, mHeight);performDraw();……
}

perfomrDraw()函数调用draw()函数开始绘制：

private void performDraw() {……boolean canUseAsync = draw(fullRedrawNeeded);……
}

ViewRootImpl.draw()函数实现：

private boolean draw(boolean fullRedrawNeeded) {Surface surface = mSurface;……if (!drawSoftware(surface, mAttachInfo, xOffset, yOffset, scalingRequired, dirty, surfaceInsets)) {return false;}……return useAsyncReport;
}

drawSoftware()软件绘制，默认是软件绘制。

drawSoftware()函数软件绘制流程：

private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff,boolean scalingRequired, Rect dirty, Rect surfaceInsets) {
​// Draw with software renderer.final Canvas canvas;……canvas = mSurface.lockCanvas(dirty);……mView.draw(canvas);……surface.unlockCanvasAndPost(canvas);
​
}

获取：通过lockCanvas函数获取Canvas对象，

绘制：再通过mView.draw(canvas)函数向在canvas上绘制，

提交：最后通过surface.unlockCanvasAndPost(canvas)函数提交Canvas。

通过lockCanvas()函数获取Canvas对象，lockCanvas()函数如何获取Canvas对象。

lockCanvas()函数实现：

/*** Gets a {@link Canvas} for drawing into this surface.** After drawing into the provided {@link Canvas}, the caller must* invoke {@link #unlockCanvasAndPost} to post the new contents to the surface.** @param inOutDirty A rectangle that represents the dirty region that the caller wants* to redraw.  This function may choose to expand the dirty rectangle if for example* the surface has been resized or if the previous contents of the surface were* not available.  The caller must redraw the entire dirty region as represented* by the contents of the inOutDirty rectangle upon return from this function.* The caller may also pass <code>null</code> instead, in the case where the* entire surface should be redrawn.* @return A canvas for drawing into the surface.** @throws IllegalArgumentException If the inOutDirty rectangle is not valid.* @throws OutOfResourcesException If the canvas cannot be locked.*/
public Canvas lockCanvas(Rect inOutDirty)throws Surface.OutOfResourcesException, IllegalArgumentException {synchronized (mLock) {checkNotReleasedLocked();if (mLockedObject != 0) {// Ideally, nativeLockCanvas() would throw in this situation and prevent the// double-lock, but that won't happen if mNativeObject was updated.  We can't// abandon the old mLockedObject because it might still be in use, so instead// we just refuse to re-lock the Surface.throw new IllegalArgumentException("Surface was already locked");}mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty);return mCanvas;}
}

通过Native层android_view_Surface.cpp的nativeLockCanvas(mNativeObject, mCanvas, inOutDirty)函数获取，mNativeOjbect参数是Java层的Surface在Native层对应的Surface对象的指针。mCanvas是Surface的变量，在lockCanvas()函数调用时mCanvas是空的。

在调用nativeLockCanvas()函数后mCanvas就有值了，最后返回mCanvas对象。

static jlong nativeLockCanvas(JNIEnv* env, jclass clazz,jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) {// (1)sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));……// (2)ANativeWindow_Buffer buffer;status_t err = surface->lock(&buffer, dirtyRectPtr);……// (3)graphics::Canvas canvas(env, canvasObj);canvas.setBuffer(&buffer, static_cast<int32_t>(surface->getBuffersDataSpace()));……// (4)// Create another reference to the surface and return it.  This reference// should be passed to nativeUnlockCanvasAndPost in place of mNativeObject,// because the latter could be replaced while the surface is locked.sp<Surface> lockedSurface(surface);lockedSurface->incStrong(&sRefBaseOwner);return (jlong) lockedSurface.get();
}

(1) 获取Native层的Surface对象。

(2) 获取Native层的Surface对象的Buffer。

(3) 将Buffer设置给Canvas，这里Canvas就有一个Buffer了。在每次都申请一个新的Buffer给Canvas对象。

(4) 向Java层返回Native的Surface对象，这里返回的是一个Long型数据，这个Long型数据是Surface指针。

获取Buffer实现，surface -> lock(&buffer, )，这里传入Buffer地址：

源码：frameworks/native/libs/gui/Surface.cpp

status_t Surface::lock(ANativeWindow_Buffer* outBuffer, ARect* inOutDirtyBounds)
{……// (1)ANativeWindowBuffer* out;status_t err = dequeueBuffer(&out, &fenceFd);……// (2)sp<GraphicBuffer> backBuffer(GraphicBuffer::getSelf(out));……// (3)void* vaddr;status_t res = backBuffer->lockAsync(GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN, newDirtyRegion.bounds(), &vaddr, fenceFd);……// (4)mLockedBuffer = backBuffer;// (5)outBuffer->bits   = vaddr;……
}

(1) 获取dequeueBuffer()函数在SurfaceFlinger的Buffer队列中获取Buffer。

(2) 创建GraphicBuffer对象backBuffer。在SharedBufferStack中有双缓冲机制，分别为FontBuffer和BackBuffer。

FontBuffer：代表当前将显示在屏幕的Buffer数据。属于前台Buffer。 BackBuffer：代表绘制的Buffer数据，是准备渲染的数据Buffer。属于后台Buffer。 　　(3) 锁定Buffer，并将Buffer地址返回，将返回的Buffer地址给Canvas的Buffer。

(4) 切换Buffer，将后台BackBuffer切换到前台，交给mLockedBuffer。FontBuffer的变量就是mLockedBuffer。

(5) 将vaddr赋值给outBuffer->bits，bits最后赋值给Canvas的Buffer，就是BkBitmap，作为Canvas的缓冲区。

dequeteBuffer()是如何获取Buffer的，dequeteBuffer()函数实现：

int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {……int buf = -1;// (1)status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight,reqFormat, reqUsage, &mBufferAge,enableFrameTimestamps ? &frameTimestamps: nullptr);……// (2)sp<GraphicBuffer>& gbuf(mSlots[buf].buffer);……// (3)if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) {if (mReportRemovedBuffers && (gbuf != nullptr)) {mRemovedBuffers.push_back(gbuf);}result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);if (result != NO_ERROR) {ALOGE("dequeueBuffer: IGraphicBufferProducer::requestBuffer failed: %d", result);mGraphicBufferProducer->cancelBuffer(buf, fence);return result;}}……// (4)*buffer = gbuf.get();
}

(1) 通过mGaphicBufferProducer->dequeteBuffer()函数在远端的Buffer slots中获得一个空闲的Buffer，返回远端Buffer地址指针。

(2) 通过gbp从本地Buffer Slots里获取Buffer，在(1)中从远端，在(2)中从本地，这里涉及远端Buffer queue与本地Buffer queue同步问题。

(3) 负责本地Buffer与远端Buffer同步，远端返回的Buffer的result是BUFFER_NEEDS_REALLOCATION或者本地的gbp是null，通过gbp的requestBuffer()获取新的远端Buffer指针地址。mGaphicBufferProducer->requestBuffer()函数。

(4) 获取Buffer。

Surface的Buffer是如何提交的？

通过surface.unlockCanvasAndPost(canvas)向远端提交更新的Buffer，unlockCanvasAndPost()函数实现：

/*** Posts the new contents of the {@link Canvas} to the surface and* releases the {@link Canvas}.** @param canvas The canvas previously obtained from {@link #lockCanvas}.*/
public void unlockCanvasAndPost(Canvas canvas) {synchronized (mLock) {unlockSwCanvasAndPost(canvas);}
}
​
private void unlockSwCanvasAndPost(Canvas canvas) {try {nativeUnlockCanvasAndPost(mLockedObject, canvas);} finally {nativeRelease(mLockedObject);mLockedObject = 0;}
}

最后调用到Native层的nativeUnlockCanvasAndPost(mLockedObject, canvas)。

Native层，nativeUnlockCanvasAndPost()函数实现：

static void nativeUnlockCanvasAndPost(JNIEnv* env, jclass clazz,jlong nativeObject, jobject canvasObj) {// (1)sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));
​// (2)// detach the canvas from the surfacegraphics::Canvas canvas(env, canvasObj);canvas.setBuffer(nullptr, ADATASPACE_UNKNOWN);// (3)// unlock surfacestatus_t err = surface->unlockAndPost();
}

(1) 获取对应Java层的Native层的Surface对象。

(2) 获取对应Java层的Native层的Canvas对象。

(3) 将本地Buffer更新到远端的Buffer queue中。

Native层更新远端Buffer queue，surface->unlockAndPost()函数实现：

源码：frameworks/native/libs/gui/Surface.cpp

status_t Surface::unlockAndPost()
{if (mLockedBuffer == nullptr) {ALOGE("Surface::unlockAndPost failed, no locked buffer");return INVALID_OPERATION;}
​int fd = -1;status_t err = mLockedBuffer->unlockAsync(&fd);ALOGE_IF(err, "failed unlocking buffer (%p)", mLockedBuffer->handle);err = queueBuffer(mLockedBuffer.get(), fd);ALOGE_IF(err, "queueBuffer (handle=%p) failed (%s)",mLockedBuffer->handle, strerror(-err));mPostedBuffer = mLockedBuffer;mLockedBuffer = nullptr;return err;
}

通过函数queueBuffer(mLockedBuffer.get(), )函数实现更新：

int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {……// (1)int i = getSlotFromBufferLocked(buffer);……// (2)status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);……return err;
}

(1) 获取Buffer的index。

(2) 通过mGraphicBufferProducer->queueBuffer(i, )函数，将本地的Buffer同步到远端Buffer queue中。

以上为车载技术中Window Display板块的surface的绘制过程与原理；更多车载技术参考，可点击《车载技术手册》查看类目学习。

Android车载学习手册​docs.qq.com/doc/DUldvclB5d0JZSVFn

三、总结