SoFunction
Updated on 2025-04-06

Creation process of rendering Surface based on Android 10

1. Surface Overview

OpenGL ES/SkiaDefining a set of specifications for drawing interfaces, why can it be cross-platform? In essence, it is necessary to establish a connection with the local window on the corresponding platform. That is to say, OpenGL ES is responsible for entering the drawing command, but it needs a "canvas" to carry the output results and finally display it to the screen. This canvas is the local window.

Therefore, each platform has different local window implementations. On the Android platform is ANativeWindow.

doubt:

  • So how to localize OpenGL? EGL to configure OpenGL ES. The key point is to provide a localization window.
  • What is the role of localization window? The local window is the bridge between OpenGL ES and the physical screen.

1.1 Brief description of Android local windows

The local windows provided by the Android graphics system can be divided into two categories:

  • FrameBufferNativeWindow

Face SF (SurfaceFlinger). It allocates the kernel through the Gralloc system call (alloc/free) of the HAL layerFrameBuffer framebuffer. This frame buffer represents the physical screen (fb* driver node, * indicates the number of screens. For example, fb0 main screen, fb1, etc.). The number of FrameBuffers is generally 2, that is, double buffering. Of course there is three times the buffer.

  • Surface

App-oriented. Corresponding toA buffer in memory, called: GraphicBuffer. It is allocated by SF. App gets a piece from SFGraphicBuffer, draw graph data (software/hardware) onto GraphicBuffer through OpenGL/Skia. In the end, SF will synthesize the GraphicBuffer data of each application, and finally passFrameBufferNativeWindowOutput to the screen.

With a holistic concept, it will be easier to understand next.

2. Introduce the SurfaceSession

2.1 Start with addView() of WindowManagerImpl

app:
()
  ()
    ViewRootImplofsetView()
      ()
      
WMS:
      new WindowState
        ()
          ()
            new SurfaceSession()

During the process of adding view to window,From WindowManagerImplofaddView(),arriveWindowManagerGlobal(The constructor will create a Session object in the system server process)addView(). Finally, it will be calledViewRootImplofsetView()method. It will be called internallyIWindowSessionofaddToDisplay()method. IWindowSession is a binder service provided by WMS (the implementation class is Session).

2.2 ()

An internal creation will be createdWindowStateObject. Calling WindowStateattach()method. Finally transferred to SessionwindowAddedLocked(), will create aSurfaceSessionObject. This is what we are looking forSurfaceFlingerA place to connect.

 SurfaceSession mSurfaceSession;
void windowAddedLocked(String packageName) {
  mPackageName = packageName;
  mRelayoutTag = "relayoutWindow: " + mPackageName;
  if (mSurfaceSession == null) {
  // A process has only one session, so the SurfaceSession object is created only once.      
      // Create SurfaceSession object      mSurfaceSession = new SurfaceSession();
      
      // Each session is stored in WMS      (this);
      
      if (mLastReportedAnimatorScale != ()) {
          (this);
      }
  }
  mNumWindow++; // Number of all windows in the process +1}

oneApplication processCorresponding to oneSessionObject, a Session object corresponds to oneSurfaceSession. WMS will store this session. In other words, WMS will keep all applications connected to SurfaceFlingerSessionStore it.

2.3 SurfaceSession creation process

An instance of this class represents a connection to SurfaceFlinger. We can create one or moreSurfaceObject.

2.3.1 Construction method

> 
private long mNativeClient; // SurfaceComposerClient*

public SurfaceSession() {
    //native method    mNativeClient = nativeCreate();
}

> frameworks/base/core/jni/android_view_SurfaceSession.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz) {
    // Create a new SurfaceComposerClient object    SurfaceComposerClient* client = new SurfaceComposerClient();
    client->incStrong((void*)nativeCreate);
    //Return the reference to the SurfaceComposerClient object to the java layer.    return reinterpret_cast<jlong>(client);
}

SurfaceComposerClientWhat is it?

2.3.2 SurfaceComposerClient

When SurfaceComposerClient is referenced for the first timeonFirstRef()method.

> frameworks/native/libs/gui/
void SurfaceComposerClient::onFirstRef() {
    //Create the sf proxy binder object sf, type ISurfaceComposer    sp<ISurfaceComposer> sf(ComposerService::getComposerService());
    if (sf != nullptr && mStatus == NO_INIT) {
        sp<ISurfaceComposerClient> conn;
        //Create an ISurfaceComposerClient object to call across processes        conn = sf->createConnection();
        if (conn != nullptr) {
            mClient = conn;
            mStatus = NO_ERROR;
        }
    }
}
  • ISurfaceComposerThe implementation class is the SurfaceFlinger object. The proxy object in the server process is ComposerService. This class defines the Binder IPC interface for accessing various SurfaceFlinger features.
  • Create aISurfaceComposerClientObject mClient, used to call across processes.

So which implementation class is the ISurfaceComposerClient?Continue to see ()

2.3.3 ()

Note that this is in the SF process.

> frameworks/native/services/surfaceflinger/
sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {
    // new client object.    return initClient(new Client(this));
}
static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) {
    status_t err = client->initCheck();
    if (err == NO_ERROR) {
        // Return this object        return client;
    }
    return nullptr;
}
> frameworks/native/services/surfaceflinger/
class Client : public BnSurfaceComposerClient{...
  class BnSurfaceComposerClient : public SafeBnInterface<ISurfaceComposerClient> {...

It turns out that the implementation class of ISurfaceComposerClient is defined in SFClient. It is also a binder service. Let's go back to the SurfaceComposerClient class, which holds the binder reference of the ISurfaceComposerClient mClient. passmClient implements communication with SF

2.3 Summary

  • In the Session class, aSurfaceSessionObject, internal reference to the c++ layerSurfaceComposerClientObject.
  • The SurfaceComposerClient object is another binder service created through SF. Reduce SF workload.
  • SurfaceComposerClientThe object is proxyed by the binder through the mClient member (ISurfaceComposerClient), and is subsequently used to create the Surface.

So far, the Session object corresponding to the application process in WMS has established a connection with SF.

3. The creation of Surface

3.1 Surface of Java layer

ViewRootImpl class member variable:

// Any thread can access this object, just lock it.public final Surface mSurface = new Surface();

The empty parameter structure is called, so the Surface at this time is just an empty shell of the Java layer.

3.2 RelayoutWindow() on the App side

When SF receives the vsync signal, it is monitored through Choreographer and distributed to the app process. After the app receives the vsync signal, doFrame(), and then calls doTraversal() to the main thread ->performTraversals():

> 
private void performTraversals() {
    // Call relayoutWindow    relayoutResult = relayoutWindow(params, viewVisibility, insetsPending);
    //...
    
    // Ask host how big it wants to be
    // Three major processes    performMeasure(childWidthMeasureSpec, childHeightMeasureSpec);
    //...
    performLayout(lp, mWidth, mHeight);
    //```
    performDraw();
    //...
}
    > 
private int relayoutWindow( params, int viewVisibility,
    boolean insetsPending) throws RemoteException {
    //...
    // 1 mWindowSession is an IWindowSession object, call relayout()     int relayoutResult = (mWindow, mSeq, params,
                (int) (() * appScale + 0.5f),
                (int) (() * appScale + 0.5f), viewVisibility,
                insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0, frameNumber,
                mTmpFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets,
                mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingDisplayCutout,
                 // The mSurfaceControl object passed here                mPendingMergedConfiguration, mSurfaceControl, mTempInsets);
                
        if (()) {
            // 2 Copy surface from mSurfaceControl            (mSurfaceControl);
        } else {
            // Invalid, it will be destroyed            destroySurface();
        }
        
     //... 
    return relayoutResult;
}
  • relayout() Pass in mSurfaceControl and call it across the process to the WMS side.
  • copyFrom() , get the Surface from mSurfaceControl.

What is SurfaceControl?

SurfaceControl can control a surface that is being displayed on the screen managed by a synthetic system. It is the linker for metadata of Buffer data and windows. Through this SurfaceControl constructed surface, the app can directly submit buffer data to the synthesis system for synthesis. In addition, the displayed properties of the buffer can also be modified through . Such as rotation, translation, and cropping.

3.3 SurfaceControl empty parametric structure

yesViewRootImplClass member variables:

private final SurfaceControl mSurfaceControl = new SurfaceControl();

SurfaceControl's empty parametric construction method does nothing!It's also an empty shell at present.

3.4 Summary

It can be seen that when the drawing starts, the WMS is called(SurfaceControl mSurfaceControl)After the method. You canSurfaceControlGet the Surface object in it.

So how does () do it internally?Keep watching ()

3.5 WMS side ()

>
@Override
public int relayout(IWindow window, int seq,  attrs,
        int requestedWidth, int requestedHeight, int viewFlags, int flags, long frameNumber,
        Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets,
        Rect outStableInsets, Rect outsets, Rect outBackdropFrame,
         cutout, MergedConfiguration mergedConfiguration,
        SurfaceControl outSurfaceControl, InsetsState outInsetsState) {
        
    // Continue to call WMS's relayoutWindow()    int res = (this, window, seq, attrs,
            requestedWidth, requestedHeight, viewFlags, flags, frameNumber,
            outFrame, outOverscanInsets, outContentInsets, outVisibleInsets,
            outStableInsets, outsets, outBackdropFrame, cutout,
            mergedConfiguration, outSurfaceControl, outInsetsState);
    return res;
}
 > 
outSurfaceControl fromappThe process is passed toWMSprocess。 Indicates the result received。It implementsparcelableinterface,跨The process is passed toWMSmiddle。
public int relayoutWindow(Session session, IWindow client, int seq, LayoutParams attrs,
        int requestedWidth, int requestedHeight, int viewVisibility, int flags,
        long frameNumber, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
        Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame,
         outCutout, MergedConfiguration mergedConfiguration,
        SurfaceControl outSurfaceControl, InsetsState outInsetsState) {
        //...
        try {
            //Call createSurfaceControl            result = createSurfaceControl(outSurfaceControl, result, win, winAnimator);
        } catch (Exception e) {
            ...
        }
        //...
        
}           
> 
private int createSurfaceControl(SurfaceControl outSurfaceControl, int result, WindowState win,
        WindowStateAnimator winAnimator) {
    if (!) {
        result |= RELAYOUT_RES_SURFACE_CHANGED;
    }
    // Declare a WindowSurfaceController    WindowSurfaceController surfaceController;
    
    try {
        //1 Create WindowSurfaceController object        surfaceController = (, );
    } finally {
    }
    if (surfaceController != null) {
        // 2 From the object to get the WindowSurfaceController, get the SurfaceController        (outSurfaceControl);
    } else {
        // For some reason there isn't a surface.  Clear the
        // caller's object so they see the same state.
        (TAG_WM, "Failed to create surface control for " + win);
        ();
    }

    return result;
}

final:

  • ()getWindowSurfaceControllerObject
  • From the WindowSurfaceController object,Get SurfaceControllerReturn to outSurfaceControl

winAnimator is the WindowStateAnimator type.

3.5.1 ()

> 
WindowSurfaceController createSurfaceLocked(int windowType, int ownerUid) {
    final WindowState w = mWin;

    if (mSurfaceController != null) {
        // If so, return        return mSurfaceController;
    }
    //...
    // Set up surface control with initial size.
    try {
        // Otherwise create a WindowSurfaceController        mSurfaceController = new WindowSurfaceController(,
                ().toString(), width, height, format, flags, this,
                windowType, ownerUid);
        // ...
    } catch (OutOfResourcesException e) {
     //...
     }
    return mSurfaceController;
}

createSurfaceLocked()It is to create a WindowSurfaceController object. WindowSurfaceControllerConstruction methodASurfaceControlObject.

3.5.2 WindowSurfaceController construction method

> 
public WindowSurfaceController(SurfaceSession s, String name, int w, int h, int format,
        int flags, WindowStateAnimator animator, int windowType, int ownerUid) {
    mAnimator = animator;
    mSurfaceW = w;
    mSurfaceH = h;

    title = name;
    mService = ;
    final WindowState win = ;
    mWindowType = windowType;
    mWindowSession = ;

    (TRACE_TAG_WINDOW_MANAGER, "new SurfaceControl");
    // 1 The makeSurface() method of WindowState is called    final  b = ()
            .setParent(())
            .setName(name)
            .setBufferSize(w, h)
            .setFormat(format)
            .setFlags(flags)
            .setMetadata(METADATA_WINDOW_TYPE, windowType)
            .setMetadata(METADATA_OWNER_UID, ownerUid);
            // 2 Create a SurfaceControl object    mSurfaceControl = ();
    (TRACE_TAG_WINDOW_MANAGER);
}
  > 
  public SurfaceControl build() {
            if (mWidth < 0 || mHeight < 0) {
                throw new IllegalStateException(
                        "width and height must be positive or unset");
            }
            if ((mWidth > 0 || mHeight > 0) && (isColorLayerSet() || isContainerLayerSet())) {
                throw new IllegalStateException(
                        "Only buffer layers can set a valid buffer size.");
            }
            // Create an object and call the parameter constructor method of SurfaceControl.            return new SurfaceControl(
                    mSession, mName, mWidth, mHeight, mFormat, mFlags, mParent, mMetadata);
        }

Calling the parameter constructor method of SurfaceControl to construct a SurfaceControl object. Remember3. 3, the member variable mSurfaceControl of ViewRootImpl, the call isSpace ginseng structure, did nothing.

However, now SurfaceControlGlycologic structureWhat did you do? Let's take a look in depth.

Parameter-like structure of SurfaceControl:

long mNativeObject; // package visibility only for  access
private SurfaceControl(SurfaceSession session, String name, int w, int h, int format, int flags,
        SurfaceControl parent, SparseIntArray metadata)
                throws OutOfResourcesException, IllegalArgumentException {
                //... 
                
                //Note that the first parameter here is SurfaceSession        mNativeObject = nativeCreate(session, name, w, h, format, flags,
                            parent != null ?  : 0, metaParcel);
}

>frameworks/base/core/jni/android_view_SurfaceControl.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj,
        jstring nameStr, jint w, jint h, jint format, jint flags, jlong parentObject,
        jobject metadataParcel) {
    ScopedUtfChars name(env, nameStr);
    //This is the SurfaceComposerClient object when the SurfaceSession was created before.  SF's proxy service.    sp<SurfaceComposerClient> client;
    if (sessionObj != NULL) {
        // mClient created earlier        client = android_view_SurfaceSession_getClient(env, sessionObj);
    } else {
        
        client = SurfaceComposerClient::getDefault();
    }
    // Parent SurfaceControl    SurfaceControl *parent = reinterpret_cast<SurfaceControl*>(parentObject);
    // Declare SurfaceControl    sp<SurfaceControl> surface;
    // metadata of the window.  Because SurfaceControl itself is the connector for surface and metadata.    LayerMetadata metadata;
    
    Parcel* parcel = parcelForJavaObject(env, metadataParcel);
    if (parcel && !parcel->objectsCount()) {
        status_t err = (parcel);
        if (err != NO_ERROR) {
          jniThrowException(env, "java/lang/IllegalArgumentException",
                            "Metadata parcel has wrong format");
        }
    }
    // Call createSurfaceChecked() of the SF process to create the surface object    status_t err = client->createSurfaceChecked(
            String8(name.c_str()), w, h, format, &surface, flags, parent, std::move(metadata));
    if (err == NAME_NOT_FOUND) {
        jniThrowException(env, "java/lang/IllegalArgumentException", NULL);
        return 0;
    } else if (err != NO_ERROR) {
        jniThrowException(env, OutOfResourcesException, NULL);
        return 0;
    }

    surface->incStrong((void *)nativeCreate);
    // Returns the jlong ​​reference of SurfaceControl    return reinterpret_cast<jlong>(());
}

>frameworks/native/libs/gui/
status_t SurfaceComposerClient::createSurfaceChecked(const String8& name, uint32_t w, uint32_t h,
                                                     PixelFormat format,
                                                     sp<SurfaceControl>* outSurface, uint32_t flags,
                                                     const sp<IBinder>& parentHandle,
                                                     LayerMetadata metadata,
                                                     uint32_t* outTransformHint) {
    sp<SurfaceControl> sur;
    status_t err = mStatus;

    if (mStatus == NO_ERROR) {
        // 
        sp<IBinder> handle;
        // gpb,buffer data producer        sp<IGraphicBufferProducer> gbp;

        uint32_t transformHint = 0;
        int32_t id = -1;
        // The specific implementation class of mClient is SF's Client service.  To create a surface        err = mClient->createSurface(name, w, h, format, flags, parentHandle, std::move(metadata),
                                     &handle, &gbp, &id, &transformHint);

        if (outTransformHint) {
            *outTransformHint = transformHint;
        }
        ALOGE_IF(err, "SurfaceComposerClient::createSurface error %s", strerror(-err));
        if (err == NO_ERROR) {
            // Finally, a native layer SurfaceControl object was created.  Note that this is the parametric structure of the native layer!  !            *outSurface =
                    new SurfaceControl(this, handle, gbp, id, w, h, format, transformHint, flags);
        }
    }
    return err;
}
Keep watchingnativeThe structure of the layer: 
>frameworks/native/libs/gui/
SurfaceControl::SurfaceControl(
        const sp<SurfaceComposerClient>& client,
        const sp<IBinder>& handle,
        const sp<IGraphicBufferProducer>& gbp,
        bool owned)
    : mClient(client), mHandle(handle), mGraphicBufferProducer(gbp), mOwned(owned)
    //Assign some member variables{
}

Summarize:

  • WMS's relayout() creates a SurfaceControl object through the WindowSurfaceControl object. At this time, the SurfaceControl object not only exists in the Java layer, but also creates a SurfaceControl object in the native layer.
  • In the native layer, the internal requests buffer, metadata and other information to SF through the previous connection surfaceSession to assign values ​​to the SurfaceControl object.

Note that the above logic is still in the native layer of WMS. andcreateSurface() across processesWhat exactly did the call do?

3.6 SF end ()

Now enter the SF process。 Clientthat isISurfaceComposerClientImplementation class.

frameworks/native/services/surfaceflinger/
status_t Client::createSurface(const String8& name, uint32_t w, uint32_t h, PixelFormat format,
                               uint32_t flags, const sp<IBinder>& parentHandle,
                               LayerMetadata metadata, sp<IBinder>* handle,
                               sp<IGraphicBufferProducer>* gbp) {
    // We rely on createLayer to check permissions.
    // Finally SF is called.  Create a layer    return mFlinger->createLayer(name, this, w, h, format, flags, std::move(metadata), handle, gbp,
                                 parentHandle);
}
> frameworks/native/services/surfaceflinger/ 
status_t SurfaceFlinger::createLayer(const String8& name, const sp<Client>& client, uint32_t w,
                                     uint32_t h, PixelFormat format, uint32_t flags,
                                     LayerMetadata metadata, sp<IBinder>* handle,
                                     sp<IGraphicBufferProducer>* gbp,
                                     const sp<IBinder>& parentHandle,
                                     const sp<Layer>& parentLayer) {
    if (int32_t(w|h) < 0) {
       // width and height cannot be negative        return BAD_VALUE;
    }
    status_t result = NO_ERROR;
    // Declare a layer object    sp<Layer> layer; 

    String8 uniqueName = getUniqueLayerName(name);
    // metadata
    if ((METADATA_WINDOW_TYPE)) {
        int32_t windowType = metadata.getInt32(METADATA_WINDOW_TYPE, 0);
        if (windowType == 441731) {
            metadata.setInt32(METADATA_WINDOW_TYPE, InputWindowInfo::TYPE_NAVIGATION_BAR_PANEL);
            primaryDisplayOnly = true;
        }
    }
    // Create different layer objects according to flag.    switch (flags & ISurfaceComposerClient::eFXSurfaceMask) {
        case ISurfaceComposerClient::eFXSurfaceBufferQueue:
            // 1 Create bufferQueue, only look at this one            result = createBufferQueueLayer(client, uniqueName, w, h, flags, std::move(metadata),
                                            format, handle, gbp, &layer);
            break;
        case ISurfaceComposerClient::eFXSurfaceBufferState:
            result = createBufferStateLayer(client, uniqueName, w, h, flags, std::move(metadata),
                                            handle, &layer);
            break;
        case ISurfaceComposerClient::eFXSurfaceColor:
        case ISurfaceComposerClient::eFXSurfaceContainer:
           ...
        default:
            result = BAD_VALUE;
            break;
    }
   ...
    // 2 Continue to call addClientLayer    result = addClientLayer(client, *handle, *gbp, layer, parentHandle, parentLayer,
                            addToCurrentState);
    ...
    return result;
}
  • Created a BufferQueueLayer object, which holds itsp<BufferLayerConsumer> mConsumerandsp<IGraphicBufferProducer> mProducer, that isGraphicBufferproducers and consumer objects.
  • JoinlayersIn the global collection.

3.6.1 ()

frameworks/native/services/surfaceflinger/

status_t SurfaceFlinger::createBufferQueueLayer(const sp&lt;Client&gt;&amp; client, const String8&amp; name,
                                                uint32_t w, uint32_t h, uint32_t flags,
                                                LayerMetadata metadata, PixelFormat&amp; format,
                                                sp&lt;IBinder&gt;* handle,
                                                sp&lt;IGraphicBufferProducer&gt;* gbp,
                                                sp&lt;Layer&gt;* outLayer) {
    // initialize the surfaces
    // Format    switch (format) {
    case PIXEL_FORMAT_TRANSPARENT:
    case PIXEL_FORMAT_TRANSLUCENT:
        format = PIXEL_FORMAT_RGBA_8888;
        break;
    case PIXEL_FORMAT_OPAQUE:
        format = PIXEL_FORMAT_RGBX_8888;
        break;
    }
    // Create layer    sp&lt;BufferQueueLayer&gt; layer = getFactory().createBufferQueueLayer(
            LayerCreationArgs(this, client, name, w, h, flags, std::move(metadata)));
    status_t err = layer-&gt;setDefaultBufferProperties(w, h, format);
    if (err == NO_ERROR) {
        // Assign gbp producer        *handle = layer-&gt;getHandle();
        *gbp = layer-&gt;getProducer();
        *outLayer = layer;
    }
    ALOGE_IF(err, "createBufferQueueLayer() failed (%s)", strerror(-err));
    return err;
}

3.6.2

frameworks/native/services/surfaceflinger/

sp<BufferQueueLayer> createBufferQueueLayer(const LayerCreationArgs& args) override {
            return new BufferQueueLayer(args);
        }

3.6.3 BufferQueueLayer construction method

Inheritance relationship:

class BufferQueueLayer : public BufferLayer, public BufferLayerConsumer::ContentsChangedListener 
{...
class BufferLayer : public Layer {...

3.6.4 ()

void BufferQueueLayer::onFirstRef() {
    BufferLayer::onFirstRef();

    // Creates a custom BufferQueue for SurfaceFlingerConsumer to use
    // Declare producers and consumers    sp&lt;IGraphicBufferProducer&gt; producer;
    sp&lt;IGraphicBufferConsumer&gt; consumer;
    // BufferQueue Initialization    BufferQueue::createBufferQueue(&amp;producer, &amp;consumer, true);
    // MonitoredProducer is packaged with producer, which is convenient for SF monitoring    mProducer = new MonitoredProducer(producer, mFlinger, this);
    {
        // Grab the SF state lock during this since it's the only safe way to access RenderEngine
        Mutex::Autolock lock(mFlinger-&gt;mStateLock);
        // Encapsulates BufferLayerConsumer encapsulates consumer        mConsumer =
           new BufferLayerConsumer(consumer, mFlinger-&gt;getRenderEngine(), mTextureName, this);
    }
    mConsumer-&gt;setConsumerUsageBits(getEffectiveUsage(0));
    mConsumer-&gt;setContentsChangedListener(this);
    mConsumer-&gt;setName(mName);

    // BufferQueueCore::mMaxDequeuedBufferCount is default to 1
    if (!mFlinger-&gt;isLayerTripleBufferingDisabled()) {
      //Set to 2        mProducer-&gt;setMaxDequeuedBufferCount(2);
    }

    if (const auto display = mFlinger-&gt;getDefaultDisplayDevice()) {
        updateTransformHint(display);
    }
}

Key points:

  • Declaredsp<IGraphicBufferProducer> producersp<IGraphicBufferConsumer> consumer;
  • Call BufferQueue::createBufferQueue(&producer, &consumer, true); to create these two objects.

3.6.5 BufferQueue::createBufferQueue()

frameworks/native/libs/gui/

void BufferQueue::createBufferQueue(sp&lt;IGraphicBufferProducer&gt;* outProducer,
        sp&lt;IGraphicBufferConsumer&gt;* outConsumer,
        bool consumerIsSurfaceFlinger) {
    // BufferQueueCore object    sp&lt;BufferQueueCore&gt; core(new BufferQueueCore());
    //BufferQueueProducer Production Object    sp&lt;IGraphicBufferProducer&gt; producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger));
    // BufferQueueConsumer consumption object    sp&lt;IGraphicBufferConsumer&gt; consumer(new BufferQueueConsumer(core));

    *outProducer = producer;
    *outConsumer = consumer;
}
 static constexpr int NUM_BUFFER_SLOTS = 64;

Therefore, the real implementer of producer and consumer isBufferQueueProducerandBufferQueueConsumer

3.7 ()

Finally, add to the global mLayers.

>frameworks/native/services/surfaceflinger/
void Client::attachLayer(const sp<IBinder>& handle, const sp<Layer>& layer)
{
Mutex::Autolock _l(mLock);
(handle, layer);
}

At this point, SF creates a layer object and adds it to the mLayers list of Client service. Hold in layersp<IGraphicBufferProducer> producerandsp<IGraphicBufferConsumer> consumer;Two objects. The specific implementer isBufferQueueProducerandBufferQueueConsumer

3.6 Summary

To review the overall process:

  • AppCall WMS's relayoutWindow(MSurfaceControl) and pass in an empty SurfaceControl object-A (empty parameter construction).
  • WMS sideAnother SurfaceControl object -B (argument construct) of the java layer is generated. At the same time, a SurfaceControl object is also generated in the native layer. The createSurface() method of SF is called internally through the IComposerClient object. Also holds a reference to IGraphicBufferProducer gbp.
  • SF endGenerates the Layer object and generates the buffer producersp<IGraphicBufferProducer>and buffer consumerssp<IGraphicBufferConsumer>

Next, it was sent from the appHow to follow SurfaceControl-A with SurfaceControl-BWhat about connecting it? Of course it is throughgetSurfaceControl()The method is here.

3.7 ()

void getSurfaceControl(SurfaceControl outSurfaceControl) {
    (mSurfaceControl);
}

mSurfaceControlIt is the one created previously in the WindowSurfaceControllerSurfaceControl-BObject. CurrentoutSurfaceControlyesApp deliveryCome here. Continue to read copyFrom():

3.8 ()

>
public void copyFrom(SurfaceControl other) {
    mName = ;
    mWidth = ;
    mHeight = ;
    assignNativeObject(nativeCopyFromSurfaceControl());
}

3.8.1 nativeCopyFromSurfaceControl()

&gt;frameworks/base/core/jni/android_view_SurfaceControl.cpp
static jlong nativeCopyFromSurfaceControl(JNIEnv* env, jclass clazz, jlong surfaceControlNativeObj) {
    // According to the reference value, get the SurfaceControl of the native layer.  This is the SurfaceControl object of WMS-B    sp&lt;SurfaceControl&gt; surface(reinterpret_cast&lt;SurfaceControl *&gt;(surfaceControlNativeObj));
    if (surface == nullptr) {
        return 0;
    }
    // Create a new native SurfaceControl object C, calling the parameter constructor method    sp&lt;SurfaceControl&gt; newSurface = new SurfaceControl(surface);
    newSurface-&gt;incStrong((void *)nativeCreate);
    return reinterpret_cast&lt;jlong&gt;(());
}

//Glycosmic structureSurfaceControl::SurfaceControl(const sp&lt;SurfaceControl&gt;&amp; other) {
    mClient = other-&gt;mClient;
    mHandle = other-&gt;mHandle;
    mGraphicBufferProducer = other-&gt;mGraphicBufferProducer;
    mTransformHint = other-&gt;mTransformHint;
    mLayerId = other-&gt;mLayerId;
    mWidth = other-&gt;mWidth;
    mHeight = other-&gt;mHeight;
    mCreateFlags = other-&gt;mCreateFlags;
}

existnativeThe layer creates a newSurfaceControlObject.

3.8.2 assignNativeObject()

private void assignNativeObject(long nativeObject) {
// What is passed is the SurfaceControl object C of the newly created native layer in the previous step.    if (mNativeObject != 0) {
        // Before release        release();
    }
    // Hold a reference.    mNativeObject = nativeObject;
}

At this point, the app endSurfaceControl of Java layerThe object is withSurfaceControl of native layerThe objects were connected. After creation is completed,Return to the app side ViewRootImpl's relayoutWindow()in the method.

3.9 ()

public void copyFrom(SurfaceControl other) {
    // Copy from surfaceControl    if (other == null) {
        throw new IllegalArgumentException("other must not be null");
    }
    // Get the native object reference of SurfaceControl    long surfaceControlPtr = ;
    if (surfaceControlPtr == 0) {
        throw new NullPointerException(
                "null SurfaceControl native object. Are you using a released SurfaceControl?");
    }
    // Get the reference of the surface object from the native layer    long newNativeObject = nativeGetFromSurfaceControl(mNativeObject, surfaceControlPtr);
    
    //Lock    synchronized (mLock) {
        // The same object will not be performed        if (newNativeObject == mNativeObject) {
            return;
        }
        if (mNativeObject != 0) {
            // Before release            nativeRelease(mNativeObject);
        }
        // Set new reference        setNativeObjectLocked(newNativeObject);
    }
}

3.9.1 nativeGetFromSurfaceControl()

frameworks/base/core/jni/android_view_Surface.cpp

static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz,
        jlong nativeObject,
        jlong surfaceControlNativeObj) {
    //Get the c++ surface object referenced by the java layer    Surface* self(reinterpret_cast&lt;Surface *&gt;(nativeObject));
    // Get the SurfaceControl object of the c++ layer    sp&lt;SurfaceControl&gt; ctrl(reinterpret_cast&lt;SurfaceControl *&gt;(surfaceControlNativeObj));

    // If the underlying IGBP's are the same, we don't need to do anything.
    // If the surface object is the same as the SurfaceControl object's gbp, there is no need to reset the value.    if (self != nullptr &amp;&amp;
            IInterface::asBinder(self-&gt;getIGraphicBufferProducer()) ==
            IInterface::asBinder(ctrl-&gt;getIGraphicBufferProducer())) {
            // Just go back        return nativeObject;
    }
    // If gbp is different, reconstruct one    sp&lt;Surface&gt; surface(ctrl-&gt;getSurface());
    if (surface != NULL) {
        surface-&gt;incStrong(&amp;sRefBaseOwner);
    }
    // Return to the reference    return reinterpret_cast&lt;jlong&gt;(());
}

The logic has been clearly commented. The question is how does getSurface() of SurfaceControl work?

3.9.2 native layer SurfaceControl::getSurface()

frameworks/native/libs/gui/

sp&lt;Surface&gt; SurfaceControl::getSurface()
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData == nullptr) {
        // Create a Surface        return generateSurfaceLocked();
    }
    return mSurfaceData;
}
mutable sp&lt;SurfaceControl&gt; mBbqChild;
sp&lt;Surface&gt; SurfaceControl::generateSurfaceLocked()
{
    uint32_t ignore;
    auto flags = mCreateFlags &amp; (ISurfaceComposerClient::eCursorWindow |
                                 ISurfaceComposerClient::eOpaque);
     // Call it into the SF process through ISurfaceComposerClient    mBbqChild = mClient-&gt;createSurface(String8("bbq-wrapper"), 0, 0, mFormat,
                                       flags, mHandle, {}, &amp;ignore);
     // mbbq is SurfaceControl    mBbq = sp&lt;BLASTBufferQueue&gt;::make("bbq-adapter", mBbqChild, mWidth, mHeight, mFormat);

    // This surface is always consumed by SurfaceFlinger, so the
    // producerControlledByApp value doesn't matter; using false.
    // Create surface    mSurfaceData = mBbq-&gt;getSurface(true);
    return mSurfaceData;
}

3.9.3 ()

frameworks/native/libs/gui/

sp<Surface> BLASTBufferQueue::getSurface(bool includeSurfaceControlHandle) {
    std::unique_lock _lock{mMutex};
    sp<IBinder> scHandle = nullptr;
    if (includeSurfaceControlHandle && mSurfaceControl) {
        scHandle = mSurfaceControl->getHandle();
    }
    return new BBQSurface(mProducer, true, scHandle, this);
}

3.9.4 BBQSurface

// Inherited surfaceclass BBQSurface : public Surface {
private:
    std::mutex mMutex;
    sp&lt;BLASTBufferQueue&gt; mBbq;
    bool mDestroyed = false;
public:
    BBQSurface(const sp&lt;IGraphicBufferProducer&gt;&amp; igbp, bool controlledByApp,
               const sp&lt;IBinder&gt;&amp; scHandle, const sp&lt;BLASTBufferQueue&gt;&amp; bbq)
          : Surface(igbp, controlledByApp, scHandle), mBbq(bbq) {}

    void allocateBuffers() override {
        uint32_t reqWidth = mReqWidth ? mReqWidth : mUserWidth;
        uint32_t reqHeight = mReqHeight ? mReqHeight : mUserHeight;
        auto gbp = getIGraphicBufferProducer();
        std::thread ([reqWidth, reqHeight, gbp=getIGraphicBufferProducer(),
                      reqFormat=mReqFormat, reqUsage=mReqUsage] () {
            // Request allocateBuffers            gbp-&gt;allocateBuffers(reqWidth, reqHeight,
                                 reqFormat, reqUsage);

        }).detach();
    }

3.10 Summary

After the SurfaceControl on the app end is connected to the SurfaceControl object in the native layer. Surface of the Java layer on the app end passescopyFrom()Method, fromnative layerThe SurfaceControl has obtained the native layerSurface Objects, internal holdingIGraphicProducer gbpObject. Can communicate with SF.

4. Summary

  • AppCall WMS' relayoutWindow(MSurfaceControl) and the one passed in isEmpty SurfaceControl object-A (empty parameter construction)
  • WMS sideAnother java layer generatedSurfaceControl object-B (parameter construct). At the same timenative layerA SurfaceControl object is also generated. The createSurface() method of SF is called internally through the IComposerClient object. at the same timeReferences that hold IGraphicBufferProducer gbp
  • SF endGeneratedLayerObject, and buffer producer is generatedsp<IGraphicBufferProducer>and buffer consumerssp<IGraphicBufferConsumer>
  • WMS sidepasscopy()Method to pass it from the App sideSurfaceControl-A objectAlso quoted from SFIGraphicBufferProducer
  • After the value of the SurfaceControl-A object on the App side is assigned, then the Surface object is passedcopyFrom()Method, letJavalayerSurfaceandNativeLayerSurfaceBuild connections. In the end, that is,SFofIGraphicBufferProducerContact was established.

This is the end of this article about the creation process of rendering Surface based on Android 10. For more related Android Surface content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!