1. Surface Overview
OpenGL ES/Skia
Defining a set of specifications for drawing interfaces, why can it be cross-platform? In essence, it is necessary to establish a connection with the local window on the corresponding platform. That is to say, OpenGL ES is responsible for entering the drawing command, but it needs a "canvas" to carry the output results and finally display it to the screen. This canvas is the local window.
Therefore, each platform has different local window implementations. On the Android platform is ANativeWindow.
doubt:
- So how to localize OpenGL? EGL to configure OpenGL ES. The key point is to provide a localization window.
- What is the role of localization window? The local window is the bridge between OpenGL ES and the physical screen.
1.1 Brief description of Android local windows
The local windows provided by the Android graphics system can be divided into two categories:
FrameBufferNativeWindow
Face SF (SurfaceFlinger). It allocates the kernel through the Gralloc system call (alloc/free) of the HAL layerFrameBuffer framebuffer
. This frame buffer represents the physical screen (fb* driver node, * indicates the number of screens. For example, fb0 main screen, fb1, etc.). The number of FrameBuffers is generally 2, that is, double buffering. Of course there is three times the buffer.
Surface
App-oriented. Corresponding toA buffer in memory
, called: GraphicBuffer. It is allocated by SF. App gets a piece from SFGraphicBuffer
, draw graph data (software/hardware) onto GraphicBuffer through OpenGL/Skia. In the end, SF will synthesize the GraphicBuffer data of each application, and finally passFrameBufferNativeWindow
Output to the screen.
With a holistic concept, it will be easier to understand next.
2. Introduce the SurfaceSession
2.1 Start with addView() of WindowManagerImpl
app: () () ViewRootImplofsetView() () WMS: new WindowState () () new SurfaceSession()
During the process of adding view to window,From WindowManagerImpl
ofaddView()
,arriveWindowManagerGlobal
(The constructor will create a Session object in the system server process)addView()
. Finally, it will be calledViewRootImpl
ofsetView()
method. It will be called internallyIWindowSession
ofaddToDisplay()
method. IWindowSession is a binder service provided by WMS (the implementation class is Session).
2.2 ()
An internal creation will be createdWindowState
Object. Calling WindowStateattach()
method. Finally transferred to SessionwindowAddedLocked()
, will create aSurfaceSession
Object. This is what we are looking forSurfaceFlinger
A place to connect.
SurfaceSession mSurfaceSession; void windowAddedLocked(String packageName) { mPackageName = packageName; mRelayoutTag = "relayoutWindow: " + mPackageName; if (mSurfaceSession == null) { // A process has only one session, so the SurfaceSession object is created only once. // Create SurfaceSession object mSurfaceSession = new SurfaceSession(); // Each session is stored in WMS (this); if (mLastReportedAnimatorScale != ()) { (this); } } mNumWindow++; // Number of all windows in the process +1}
oneApplication process
Corresponding to oneSession
Object, a Session object corresponds to oneSurfaceSession
. WMS will store this session. In other words, WMS will keep all applications connected to SurfaceFlingerSession
Store it.
2.3 SurfaceSession creation process
An instance of this class represents a connection to SurfaceFlinger. We can create one or moreSurface
Object.
2.3.1 Construction method
> private long mNativeClient; // SurfaceComposerClient* public SurfaceSession() { //native method mNativeClient = nativeCreate(); } > frameworks/base/core/jni/android_view_SurfaceSession.cpp static jlong nativeCreate(JNIEnv* env, jclass clazz) { // Create a new SurfaceComposerClient object SurfaceComposerClient* client = new SurfaceComposerClient(); client->incStrong((void*)nativeCreate); //Return the reference to the SurfaceComposerClient object to the java layer. return reinterpret_cast<jlong>(client); }
SurfaceComposerClient
What is it?
2.3.2 SurfaceComposerClient
When SurfaceComposerClient is referenced for the first timeonFirstRef()
method.
> frameworks/native/libs/gui/ void SurfaceComposerClient::onFirstRef() { //Create the sf proxy binder object sf, type ISurfaceComposer sp<ISurfaceComposer> sf(ComposerService::getComposerService()); if (sf != nullptr && mStatus == NO_INIT) { sp<ISurfaceComposerClient> conn; //Create an ISurfaceComposerClient object to call across processes conn = sf->createConnection(); if (conn != nullptr) { mClient = conn; mStatus = NO_ERROR; } } }
-
ISurfaceComposer
The implementation class is the SurfaceFlinger object. The proxy object in the server process is ComposerService. This class defines the Binder IPC interface for accessing various SurfaceFlinger features. - Create a
ISurfaceComposerClient
Object mClient, used to call across processes.
So which implementation class is the ISurfaceComposerClient?Continue to see ()
。
2.3.3 ()
Note that this is in the SF process.
> frameworks/native/services/surfaceflinger/ sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() { // new client object. return initClient(new Client(this)); } static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) { status_t err = client->initCheck(); if (err == NO_ERROR) { // Return this object return client; } return nullptr; } > frameworks/native/services/surfaceflinger/ class Client : public BnSurfaceComposerClient{... class BnSurfaceComposerClient : public SafeBnInterface<ISurfaceComposerClient> {...
It turns out that the implementation class of ISurfaceComposerClient is defined in SFClient
. It is also a binder service. Let's go back to the SurfaceComposerClient class, which holds the binder reference of the ISurfaceComposerClient mClient. passmClient implements communication with SF
。
2.3 Summary
- In the Session class, a
SurfaceSession
Object, internal reference to the c++ layerSurfaceComposerClient
Object. - The SurfaceComposerClient object is another binder service created through SF. Reduce SF workload.
-
SurfaceComposerClient
The object is proxyed by the binder through the mClient member (ISurfaceComposerClient), and is subsequently used to create the Surface.
So far, the Session object corresponding to the application process in WMS has established a connection with SF.
3. The creation of Surface
3.1 Surface of Java layer
ViewRootImpl class member variable:
// Any thread can access this object, just lock it.public final Surface mSurface = new Surface();
The empty parameter structure is called, so the Surface at this time is just an empty shell of the Java layer.
3.2 RelayoutWindow() on the App side
When SF receives the vsync signal, it is monitored through Choreographer and distributed to the app process. After the app receives the vsync signal, doFrame(), and then calls doTraversal() to the main thread ->performTraversals()
:
> private void performTraversals() { // Call relayoutWindow relayoutResult = relayoutWindow(params, viewVisibility, insetsPending); //... // Ask host how big it wants to be // Three major processes performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); //... performLayout(lp, mWidth, mHeight); //``` performDraw(); //... } > private int relayoutWindow( params, int viewVisibility, boolean insetsPending) throws RemoteException { //... // 1 mWindowSession is an IWindowSession object, call relayout() int relayoutResult = (mWindow, mSeq, params, (int) (() * appScale + 0.5f), (int) (() * appScale + 0.5f), viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0, frameNumber, mTmpFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets, mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingDisplayCutout, // The mSurfaceControl object passed here mPendingMergedConfiguration, mSurfaceControl, mTempInsets); if (()) { // 2 Copy surface from mSurfaceControl (mSurfaceControl); } else { // Invalid, it will be destroyed destroySurface(); } //... return relayoutResult; }
-
relayout()
Pass in mSurfaceControl and call it across the process to the WMS side. -
copyFrom()
, get the Surface from mSurfaceControl.
What is SurfaceControl?
SurfaceControl can control a surface that is being displayed on the screen managed by a synthetic system. It is the linker for metadata of Buffer data and windows. Through this SurfaceControl constructed surface, the app can directly submit buffer data to the synthesis system for synthesis. In addition, the displayed properties of the buffer can also be modified through . Such as rotation, translation, and cropping.
3.3 SurfaceControl empty parametric structure
yesViewRootImpl
Class member variables:
private final SurfaceControl mSurfaceControl = new SurfaceControl();
SurfaceControl's empty parametric construction method does nothing!It's also an empty shell at present.
3.4 Summary
It can be seen that when the drawing starts, the WMS is called(SurfaceControl mSurfaceControl)
After the method. You canSurfaceControl
Get the Surface object in it.
So how does () do it internally?Keep watching ()
。
3.5 WMS side ()
> @Override public int relayout(IWindow window, int seq, attrs, int requestedWidth, int requestedHeight, int viewFlags, int flags, long frameNumber, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets, Rect outStableInsets, Rect outsets, Rect outBackdropFrame, cutout, MergedConfiguration mergedConfiguration, SurfaceControl outSurfaceControl, InsetsState outInsetsState) { // Continue to call WMS's relayoutWindow() int res = (this, window, seq, attrs, requestedWidth, requestedHeight, viewFlags, flags, frameNumber, outFrame, outOverscanInsets, outContentInsets, outVisibleInsets, outStableInsets, outsets, outBackdropFrame, cutout, mergedConfiguration, outSurfaceControl, outInsetsState); return res; } > outSurfaceControl fromappThe process is passed toWMSprocess。 Indicates the result received。It implementsparcelableinterface,跨The process is passed toWMSmiddle。 public int relayoutWindow(Session session, IWindow client, int seq, LayoutParams attrs, int requestedWidth, int requestedHeight, int viewVisibility, int flags, long frameNumber, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame, outCutout, MergedConfiguration mergedConfiguration, SurfaceControl outSurfaceControl, InsetsState outInsetsState) { //... try { //Call createSurfaceControl result = createSurfaceControl(outSurfaceControl, result, win, winAnimator); } catch (Exception e) { ... } //... } > private int createSurfaceControl(SurfaceControl outSurfaceControl, int result, WindowState win, WindowStateAnimator winAnimator) { if (!) { result |= RELAYOUT_RES_SURFACE_CHANGED; } // Declare a WindowSurfaceController WindowSurfaceController surfaceController; try { //1 Create WindowSurfaceController object surfaceController = (, ); } finally { } if (surfaceController != null) { // 2 From the object to get the WindowSurfaceController, get the SurfaceController (outSurfaceControl); } else { // For some reason there isn't a surface. Clear the // caller's object so they see the same state. (TAG_WM, "Failed to create surface control for " + win); (); } return result; }
final:
- ()get
WindowSurfaceController
Object - From the WindowSurfaceController object,
Get SurfaceController
,Return to outSurfaceControl
。
winAnimator is the WindowStateAnimator type.
3.5.1 ()
> WindowSurfaceController createSurfaceLocked(int windowType, int ownerUid) { final WindowState w = mWin; if (mSurfaceController != null) { // If so, return return mSurfaceController; } //... // Set up surface control with initial size. try { // Otherwise create a WindowSurfaceController mSurfaceController = new WindowSurfaceController(, ().toString(), width, height, format, flags, this, windowType, ownerUid); // ... } catch (OutOfResourcesException e) { //... } return mSurfaceController; }
createSurfaceLocked()
It is to create a WindowSurfaceController object. WindowSurfaceControllerConstruction method
ASurfaceControl
Object.
3.5.2 WindowSurfaceController construction method
> public WindowSurfaceController(SurfaceSession s, String name, int w, int h, int format, int flags, WindowStateAnimator animator, int windowType, int ownerUid) { mAnimator = animator; mSurfaceW = w; mSurfaceH = h; title = name; mService = ; final WindowState win = ; mWindowType = windowType; mWindowSession = ; (TRACE_TAG_WINDOW_MANAGER, "new SurfaceControl"); // 1 The makeSurface() method of WindowState is called final b = () .setParent(()) .setName(name) .setBufferSize(w, h) .setFormat(format) .setFlags(flags) .setMetadata(METADATA_WINDOW_TYPE, windowType) .setMetadata(METADATA_OWNER_UID, ownerUid); // 2 Create a SurfaceControl object mSurfaceControl = (); (TRACE_TAG_WINDOW_MANAGER); } > public SurfaceControl build() { if (mWidth < 0 || mHeight < 0) { throw new IllegalStateException( "width and height must be positive or unset"); } if ((mWidth > 0 || mHeight > 0) && (isColorLayerSet() || isContainerLayerSet())) { throw new IllegalStateException( "Only buffer layers can set a valid buffer size."); } // Create an object and call the parameter constructor method of SurfaceControl. return new SurfaceControl( mSession, mName, mWidth, mHeight, mFormat, mFlags, mParent, mMetadata); }
Calling the parameter constructor method of SurfaceControl to construct a SurfaceControl object. Remember3. 3
, the member variable mSurfaceControl of ViewRootImpl, the call isSpace ginseng structure
, did nothing.
However, now SurfaceControlGlycologic structure
What did you do? Let's take a look in depth.
Parameter-like structure of SurfaceControl:
long mNativeObject; // package visibility only for access private SurfaceControl(SurfaceSession session, String name, int w, int h, int format, int flags, SurfaceControl parent, SparseIntArray metadata) throws OutOfResourcesException, IllegalArgumentException { //... //Note that the first parameter here is SurfaceSession mNativeObject = nativeCreate(session, name, w, h, format, flags, parent != null ? : 0, metaParcel); } >frameworks/base/core/jni/android_view_SurfaceControl.cpp static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj, jstring nameStr, jint w, jint h, jint format, jint flags, jlong parentObject, jobject metadataParcel) { ScopedUtfChars name(env, nameStr); //This is the SurfaceComposerClient object when the SurfaceSession was created before. SF's proxy service. sp<SurfaceComposerClient> client; if (sessionObj != NULL) { // mClient created earlier client = android_view_SurfaceSession_getClient(env, sessionObj); } else { client = SurfaceComposerClient::getDefault(); } // Parent SurfaceControl SurfaceControl *parent = reinterpret_cast<SurfaceControl*>(parentObject); // Declare SurfaceControl sp<SurfaceControl> surface; // metadata of the window. Because SurfaceControl itself is the connector for surface and metadata. LayerMetadata metadata; Parcel* parcel = parcelForJavaObject(env, metadataParcel); if (parcel && !parcel->objectsCount()) { status_t err = (parcel); if (err != NO_ERROR) { jniThrowException(env, "java/lang/IllegalArgumentException", "Metadata parcel has wrong format"); } } // Call createSurfaceChecked() of the SF process to create the surface object status_t err = client->createSurfaceChecked( String8(name.c_str()), w, h, format, &surface, flags, parent, std::move(metadata)); if (err == NAME_NOT_FOUND) { jniThrowException(env, "java/lang/IllegalArgumentException", NULL); return 0; } else if (err != NO_ERROR) { jniThrowException(env, OutOfResourcesException, NULL); return 0; } surface->incStrong((void *)nativeCreate); // Returns the jlong reference of SurfaceControl return reinterpret_cast<jlong>(()); } >frameworks/native/libs/gui/ status_t SurfaceComposerClient::createSurfaceChecked(const String8& name, uint32_t w, uint32_t h, PixelFormat format, sp<SurfaceControl>* outSurface, uint32_t flags, const sp<IBinder>& parentHandle, LayerMetadata metadata, uint32_t* outTransformHint) { sp<SurfaceControl> sur; status_t err = mStatus; if (mStatus == NO_ERROR) { // sp<IBinder> handle; // gpb,buffer data producer sp<IGraphicBufferProducer> gbp; uint32_t transformHint = 0; int32_t id = -1; // The specific implementation class of mClient is SF's Client service. To create a surface err = mClient->createSurface(name, w, h, format, flags, parentHandle, std::move(metadata), &handle, &gbp, &id, &transformHint); if (outTransformHint) { *outTransformHint = transformHint; } ALOGE_IF(err, "SurfaceComposerClient::createSurface error %s", strerror(-err)); if (err == NO_ERROR) { // Finally, a native layer SurfaceControl object was created. Note that this is the parametric structure of the native layer! ! *outSurface = new SurfaceControl(this, handle, gbp, id, w, h, format, transformHint, flags); } } return err; } Keep watchingnativeThe structure of the layer: >frameworks/native/libs/gui/ SurfaceControl::SurfaceControl( const sp<SurfaceComposerClient>& client, const sp<IBinder>& handle, const sp<IGraphicBufferProducer>& gbp, bool owned) : mClient(client), mHandle(handle), mGraphicBufferProducer(gbp), mOwned(owned) //Assign some member variables{ }
Summarize:
- WMS's relayout() creates a SurfaceControl object through the WindowSurfaceControl object. At this time, the SurfaceControl object not only exists in the Java layer, but also creates a SurfaceControl object in the native layer.
- In the native layer, the internal requests buffer, metadata and other information to SF through the previous connection surfaceSession to assign values to the SurfaceControl object.
Note that the above logic is still in the native layer of WMS. andcreateSurface() across processes
What exactly did the call do?
3.6 SF end ()
Now enter the SF process
。 Client
that isISurfaceComposerClient
Implementation class.
frameworks/native/services/surfaceflinger/ status_t Client::createSurface(const String8& name, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, const sp<IBinder>& parentHandle, LayerMetadata metadata, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp) { // We rely on createLayer to check permissions. // Finally SF is called. Create a layer return mFlinger->createLayer(name, this, w, h, format, flags, std::move(metadata), handle, gbp, parentHandle); } > frameworks/native/services/surfaceflinger/ status_t SurfaceFlinger::createLayer(const String8& name, const sp<Client>& client, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, LayerMetadata metadata, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, const sp<IBinder>& parentHandle, const sp<Layer>& parentLayer) { if (int32_t(w|h) < 0) { // width and height cannot be negative return BAD_VALUE; } status_t result = NO_ERROR; // Declare a layer object sp<Layer> layer; String8 uniqueName = getUniqueLayerName(name); // metadata if ((METADATA_WINDOW_TYPE)) { int32_t windowType = metadata.getInt32(METADATA_WINDOW_TYPE, 0); if (windowType == 441731) { metadata.setInt32(METADATA_WINDOW_TYPE, InputWindowInfo::TYPE_NAVIGATION_BAR_PANEL); primaryDisplayOnly = true; } } // Create different layer objects according to flag. switch (flags & ISurfaceComposerClient::eFXSurfaceMask) { case ISurfaceComposerClient::eFXSurfaceBufferQueue: // 1 Create bufferQueue, only look at this one result = createBufferQueueLayer(client, uniqueName, w, h, flags, std::move(metadata), format, handle, gbp, &layer); break; case ISurfaceComposerClient::eFXSurfaceBufferState: result = createBufferStateLayer(client, uniqueName, w, h, flags, std::move(metadata), handle, &layer); break; case ISurfaceComposerClient::eFXSurfaceColor: case ISurfaceComposerClient::eFXSurfaceContainer: ... default: result = BAD_VALUE; break; } ... // 2 Continue to call addClientLayer result = addClientLayer(client, *handle, *gbp, layer, parentHandle, parentLayer, addToCurrentState); ... return result; }
- Created a BufferQueueLayer object, which holds it
sp<BufferLayerConsumer> mConsumer
andsp<IGraphicBufferProducer> mProducer
, that isGraphicBuffer
producers and consumer objects. - Join
layers
In the global collection.
3.6.1 ()
frameworks/native/services/surfaceflinger/
status_t SurfaceFlinger::createBufferQueueLayer(const sp<Client>& client, const String8& name, uint32_t w, uint32_t h, uint32_t flags, LayerMetadata metadata, PixelFormat& format, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, sp<Layer>* outLayer) { // initialize the surfaces // Format switch (format) { case PIXEL_FORMAT_TRANSPARENT: case PIXEL_FORMAT_TRANSLUCENT: format = PIXEL_FORMAT_RGBA_8888; break; case PIXEL_FORMAT_OPAQUE: format = PIXEL_FORMAT_RGBX_8888; break; } // Create layer sp<BufferQueueLayer> layer = getFactory().createBufferQueueLayer( LayerCreationArgs(this, client, name, w, h, flags, std::move(metadata))); status_t err = layer->setDefaultBufferProperties(w, h, format); if (err == NO_ERROR) { // Assign gbp producer *handle = layer->getHandle(); *gbp = layer->getProducer(); *outLayer = layer; } ALOGE_IF(err, "createBufferQueueLayer() failed (%s)", strerror(-err)); return err; }
3.6.2
frameworks/native/services/surfaceflinger/
sp<BufferQueueLayer> createBufferQueueLayer(const LayerCreationArgs& args) override { return new BufferQueueLayer(args); }
3.6.3 BufferQueueLayer construction method
Inheritance relationship:
class BufferQueueLayer : public BufferLayer, public BufferLayerConsumer::ContentsChangedListener {... class BufferLayer : public Layer {...
3.6.4 ()
void BufferQueueLayer::onFirstRef() { BufferLayer::onFirstRef(); // Creates a custom BufferQueue for SurfaceFlingerConsumer to use // Declare producers and consumers sp<IGraphicBufferProducer> producer; sp<IGraphicBufferConsumer> consumer; // BufferQueue Initialization BufferQueue::createBufferQueue(&producer, &consumer, true); // MonitoredProducer is packaged with producer, which is convenient for SF monitoring mProducer = new MonitoredProducer(producer, mFlinger, this); { // Grab the SF state lock during this since it's the only safe way to access RenderEngine Mutex::Autolock lock(mFlinger->mStateLock); // Encapsulates BufferLayerConsumer encapsulates consumer mConsumer = new BufferLayerConsumer(consumer, mFlinger->getRenderEngine(), mTextureName, this); } mConsumer->setConsumerUsageBits(getEffectiveUsage(0)); mConsumer->setContentsChangedListener(this); mConsumer->setName(mName); // BufferQueueCore::mMaxDequeuedBufferCount is default to 1 if (!mFlinger->isLayerTripleBufferingDisabled()) { //Set to 2 mProducer->setMaxDequeuedBufferCount(2); } if (const auto display = mFlinger->getDefaultDisplayDevice()) { updateTransformHint(display); } }
Key points:
- Declared
sp<IGraphicBufferProducer> producer
;sp<IGraphicBufferConsumer> consumer
; - Call BufferQueue::
createBufferQueue
(&producer, &consumer, true); to create these two objects.
3.6.5 BufferQueue::createBufferQueue()
frameworks/native/libs/gui/
void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer, sp<IGraphicBufferConsumer>* outConsumer, bool consumerIsSurfaceFlinger) { // BufferQueueCore object sp<BufferQueueCore> core(new BufferQueueCore()); //BufferQueueProducer Production Object sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger)); // BufferQueueConsumer consumption object sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core)); *outProducer = producer; *outConsumer = consumer; } static constexpr int NUM_BUFFER_SLOTS = 64;
Therefore, the real implementer of producer and consumer isBufferQueueProducer
andBufferQueueConsumer
。
3.7 ()
Finally, add to the global mLayers.
>frameworks/native/services/surfaceflinger/ void Client::attachLayer(const sp<IBinder>& handle, const sp<Layer>& layer) { Mutex::Autolock _l(mLock); (handle, layer); }
At this point, SF creates a layer object and adds it to the mLayers list of Client service. Hold in layersp<IGraphicBufferProducer> producer
andsp<IGraphicBufferConsumer> consumer
;Two objects. The specific implementer isBufferQueueProducer
andBufferQueueConsumer
。
3.6 Summary
To review the overall process:
-
App
Call WMS's relayoutWindow(MSurfaceControl) and pass in an empty SurfaceControl object-A (empty parameter construction). -
WMS side
Another SurfaceControl object -B (argument construct) of the java layer is generated. At the same time, a SurfaceControl object is also generated in the native layer. The createSurface() method of SF is called internally through the IComposerClient object. Also holds a reference to IGraphicBufferProducer gbp. -
SF end
Generates the Layer object and generates the buffer producersp<IGraphicBufferProducer>
and buffer consumerssp<IGraphicBufferConsumer>
。
Next, it was sent from the appHow to follow SurfaceControl-A with SurfaceControl-B
What about connecting it? Of course it is throughgetSurfaceControl()
The method is here.
3.7 ()
void getSurfaceControl(SurfaceControl outSurfaceControl) { (mSurfaceControl); }
mSurfaceControl
It is the one created previously in the WindowSurfaceControllerSurfaceControl-B
Object. CurrentoutSurfaceControl
yesApp delivery
Come here. Continue to read copyFrom():
3.8 ()
> public void copyFrom(SurfaceControl other) { mName = ; mWidth = ; mHeight = ; assignNativeObject(nativeCopyFromSurfaceControl()); }
3.8.1 nativeCopyFromSurfaceControl()
>frameworks/base/core/jni/android_view_SurfaceControl.cpp static jlong nativeCopyFromSurfaceControl(JNIEnv* env, jclass clazz, jlong surfaceControlNativeObj) { // According to the reference value, get the SurfaceControl of the native layer. This is the SurfaceControl object of WMS-B sp<SurfaceControl> surface(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj)); if (surface == nullptr) { return 0; } // Create a new native SurfaceControl object C, calling the parameter constructor method sp<SurfaceControl> newSurface = new SurfaceControl(surface); newSurface->incStrong((void *)nativeCreate); return reinterpret_cast<jlong>(()); } //Glycosmic structureSurfaceControl::SurfaceControl(const sp<SurfaceControl>& other) { mClient = other->mClient; mHandle = other->mHandle; mGraphicBufferProducer = other->mGraphicBufferProducer; mTransformHint = other->mTransformHint; mLayerId = other->mLayerId; mWidth = other->mWidth; mHeight = other->mHeight; mCreateFlags = other->mCreateFlags; }
existnative
The layer creates a newSurfaceControl
Object.
3.8.2 assignNativeObject()
private void assignNativeObject(long nativeObject) { // What is passed is the SurfaceControl object C of the newly created native layer in the previous step. if (mNativeObject != 0) { // Before release release(); } // Hold a reference. mNativeObject = nativeObject; }
At this point, the app endSurfaceControl of Java layer
The object is withSurfaceControl of native layer
The objects were connected. After creation is completed,Return to the app side ViewRootImpl's relayoutWindow()
in the method.
3.9 ()
public void copyFrom(SurfaceControl other) { // Copy from surfaceControl if (other == null) { throw new IllegalArgumentException("other must not be null"); } // Get the native object reference of SurfaceControl long surfaceControlPtr = ; if (surfaceControlPtr == 0) { throw new NullPointerException( "null SurfaceControl native object. Are you using a released SurfaceControl?"); } // Get the reference of the surface object from the native layer long newNativeObject = nativeGetFromSurfaceControl(mNativeObject, surfaceControlPtr); //Lock synchronized (mLock) { // The same object will not be performed if (newNativeObject == mNativeObject) { return; } if (mNativeObject != 0) { // Before release nativeRelease(mNativeObject); } // Set new reference setNativeObjectLocked(newNativeObject); } }
3.9.1 nativeGetFromSurfaceControl()
frameworks/base/core/jni/android_view_Surface.cpp
static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz, jlong nativeObject, jlong surfaceControlNativeObj) { //Get the c++ surface object referenced by the java layer Surface* self(reinterpret_cast<Surface *>(nativeObject)); // Get the SurfaceControl object of the c++ layer sp<SurfaceControl> ctrl(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj)); // If the underlying IGBP's are the same, we don't need to do anything. // If the surface object is the same as the SurfaceControl object's gbp, there is no need to reset the value. if (self != nullptr && IInterface::asBinder(self->getIGraphicBufferProducer()) == IInterface::asBinder(ctrl->getIGraphicBufferProducer())) { // Just go back return nativeObject; } // If gbp is different, reconstruct one sp<Surface> surface(ctrl->getSurface()); if (surface != NULL) { surface->incStrong(&sRefBaseOwner); } // Return to the reference return reinterpret_cast<jlong>(()); }
The logic has been clearly commented. The question is how does getSurface() of SurfaceControl work?
3.9.2 native layer SurfaceControl::getSurface()
frameworks/native/libs/gui/
sp<Surface> SurfaceControl::getSurface() { Mutex::Autolock _l(mLock); if (mSurfaceData == nullptr) { // Create a Surface return generateSurfaceLocked(); } return mSurfaceData; } mutable sp<SurfaceControl> mBbqChild; sp<Surface> SurfaceControl::generateSurfaceLocked() { uint32_t ignore; auto flags = mCreateFlags & (ISurfaceComposerClient::eCursorWindow | ISurfaceComposerClient::eOpaque); // Call it into the SF process through ISurfaceComposerClient mBbqChild = mClient->createSurface(String8("bbq-wrapper"), 0, 0, mFormat, flags, mHandle, {}, &ignore); // mbbq is SurfaceControl mBbq = sp<BLASTBufferQueue>::make("bbq-adapter", mBbqChild, mWidth, mHeight, mFormat); // This surface is always consumed by SurfaceFlinger, so the // producerControlledByApp value doesn't matter; using false. // Create surface mSurfaceData = mBbq->getSurface(true); return mSurfaceData; }
3.9.3 ()
frameworks/native/libs/gui/
sp<Surface> BLASTBufferQueue::getSurface(bool includeSurfaceControlHandle) { std::unique_lock _lock{mMutex}; sp<IBinder> scHandle = nullptr; if (includeSurfaceControlHandle && mSurfaceControl) { scHandle = mSurfaceControl->getHandle(); } return new BBQSurface(mProducer, true, scHandle, this); }
3.9.4 BBQSurface
// Inherited surfaceclass BBQSurface : public Surface { private: std::mutex mMutex; sp<BLASTBufferQueue> mBbq; bool mDestroyed = false; public: BBQSurface(const sp<IGraphicBufferProducer>& igbp, bool controlledByApp, const sp<IBinder>& scHandle, const sp<BLASTBufferQueue>& bbq) : Surface(igbp, controlledByApp, scHandle), mBbq(bbq) {} void allocateBuffers() override { uint32_t reqWidth = mReqWidth ? mReqWidth : mUserWidth; uint32_t reqHeight = mReqHeight ? mReqHeight : mUserHeight; auto gbp = getIGraphicBufferProducer(); std::thread ([reqWidth, reqHeight, gbp=getIGraphicBufferProducer(), reqFormat=mReqFormat, reqUsage=mReqUsage] () { // Request allocateBuffers gbp->allocateBuffers(reqWidth, reqHeight, reqFormat, reqUsage); }).detach(); }
3.10 Summary
After the SurfaceControl on the app end is connected to the SurfaceControl object in the native layer. Surface of the Java layer on the app end passescopyFrom()
Method, fromnative layer
The SurfaceControl has obtained the native layerSurface Objects
, internal holdingIGraphicProducer gbp
Object. Can communicate with SF.
4. Summary
-
App
Call WMS' relayoutWindow(MSurfaceControl) and the one passed in isEmpty SurfaceControl object-A (empty parameter construction)
。 -
WMS side
Another java layer generatedSurfaceControl object-B (parameter construct)
. At the same timenative layer
A SurfaceControl object is also generated. The createSurface() method of SF is called internally through the IComposerClient object. at the same timeReferences that hold IGraphicBufferProducer gbp
。 -
SF end
GeneratedLayer
Object, and buffer producer is generatedsp<IGraphicBufferProducer>
and buffer consumerssp<IGraphicBufferConsumer>
。 -
WMS side
passcopy()
Method to pass it from the App sideSurfaceControl-A object
Also quoted from SFIGraphicBufferProducer
。 - After the value of the SurfaceControl-A object on the App side is assigned, then the Surface object is passed
copyFrom()
Method, letJava
layerSurface
andNative
LayerSurface
Build connections. In the end, that is,SF
ofIGraphicBufferProducer
Contact was established.
This is the end of this article about the creation process of rendering Surface based on Android 10. For more related Android Surface content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!