Detailed explanation of how Android SurfaceView and TextureView is used

Surface

The official explanation for Surface is: the handle on the original buffer managed by the screen synthesizer, the so-called native buffer, is used to save the pixel data of the current window, that is, the native buffer and the contents in it can be obtained through Surface. Surface corresponds to a screen buffer, each Window corresponds to a Surface, and any View is drawn on the Surface. Canvas in Surface is used to provide drawings.

SurfaceView

SurfaceView is different from ordinary Views. It has its own Surface. It works by creating a new window that is different from the application window, separated from the host window, and can handle business in a separate thread, not controlled by the View's attributes, and cannot perform translation and scaling and other conversions. It uses the "double buffering" mechanism to achieve efficient interface refresh effect.

Double buffering technology is to process the image to be processed in memory and then display it on the screen. Double buffering is mainly used to solve the flicker caused by repeated local flushing. First draw the things you want to draw into a memory area, and then draw the whole at one time.

Let’s take a look at the specific use of SurfaceView through a camera preview function

private lateinit var surfaceHolder: SurfaceHolder

Get the camera management class, open a specific camera, and create a preview request in the successful callback.

        val cameraManager = getSystemService(Context.CAMERA_SERVICE) as CameraManager
        val surfaceView = findViewById<SurfaceView>()
        surfaceHolder = 
        (true)
        (object :  {
            override fun surfaceCreated(p0: SurfaceHolder) {
                if ((
                        this@SurfaceActivity,
                        
                    ) != PackageManager.PERMISSION_GRANTED
                ) {
                    //No camera permission                    return
                }
                try {
                    //Get the list of available camera devices                    val cameraIdList = 
                    //Open the camera                    (
                        cameraIdList[0],
                        //Camel creates monitoring                        object : () {
                            override fun onOpened(p0: CameraDevice) { //Open the camera                                try {
                                    startPreview(p0)
                                } catch (e: CameraAccessException) {
                                    ()
                                }
                            }
                            override fun onDisconnected(p0: CameraDevice) { //Close the camera                                ()
                            }
                            override fun onError(p0: CameraDevice, p1: Int) {
                                (tag, " onError")
                            }
                        },
                        null
                    )
                } catch (e: CameraAccessException) {
                    ()
                }
            }
            override fun surfaceChanged(p0: SurfaceHolder, p1: Int, p2: Int, p3: Int) {
                (tag, "surfaceChanged")
            }
            override fun surfaceDestroyed(p0: SurfaceHolder) {
                (tag, "surfaceDestroyed")
            }
        })

Start preview

    private fun startPreview(cameraDevice: CameraDevice) {
        val captureRequest = (CameraDevice.TEMPLATE_PREVIEW)
        ()
        (
            listOf(),
            //Session status callback            object : () {
                override fun onConfigured(p0: CameraCaptureSession) {
                    try {
                        // Create preview required                        val previewRequestBuilder =
                            (CameraDevice.TEMPLATE_PREVIEW)
                        ()
                        val previewRequest = ()
                        (previewRequest, null, null)
                    } catch (e: CameraAccessException) {
                        ()
                    }
                }
                override fun onConfigureFailed(p0: CameraCaptureSession) {
                    (tag, "onConfigureFailed")
                }
            }, null
        )
    }

In this way, the simple camera preview function will be released. Of course, we also need to apply for CAMERA permission, omitted here.

TextureView

TextureView can be said to be a View object that combines View and SurfaceTexture. A View that can output the content stream as an external texture. It can only be used to open a hardware-accelerated window. TextureView will not create an independent window, but can perform animations such as translation and rotation like ordinary Views. TextureView can only be used in the API after Android 4.0. However, Android devices basically do not have versions before Android 4.0, so there is no need to worry about this.

The use of TextureView is also relatively simple. You need to get its SurfaceTexture and then you can render it. Here is a simple video playback example to see how it is used.

        val textureView = findViewById<TextureView>()
        val mediaPlayer = MediaPlayer()
         = object :  {
            override fun onSurfaceTextureAvailable(p0: SurfaceTexture, p1: Int, p2: Int) {
                with(mediaPlayer) {
                    setDataSource()
                    setSurface(Surface(p0))
                    prepare()
                    start()
                    (tag, "setOnPreparedListener")
                    setOnPreparedListener {
                        ()
                    }
                }
            }
            override fun onSurfaceTextureSizeChanged(p0: SurfaceTexture, p1: Int, p2: Int) {
            }
            override fun onSurfaceTextureDestroyed(p0: SurfaceTexture): Boolean {
                ()
                ()
                return true
            }
            override fun onSurfaceTextureUpdated(p0: SurfaceTexture) {
            }
        }

SurfaceTexture

SurfaceTexture is a combination of Surface and OpenGL ES textures. It is used to provide the Surface output to OpenGL ES textures. Unlike SurfaceView, it does not directly display the image stream processing, but instead converts it to GL external textures. Therefore, it can be used for secondary processing of image stream data, such as Camera filters, desktop special effects, etc., but this will cause several frame delays. At the same time, since it manages BufferQueue itself, the memory consumption will be slightly larger. For example, after Camera's preview data becomes a texture, it can be displayed through the SurfaceTexture to the TextureView as a hardware acceleration layer in the View level.

The difference between SurfaceView and TextureView

• SurfaceView is output directly and has its own independent Surface. Its rendering can be placed in a separate thread. Its disadvantage is that it cannot be transformed or animation.
• TextureView is a text that can output content streams as external textures. It must be a hardware acceleration layer, and the display screen updates will have a delay of 1 to 3 frames.
• The TextureView itself also contains SurfaceTexture. Compared with the SurfaceView+SurfaceTexture combination, it can also convert the image on the content stream into texture output. The difference is that the TextureView is drawn in the View level, while the SurfaceView+SurfaceTexture is drawn on a separate Surface.

Personally, I think that for games that need to constantly update canvas, SurfaceView is the best choice, and TextureView is more suitable for the development of video players or camera applications.

This is the end of this article about the detailed explanation of how to use Android SurfaceView and TextureView. For more related contents of Android SurfaceView and TextureView, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!