Android Camera API functions and usage guide

A StreamConfigurationMap

1. The role of StreamConfigurationMap

StreamConfigurationMapIt is a core class in the Android Camera2 API.Used to describe the output stream configuration supported by the camera device, including the following information:

• Supported formats and resolutions: For example, YUV_420_888, JPEG, RAW and other formats and their corresponding resolution list.
• Input/Output Stream Combination Rules: For example, which formats can be used for preview, photographing or video recording at the same time.
• Hardware capability limitations: for example, whether hardware-level image processing (such as YUV reprocessing), dynamic range mode (HDR), etc. are supported.
• Frame rate and duration limits: such as the minimum/maximum frame rate supported at certain resolutions, or the maximum duration of video recording.

Apply throughCameraCharacteristicsGetStreamConfigurationMap, and properly configured based on thisCaptureSession(For example, select a compatible resolution for preview and taking photos).

2. Interaction with cameraprovider

(1) Data source

• CameraProvider provides raw data:

Camera HAL implementation (e.g.@2.4)passgetCameraCharacteristics()Methods report hardware capabilities to the framework, including supported stream configurations (format, resolution, dynamic range, etc.).

• The frame is encapsulated as StreamConfigurationMap：

Framework layer (e.g.CameraService) parse the metadata returned by HAL and convert it into a direct useable by the application.StreamConfigurationMap。

(2) Configuration process example

• Application request camera capability：
  • Application call(cameraId)。
• Framework query HAL：
  • CameraServicepassCameraProviderThe HAL interface obtains the metadata of the camera.
• HAL Returns to the underlying configuration：
  • CameraProviderFrom hardware or board-level configuration files (e.g.cam_board.xml) reads supported stream configurations and passes them to the framework.
• Build StreamConfigurationMap：
  • The framework will raw data (e.g.SCALER_AVAILABLE_STREAM_CONFIGURATIONS) encapsulated asStreamConfigurationMapObject.
• Application usage configuration：
  • Application basedStreamConfigurationMapSelect a compatible stream combination (e.g. both support1080p@30fpsPreview and12MPPhotograph).

(3) Corresponding to key data structures

3. Actual examples

Assume a camera device supports the following configuration (throughCameraProviderReport):

• YUV_420_888: 1920x1080@30fps, 1280x720@60fps
• JPEG: 4000x3000@10fps

butStreamConfigurationMapThe following information will be generated:

// Get the resolution supported by YUV formatSize[] yuvSizes = (ImageFormat.YUV_420_888);
// Result: [1920x1080, 1280x720]// Check whether hardware-level YUV reprocessing is supportedboolean isReprocessSupported = (
    CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_YUV_REPROCESSING);

StreamConfigurationMapIt is the interface for the application layer to obtain the camera output capability, andCameraProviderIt is a provider of underlying hardware capabilities. The former relies on the metadata reported by the latter, and the two realize togetherTransparent mapping of hardware capabilities to application interfaces, is one of the core designs of the Android Camera system hierarchical architecture.

In the Android Camera System (Camera2 API),OutputConfigurationandStreamConfigurationMapare two key classes related to camera output stream configurations that play different roles in camera workflow. The following is their detailed introduction and relationship analysis:

2. OutputConfiguration

1 Function

effect：

OutputConfigurationis a class introduced by Android 5.0 (API 21) forDefine the configuration of a single camera output stream. Its core functions include:

• Encapsulated output target (e.g.SurfaceorSurfaceView）。
• Configure the physical camera (specify the physical sensor in a multi-camera device).
• Manage shared output streams (allow multipleSurfaceShare the same output stream).
• Use scenarios：
• When createdCameraCaptureSessionWhen a group needs to be passed inOutputConfigurationObject, describing all output streams.
• Key Methods：
  • addSurface(Surface surface): Add sharedSurface。
  • setPhysicalCameraId(String id): Specify the physical camera (for dual-camera/multi-camera devices).
  • getSurfaces(): Get the associatedSurfaceList.
• Example：

// Create ImageReader and get SurfaceImageReader imageReader = (width, height, format, maxImages);
Surface imageSurface = ();
// Create OutputConfigurationOutputConfiguration outputConfig = new OutputConfiguration(imageSurface);
// Optional: Configure a shared Surface or physical camera// (anotherSurface);
// ("2");
// Create CameraCaptureSessionList<OutputConfiguration> outputConfigs = new ArrayList<>();
(outputConfig);
(outputConfigs, callback, handler);

2. Relationship with StreamConfigurationMap

Collaboration process：

• Step 1:passStreamConfigurationMapQuery the output parameters (format, resolution, etc.) supported by the device.
• Step 2: Create an output target based on legal parameters (such asImageReaderorSurfaceView）。
• Step 3: Encapsulate the output target toOutputConfiguration, used to createCameraCaptureSession。

Dependencies：

• OutputConfigurationThe parameters (such as format, resolution) must comply withStreamConfigurationMapConstraints, otherwise session creation will fail.
• StreamConfigurationMapProvide theoretical support,OutputConfigurationResponsible for actual configuration.

Extended features：

• OutputConfigurationSupports advanced features (such as multi-camera sharing output stream), andStreamConfigurationMapDescribe only hardware capabilities.
• After Android 10 (API 29),OutputConfigurationAdded support for dynamic resolution and physical camera binding.

3. Frequently Asked Questions

Q1: Why do you need to use both?
A: StreamConfigurationMapIt is the "capability manual" of the camera device, telling developers what output configurations the hardware supports.OutputConfigurationIt is the actual construction of the output stream, binding legal parameters to the specificSurface`。

The two work together to ensure the correctness and efficiency of the camera output stream

Q2: How to avoidInvalidSurfaceException？
A: make sureOutputConfigurationofSurfaceParameters (format, resolution) areStreamConfigurationMapwithin the legal scope.

Q3: How to deal with multi-camera scenarios?
A: use()Specify the physical camera and passStreamConfigurationMapCheck whether the camera supports target parameters.

Summarize StreamConfigurationMapIt is the "capability manual" of the camera device, telling developers what output configurations the hardware supports.OutputConfigurationIt is the actual construction of the output stream, binding legal parameters to the specificSurface. The two are combined to ensure the correctness and efficiency of the camera output stream and are an indispensable component in the Camera2 API.

In the Android Camera2 API,CameraCaptureSessionIt is the core component that manages camera data flow and captures requests. It's withOutputConfigurationClosely related, they jointly determine the configuration and operating mechanism of the camera's output target (such as preview, photography, video recording, etc.). The following is a detailed explanation and itsOutputConfigurationRelationship:

Three CameraCaptureSession

1. Function

CameraCaptureSessionIt's a camera device (CameraDevice) and output target (Surface) The bridge between:

Manage output streams: Bind multipleSurface(As previewedSurfaceView, taking picturesImageReader) and make sure the data is transferred to these destinations correctly.

Submit a capture request:passcapture()orsetRepeatingRequest()Send a request (CaptureRequest) and control the camera's behavior (such as autofocus, exposure, frame rate, etc.).

Handle asynchronous events: Monitor the camera status (such as focus completion, frame capture completion) and call back to the application.

2 Life cycle

• create:pass()orcreateCaptureSessionByOutputConfigurations()create.
• Activity: Capture request can be submitted, and the camera data flows to boundSurface。
• closure: Calledclose()Release resources and no more requests are sent.

3. Create CameraCaptureSession

When creating a session, you need to specify a set of output targets (SurfaceorOutputConfiguration). Two methods:

Traditional way (based onSurfaceList):

(
    List<Surface> outputs,   // Directly pass the Surface list     callback, 
    Handler handler
);

Suitable for simple scenarios, but with limited flexibility (e.g., multi-camera or dynamic resolution is not supported).

based onOutputConfigurationWay (API 21+, extensions are enhanced in subsequent versions):

(
    List<OutputConfiguration> outputConfigurations,  // Encapsulated advanced configuration of Surface     callback,
    Handler handler
);

Supports more complex configurations (such as shared streaming, physical camera binding, dynamic resolution, etc.).

4. Relationship with OutputConfiguration

(1) OutputConfiguration is the input to the session

• Function:OutputConfigurationEncapsulated one or moreSurfaceConfiguration information for defining the characteristics of the output stream:
  • Single or sharedSurface:passaddSurface()Add multipleSurface, share the same data stream (such as preview and video sharing the same frame of data).
  • Physical camera binding: In dual-camera/multi-camera equipment,setPhysicalCameraId()Specifies which physical camera the output stream comes from.
  • Dynamic resolution(API 23+): Allows dynamic resolution adjustments when the session is running (requires hardware support).
• Advantages: Compared to direct deliverySurfaceList,OutputConfigurationProvides finer granular control capabilities.

(2) The process of creating a session

• Query device support:passStreamConfigurationMapConfirm the format and resolution supported by the camera.
• Create an output target: Created based on legal parametersSurface(likeImageReader、SurfaceView）。
• Encapsulated as OutputConfiguration:WillSurfaceIts additional configurations (such as physical cameras) are packaged toOutputConfiguration。
• Create a session: CalledcreateCaptureSessionByOutputConfigurations(), incomingOutputConfigurationList.

(3) Key constraints

• Immutability:onceCameraCaptureSessionCreated successfully, its boundOutputConfigurationNot modified (such as new or removedSurface). If you need to change, the current session must be closed and recreated.
• Hardware limitations：OutputConfigurationThe configuration (such as resolution, format) must comply withStreamConfigurationMapsupport range.

5. Example: Create a session using OutputConfiguration

// Create two output targets: Preview Surface and Photo ImageReaderSurfaceView surfaceView = ... // Preview SurfaceViewImageReader imageReader = (4032, 3024, , 3);
// Encapsulated as OutputConfigurationOutputConfiguration previewConfig = new OutputConfiguration(().getSurface());
OutputConfiguration captureConfig = new OutputConfiguration(());
// Optional: Configure shared streams or physical cameras// (anotherSurface); // Share the same stream// ("1"); // Specify the physical cameraList<OutputConfiguration> outputConfigs = new ArrayList<>();
(previewConfig);
(captureConfig);
// Create CameraCaptureSession(outputConfigs, 
    new () {
        @Override
        public void onConfigured(@NonNull CameraCaptureSession session) {
            // The session is ready, you can submit CaptureRequest        }
        @Override
        public void onConfigureFailed(@NonNull CameraCaptureSession session) {
            // Configuration failed (incorrect parameters or hardware problems)        }
    }, 
    null // Optional Handler);

6. Advanced features and compatibility

• Shared Surfaces：
  • MultipleSurfaceShare the same output stream (such as preview and AI analytics shared data) to reduce memory and power consumption.
  • Implementation method: through()Add multipleSurface。
• Dynamic Resolution（API 23+）：
  • Allows dynamic adjustment of output resolution when the session is running (depending on device support).
  • pass()Configuration.
• Multi-camera support（API 28+）：
  • In dual camera device,()Specifies the output stream source.

7. FAQs and Solutions

• Question 1：createCaptureSessionFailed, error isIllegalArgumentException。
• reason：OutputConfigurationThe parameters are illegal (such as the resolution is beyond the support range).
• solve:examineStreamConfigurationMaplegal parameters.
• Question 2: The output stream cannot be modified at runtime.
• reason：CameraCaptureSessionThe configuration is immutable.
• solve: Close the current session and recreate a new session.
• Question 3: Data flow is chaotic in multi-camera scenarios.
• reason: Not specified correctlysetPhysicalCameraId()。
• solve: Make sure everyOutputConfigurationBind to the correct physical camera.

Summarize

• CameraCaptureSession: is the core controller of the camera data flow, responsible for managing output targets and capturing requests.
• andOutputConfigurationRelationship:
  • OutputConfigurationis the input of the session, which defines the specific configuration of each output stream (e.g.Surface, physical camera).
  • CameraCaptureSessionpassOutputConfigurationThe configuration of the data flow ensures that the data flow complies with the hardware capabilities (byStreamConfigurationMapdefinition).
  • useOutputConfigurationAdvanced features (such as shared streaming, multi-camera) can be enabled, while traditionalSurfaceList mode functions are limited.

By reasonable useOutputConfiguration, developers can more flexibly configure camera output streams to meet the needs of complex scenarios (such as multi-camera, dynamic resolution, and low-power shared streams).

This is the end of this article about the introduction of Android Camera API. For more related content on Android Camera API, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!