稀有猿诉

十年磨一剑,历炼出锋芒,说话千百句,不如码二行。

Camera 2教程之预览与加强

前一篇文章讲解了如何使用这套新的API,但仍有很多可以提升的空间,这篇重点来讲讲,如何提升预览画质和做一些加强。

线程模型

因为相机是属于硬件,操作起来可能会耗时,这套新的API也特别注意,因此加了很多异步化处理,所有的请求结果全是通过回调来进行的,并且需要调用者来指定一个回调所使用的线程。因此,我们需要一个专门用于camera操作的线程,用HandlerThread就可以,并把它控制在Activity的生命周期之中,比如在onCreate时启动此线程,在onDestroy时关闭。

1
2
3
4
5
6
7
8
9
10
11
@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    attachCameraThread();
}

@Override
protected void onDestroy() {
    super.onDestroy();
     detachCameraThread();
}

所有与camera相关的操作,均要在相机专属的HandlerThread中调用,与其他线程(如主线程)的交互均通过回调处理。

关键对象

为了进一步的封装和方便管理,需要两个关键对象的封装。

CameraContext

一个是CameraContext,负责管理相机的线程,外部所有的方法调用均应该通过它来进行,我们的目的是要把所有的相机相关操作封装在自己的线程里面,因此,暴露给外面的接口,必须统一,并且在开放的方法中加入线程检查,如果还没有启动HandlerThread,就报错,调用者需要先attachThread:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
public class CameraContext {
    private static final String LOG_TAG = CameraContext.class.getSimpleName();

    private final CameraManagerWrapper cameraManager;
    private volatile HandlerThread cameraManageThread;
    private volatile CameraThreadHandler cameraThreadHandler;

    private volatile CameraAgent currentCamera;

    public CameraContext(Context context) {
        cameraManager = new CameraManagerWrapper(context);
    }

    public void attachThread() {
        Log.d(LOG_TAG, "attachThread");
        if (cameraThreadHandler != null) {
            cameraThreadHandler.removeCallbacksAndMessages(null);
        }
        if (cameraManageThread != null && cameraManageThread.isAlive()) {
            cameraManageThread.quitSafely();
        }
        cameraManageThread = new HandlerThread("Camera Management Thread");
        cameraManageThread.start();
        cameraThreadHandler = new CameraThreadHandler(cameraManageThread.getLooper());
    }

    public void detachThread() {
        Log.d(LOG_TAG, "detachThread");
        if (cameraManageThread != null && cameraManageThread.isAlive()) {
             cameraManageThread.quitSafely();
        }
    }

    private void checkThread() {
        if (cameraManageThread == null || !cameraManageThread.isAlive()) {
            throw new CameraSetupException("attachThread must be called before any other method invocations.");
        }
    }

    public void openCamera(Consumer<String> consumer) {
        checkThread();
        Runnable actionOpen = () -> {
              // ...
        };
        Message msg = Message.obtain(cameraThreadHandler, actionOpen);
        msg.what = CameraThreadHandler.MSG_OPEN_CAMERA;
        msg.sendToTarget();
    }

    public void closeCamera() {
        checkThread();
        Runnable actionClose = null;
        Message msgClose = Message.obtain(cameraThreadHandler, actionClose);
        msgClose.what = CameraThreadHandler.MSG_CLOSE_CAMERA;
        msgClose.sendToTarget();
    }
}

CameraAgent

还需要对CameraDevice进行封装,把CameraCaptureSession,以及RequestBuilder,封装在内,并且在三大回调Device State Callback,Session State Callback以及Session Capture Callback也都封装在内,因此这些东西的生命周期全都是在CameraDevice内部的。

设计要点:

  • CameraAgent不开放接口给外部,它只能开放给CameraContext使用
  • 对象是对camera device的完整封装,对象本身一直可用,与CameraDevice是否打开无直接关系
  • 随时可以查询静态配置属性,创建对象时就要传入id和CameraCharacteristics
  • 有连接状态,也即打开对应的CameraDevice,动态配置属性查询,以及像启动预览,必须要是连接状态
  • 拍照应该在预览状态内
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
class CameraAgent {
    private static String LOG_TAG = CameraAgent.class.getSimpleName();
    private final String id;
    private final CameraCharacteristics characteristics;
    private final CameraContext.CameraThreadHandler cameraHandler;
    private final Executor executor;

    private Optional<CameraDevice> cameraDevice;
    private Optional<CameraCaptureSession> captureSession;

    private CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
        @Override
        public void onOpened(@NonNull CameraDevice camera) {
            Log.d(LOG_TAG, "Device Status: onOpened");
            cameraDevice = Optional.of(camera);
        }

        @Override
        public void onDisconnected(@NonNull CameraDevice camera) {
            Log.d(LOG_TAG, "Device Status: onDisconnected");
            cameraDevice = Optional.empty();
        }

        @Override
        public void onError(@NonNull CameraDevice camera, int error) {
            cameraDevice = Optional.empty();
        }
    };

    private CameraCaptureSession.StateCallback regularSessionCallback = new CameraCaptureSession.StateCallback() {
        @Override
        public void onConfigured(@NonNull CameraCaptureSession session) {
            Log.d(LOG_TAG, "Session State: onConfigured");
            captureSession = Optional.of(session);
        }

        @Override
        public void onConfigureFailed(@NonNull CameraCaptureSession session) {
            captureSession = Optional.empty();
        }

        @Override
        public void onClosed(@NonNull CameraCaptureSession session) {
            Log.d(LOG_TAG, "Session State: onClosed");
            super.onClosed(session);
            captureSession = Optional.empty();
        }
    };

    private CameraCaptureSession.CaptureCallback captureCallback = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureStarted(@NonNull CameraCaptureSession session, @NonNull CaptureRequest request, long timestamp, long frameNumber) {
            super.onCaptureStarted(session, request, timestamp, frameNumber);
        }
    };

    private CaptureRequest.Builder requestBuilder;

    CameraAgent(String id, CameraCharacteristics cameraCharacteristics, CameraContext.CameraThreadHandler handler) {
        this.id = id;
        this.characteristics = cameraCharacteristics;
        this.cameraHandler = handler;
        executor = command -> handler.post(command);

        cameraDevice = Optional.empty();
        captureSession = Optional.empty();
        previewSize = Optional.empty();
        errorCode = 0;
    }

    String getId() {
        return id;
    }

    boolean connected() {
        return cameraDevice.isPresent();
    }

    CameraDevice.StateCallback getDeviceStateCallback() {
        return stateCallback;
    }

    void connect(@NonNull CameraManagerWrapper wrapper) {
        Log.d(LOG_TAG, "connect");
        wrapper.openCamera(this, cameraHandler);
    }

    void disconnect() {
        Log.d(LOG_TAG, "disconnect");
        if (captureSession.isPresent()) {
            captureSession.get().close();
        }
        if (cameraDevice.isPresent()) {
            cameraDevice.get().close();
        }
    }

    void startPreview() {
    }

    void stopPreview() {
    }
}

启动预览

新的这套API并没有直接设置预览大小或者图片大小的地方,camera的输出都是Surface,底层是通过Surface的大小来做具体的尺寸。本质上都是数据在流动,其实都是buffer,Surface也是buffer,设定了Surface的大小,也就确定了输出buffer的大小,camera硬件也就知道了大小。

用SurfaceView就可以

预览是camera的输出,是另外一些组件的输入,API设计已做好了衔接,Surface就是中间的桥梁,Surface作为SurfaceView(可理解为屏幕)的输入,它可以作为camera的输出,由此便把camera的预览显示了出来。在布局文件中用SurfaceView来充当Activity的布局,由此便能得到Surface,再把它塞给camera即可。

需要重点说一下尺寸的约束,想让预览的Surface反应camera预览的尺寸,因此SurfaceView要是wrap_content的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/black"
    tools:context=".CameraActivity">

    <net.toughcoder.effectivecamera.AutoFitSurfaceView
        android:id="@+id/view_finder"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintLeft_toRightOf="parent"
        app:layout_constraintBottom_toBottomOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>

如何选择预览尺寸

输入的约束条件是一个比例,比如流行的预览比例是4:3,16:9或者全屏,这个比例是长边与短边的比值,这个可以作为用户体验层面的一个约束,或者叫做设置,尺寸的选择应该遵守此约束。

另外一个约束就是屏幕尺寸,预览的大小应该能刚好满足屏幕尺寸即可,超出屏幕其实就浪费了,没有必要。

所以,选择预览尺寸的策略就是保证比例和刚好满足屏幕。

每个camera都有支持的一组预览尺寸,按照 我们的策略从其中选择一个就可以了。预览尺寸从静态配置中就可以读得到,不需要连接状态,因此,可以在创建好CameraAgent对象后就可以进行尺寸选择,屏幕尺寸随时可获利,比例约束是一个设置随时可读取,因此这是可行的。

当选择好了预览尺寸后,要把它设置到SurfaceView中去,以让SurfaceView调整自身的大小。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
/*
 * Strategies:
 *  1) preview size should not be bigger than screen, which is not necessary.
 *  2) ratio should match.
 *  3) pick the largest one.
 *  4) if not found, use screen size.
 */
private Optional<CameraSize> calculatePreviewSize(Point screenSize, float ratio) {
    int width = Math.min(screenSize.x, screenSize.y);
    int height = Math.min(Math.round(width * ratio), screenSize.y);
    final int limit = screenSize.x * screenSize.y;
    Log.d(LOG_TAG, "screen size " + screenSize.x + " x " + screenSize.y +
            ", ratio " + ratio + ", desired width->" + width + ", height->" + height);

    StreamConfigurationMap streamMap = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
    Size[] surfaceSizes = streamMap.getOutputSizes(Surface.class);
    if (surfaceSizes == null) {
        surfaceSizes = streamMap.getOutputSizes(ImageFormat.PRIVATE);
    }
    List<CameraSize> supportedSize = Arrays.asList(surfaceSizes)
            .stream()
            .map(CameraSize::new)
            .filter(size -> size.height * size.width <= limit)
            .sorted(CameraSize::compare)
            .collect(Collectors.toList());
    Log.d(LOG_TAG, "supportedSize " + supportedSize);
    return supportedSize.stream().filter(size -> size.matchRatio(ratio)).findFirst();

确保关闭

相机是一种硬件资源,当退出的时候要能确保它是关闭状态的,也就是说要确保CameraAgent#disconnect能执行,且要执行完成,执行完成的意思是,你需要收到onDisconnected的回调。

这就要求我们在detachThread,即退出相机线程的时候,要小心处理好尚未来得及执行(如有)的操作,因为所有的操作都会转到相机线程中去,是通过消息队列,所以操作可能还在排队中尚未真正执行。

具体做法是,直接移除掉未得到执行的open和其他操作。如果有pending状态的关闭,则要先让其执行,并且把关闭线程的操作放到CameraDevice State Callcback的onDisconnect中,也就是说待CameraDevice完全关闭完成后,才可以终止相机线程:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public void detachThread() {
    Log.d(LOG_TAG, "detachThread");
    if (cameraThreadHandler != null) {
        // Drop all pending open actions
        cameraThreadHandler.removeMessages(CameraThreadHandler.MSG_OPEN_CAMERA);
        if (cameraThreadHandler.hasMessages(CameraThreadHandler.MSG_CLOSE_CAMERA)) {
            // Ensure close actions are dispatched.
            try {
                Thread.sleep(50);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        // Drop all other messages.
        cameraThreadHandler.removeCallbacksAndMessages(null);
    }
    // TODO: technically speaking, should do this in handler thread
    // since all connect/disconnect are done inside handler thread
    // status might not be synced with caller's thread.
    if (cameraManageThread != null && cameraManageThread.isAlive()) {
        if (currentCamera != null && currentCamera.connected()) {
            currentCamera.addDisconnectedAction(() -> cameraManageThread.quitSafely());
        } else {
            cameraManageThread.quitSafely();
        }
    }
}

Comments