Camera.takePicture () process analysis

King of heaven and earth tiger 626 2021-04-07 03:04:10 阅读数:36

本文一共[544]字,预计阅读时长:1分钟~
camera.takepicture camera takepicture process analysis

Set the callback function , Mainly send back the data information

  /** Set the notification and data callbacks */
    void setCallbacks(notify_callback notify_cb,
                      data_callback data_cb,
                      data_callback_timestamp data_cb_timestamp,
                      void* user)
    {
        mNotifyCb = notify_cb;
        mDataCb = data_cb;
        mDataCbTimestamp = data_cb_timestamp;
        mCbUser = user;

        ALOGV("%s(%s)", __FUNCTION__, mName.string());

        if (mDevice->ops->set_callbacks) {
            mDevice->ops->set_callbacks(mDevice,
                                   __notify_cb,
                                   __data_cb,
                                   __data_cb_timestamp,
                                   __get_memory,
                                   this);

        }
    }

android_hardware_Camera.cpp file

// Taking pictures

static void android_hardware_Camera_takePicture(JNIEnv *env, jobject thiz, jint msgType)
{
    ALOGV("takePicture");
    JNICameraContext* context;
    sp<Camera> camera = get_native_camera(env, thiz, &context);
    if (camera == 0) return;

    /*
     * When CAMERA_MSG_RAW_IMAGE is requested, if the raw image callback
     * buffer is available, CAMERA_MSG_RAW_IMAGE is enabled to get the
     * notification _and_ the data; otherwise, CAMERA_MSG_RAW_IMAGE_NOTIFY
     * is enabled to receive the callback notification but no data.
     *
     * Note that CAMERA_MSG_RAW_IMAGE_NOTIFY is not exposed to the
     * Java application.
     */
    if (msgType & CAMERA_MSG_RAW_IMAGE) {
        ALOGV("Enable raw image callback buffer");
        if (!context->isRawImageCallbackBufferAvailable()) {
            ALOGV("Enable raw image notification, since no callback buffer exists");
            msgType &= ~CAMERA_MSG_RAW_IMAGE;
            msgType |= CAMERA_MSG_RAW_IMAGE_NOTIFY;
        }
    }

    if (camera->takePicture(msgType) != NO_ERROR) {
        jniThrowRuntimeException(env, "takePicture failed");
        return;
    }

}

Camera.cpp file

// take a picture
status_t Camera::takePicture(int msgType)
{
    ALOGV("takePicture: 0x%x", msgType);
    sp <ICamera> c = mCamera;
    if (c == 0) return NO_INIT;
    return c->takePicture(msgType);
}

Location :frameworks/av/camera/ICamera.cpp
takePicture():
utilize Binder The mechanism sends corresponding instructions to the server .
What is actually called is CameraClient::takePicture() function .

  // take a picture - returns an IMemory (ref-counted mmap)
    status_t takePicture(int msgType)
    {
        ALOGV("takePicture: 0x%x", msgType);
        Parcel data, reply;
        data.writeInterfaceToken(ICamera::getInterfaceDescriptor());
        data.writeInt32(msgType);
        remote()->transact(TAKE_PICTURE, data, &reply);
        status_t ret = reply.readInt32();
        return ret;
    }

Location :frameworks/av/services/camera/libcameraservice/api1/CameraClient.cpp
takePicture():
Be careful ,CAMERA_MSG_RAW_IMAGE Command and CAMERA_MSG_RAW_IMAGE_NOTIFY The command cannot be valid at the same time , It needs to be checked .
Filter incoming instructions , Just stay with takePicture() Operation related .
call CameraHardwareInterface Medium takePicture() Interface .

// take a picture - image is returned in callback
status_t CameraClient::takePicture(int msgType) {
    LOG1("takePicture (pid %d): 0x%x", getCallingPid(), msgType);

    Mutex::Autolock lock(mLock);
    status_t result = checkPidAndHardware();
    if (result != NO_ERROR) return result;

    if ((msgType & CAMERA_MSG_RAW_IMAGE) &&
        (msgType & CAMERA_MSG_RAW_IMAGE_NOTIFY)) {
        ALOGE("CAMERA_MSG_RAW_IMAGE and CAMERA_MSG_RAW_IMAGE_NOTIFY"
                " cannot be both enabled");
        return BAD_VALUE;
    }

    // We only accept picture related message types
    // and ignore other types of messages for takePicture().
    int picMsgType = msgType
                        & (CAMERA_MSG_SHUTTER |
                           CAMERA_MSG_POSTVIEW_FRAME |
                           CAMERA_MSG_RAW_IMAGE |
                           CAMERA_MSG_RAW_IMAGE_NOTIFY |
                           CAMERA_MSG_COMPRESSED_IMAGE);

    enableMsgType(picMsgType);

    return mHardware->takePicture();
}

 

Location :frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h
takePicture():
adopt mDevice Function pointer set in , call HAL Corresponding to the specific platform in the layer takePicture Implementation logic of operation .
Next is the process related to the specific platform , This is not the main part for me , And in the last note, there has been a more in-depth exploration , So we will not continue to dig down here .
It's time to control the process HAL After the layer , Again to Linux Drivers Send control command , So as to make concrete Camera Equipment execution instructions , And get the data .
 

 /**
     * Take a picture.
     */
    status_t takePicture()
    {
        ALOGV("%s(%s)", __FUNCTION__, mName.string());
        if (mDevice->ops->take_picture)
            return mDevice->ops->take_picture(mDevice);
        return INVALID_OPERATION;
    }

Because the data flow is through callback Functionally implemented , So when I explore its process, I analyze it from the bottom to the top .

Location :frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h
Here we only choose dataCallback Analysis of relevant processes .
__data_cb():
The callback function is implemented in the same file setCallbacks() Function .
Camera After the device gets the data , It's going to move up , stay HAL This callback function will be called in layer .
Through function pointers mDataCb Call the callback passed in from the previous layer , To upload data .
This mDataCb The pointer corresponds to , yes CameraClient Class dataCallback()
 static void __data_cb(int32_t msg_type,
                          const camera_memory_t *data, unsigned int index,
                          camera_frame_metadata_t *metadata,
                          void *user)
    {
        ALOGV("%s", __FUNCTION__);
        CameraHardwareInterface *__this =
                static_cast<CameraHardwareInterface *>(user);
        sp<CameraHeapMemory> mem(static_cast<CameraHeapMemory *>(data->handle));
        if (index >= mem->mNumBufs) {
            ALOGE("%s: invalid buffer index %d, max allowed is %d", __FUNCTION__,
                 index, mem->mNumBufs);
            return;
        }
        __this->mDataCb(msg_type, mem->mBuffers[index], metadata, __this->mCbUser);
    }

Location :frameworks/av/services/camera/libcameraservice/api1/CameraClient.cpp
dataCallback():
This callback is implemented in this file initialize() Function to CameraHardwareInterface in .
After starting this callback , From Cookie Get connected clients .
according to msgType, Start the corresponding handle operation .
Choose... From one of the branches handle Function to see .

void CameraClient::dataCallback(int32_t msgType,
        const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata, void* user) {
    LOG2("dataCallback(%d)", msgType);

    sp<CameraClient> client = static_cast<CameraClient*>(getClientFromCookie(user).get());
    if (client.get() == nullptr) return;

    if (!client->lockIfMessageWanted(msgType)) return;
    if (dataPtr == 0 && metadata == NULL) {
        ALOGE("Null data returned in data callback");
        client->handleGenericNotify(CAMERA_MSG_ERROR, UNKNOWN_ERROR, 0);
        return;
    }

    switch (msgType & ~CAMERA_MSG_PREVIEW_METADATA) {
        case CAMERA_MSG_PREVIEW_FRAME:
            client->handlePreviewData(msgType, dataPtr, metadata);
            break;
        case CAMERA_MSG_POSTVIEW_FRAME:
            client->handlePostview(dataPtr);
            break;
        case CAMERA_MSG_RAW_IMAGE:
            client->handleRawPicture(dataPtr);

            break;
        case CAMERA_MSG_COMPRESSED_IMAGE:
            client->handleCompressedPicture(dataPtr);
            break;
        default:
            client->handleGenericData(msgType, dataPtr, metadata);
            break;
    }
}

handleRawPicture():
stay open In the process ,connect() When a function is called ,mRemoteCallback Has been set as a client instance , It corresponds to ICameraClient A strong pointer to .
Through this example , Here is based on Binder Mechanism to start the client's dataCallback.
Client's dataCallback It is realized in Camera Class .
 

// picture callback - raw image ready
void CameraClient::handleRawPicture(const sp<IMemory>& mem) {
    disableMsgType(CAMERA_MSG_RAW_IMAGE);

    ssize_t offset;
    size_t size;
    sp<IMemoryHeap> heap = mem->getMemory(&offset, &size);

    sp<ICameraClient> c = mRemoteCallback;
    mLock.unlock();
    if (c != 0) {
        c->dataCallback(CAMERA_MSG_RAW_IMAGE, mem, NULL);
    }
}

Location :frameworks/av/camera/Camera.cpp
dataCallback():
call CameraListener Of postData Interface , Continue to move data up .
postData The implementation of the interface is in android_hardware_Camera.cpp in .
// callback from camera service when frame or image is ready
void Camera::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr,
                          camera_frame_metadata_t *metadata)
{
    sp<CameraListener> listener;
    {
        Mutex::Autolock _l(mLock);
        listener = mListener;
    }
    if (listener != NULL) {
        listener->postData(msgType, dataPtr, metadata);
    }
}

Location :frameworks/base/core/jni/android_hardware_Camera.cpp
postData():
yes JNICameraContext class , This class inherits CameraListener.
First get the virtual machine pointer .
Then filter out CAMERA_MSG_PREVIEW_METADATA Information .
Enter branch processing .
For data transmission path , The point is copyAndPost() function .

void JNICameraContext::postData(int32_t msgType, const sp<IMemory>& dataPtr,
                                camera_frame_metadata_t *metadata)
{
    // VM pointer will be NULL if object is released
    Mutex::Autolock _l(mLock);
    JNIEnv *env = AndroidRuntime::getJNIEnv();
    if (mCameraJObjectWeak == NULL) {
        ALOGW("callback on dead camera object");
        return;
    }

    int32_t dataMsgType = msgType & ~CAMERA_MSG_PREVIEW_METADATA;

    // return data based on callback type
    switch (dataMsgType) {
        case CAMERA_MSG_VIDEO_FRAME:
            // should never happen
            break;

        // For backward-compatibility purpose, if there is no callback
        // buffer for raw image, the callback returns null.
        case CAMERA_MSG_RAW_IMAGE:
            ALOGV("rawCallback");
            if (mRawImageCallbackBuffers.isEmpty()) {
                env->CallStaticVoidMethod(mCameraJClass, fields.post_event,
                        mCameraJObjectWeak, dataMsgType, 0, 0, NULL);
            } else {
                copyAndPost(env, dataPtr, dataMsgType);
            }
            break;

        // There is no data.
        case 0:
            break;

        default:
            ALOGV("dataCallback(%d, %p)", dataMsgType, dataPtr.get());
            copyAndPost(env, dataPtr, dataMsgType);
            break;
    }

    // post frame metadata to Java
    if (metadata && (msgType & CAMERA_MSG_PREVIEW_METADATA)) {
        postMetadata(env, CAMERA_MSG_PREVIEW_METADATA, metadata);
    }
}

copyAndPost():
First of all confirm Memory Is there any data in .
apply Java Byte array (jbyteArray, jbyte*), And will Memory Data is given to it .
The point is this function :
env->CallStaticVoidMethod(mCameraJClass, fields.post_event, mCameraJObjectWeak, msgType, 0, 0, obj);
Its function is to transmit images to Java End .
Through fields post_event, stay c++ Call in Java Methods , And pass in the corresponding parameters .
Finally call to Java Terminal postEventFromNative() Method .
 

void JNICameraContext::copyAndPost(JNIEnv* env, const sp<IMemory>& dataPtr, int msgType)
{
    jbyteArray obj = NULL;

    // allocate Java byte array and copy data
    if (dataPtr != NULL) {
        ssize_t offset;
        size_t size;
        sp<IMemoryHeap> heap = dataPtr->getMemory(&offset, &size);
        ALOGV("copyAndPost: off=%zd, size=%zu", offset, size);
        uint8_t *heapBase = (uint8_t*)heap->base();

        if (heapBase != NULL) {
            const jbyte* data = reinterpret_cast<const jbyte*>(heapBase + offset);

            if (msgType == CAMERA_MSG_RAW_IMAGE) {
                obj = getCallbackBuffer(env, &mRawImageCallbackBuffers, size);
            } else if (msgType == CAMERA_MSG_PREVIEW_FRAME && mManualBufferMode) {
                obj = getCallbackBuffer(env, &mCallbackBuffers, size);

                if (mCallbackBuffers.isEmpty()) {
                    ALOGV("Out of buffers, clearing callback!");
                    mCamera->setPreviewCallbackFlags(CAMERA_FRAME_CALLBACK_FLAG_NOOP);
                    mManualCameraCallbackSet = false;

                    if (obj == NULL) {
                        return;
                    }
                }
            } else {
                ALOGV("Allocating callback buffer");
                obj = env->NewByteArray(size);
            }

            if (obj == NULL) {
                ALOGE("Couldn't allocate byte array for JPEG data");
                env->ExceptionClear();
            } else {
                env->SetByteArrayRegion(obj, 0, size, data);
            }
        } else {
            ALOGE("image heap is NULL");
        }
    }

    // post image data to Java
    env->CallStaticVoidMethod(mCameraJClass, fields.post_event,
            mCameraJObjectWeak, msgType, 0, 0, obj);
    if (obj) {
        env->DeleteLocalRef(obj);
    }
}

Location :frameworks/base/core/java/android/hardware/Camera.java
Both of the following methods are EventHandler Members of , This class inherits Handler class .
postEventFromNative():
First determine Camera Is it instantiated .
After the confirmation , adopt Camera Members of mEventHandler Of obtainMessage Method from Native The data obtained in the environment is encapsulated as Message An instance of a class , And then call sendMessage() Method to transfer data out

   private static void postEventFromNative(Object camera_ref,
                                            int what, int arg1, int arg2, Object obj)
    {
        Camera c = (Camera)((WeakReference)camera_ref).get();
        if (c == null)
            return;

        if (c.mEventHandler != null) {
            Message m = c.mEventHandler.obtainMessage(what, arg1, arg2, obj);
            c.mEventHandler.sendMessage(m);
        }
    }

handleMessage():
sendMessage() Method will be used to process the outgoing data , And send it to the corresponding callback class .
Notice several different callback classes (mRawImageCallback、mJpegCallback etc. ) There are onPictureTaken() Method , By calling this method , The data transmitted from the bottom layer to the top layer finally Java Application , Upper application through parsing Message You can get the images , So that subsequent operations can be carried out .
This is the end of the data flow I analyzed .

Summary
In this note , We from Camera.takePicture() Approach , Contact what you learned before Open technological process , Will the whole Camera The process is simply followed up .
Whether it's control flow or data flow , All of them need to carry out the next step through five levels . Control flow is the flow of commands from the top to the bottom , And data flow is the flow of data from the bottom to the top .
If you want to customize a data processing C++ Function library , And add it to the camera , We can pass the HAL Layer to make some changes , take RAW The image flows to our processing , I'll deal with the RAW The image returns HAL layer ( Need to be in HAL Layer pair RAW Format some processing to upload the image ), Finally, the image is transferred to the top-level application through the normal callback process , Then we can realize our custom function .
thus , For the entire Camera Framework , And how it works , We already have a clear understanding of .
 

版权声明:本文为[King of heaven and earth tiger 626]所创,转载请带上原文链接,感谢。 https://netfreeman.com/2021/04/20210407030145544r.html