引言
为了给用户提供更好的成像效果，现在的手机都会接入一些第三方的图像处理算法。MTK平台的HAL3也在P2这一层提供接入的plugin。按图像处理算法需要的帧数和摄像头数量，大体可以分为三类：
-
单帧算法：
常见的单帧算法有：美颜算法(瘦脸、磨皮、大眼)、广角镜头畸变校正算法、附加表情算法、单摄背景虚化算法(伪双摄算法)等等，仅需单帧图像输入的算法都属于单帧算法。一般情况下，输入一帧图像，算法处理完输出一帧处理后的图像。 -
多帧算法：
常见的多帧算法有：MFNR（多帧降噪）、HDR（高动态范围）等等，需要连续多帧图像输入的算法都属于多帧算法。一般情况下，输入连续多帧图像，算法处理完输出一帧处理后的图像。 -
双摄算法：
最常见的双摄算法是双摄景深算法或者叫双摄背景虚化算法，除此之外，也有彩色+黑白用于增强夜拍效果的双摄算法。单帧算法和多帧算法仅需要获取一个摄像头的图像。而双摄算法需要获取主、辅两个摄像头的图像，并且一般还会要求主、辅摄像头同步。分别获取主、辅摄像头的两帧同步图像，处理后输出一帧主摄图像，用户也仅能看到主摄图像。
根据这个大体上的分类，MTK HAL算法集成系列文章共三篇：
- MTK HAL算法集成之单帧算法
- MTK HAL算法集成之多帧算法
- MTK HAL算法集成之双摄算法
本文是其中的第一篇。这个系列文章均基于Android 9.0，MT6763平台，HAL版本是HAL3。
1. 算法集成前的准备
在开展集成工作之前，首先要对算法有一个基本的评估，并且对于集成也应有一定的要求。
1. 1 算法要求及评估
- 处理效果好，不能比竞品差，超过竞品更佳。（这条和camera调试的主观效果一样，主观性较强，往往一厢情愿，具体看项目要求吧）
- 各个场景及压力测试下效果稳定。
- 处理后照片无色差、锐度和饱和度无损失，或者损失在可接受范围。
- 达到可接受的分辨率，最好可达到摄像头的最大分辨率。
- 处理时间越快越好，不超过竞品时间、不超过项目和产品的目标时间。
- 无内存泄露，占用内存少。
- 提供必要的集成说明文档，包括算法类型、输入及输出图像要求、输入参数要求等等。
注意：如果有条件，处理时间、内存占用、分辨率等等可量化的指标可要求算法提供方给出具体的参考数据，以便集成完后测试验证。
1.2 算法集成要求
- 编译时可根据项目控制是否集成算法。
- 运行时可以用参数控制是否启用算法。
- 集成算法库正常运行、压力测试下效果稳定、无内存泄露。
1.3 算法集成的步骤
(1). 根据算法选择feature类型，如果与MTK提供的feature不能对号入座，则需要添加自定义feature。
(2). 将算法对应的feature类型添加到scenario配置表。
(3). 根据算法选择plugin类型，编写CPP文件实现plugin，挂载算法。
(4). 如果算法不能复用Android和MTK提供的metadata，则还需要为算法配置自定义的metadata以便APP控制是否启用算法。
首先，我准备了一个libwatermark.so，它仅仅实现了一个添加水印的功能，用它来模拟第三方的单帧算法库。如果想了解添加水印的实现代码，可以参考我另外一篇文章：Android 实现图片加水印或logo。接下来，我们就按照集成步骤，逐步详细讲解。
2. 为算法选择feature
2.1 MTK提供的feature
MTK在mtk_feature_type.h和customer_feature_type.h已经提供了一些feature。
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/mtk/mtk_feature_type.h：
NO_FEATURE_NORMAL = 0ULL,
// MTK (bit 0-31)
MTK_FEATURE_MFNR = 1ULL << 0,
MTK_FEATURE_HDR = 1ULL << 1,
MTK_FEATURE_REMOSAIC = 1ULL << 2,
MTK_FEATURE_ABF = 1ULL << 3,
MTK_FEATURE_NR = 1ULL << 4,
MTK_FEATURE_FB = 1ULL << 5,
MTK_FEATURE_CZ = 1ULL << 6,
MTK_FEATURE_DRE = 1ULL << 7,
MTK_FEATURE_DEPTH = 1ULL << 8,
MTK_FEATURE_BOKEH = 1ULL << 9,
MTK_FEATURE_VSDOF = (MTK_FEATURE_DEPTH|MTK_FEATURE_BOKEH),
MTK_FEATURE_FSC = 1ULL << 10,
MTK_FEATURE_3DNR = 1ULL << 11,
MTK_FEATURE_EIS = 1ULL << 12,
MTK_FEATURE_AINR = 1ULL << 13,
MTK_FEATURE_DUAL_YUV = 1ULL << 14,
MTK_FEATURE_DUAL_HWDEPTH = 1ULL << 15,
MTK_FEATURE_AIS = 1ULL << 16,
MTK_FEATURE_HFG = 1ULL << 17,
MTK_FEATURE_DCE = 1ULL << 18,
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：
// ThirdParty (bit 32-63)
TP_FEATURE_HDR = 1ULL << 32,
TP_FEATURE_MFNR = 1ULL << 33,
TP_FEATURE_EIS = 1ULL << 34,
TP_FEATURE_FB = 1ULL << 35,
TP_FEATURE_FILTER = 1ULL << 36,
TP_FEATURE_DEPTH = 1ULL << 37,
TP_FEATURE_BOKEH = 1ULL << 38,
TP_FEATURE_VSDOF = (TP_FEATURE_DEPTH|TP_FEATURE_BOKEH),
TP_FEATURE_FUSION = 1ULL << 39,
TP_FEATURE_HDR_DC = 1ULL << 40, // used by DualCam
TP_FEATURE_DUAL_YUV = 1ULL << 41,
TP_FEATURE_DUAL_HWDEPTH = 1ULL << 42,
TP_FEATURE_PUREBOKEH = 1ULL << 43,
TP_FEATURE_RAW_HDR = 1ULL << 44,
TP_FEATURE_RELIGHTING = 1ULL << 45,
MTK提供的这些feature可以满足绝大多数算法的集成，在可以对号入座的情况下，我们直接使用已有feature即可。如果不能够满足我们的要求，可以参考下节内容添加新的feature。
2.2 添加自定义feature
本来单帧算法对应的feature可以选择MTK提供的MTK_FEATURE_FB和TP_FEATURE_FB，但是为了讲解如何添加新feature，我们选择添加一个自定义feature：TP_FEATURE_WATERMARK。
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
old mode 100644
new mode 100755
index a41fd864f5..17bc35eea8
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
@@ -59,6 +59,7 @@ enum eFeatureIndexCustomer {
TP_FEATURE_PUREBOKEH = 1ULL << 43,
TP_FEATURE_RAW_HDR = 1ULL << 44,
TP_FEATURE_RELIGHTING = 1ULL << 45,
+ TP_FEATURE_WATERMARK = 1ULL << 46,
// TODO: reserve for customer feature index (bit 32-63)
};
vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
old mode 100644
new mode 100755
index e32f80a609..47273b01c7
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
@@ -599,6 +599,7 @@ const char* FeatID2Name(FeatureID_T fid)
case FID_FUSION_3RD_PARTY: return "fusion_3rd_party";
case FID_PUREBOKEH_3RD_PARTY: return "purebokeh_3rd_party";
case FID_RELIGHTING_3RD_PARTY: return "relighting_3rd_party";
+ case FID_WATERMARK_3RD_PARTY: return "watermark_3rd_party";
default: return "unknown";
};
vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
index 8bb794ba02..d4343aaccf 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
@@ -779,7 +779,8 @@ MBOOL YUVNode::onInit()
featId = FID_FB_3RD_PARTY;
else if (rProperty.mFeatures & TP_FEATURE_RELIGHTING)
featId = FID_RELIGHTING_3RD_PARTY;
-
+ else if (rProperty.mFeatures & TP_FEATURE_WATERMARK)
+ featId = FID_WATERMARK_3RD_PARTY;
if (featId != NULL_FEATURE) {
MY_LOGD_IF(mLogLevel, "%s finds plugin:%s, priority:%d",
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
old mode 100644
new mode 100755
index 2f1ad8a665..ab47aae456
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
@@ -172,6 +172,7 @@ enum CaptureFeatureFeatureID {
FID_FUSION_3RD_PARTY,
FID_PUREBOKEH_3RD_PARTY,
FID_RELIGHTING_3RD_PARTY,
+ FID_WATERMARK_3RD_PARTY,
NUM_OF_FEATURE,
NULL_FEATURE = 0xFF,
};
vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp
old mode 100644
new mode 100755
index cc1dc549fd..00559cbc30
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp
@@ -428,6 +428,9 @@ MBOOL CaptureProcessor::onEnque(const sp<P2FrameRequest> &pP2Frame)
pCapRequest->addFeature(FID_HFG);
if (feature & MTK_FEATURE_DCE)
pCapRequest->addFeature(FID_DCE);
+ if (feature & TP_FEATURE_WATERMARK)
+ pCapRequest->addFeature(FID_WATERMARK_3RD_PARTY);
+
}
}
3. 将算法对应的feature添加到scenario配置表
在我们打开camera进行预览和拍照的时候，MTK HAL3会执行vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/policy/FeatureSettingPolicy.cpp的代码，会分别调用
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/scenario_mgr.cpp的
get_streaming_scenario函数和get_capture_scenario函数。它们会读取一个scenario的feature配置表，遍历所有的feature，决定哪些feature会被执行。这个配置表中有许多的scenario，一个scenario可能对应多个feature。因此添加自定义feature后，还需将自定义的feature添加到配置表中。MTK feature 对应的配置表是 gMtkScenarioFeaturesMaps，customer feature 对应的配置表是 gCustomerScenarioFeaturesMaps。
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp
old mode 100644
new mode 100755
index f8d081e433..577f85797e
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp
@@ -93,30 +93,30 @@ using namespace NSCam::v3::pipeline::policy::scenariomgr;
// #define <feature combination> (key feature | post-processing features | ...)
//
// single cam capture feature combination
-#define TP_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB)
+#define TP_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
#define TP_FEATURE_COMBINATION_CSHOT (NO_FEATURE_NORMAL | MTK_FEATURE_CZ| MTK_FEATURE_HFG)
-#define TP_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB)
+#define TP_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
#define TP_FEATURE_COMBINATION_PRO (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE)
-#define TP_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB)
+#define TP_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK)
// dual cam capture feature combination
// the VSDOF means the combination of Bokeh feature and Depth feature
-#define TP_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF)
-#define TP_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF)
-#define TP_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF)
-#define TP_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_FUSION)
-#define TP_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_PUREBOKEH)
+#define TP_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_FUSION| TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_PUREBOKEH| TP_FEATURE_WATERMARK)
// streaming feature combination (TODO: it should be refined by streaming scenario feature)
-#define TP_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB)
-#define TP_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV)
-#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH)
-#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB)
+#define TP_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV|TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH|TP_FEATURE_WATERMARK)
+#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK)
// ======================================================================================================
//
/******************************************************************************
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp
old mode 100644
new mode 100755
index 011f551354..f14ff8a6e2
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp
@@ -89,29 +89,29 @@ using namespace NSCam::v3::pipeline::policy::scenariomgr;
// #define <feature combination> (key feature | post-processing features | ...)
//
// single cam capture feature combination
-#define MTK_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB)
+#define MTK_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
#define MTK_FEATURE_COMBINATION_CSHOT (NO_FEATURE_NORMAL | MTK_FEATURE_CZ| MTK_FEATURE_HFG)
-#define MTK_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB)
+#define MTK_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK)
// dual cam capture feature combination
// the VSDOF means the combination of Bokeh feature and Depth feature
-#define MTK_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF)
-#define MTK_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF)
-#define MTK_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF)
-#define MTK_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_FUSION)
-#define MTK_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_PUREBOKEH)
+#define MTK_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_FUSION| TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_PUREBOKEH| TP_FEATURE_WATERMARK)
// streaming feature combination (TODO: it should be refined by streaming scenario feature)
-#define MTK_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB)
-#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV)
-#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH)
-#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB)
+#define MTK_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV|TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH|TP_FEATURE_WATERMARK)
+#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK)
// ======================================================================================================
//
/******************************************************************************
注意：
MTK在Android Q（10.0）上优化了scenario配置表的客制化，Android Q及更高版本，scenario需要在：
vendor/mediatek/proprietary/custom/[platform]/hal/camera/camera_custom_feature_table.cpp中配置，[platform]是诸如mt6580，mt6763之类的。
将自定义feature添加到scenario配置表时，不可贪多，只要添加到合适的scenario就行，多了可能多个算法会有冲突。如果仅在简单场景，添加到MTK_FEATURE_COMBINATION_SINGLE和TP_FEATURE_COMBINATION_SINGLE就可以满足绝大多数需求。(2021-02-02更新)
4. 挂载算法
4.1 为算法选择plugin
MTK HAL3在vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/plugin/PipelinePluginType.h 中将三方算法的挂载点大致分为以下几类：
- BokehPlugin： Bokeh算法挂载点，双摄景深算法的虚化部分。
- DepthPlugin： Depth算法挂载点，双摄景深算法的计算深度部分。
- FusionPlugin： Depth和Bokeh放在1个算法中，即合并的双摄景深算法挂载点。
- JoinPlugin： Streaming相关算法挂载点，预览算法都挂载在这里。
- MultiFramePlugin： 多帧算法挂载点，包括YUV与RAW，例如MFNR/HDR
- RawPlugin： RAW算法挂载点，例如remosaic
- YuvPlugin： Yuv单帧算法挂载点，例如美颜、广角镜头畸变校正等
对号入座，将要集成的算法选择相应的plugin。这里是单帧算法，所以预览我们选择JoinPlugin，拍照选择YuvPlugin。
4.2 编写算法集成文件
参照FBImpl.cpp和sample_streaming_fb.cpp中分别实现拍照和预览。目录结构如下：
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/tp_watermark/
├── Android.mk
├── include
│ └── watermark.h
├── lib
│ ├── arm64-v8a
│ │ └── libwatermark.so
│ └── armeabi-v7a
│ └── libwatermark.so
├── res
│ └── watermark.rgba
├── WatermarkCapture.cpp
└── WatermarkPreview.cpp
文件说明：
- Android.mk中配置算法库、头文件、集成的源代码CPP文件编译成库libmtkcam.plugin.tp_watermark，供libmtkcam_3rdparty.customer依赖调用。
- 集成的源代码CPP文件，WatermarkCapture.cpp用于拍照，WatermarkPreview.cpp用于预览。
- libwatermark.so实现了添加水印的功能，libwatermark.so用来模拟需要接入的第三方算法库。watermark.h是头文件。
- watermark.rgba是对应的水印文件。
4.2.1 添加全局宏控
为了能控制某个项目是否集成此算法，我们在device/mediateksample/k63v2_64_bsp/ProjectConfig.mk中添加一个宏，用于控制新接入算法的编译：
QXT_WATERMARK_SUPPORT = yes
当某个项目不需要新接入的算法时，将device/mediateksample/[platform]/ProjectConfig.mk的QXT_WA_SUPPORT的值设为 no 就可以了。
4.2.2 mtkcam3/3rdparty/customer/tp_watermark/Android.mk
ifeq ($(QXT_WATERMARK_SUPPORT),yes)
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := libwatermark
LOCAL_SRC_FILES_32 := lib/armeabi-v7a/libwatermark.so
LOCAL_SRC_FILES_64 := lib/arm64-v8a/libwatermark.so
LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
LOCAL_MODULE_SUFFIX := .so
LOCAL_PROPRIETARY_MODULE := true
LOCAL_MULTILIB := both
include $(BUILD_PREBUILT)
################################################################################
include $(CLEAR_VARS)
#-----------------------------------------------------------
include $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam/mtkcam.mk
#-----------------------------------------------------------
LOCAL_SRC_FILES += WatermarkCapture.cpp
LOCAL_SRC_FILES += WatermarkPreview.cpp
#-----------------------------------------------------------
LOCAL_C_INCLUDES += $(MTKCAM_C_INCLUDES)
LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam3/include
LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam/include
#
LOCAL_C_INCLUDES += system/media/camera/include
LOCAL_C_INCLUDES += $(TOP)/external/libyuv/files/include/
LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam3/3rdparty/customer/tp_watermark/include
#-----------------------------------------------------------
LOCAL_CFLAGS += $(MTKCAM_CFLAGS)
#
#-----------------------------------------------------------
LOCAL_STATIC_LIBRARIES +=
#
LOCAL_WHOLE_STATIC_LIBRARIES +=
#-----------------------------------------------------------
LOCAL_SHARED_LIBRARIES += liblog
LOCAL_SHARED_LIBRARIES += libutils
LOCAL_SHARED_LIBRARIES += libcutils
LOCAL_SHARED_LIBRARIES += libmtkcam_modulehelper
LOCAL_SHARED_LIBRARIES += libmtkcam_stdutils
LOCAL_SHARED_LIBRARIES += libmtkcam_pipeline
LOCAL_SHARED_LIBRARIES += libmtkcam_metadata
LOCAL_SHARED_LIBRARIES += libmtkcam_metastore
LOCAL_SHARED_LIBRARIES += libmtkcam_streamutils
LOCAL_SHARED_LIBRARIES += libmtkcam_imgbuf
LOCAL_SHARED_LIBRARIES += libyuv.vendor
#-----------------------------------------------------------
LOCAL_HEADER_LIBRARIES := libutils_headers liblog_headers libhardware_headers
#-----------------------------------------------------------
LOCAL_MODULE := libmtkcam.plugin.tp_watermark
LOCAL_PROPRIETARY_MODULE := true
LOCAL_MODULE_OWNER := mtk
LOCAL_MODULE_TAGS := optional
include $(MTK_STATIC_LIBRARY)
################################################################################
include $(call all-makefiles-under,$(LOCAL_PATH))
endif
4.2.3 mtkcam3/3rdparty/customer/tp_watermark/WatermarkCapture.cpp
主要函数介绍：
- 在property函数中feature类型设置我们在第三步中添加的TP_FEATURE_WATERMARK，并设置名称、优先级等等属性。
- 在negotiate函数中配置算法需要的输入、输出图像的格式、尺寸。
- 在negotiate函数或者process函数中获取上层传下来的metadata参数，根据参数决定算法是否运行，或者将参数传给算法。
- 在process函数中接入算法。
注意：
MTK原文：
negotiate函数设置格式时，一个挂载点如果挂载多个同类型的plugin，则只有第一个 plugin 中的 negotiate 中的 input buffer 设定有效。
在YUVNode 下挂载单帧 YUV plugin时，一定要确保 MTK 平台的SWNR plugin 的 negotiate 直接返回不OK，不做任何 accepted format 等的设定。否则，可能会出现因 SWNR plugin和三方plugin negotiate时设定的 accepted format 不一致而导致的三方 plugin 拿不到它想要的 format 的buffer。
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
old mode 100644
new mode 100755
index 0ae951cc83..c4819068f7
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
@@ -340,7 +340,7 @@ negotiate(Selection& sel)
sel.mOMetadataApp.setRequired(false);
sel.mOMetadataHal.setRequired(true);
- return OK;
+ return -EINVAL;//OK;
}
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/tp_watermark/WatermarkCapture.cpp：
#define LOG_TAG "WatermarkCapture"
//
#include <mtkcam/utils/std/Log.h>
//
#include <stdlib.h>
#include <utils/Errors.h>
#include <utils/List.h>
#include <utils/RefBase.h>
#include <sstream>
//
#include <mtkcam/utils/metadata/client/mtk_metadata_tag.h>
#include <mtkcam/utils/metadata/hal/mtk_platform_metadata_tag.h>
//
//
#include <mtkcam/utils/imgbuf/IIonImageBufferHeap.h>
//
#include <mtkcam/drv/IHalSensor.h>
#include <mtkcam/utils/std/Format.h>
//
#include <mtkcam3/pipeline/hwnode/NodeId.h>
#include <mtkcam/utils/metastore/ITemplateRequest.h>
#include <mtkcam/utils/metastore/IMetadataProvider.h>
#include <mtkcam3/3rdparty/plugin/PipelinePlugin.h>
#include <mtkcam3/3rdparty/plugin/PipelinePluginType.h>
#include <stdlib.h>
#include <watermark.h>
#include <mtkcam/utils/std/Time.h>
#include <time.h>
#include <libyuv.h>
//
using namespace NSCam;
using namespace android;
using namespace std;
using namespace NSCam::NSPipelinePlugin;
/******************************************************************************
*
******************************************************************************/
#define MY_LOGV(fmt, arg...) CAM_LOGV("(%d)[%s] " fmt, ::gettid(), __FUNCTION__, ##arg)
#define MY_LOGD(fmt, arg...) CAM_LOGD("(%d)[%s] " fmt, ::gettid(), __FUNCTION__, ##arg)
#define MY_LOGI(fmt, arg...) CAM_LOGI("(%d)[%s] " fmt, ::gettid(), __FUNCTION__, ##arg)
#define MY_LOGW(fmt, arg...) CAM_LOGW("(%d)[%s] " fmt, ::gettid(), __FUNCTION__, ##arg)
#define MY_LOGE(fmt, arg...) CAM_LOGE("(%d)[%s] " fmt, ::gettid(), __FUNCTION__, ##arg)
//
#define FUNCTION_IN MY_LOGD("%s +", __FUNCTION__)
#define FUNCTION_OUT MY_LOGD("%s -", __FUNCTION__)
//systrace
#if 1
#ifndef ATRACE_TAG
#define ATRACE_TAG ATRACE_TAG_CAMERA
#endif
#include <utils/Trace.h>
#define WATERMARK_TRACE_CALL() ATRACE_CALL()
#define WATERMARK_TRACE_NAME(name) ATRACE_NAME(name)
#define WATERMARK_TRACE_BEGIN(name) ATRACE_BEGIN(name)
#define WATERMARK_TRACE_END() ATRACE_END()
#else
#define WATERMARK_TRACE_CALL()
#define WATERMARK_TRACE_NAME(name)
#define WATERMARK_TRACE_BEGIN(name)
#define WATERMARK_TRACE_END()
#endif
template <class T>
inline bool
tryGetMetadata(IMetadata const *pMetadata, MUINT32 tag, T& rVal)
{
if(pMetadata == nullptr) return MFALSE;
IMetadata::IEntry entry = pMetadata->entryFor(tag);
if(!entry.isEmpty())
{
rVal = entry.itemAt(0,Type2Type<T>());
return true;
}
else
{
#define var(v) #v
#define type(t) #t
MY_LOGW("no metadata %s in %s", var(tag), type(pMetadata));
#undef type
#undef var
}
return false;
}
/******************************************************************************
*
******************************************************************************/
class WatermarkCapture : public YuvPlugin::IProvider {
public:
typedef YuvPlugin::Property Property;
typedef YuvPlugin::Selection Selection;
typedef YuvPlugin::Request::Ptr RequestPtr;
typedef YuvPlugin::RequestCallback::Ptr RequestCallbackPtr;
private:
int mOpenid;
MBOOL mEnable = 1;
MBOOL mDump = 0;
unsigned char *mSrcRGBA = nullptr;
unsigned char *mWatermarkRGBA = nullptr;
int mWatermarkWidth = 0;
int mWatermarkHeight = 0;
public:
WatermarkCapture();
~WatermarkCapture();
void init();
void uninit();
void abort(vector <RequestPtr> &pRequests);
void set(MINT32 iOpenId, MINT32 iOpenId2);
const Property &property();
MERROR negotiate(Selection &sel);
MERROR process(RequestPtr pRequest, RequestCallbackPtr pCallback);
};
WatermarkCapture::WatermarkCapture() : mOpenid(-1) {
FUNCTION_IN;
mEnable = property_get_bool("vendor.debug.camera.watermark.capture.enable", 1);
mDump = property_get_bool("vendor.debug.camera.watermark.capture.dump", 0);
FUNCTION_OUT;
}
WatermarkCapture::~WatermarkCapture() {
FUNCTION_IN;
FUNCTION_OUT;
}
void WatermarkCapture::init() {
FUNCTION_IN;
mWatermarkWidth = 180;
mWatermarkHeight = 640;
int watermarkSize = mWatermarkWidth * mWatermarkHeight * 4;
mWatermarkRGBA = (unsigned char *) malloc(watermarkSize);
FILE *fp;
char path[256];
snprintf(path, sizeof(path), "/vendor/res/images/watermark.rgba");
if ((fp = fopen(path, "r")) == NULL) {
MY_LOGE("Failed to open /vendor/res/images/watermark.rgba");
}
fread(mWatermarkRGBA, 1, watermarkSize, fp);
fclose(fp);
FUNCTION_OUT;
}
void WatermarkCapture::uninit() {
FUNCTION_IN;
free(mWatermarkRGBA);
FUNCTION_OUT;
}
void WatermarkCapture::abort(vector <RequestPtr> &pRequests) {
FUNCTION_IN;
(void)pRequests;
FUNCTION_OUT;
}
void WatermarkCapture::set(MINT32 iOpenId, MINT32 iOpenId2) {
FUNCTION_IN;
MY_LOGD("set openId:%d openId2:%d", iOpenId, iOpenId2);
mOpenid = iOpenId;
FUNCTION_OUT;
}
const WatermarkCapture::Property &WatermarkCapture::property() {
FUNCTION_IN;
static Property prop;
static bool inited;
if (!inited) {
prop.mName = "TP_WATERMARK";
prop.mFeatures = TP_FEATURE_WATERMARK;
prop.mInPlace = MTRUE;
prop.mFaceData = eFD_Current;
prop.mPosition = 0;
inited = true;
}
FUNCTION_OUT;
return prop;
}
MERROR WatermarkCapture::negotiate(Selection &sel) {
FUNCTION_IN;
if (!mEnable) {
MY_LOGD("Force off TP_WATERMARK");
FUNCTION_OUT;
return -EINVAL;
}
sel.mIBufferFull
.setRequired(MTRUE)
.addAcceptedFormat(eImgFmt_I420)
.addAcceptedSize(eImgSize_Full);
sel.mIMetadataDynamic.setRequired(MTRUE);
sel.mIMetadataApp.setRequired(MTRUE);
sel.mIMetadataHal.setRequired(MTRUE);
sel.mOMetadataApp.setRequired(MTRUE);
sel.mOMetadataHal.setRequired(MTRUE);
FUNCTION_OUT;
return OK;
}
MERROR WatermarkCapture::process(RequestPtr pRequest,
RequestCallbackPtr pCallback = nullptr) {
FUNCTION_IN;
WATERMARK_TRACE_CALL();
MBOOL needRun = MFALSE;
if (pRequest->mIBufferFull != nullptr && pRequest->mOBufferFull != nullptr) {
IImageBuffer *pIBufferFull = pRequest->mIBufferFull->acquire();
IImageBuffer *pOBufferFull = pRequest->mOBufferFull->acquire();
if (pRequest->mIMetadataDynamic != nullptr) {
IMetadata *meta = pRequest->mIMetadataDynamic->acquire();
if (meta != NULL)
MY_LOGD("[IN] Dynamic metadata count: %d", meta->count());
else
MY_LOGD("[IN] Dynamic metadata empty");
}
int frameNo = 0, requestNo = 0;
if (pRequest->mIMetadataHal != nullptr) {
IMetadata *pIMetataHAL = pRequest->mIMetadataHal->acquire();
if (pIMetataHAL != NULL) {
MY_LOGD("[IN] HAL metadata count: %d", pIMetataHAL->count());
if (!tryGetMetadata<int>(pIMetataHAL, MTK_PIPELINE_FRAME_NUMBER, frameNo)) {
frameNo = 0;
}
if (!tryGetMetadata<int>(pIMetataHAL, MTK_PIPELINE_REQUEST_NUMBER, requestNo)) {
requestNo = 0;
}
MY_LOGD("frameNo: %d, requestNo: %d", frameNo, requestNo);
} else {
MY_LOGD("[IN] HAL metadata empty");
}
}
if (pRequest->mIMetadataApp != nullptr) {
IMetadata *pIMetadataApp = pRequest->mIMetadataApp->acquire();
MINT32 mode = 0;
if (!tryGetMetadata<MINT32>(pIMetadataApp, QXT_FEATURE_WATERMARK, mode)) {
mode = 0;
}
needRun = mode == 1 ? 1 : 0;
}
MY_LOGD("needRun: %d", needRun);
int width = pIBufferFull->getImgSize().w;
int height = pIBufferFull->getImgSize().h;
MINT inFormat = pIBufferFull->getImgFormat();
if (needRun && inFormat == NSCam::eImgFmt_I420) {
uint32_t currentTime = (NSCam::Utils::TimeTool::getReadableTime()) % 1000;
time_t timep;
time (&timep);
char currentDate[20];
strftime(currentDate, sizeof(currentDate), "%Y%m%d_%H%M%S", localtime(&timep));
//dump input I420
if (mDump) {
char path[256];
snprintf(path, sizeof(path), "/data/vendor/camera_dump/capture_in_frame%d_%dx%d_%s_%d.i420",
frameNo, width, height, currentDate, currentTime);
pIBufferFull->saveToFile(path);
}
nsecs_t t1 = systemTime(CLOCK_MONOTONIC);
if (mSrcRGBA == NULL) {
mSrcRGBA = (unsigned char *) malloc(width * height * 4);
}
//convert I420 to RGBA
libyuv::I420ToABGR((unsigned char *) (pIBufferFull->getBufVA(0)), width,
(unsigned char *) (pIBufferFull->getBufVA(1)), width >> 1,
(unsigned char *) (pIBufferFull->getBufVA(2)), width >> 1,
mSrcRGBA, width * 4,
width, height);
nsecs_t t2 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Prepare src cost %02ld ms", ns2ms(t2 - t1));
Watermark::add(mSrcRGBA, width, height, mWatermarkRGBA, mWatermarkWidth, mWatermarkHeight, (width - mWatermarkWidth) / 2, (height - mWatermarkHeight) / 2);
nsecs_t t3 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Add watermark cost %02ld ms", ns2ms(t3 - t2));
//convert RGBA to I420
libyuv::ABGRToI420(mSrcRGBA, width * 4,
(unsigned char *) (pOBufferFull->getBufVA(0)), width,
(unsigned char *) (pOBufferFull->getBufVA(1)), width >> 1,
(unsigned char *) (pOBufferFull->getBufVA(2)), width >> 1,
width, height);
nsecs_t t4 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Copy in to out cost %02ld ms", ns2ms(t4 - t3));
//dump output I420
if (mDump) {
char path[256];
snprintf(path, sizeof(path), "/data/vendor/camera_dump/capture_out_frame%d_%dx%d_%s_%d.i420",
frameNo, width, height, currentDate, currentTime);
pOBufferFull->saveToFile(path);
}
free(mSrcRGBA);
} else {
if (!needRun) {
MY_LOGE("No need run, skip add watermark for capture.");
} else if (inFormat != NSCam::eImgFmt_YV12) {
MY_LOGE("Unsupported format, skip add watermark for capture.");
} else {
MY_LOGE("Unknown exception, skip add watermark for capture.");
}
memcpy((unsigned char *) (pOBufferFull->getBufVA(0)),
(unsigned char *) (pIBufferFull->getBufVA(0)),
pIBufferFull->getBufSizeInBytes(0));
memcpy((unsigned char *) (pOBufferFull->getBufVA(1)),
(unsigned char *) (pIBufferFull->getBufVA(1)),
pIBufferFull->getBufSizeInBytes(1));
memcpy((unsigned char *) (pOBufferFull->getBufVA(2)),
(unsigned char *) (pIBufferFull->getBufVA(2)),
pIBufferFull->getBufSizeInBytes(2));
}
pRequest->mIBufferFull->release();
pRequest->mOBufferFull->release();
if (pRequest->mIMetadataDynamic != nullptr) {
pRequest->mIMetadataDynamic->release();
}
if (pRequest->mIMetadataHal != nullptr) {
pRequest->mIMetadataHal->release();
}
if (pRequest->mIMetadataApp != nullptr) {
pRequest->mIMetadataApp->release();
}
}
if (pCallback != nullptr) {
MY_LOGD("callback request");
pCallback->onCompleted(pRequest, 0);
}
FUNCTION_OUT;
return OK;
}
REGISTER_PLUGIN_PROVIDER(Yuv, WatermarkCapture);
4.2.4 mtkcam3/3rdparty/customer/tp_watermark/WatermarkPreview.cpp
#include <mtkcam3/3rdparty/plugin/PipelinePluginType.h>
#include <mtkcam/utils/metadata/hal/mtk_platform_metadata_tag.h>
#include <mtkcam/utils/metadata/client/mtk_metadata_tag.h>
#include <cutils/properties.h>
#include <watermark.h>
#include <mtkcam/utils/std/Time.h>
#include <time.h>
#include <libyuv.h>
#include <dlfcn.h>
using NSCam::NSPipelinePlugin::Interceptor;
using NSCam::NSPipelinePlugin::PipelinePlugin;
using NSCam::NSPipelinePlugin::PluginRegister;
using NSCam::NSPipelinePlugin::Join;
using NSCam::NSPipelinePlugin::JoinPlugin;
using namespace NSCam::NSPipelinePlugin;
using NSCam::MSize;
using NSCam::MERROR;
using NSCam::IImageBuffer;
using NSCam::IMetadata;
using NSCam::Type2Type;
#ifdef LOG_TAG
#undef LOG_TAG
#endif // LOG_TAG
#define LOG_TAG "WatermarkPreview"
#include <log/log.h>
#include <android/log.h>
#define MY_LOGI(fmt, arg...) ALOGI("[%s] " fmt, __FUNCTION__, ##arg)
#define MY_LOGD(fmt, arg...) ALOGD("[%s] " fmt, __FUNCTION__, ##arg)
#define MY_LOGW(fmt, arg...) ALOGW("[%s] " fmt, __FUNCTION__, ##arg)
#define MY_LOGE(fmt, arg...) ALOGE("[%s] " fmt, __FUNCTION__, ##arg)
#define FUNCTION_IN MY_LOGD("%s +", __FUNCTION__)
#define FUNCTION_OUT MY_LOGD("%s -", __FUNCTION__)
template <class T>
inline bool
tryGetMetadata(IMetadata const *pMetadata, MUINT32 tag, T& rVal)
{
if(pMetadata == nullptr) return MFALSE;
IMetadata::IEntry entry = pMetadata->entryFor(tag);
if(!entry.isEmpty())
{
rVal = entry.itemAt(0,Type2Type<T>());
return true;
}
else
{
#define var(v) #v
#define type(t) #t
MY_LOGW("no metadata %s in %s", var(tag), type(pMetadata));
#undef type
#undef var
}
return false;
}
class WatermarkPreview : public JoinPlugin::IProvider {
public:
typedef JoinPlugin::Property Property;
typedef JoinPlugin::Selection Selection;
typedef JoinPlugin::Request::Ptr RequestPtr;
typedef JoinPlugin::RequestCallback::Ptr RequestCallbackPtr;
private:
bool mDisponly = false;
bool mInplace = false;
int mOpenID1 = 0;
int mOpenID2 = 0;
MBOOL mEnable = 1;
MBOOL mDump = 0;
unsigned char *mSrcRGBA = nullptr;
unsigned char *mWatermarkRGBA = nullptr;
int mWatermarkWidth = 0;
int mWatermarkHeight = 0;
public:
WatermarkPreview();
~WatermarkPreview();
void init();
void uninit();
void abort(std::vector <RequestPtr> &pRequests);
void set(MINT32 openID1, MINT32 openID2);
const Property &property();
MERROR negotiate(Selection &sel);
MERROR process(RequestPtr pRequest, RequestCallbackPtr pCallback);
private:
MERROR getConfigSetting(Selection &sel);
MERROR getP1Setting(Selection &sel);
MERROR getP2Setting(Selection &sel);
};
WatermarkPreview::WatermarkPreview() {
FUNCTION_IN;
mEnable = property_get_bool("vendor.debug.camera.watermark.preview.enable", 1);
mDump = property_get_bool("vendor.debug.camera.watermark.preview.dump", 0);
FUNCTION_OUT;
}
WatermarkPreview::~WatermarkPreview() {
FUNCTION_IN;
FUNCTION_OUT;
}
void WatermarkPreview::init() {
FUNCTION_IN;
mWatermarkWidth = 180;
mWatermarkHeight = 640;
int watermarkSize = mWatermarkWidth * mWatermarkHeight * 4;
mWatermarkRGBA = (unsigned char *) malloc(watermarkSize);
FILE *fp;
char path[256];
snprintf(path, sizeof(path), "/vendor/res/images/watermark.rgba");
if ((fp = fopen(path, "r")) == NULL) {
MY_LOGE("Failed to open /vendor/res/images/watermark.rgba");
}
fread(mWatermarkRGBA, 1, watermarkSize, fp);
fclose(fp);
FUNCTION_OUT;
}
void WatermarkPreview::uninit() {
FUNCTION_IN;
free(mSrcRGBA);
free(mWatermarkRGBA);
FUNCTION_OUT;
}
void WatermarkPreview::abort(std::vector <RequestPtr> &pRequests) {
FUNCTION_IN;
(void)pRequests;
FUNCTION_OUT;
}
void WatermarkPreview::set(MINT32 openID1, MINT32 openID2) {
FUNCTION_IN;
MY_LOGD("set openID1:%d openID2:%d", openID1, openID2);
mOpenID1 = openID1;
mOpenID2 = openID2;
FUNCTION_OUT;
}
const WatermarkPreview::Property &WatermarkPreview::property() {
FUNCTION_IN;
static Property prop;
static bool inited;
if (!inited) {
prop.mName = "TP_WATERMARK";
prop.mFeatures = TP_FEATURE_WATERMARK;
//prop.mInPlace = MTRUE;
//prop.mFaceData = eFD_Current;
//prop.mPosition = 0;
inited = true;
}
FUNCTION_OUT;
return prop;
}
MERROR WatermarkPreview::negotiate(Selection &sel) {
FUNCTION_IN;
MERROR ret = OK;
if (sel.mSelStage == eSelStage_CFG) {
ret = getConfigSetting(sel);
} else if (sel.mSelStage == eSelStage_P1) {
ret = getP1Setting(sel);
} else if (sel.mSelStage == eSelStage_P2) {
ret = getP2Setting(sel);
}
FUNCTION_OUT;
return ret;
}
MERROR WatermarkPreview::process(RequestPtr pRequest, RequestCallbackPtr pCallback) {
FUNCTION_IN;
(void) pCallback;
MERROR ret = -EINVAL;
MBOOL needRun = MFALSE;
IImageBuffer *in = NULL, *out = NULL;
if (pRequest->mIBufferMain1 != NULL && pRequest->mOBufferMain1 != NULL) {
in = pRequest->mIBufferMain1->acquire();
out = pRequest->mOBufferMain1->acquire();
int frameNo = 0, requestNo = 0;
if (pRequest->mIMetadataHal1 != nullptr) {
IMetadata *pIMetataHAL1 = pRequest->mIMetadataHal1->acquire();
if (pIMetataHAL1 != NULL) {
if (!tryGetMetadata<int>(pIMetataHAL1, MTK_PIPELINE_FRAME_NUMBER, frameNo)) {
frameNo = 0;
}
if (!tryGetMetadata<int>(pIMetataHAL1, MTK_PIPELINE_REQUEST_NUMBER, requestNo)) {
requestNo = 0;
}
pRequest->mIMetadataHal1->release();
MY_LOGD("frameNo: %d, requestNo: %d", frameNo, requestNo);
} else {
MY_LOGD("HAL metadata empty");
}
}
MY_LOGD("in[%d](%dx%d)=%p out[%d](%dx%d)=%p",
in->getPlaneCount(), in->getImgSize().w, in->getImgSize().h, in,
out->getPlaneCount(), out->getImgSize().w, out->getImgSize().h, out);
if (pRequest->mIMetadataApp != nullptr) {
IMetadata *pIMetadataApp = pRequest->mIMetadataApp->acquire();
MINT32 mode = 0;
if (!tryGetMetadata<MINT32>(pIMetadataApp, QXT_FEATURE_WATERMARK, mode)) {
mode = 0;
}
needRun = mode == 1 ? 1 : 0;
pRequest->mIMetadataApp->release();
}
MY_LOGD("needRun: %d", needRun);
int width = in->getImgSize().w;
int height = in->getImgSize().h;
MINT inFormat = in->getImgFormat();
if (needRun && inFormat == NSCam::eImgFmt_YV12) {
uint32_t currentTime = (NSCam::Utils::TimeTool::getReadableTime()) % 1000;
time_t timep;
time (&timep);
char currentDate[20];
strftime(currentDate, sizeof(currentDate), "%Y%m%d_%H%M%S", localtime(&timep));
//dump input YV12
if (mDump) {
char path[256];
snprintf(path, sizeof(path), "/data/vendor/camera_dump/preview_in_frame%d_%dx%d_%s_%d.yv12",
frameNo, width, height, currentDate, currentTime);
in->saveToFile(path);
}
nsecs_t t1 = systemTime(CLOCK_MONOTONIC);
if (mSrcRGBA == NULL) {
mSrcRGBA = (unsigned char *) malloc(width * height * 4);
}
//convert YV12 to RGBA
libyuv::I420ToABGR((unsigned char *)(in->getBufVA(0)), width,
(unsigned char *)(in->getBufVA(2)), width >> 1,
(unsigned char *)(in->getBufVA(1)), width >> 1,
mSrcRGBA, width * 4,
width, height);
nsecs_t t2 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Prepare src cost %02ld ms", ns2ms(t2 - t1));
Watermark::add(mSrcRGBA, width, height, mWatermarkRGBA, mWatermarkWidth, mWatermarkHeight, (width - mWatermarkWidth) / 2, (height - mWatermarkHeight) / 2);
nsecs_t t3 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Add watermark cost %02ld ms", ns2ms(t3 - t2));
//convert RGBA to YV12
libyuv::ABGRToI420(mSrcRGBA, width * 4,
(unsigned char *)(out->getBufVA(0)), width,
(unsigned char *)(out->getBufVA(2)), width >> 1,
(unsigned char *)(out->getBufVA(1)), width >> 1,
width, height);
nsecs_t t4 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Copy in to out cost %02ld ms", ns2ms(t4 - t3));
//dump output YV12
if (mDump) {
char path[256];
snprintf(path, sizeof(path), "/data/vendor/camera_dump/preview_out_frame%d_%dx%d_%s_%d.yv12",
frameNo, width, height, currentDate, currentTime);
out->saveToFile(path);
}
} else {
if (!needRun) {
MY_LOGE("No need run, skip add watermark for preview.");
} else if (inFormat != NSCam::eImgFmt_YV12) {
MY_LOGE("Unsupported format, skip add watermark for preview.");
} else {
MY_LOGE("Unknown exception, skip add watermark for preview.");
}
memcpy((unsigned char *) (out->getBufVA(0)),
(unsigned char *)(in->getBufVA(0)),
in->getBufSizeInBytes(0));
memcpy((unsigned char *) (out->getBufVA(1)),
(unsigned char *)(in->getBufVA(1)),
in->getBufSizeInBytes(1));
memcpy((unsigned char *) (out->getBufVA(2)),
(unsigned char *)(in->getBufVA(2)),
in->getBufSizeInBytes(2));
}
pRequest->mIBufferMain1->release();
pRequest->mOBufferMain1->release();
ret = OK;
}
FUNCTION_OUT;
return ret;
}
MERROR WatermarkPreview::getConfigSetting(Selection &sel) {
MY_LOGI("max out size(%dx%d)",
sel.mCfgInfo.mMaxOutSize.w, sel.mCfgInfo.mMaxOutSize.h);
mDisponly = property_get_bool("vendor.debug.tpi.s.fb.disponly", 0);
mInplace = mDisponly || property_get_bool("vendor.debug.tpi.s.fb.inplace", 0);
sel.mCfgOrder = 3;
sel.mCfgJoinEntry = eJoinEntry_S_YUV;
sel.mCfgInplace = mInplace;
sel.mCfgEnableFD = MTRUE;
sel.mCfgRun = mEnable;
sel.mIBufferMain1.setRequired(MTRUE);
if (!mDisponly && property_get_bool("vendor.debug.tpi.s.fb.nv21", 0)) {
sel.mIBufferMain1.addAcceptedFormat(NSCam::eImgFmt_NV21);
}
if (!mDisponly && property_get_bool("vendor.debug.tpi.s.fb.size", 0)) {
sel.mIBufferMain1.setSpecifiedSize(sel.mCfgInfo.mMaxOutSize);
}
sel.mOBufferMain1.setRequired(MTRUE);
sel.mIBufferMain1.addAcceptedFormat(NSCam::eImgFmt_YV12);
sel.mIBufferMain1.addAcceptedSize(eImgSize_Full);
IMetadata *meta = sel.mIMetadataApp.getControl().get();
MY_LOGD("sessionMeta=%p", meta);
return OK;
}
MERROR WatermarkPreview::getP1Setting(Selection &sel) {
(void) sel;
return OK;
}
MERROR WatermarkPreview::getP2Setting(Selection &sel) {
MBOOL run = MTRUE;
sel.mP2Run = run;
return OK;
}
REGISTER_PLUGIN_PROVIDER(Join, WatermarkPreview);
4.2.5 mtkcam3/3rdparty/customer/Android.mk
最终vendor.img需要的目标共享库是libmtkcam_3rdparty.customer.so。因此，我们还需要修改Android.mk，使模块libmtkcam_3rdparty.customer依赖libmtkcam.plugin.tp_watermark。vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk
old mode 100644
new mode 100755
index ce060c39f9..ff5763d3c2
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk
@@ -70,6 +70,13 @@ LOCAL_WHOLE_STATIC_LIBRARIES += libmtkcam.plugin.tp_purebokeh
LOCAL_SHARED_LIBRARIES += libcam.iopipe
LOCAL_SHARED_LIBRARIES += libmtkcam_modulehelper
endif
+
+ifeq ($(QXT_WATERMARK_SUPPORT), yes)
+LOCAL_SHARED_LIBRARIES += libwatermark
+LOCAL_SHARED_LIBRARIES += libyuv.vendor
+LOCAL_WHOLE_STATIC_LIBRARIES += libmtkcam.plugin.tp_watermark
+endif
+
# for app super night ev decision (experimental for customer only)
LOCAL_WHOLE_STATIC_LIBRARIES += libmtkcam.control.customersupernightevdecision
4.2.6 预置水印文件
diff --git a/device/mediateksample/k63v2_64_bsp/device.mk b/device/mediateksample/k63v2_64_bsp/device.mk
index 2619000c72..048c33462e 100644
--- a/device/mediateksample/k63v2_64_bsp/device.mk
+++ b/device/mediateksample/k63v2_64_bsp/device.mk
@@ -98,6 +98,9 @@ PRODUCT_COPY_FILES += vendor/mediatek/proprietary/custom/k63v2_64_bsp/factory/re
PRODUCT_COPY_FILES += vendor/mediatek/proprietary/custom/k63v2_64_bsp/factory/res/images/lcd_test_01.png:$(TARGET_COPY_OUT_VENDOR)/res/images/lcd_test_01.png:mtk
PRODUCT_COPY_FILES += vendor/mediatek/proprietary/custom/k63v2_64_bsp/factory/res/images/lcd_test_02.png:$(TARGET_COPY_OUT_VENDOR)/res/images/lcd_test_02.png:mtk
+ifeq ($(QXT_WATERMARK_SUPPORT),yes)
+PRODUCT_COPY_FILES += vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/tp_watermark/res/watermark.rgba::$(TARGET_COPY_OUT_VENDOR)/res/images/watermark.rgba
+endif
# overlay has priorities. high <-> low.
camera hal进程为mtk_camera_hal，它要读取/vendor/res/images/watermark.rgba，读取需要vendor_file SELinux权限。这里为mtk_camera_hal配置SELinux权限：
diff --git a/device/mediatek/sepolicy/bsp/non_plat/mtk_hal_camera.te b/device/mediatek/sepolicy/bsp/non_plat/mtk_hal_camera.te
index 8de5d0a437..7ebd9a03e5 100644
--- a/device/mediatek/sepolicy/bsp/non_plat/mtk_hal_camera.te
+++ b/device/mediatek/sepolicy/bsp/non_plat/mtk_hal_camera.te
@@ -92,6 +92,7 @@ allow mtk_hal_camera sysfs_boot_mode:file { read open };
# Purpose: NDD
allow mtk_hal_camera vendor_data_file:dir create_dir_perms;
allow mtk_hal_camera vendor_data_file:file create_file_perms;
+allow mtk_hal_camera vendor_file:file { read getattr open };
5. 自定义metadata
添加metadata是为了让APP层能够通过metadata传递相应的参数给HAL层。APP层是通过CaptureRequest.Builder.set(@NonNull Key<T> key, T value)来设置参数的。
由于我们是自定义的feature，无法复用MTK提供的metadata，因此，我们需要自定义metadata。
vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag.h b/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag.h
index 22d4aa2bf2..b020352092 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag.h
@@ -89,6 +89,7 @@ typedef enum mtk_camera_metadata_section {
MTK_BGSERVICE_FEATURE = 12,
MTK_CONFIGURE_SETTING = 13,
MTK_FLASH_FEATURE = 14,
+ QXT_FEATURE = 15,
MTK_VENDOR_SECTION_COUNT,
} mtk_camera_metadata_section_t;
@@ -146,6 +147,7 @@ typedef enum mtk_camera_metadata_section_start {
MTK_CONFIGURE_SETTING_START = (MTK_CONFIGURE_SETTING + MTK_VENDOR_TAG_SECTION) << 16,
MTK_FLASH_FEATURE_START = (MTK_FLASH_FEATURE + MTK_VENDOR_TAG_SECTION) << 16,
+ QXT_FEATURE_START = (QXT_FEATURE + MTK_VENDOR_TAG_SECTION) << 16,
} mtk_camera_metadata_section_start_t;
@@ -599,6 +601,8 @@ typedef enum mtk_camera_metadata_tag {
MTK_FLASH_FEATURE_CALIBRATION_RESULT, // flash calibration result
MTK_FLASH_FEATURE_END,
+ QXT_FEATURE_WATERMARK = QXT_FEATURE_START,
+ QXT_FEATURE_END,
} mtk_camera_metadata_tag_t;
/**
vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag_info.inl：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag_info.inl b/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag_info.inl
index 15449c433d..1b4fc75a0e 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag_info.inl
+++ b/vendor/mediatek/proprietary/hardware/mtkcam/include/mtkcam/utils/metadata/client/mtk_metadata_tag_info.inl
@@ -91,6 +91,11 @@ _IMP_SECTION_INFO_(MTK_DISTORTION_CORRECTION_INFO, "mtk.distortionCorrection")
_IMP_SECTION_INFO_(MTK_IOPIPE_INFO, "mtk.iopipe.info")
_IMP_SECTION_INFO_(MTK_HAL_INFO, "mtk.hal.info")
+_IMP_SECTION_INFO_(QXT_FEATURE, "com.qxt.camera")
+
+_IMP_TAG_INFO_( QXT_FEATURE_WATERMARK,
+ MINT32, "watermark")
+
/******************************************************************************
*
******************************************************************************/
vendor/mediatek/proprietary/hardware/mtkcam/utils/metadata/vendortag/VendorTagTable.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam/utils/metadata/vendortag/VendorTagTable.h b/vendor/mediatek/proprietary/hardware/mtkcam/utils/metadata/vendortag/VendorTagTable.h
index 2481492f90..33e581adfd 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam/utils/metadata/vendortag/VendorTagTable.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam/utils/metadata/vendortag/VendorTagTable.h
@@ -377,6 +377,16 @@ static auto& _FlashFeature_()
}
+static auto& _QxtFeature_()
+{
+ static const std::map<uint32_t, VendorTag_t>
+ sInst = {
+ _TAG_(QXT_FEATURE_WATERMARK,
+ "watermark", TYPE_INT32),
+ };
+ //
+ return sInst;
+}
/******************************************************************************
*
@@ -460,6 +470,10 @@ static auto& getGlobalSections()
MTK_FLASH_FEATURE_END,
_FlashFeature_() ),
+ _SECTION_( "com.qxt.camera",
+ QXT_FEATURE_START,
+ QXT_FEATURE_END,
+ _QxtFeature_() ),
};
// append custom vendor tags sections to mtk sections
vendor/mediatek/proprietary/hardware/mtkcam/utils/metastore/metadataprovider/constructStaticMetadata.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam/utils/metastore/metadataprovider/constructStaticMetadata.cpp b/vendor/mediatek/proprietary/hardware/mtkcam/utils/metastore/metadataprovider/constructStaticMetadata.cpp
index edd5b5f1b9..591b25b162 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam/utils/metastore/metadataprovider/constructStaticMetadata.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam/utils/metastore/metadataprovider/constructStaticMetadata.cpp
@@ -578,6 +578,19 @@ updateData(IMetadata &rMetadata)
}
}
#endif
+
+#if 1
+ {
+ IMetadata::IEntry qxtAvailRequestEntry = rMetadata.entryFor(MTK_REQUEST_AVAILABLE_REQUEST_KEYS);
+ qxtAvailRequestEntry.push_back(QXT_FEATURE_WATERMARK , Type2Type< MINT32 >());
+ rMetadata.update(qxtAvailRequestEntry.tag(), qxtAvailRequestEntry);
+
+ IMetadata::IEntry qxtAvailSessionEntry = rMetadata.entryFor(MTK_REQUEST_AVAILABLE_SESSION_KEYS);
+ qxtAvailSessionEntry.push_back(QXT_FEATURE_WATERMARK , Type2Type< MINT32 >());
+ rMetadata.update(qxtAvailSessionEntry.tag(), qxtAvailSessionEntry);
+ }
+#endif
+
// update multi-cam feature mode to static metadata
// vendor tag
{
前面这些步骤完成之后，集成工作就基本完成了。我们需要重新编译一下系统源码，为节约时间，也可以只编译vendor.img。趁着编译的时间，我们可以写一个demo来验证算法是否集成成功了。
6. APP调用算法
WatermarkActivity：
public class WatermarkActivity extends BaseActivity {
private static final String TAG = WatermarkActivity.class.getSimpleName();
/*
* 16:9 picture size: 3840x2160 preview size 1280x720
* 4:3 picture size: 3264x2448 preview size 960x720
* Now is 4:3
*/
private static final int PREVIEW_WIDTH = 1280;
private static final int PREVIEW_HEIGHT = 720;
private static final int CAPTURE_WIDTH = 3264;
private static final int CAPTURE_HEIGHT = 2448;
private static final String IMAGE_PATH =
Environment.getExternalStorageDirectory().getAbsolutePath()
+ File.separator + "DCIM" + File.separator + "Camera";
private static final String CAMERA_ID = "0";
private static final String KEY_WATERMARK = "com.qxt.camera.watermark";
private static final String SP_NAME = "watermark";
private static final String SP_STATE_KEY = "state";
private AutoFitTextureView mTextureView;
private ProgressBar mProgressBar;
private Handler mMainHandler;
private Handler mCameraHandler;
private HandlerThread mCameraHandlerThread;
private CameraManager mCameraManager;
private CaptureRequest.Builder mPreviewBuilder;
private CameraDevice mCameraDevice;
private CameraCaptureSession mCameraCaptureSession;
private MediaActionSound mCameraSound;
private String mTakePictureTime;
private SimpleDateFormat mDateFormat = new SimpleDateFormat(
"yyyyMMdd_HHmmss", Locale.getDefault());
private ImageReader mCaptureImageReader;
private Surface mSurface;
public CaptureRequest.Key<int[]> mVendorKey;
private int mVendorKeyEnable;
private SharedPreferences mSharedPref;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
getWindow().setFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON,
WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
setContentView(R.layout.activity_watermark);
mProgressBar = findViewById(R.id.progressbar);
mTextureView = findViewById(R.id.texture);
mTextureView.setAspectRatio(PREVIEW_HEIGHT, PREVIEW_WIDTH);
mCameraSound = new MediaActionSound();
mCameraSound.load(MediaActionSound.SHUTTER_CLICK);
mCameraManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
mMainHandler = new Handler();
initVendorTag();
mSharedPref = getSharedPreferences(SP_NAME, Context.MODE_PRIVATE);
mVendorKeyEnable = mSharedPref.getInt(SP_STATE_KEY, 0);
getCameraCharacteristics(CAMERA_ID);
}
@Override
public boolean onCreateOptionsMenu(Menu menu) {
getMenuInflater().inflate(R.menu.menu_watermark, menu);
Switch s = menu.findItem(R.id.action_watermark)
.getActionView().findViewById(R.id.switch_watermark);
s.setChecked(mVendorKeyEnable > 0);
s.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
@Override
public void onCheckedChanged(CompoundButton btn, boolean isChecked) {
if (isChecked) {
mVendorKeyEnable = 1;
} else {
mVendorKeyEnable = 0;
}
mSharedPref.edit().putInt(SP_STATE_KEY, mVendorKeyEnable).commit();
if (mPreviewBuilder != null && mCameraCaptureSession != null) {
try {
mCameraCaptureSession.stopRepeating();
setVendorTag(mPreviewBuilder);
mCameraCaptureSession.setRepeatingRequest(mPreviewBuilder.build(),
mSessionCaptureCallback, mCameraHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
LogUtils.d(TAG, "[onCheckedChanged] isChecked=" + isChecked
+ ", mWideAngleEnable=" + mVendorKeyEnable);
}
});
return true;
}
@Override
protected void onResume() {
super.onResume();
initLooper();
if (mTextureView.isAvailable()) {
openCamera();
} else {
mTextureView.setSurfaceTextureListener(mSurfaceTextureListener);
}
}
@Override
protected void onPause() {
super.onPause();
closeCamera();
stopLooper();
}
@Override
protected void onDestroy() {
super.onDestroy();
}
public void onClick(View view) {
if (view != null && view.getId() == R.id.btn_capture) {
takePicture();
}
}
private void initLooper() {
mCameraHandlerThread = new HandlerThread("WideAngleCamera");
mCameraHandlerThread.start();
mCameraHandler = new Handler(mCameraHandlerThread.getLooper());
}
private void stopLooper() {
try {
mCameraHandlerThread.quit();
mCameraHandlerThread.join();
mCameraHandlerThread = null;
mCameraHandler = null;
} catch (Exception e) {
e.printStackTrace();
}
}
@SuppressLint("MissingPermission")
private void openCamera() {
try {
mCameraManager.openCamera(CAMERA_ID, new CameraDevice.StateCallback() {
@Override
public void onOpened(@NonNull CameraDevice camera) {
mCameraDevice = camera;
createCameraPreviewSession();
}
@Override
public void onDisconnected(@NonNull CameraDevice camera) {
LogUtils.d(TAG, "onDisconnected");
camera.close();
mCameraDevice = null;
}
@Override
public void onError(@NonNull CameraDevice camera, int error) {
LogUtils.d(TAG, "onError error=" + error);
camera.close();
mCameraDevice = null;
}
}, mCameraHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
private void closeCamera() {
try {
if (null != mCameraCaptureSession) {
mCameraCaptureSession.close();
mCameraCaptureSession = null;
}
if (null != mCameraDevice) {
mCameraDevice.close();
mCameraDevice = null;
}
if (null != mCaptureImageReader) {
mCaptureImageReader.close();
mCaptureImageReader = null;
}
} catch (Exception e) {
e.printStackTrace();
}
}
private void createCameraPreviewSession() {
if (isFinishing() || isDestroyed() || mCameraDevice == null) {
return;
}
try {
mCaptureImageReader = ImageReader.newInstance(CAPTURE_WIDTH,
CAPTURE_HEIGHT, ImageFormat.YUV_420_888, 2);
mCaptureImageReader.setOnImageAvailableListener(
mCaptureOnImageAvailableListener, mCameraHandler);
mPreviewBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
setVendorTag(mPreviewBuilder);
mPreviewBuilder.addTarget(mSurface);
mCameraDevice.createCaptureSession(Arrays.asList(mSurface,
mCaptureImageReader.getSurface()),
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession session) {
if (isFinishing() || isDestroyed() || mCameraDevice == null) {
return;
}
try {
mCameraCaptureSession = session;
mPreviewBuilder.set(CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_AUTO);
mPreviewBuilder.set(CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
mCameraCaptureSession.setRepeatingRequest(mPreviewBuilder.build(),
mSessionCaptureCallback, mCameraHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
@Override
public void onConfigureFailed(@NonNull CameraCaptureSession session) {
}
}, mCameraHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
private void takePicture() {
try {
mTakePictureTime = mDateFormat.format(System.currentTimeMillis());
final CaptureRequest.Builder captureBuilder =
mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
setVendorTag(captureBuilder);
Surface surface = mCaptureImageReader.getSurface();
captureBuilder.addTarget(surface);
mCameraCaptureSession.capture(captureBuilder.build(),
new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
super.onCaptureCompleted(session, request, result);
}
}, mCameraHandler);
if (mCameraSound != null) {
mCameraSound.play(MediaActionSound.SHUTTER_CLICK);
}
mProgressBar.setVisibility(View.VISIBLE);
} catch (Exception e) {
e.printStackTrace();
}
}
private void notifyPictureTaken() {
mProgressBar.setVisibility(View.GONE);
Toast toast = Toast.makeText(WatermarkActivity.this,
getString(R.string.image_saved, IMAGE_PATH), Toast.LENGTH_SHORT);
toast.setGravity(Gravity.CENTER, 0, 0);
toast.show();
}
@SuppressWarnings("unused")
private void getCameraCharacteristics(String cameraId) {
try {
CameraCharacteristics cs = mCameraManager.getCameraCharacteristics(cameraId);
StreamConfigurationMap map = cs.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
if (map != null) {
//获取图像输出的尺寸
Size[] pictureSize = map.getOutputSizes(ImageFormat.JPEG);
Size[] previewSize = map.getOutputSizes(SurfaceTexture.class);
StringBuilder pictureBuilder = new StringBuilder("picture sizes: ");
for (Size size : pictureSize) {
pictureBuilder.append(size);
pictureBuilder.append(", ");
}
LogUtils.d(TAG, pictureBuilder.toString());
StringBuilder previewBuilder = new StringBuilder("preview sizes: ");
for (Size size : previewSize) {
previewBuilder.append(size);
previewBuilder.append(", ");
}
LogUtils.d(TAG, previewBuilder.toString());
}
} catch (Exception e) {
e.printStackTrace();
}
}
TextureView.SurfaceTextureListener mSurfaceTextureListener
= new TextureView.SurfaceTextureListener() {
@Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int w, int h) {
LogUtils.d(TAG, "onSurfaceAvaliable, width:" + w + ", height:" + h);
surfaceTexture.setDefaultBufferSize(PREVIEW_WIDTH, PREVIEW_HEIGHT);
mSurface = new Surface(surfaceTexture);
openCamera();
}
@Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int w, int h) {
LogUtils.d(TAG, "onSurfaceTextureSizeChanged, width:" + w + ", height:" + h);
}
@Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
LogUtils.d(TAG, "onSurfaceTextureDestroyed");
mSurface = null;
return false;
}
@Override
public void onSurfaceTextureUpdated(SurfaceTexture surface) {
}
};
private final CameraCaptureSession.CaptureCallback mSessionCaptureCallback
= new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
super.onCaptureCompleted(session, request, result);
mCameraCaptureSession = session;
}
};
private final ImageReader.OnImageAvailableListener mCaptureOnImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(final ImageReader reader) {
LogUtils.d(TAG, "capture onImageAvailable");
Image image = reader.acquireLatestImage();
if (image == null) return;
ImageUtils.saveImage(WatermarkActivity.this, image, IMAGE_PATH,
"WIDE_" + mTakePictureTime, ImageUtils.ROTATE_90);
image.close();
LogUtils.d(TAG, "saved");
mMainHandler.post(new Runnable() {
@Override
public void run() {
notifyPictureTaken();
}
});
}
};
private void initVendorTag() {
try {
CameraCharacteristics c = mCameraManager.getCameraCharacteristics(CAMERA_ID);
mVendorKey = CameraUtils.getSessionKey(c, KEY_WATERMARK);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
private void setVendorTag(CaptureRequest.Builder builder) {
if (mVendorKey != null) {
builder.set(mVendorKey, new int[]{mVendorKeyEnable});
LogUtils.d(TAG, "[setVendorTag] set watermark to " + mVendorKeyEnable);
}
}
}
CameraUtils：
public class CameraUtils {
private static final String TAG = CameraUtils.class.getSimpleName();
@RequiresApi(api = Build.VERSION_CODES.P)
public static CaptureRequest.Key<int[]> getSessionKey(
CameraCharacteristics cs, String key) {
if (cs == null) {
LogUtils.i(TAG, "[getSessionKey] CameraCharacteristics is null");
return null;
}
CaptureRequest.Key<int[]> targetKey = null;
List<CaptureRequest.Key<?>> sessionKeys = cs.getAvailableSessionKeys();
if (sessionKeys == null) {
LogUtils.i(TAG, "[getSessionKey] No keys!");
return null;
}
for (CaptureRequest.Key<?> sessionKey : sessionKeys) {
if (sessionKey.getName().equals(key)) {
LogUtils.i(TAG, "[getSessionKey] key :" + key);
targetKey = (CaptureRequest.Key<int[]>) sessionKey;
break;
}
}
return targetKey;
}
}
为样机刷入系统整包或者vendor.img，开机后，安装demo验证。我们来拍一张电脑显示器看看效果：
预览：
预览调用水印算法.png
拍照：
拍照调用加水印算法.jpg
7. 遇到的问题及解决方法
问题1：
如果process函数中buffer的acquire和release没有成对出现，也就是buffer没正常release，那么就会出现连续拍多张之后，算法未被调用的情况。
问题1解决方法：
YUVNode.cpp中加入一个保险的代码，万一集成代码中忘记release，在YUVNode中release。
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
index 8bb794ba02..d4343aaccf 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
@@ -1050,9 +1051,11 @@ MBOOL YUVNode::onRequestProcess(RequestPtr& pRequest)
auto pPlgRequest = mPlugin->createRequest();
- pPlgRequest->mIBufferFull = (iBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, INPUT) : iBufferFullHandle;
+ //pPlgRequest->mIBufferFull = (iBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, INPUT) : iBufferFullHandle;
+ pPlgRequest->mIBufferFull = (iBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, INPUT) : std::move(iBufferFullHandle);
pPlgRequest->mIBufferClean = PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_PURE_YUV, INPUT);
- pPlgRequest->mOBufferFull = (oBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, OUTPUT) : oBufferFullHandle;
+ //pPlgRequest->mOBufferFull = (oBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, OUTPUT) : oBufferFullHandle;
+ pPlgRequest->mOBufferFull = (oBufferFullHandle == NULL) ? PluginHelper::CreateBuffer(pNodeReq, TID_MAN_FULL_YUV, OUTPUT) : std::move(oBufferFullHandle);
pPlgRequest->mIMetadataDynamic = PluginHelper::CreateMetadata(pNodeReq, MID_MAN_IN_P1_DYNAMIC);
pPlgRequest->mIMetadataApp = PluginHelper::CreateMetadata(pNodeReq, MID_MAN_IN_APP);
问题2：
算法需要RGB数据，HAL层是YUV数据，使用openGL和各类RGB转换公式进行YUV和RGB互转后，最终照片有色差。
问题2解决方法：
使用libyuv进行转换，libyuv转换效率非常高，经测试，libyuv比公式法和opencv都要快，并且没有色差。android源码本身已集成libyuv，使用起来也非常方便。
Android.mk:
LOCAL_C_INCLUDES += $(TOP)/external/libyuv/files/include/
LOCAL_SHARED_LIBRARIES += libyuv.vendor
如不清楚libyuv的使用，请参考本人的另外一篇文章：YUV420转RGBA之使用libyuv
8. 结语
现公司是家ODM公司，2018年起公司开始做AI算法，我被招来做android平台上的算法集成工作。然而我虽有几年App和Frameworks的工作经验，之前却一直是以java作为主要开发语言，C/C++语言的开发工作甚至是大学时代的事情，而部门当中会android开发的也仅有我一人。在这种情形下，重新学C/C++，并摸索着开始集成算法，真的非常具有挑战性。
当时我们公司与某公司有合作，我们刚开始寄希望于该公司可以提供技术支援，以帮助我们完成算法集成的工作。然而，该公司不但有手机OS，也有自家的AI算法部门，在技术支援上对方不愿意配合。对方的想法也很容易理解，他们希望我们的算法无法商用，这样他们就能够顺利推销他们的算法了。
在一段时间的学习和实践之后，我也终于基本掌握了算法集成的方法和步骤。很感激部门leader L老师能够给予我如此多的耐心和支持。回过头来看，算法集成并不复杂，甚至没有任何技术原理上的东西，只是一些繁琐的步骤。于是，在项目不忙的情况下，写下本文。希望以此记录自己的心路历程，也期待可以帮助到有需要的人。本文的主要内容最初于2019年就写好了，但是由于某些原因，时至今日才整理发出来，不管怎样，但愿好文不怕晚。
9. 本文参考
本文主要参考MTK-Online的Camera quick start部分，MTK在MTK-Online上有详细的文章及教程（为MTK点赞）：
https://online.mediatek.com/QuickStart/2a17666a-9d46-4686-9222-610ec0f087cc
欢迎交流、点赞、转载，码字不易，转载请注明出处。
网友评论