对FasterRCNN的改进

1. Automated pulmonary nodule detection in CT images using deep convolutional neural networks_PR_2019

Firstly, we adjust the structure of Faster RCNN with two region proposal networksand a deconvolutional layer to detect nodule candidates, and then three models are trained for three kinds of slicesfor later result fusion.
Secondly, a boosting architecturebased on 2D CNN is designed for false positive reduction, which is a classifier to distinguish true nodules from the candidates. The misclassified samples are still kept for retraining a model which boosts the sensitivity for pulmonary nodule detection.
Finally, the results of these networks are fused to vote out the final classification results.

通过对VGG16的第五层解卷积然后结合第三层进行双RPN网络得到rpn网络的四个loss
根据肿块的先验信息设计了七个anchor, 这样改为了解决感知域问题
进行二维检测为了解决计算和内存的问题，小尺度肿块能够检测到的关键是感知域是否合适，但是一系列的pooling之后感知域会增大，利用双RPN网络因为作者认为每个RPN网络有不同的性质能够利用的特征不同。设计了七个比例为1的anchor，最终的loss有六部分组成两个RPN网络和最终的结果

框架
训练三个模型都是针对axis轴的图像，只是图像的位置不同，同时三通道的图像由当前层和前后两层组成(这样做图像不是都差不多为什么用三个模型)

To fully use of the spatial information of the CT scans, in the training process, we extract not only the middle slice of the nodule, but also two neighboring slices. Three models are trained with this three kinds of slices separately. The first model is trained with the middle slice and two neighboring slices. The second model is trained withtop neighboring slice and its two neighboring slices. The third model is trained with thebottom neighboring slice and its two neighboring slices.When testing, the slices are input into the three models separately, and then we merge the detection results of the three models to get the nodule candidates.

用FasterRCNN提取后使用boosting方法去除假阳性，通过训练三个弱的分类器，弱分类器投票得到好的结果

从二维检测结果中选择九个面（常见的方法），对于分辨率不同的处理成各项同性的size。
我所理解的boosting方法：

1. 在二维检测的结果上进行数据集建立，由于检测结果中fp/tp = 500/1，对数据集进行扩增进行类间平衡
2. 训练三个弱分类器，每个分类器都在上一个的结果基础上进行训练

boosting and vote

几个问题：

1. 二维的检测结果如何变成三维?

For the acquisition of candidate nodules, we obtain the 2D faces (35 × 35) based on the extraction of different slices in a 3D cube block

2. 数据集建立的时候为什么要对FP进行处理后留下误分类的,图(a)中的下半部分,难道是出去一些数据平衡两类比例?

boosting

结果根据肿瘤大小分析

结果

2. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images_2018

本文利用VGG16的三层和五层进行连接作为特征提取,并且在输出引入空间连接层,能够解决小目标问题,改进的理论支撑是parseNet论文

In our study, inspired by state-of-the-art object detection network Faster R-CNN, we develop a detector which is more suitable for thyroid papillary carcinoma detection in ultrasound images. In order to improve the accuracy of the detection, weadd a spatial constrained layerto CNN so that the detector can extract the features of surrounding region in which the cancer regions are residing.

连接VGG的三层和五层,并添加空间约束层,空间约束层的部分没有看懂,大概是讲希望网络学习到更多空间上的信息,利用loss的修改实现

结构

本文试了各种层间链接,得出结论三层和五层连接最好.但是有篇论文用来四层和五层的连接

连接结果

3. DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification

两个三维网络分别进行检测和分类

DeepLung contains two parts, nodule detection and classification. Considering the 3D nature of lung CT data, two 3D networks are designed for the nodule detection and classification respectively. Specifically, a 3D Faster R-CNN is designed for nodule detection with a U-net-like encoder-decoder structure to effectively learn nodule features. For nodule classification, gradient boosting machine (GBM) with 3D dual path network (DPN) features is proposed.

主要分检测和分类两部分,文献综述不错
最后一层是作为特征输入到RPN网络,包括四个resNET18的block,利用Unet思想连接,对图像resize到很小.

3D FasterRCNN

In the last layer, we design 3 anchors, 5, 10, 20, for scale references which are designed based on the distribution of nodule sizes. For each anchor, there are 5 parts in the loss function, classification loss Lcls for whether the current box is a nodule or not, regression loss Lreg for nodule coordinates x; y; z and nodule size d

anchor的位置通过xyz和d确定,也就是说是立方体
用三维的resNET和Unet结构进行特征提取,然后进行三维的FasterRCNN
https://github.com/wentaozhu/DeepLung

结果

4. Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks_2017

先在二维的axis面检测,然后三维去除假阳性
将VGG16的第五层进行解卷积,作为特征送入RPN网络;然后利用三维的网络进行假阳性去除

We first introduce a deconvolutional structureto Faster Region-based Convolutional Neural Network (Faster R-CNN) for candidate detection on axial slices. Then, a three-dimensional DCNN is presented for the subsequent false positive reduction.

本文解决了:

1. 肿块形装等变化很大的问题(没啥新意)
2. 在三维的研究很少

流程

输入是三通道的Image用当前层和前后两层组成三通道(也有文章用有一定间隔的层作为三通道)

为什么改进VGG16因为不能很好的表达特征

To address this problem, we add a deconvolutional layer, whose kernel size, stride size, padding size and kernel number are 4, 4, 2 and 512 respectively, after the last layer of the original feature extractor.

如果没有outputpadding的话输出是(W/4-4,H/4-4)*512

解卷积输出公式

anchor的比例都是1

To fit the size of nodules, we design six anchors with different size for each sliding window: 4 × 4, 6 × 6, 10 × 10, 16 × 16, 22 × 22, and 32 × 32

三维上用自己搭建的三维网络进行假阳性去除

This network contains six 3D convolutional layers which are followed by Rectified Linear Unit (ReLU) activation layers, three 3D max-pooling layers, three fully connected layers, and a final 2-way softmax activation layer to classify the candidates from nodules to none-nodules. Moreover, dropout layers are added after max-pooling layers and fully-connected layers to avoid overfitting. We initialize the parameters of the proposed 3D DCNN by the same strategy using in the literature [3]

三维网络

在二维网络上的结果recall = 0.946原始FasterRCNN = 0.817

5. Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images_TMI_2018

重点在于多示例学习的弱监督,利用FasterRCNN作为网络,修改loss实现弱监督