REF:
https://neptune.ai/blog/image-segmentation-tips-and-tricks-from-kaggle-competitions#ensembling-methods
【后续出中文版】

一、External Data

• Use of the LUng Node Analysis Grand Challenge data because it contains detailed annotations from radiologists
• Use of the LIDC-IDRI data because it had radiologist descriptions of each tumor that they found
• Use Flickr CC, Wikipedia Commons datasets
• Use Human Protein Atlas Dataset
• Use IDRiD dataset
  二、Data Exploration and Gaining insights
• Clustering of 3d segmentation with the 0.5 threshold
• Identify if there is a substantial difference in train/test label distributions

三、Preprocessing

• Perform blob Detection using the Difference of Gaussian (DoG) method. Used the implementation available in skimage package.
• Use of patch-based inputs for training in order to reduce the time of training
• Use cudf for loading data instead of Pandas because it has a faster reader
• Ensure that all the images have the same orientation
• Apply contrast limited adaptive histogram equalization
• Use OpenCV for all general image preprocessing
• Employ automatic active learning and adding manual annotations
• Resize all images to the same resolution in order to apply the same model to scans of different thicknesses
• Convert scan images into normalized 3D numpy arrays
• Apply single Image Haze Removal using Dark Channel Prior
• Convert all data to Hounsfield units
• Find duplicate images using pair-wise correlation on RGBY
• Make labels more balanced by developing a sampler
• Apply pseudo labeling to test data in order to improve score
• Scale down images/masks to 320×480
• Histogram equalization (CLAHE) with kernel size 32×32
• Convert DCM to PNG
• Calculate the md5 hash for each image when there are duplicate images

四、Data Augmentations

• Use albumentations package for augmentations
• Apply random rotation by 90 degrees
• Use horizontal, vertical or both flips
• Attempt heavy geometric transformations: Elastic Transform, Perspective
  Transform, Piecewise Affine transforms, pincushion distortion
• Apply random HSV
• Use of loss-less augmentation for generalization to prevent loss of useful image information
• Apply channel shuffling
• Do data augmentation based on class frequency
• Apply gaussian noise
• Use lossless permutations of 3D images for data augmentation
• Rotate by a random angle from 0 to 45 degrees
• Scale by a random factor from 0.8 to 1.2
• Brightness changing
• Randomly change hue, saturation and value
• Apply D4 augmentations
• Contrast limited adaptive histogram equalization
• Use the AutoAugment augmentation strategy

五、Modeling

1、Architectures

• Use of a U-net based architecture. Adopted the concepts and applied them to 3D input tensors
• Employing automatic active learning and adding manual annotations
• The inception-ResNet v2 architecture for training features with different receptive fields
• Siamese networks with adversarial training
• ResNet50, Xception, *Inception ResNet *v2 x 5 with Dense (FC) layer as the final layer
• Use of a global max-pooling layer which returns a fixed-length output no matter the input size
• Use of stacked dilated convolutions
• VoxelNet
• Replace plus sign in LinkNet skip connections with concat and conv1x1
• Generalized mean pooling
• Keras NASNetLarge to train the model from scratch using 224x224x3
• Use of the 3D convnet to slide over the images
• Imagenet-pre-trained ResNet152 as the feature extractor
• Replace the final fully-connected layers of ResNet by 3 fully connected layers with dropout
• Use ConvTranspose in the decoder
• Applying the VGG baseline architecture
• Implementing the C3D network with adjusted receptive fields and a 64 unit bottleneck layer on the end of the network
• Use of UNet type architectures with pre-trained weights to improve convergence and performance of binary segmentation on 8-bit RGB input images
• LinkNet since it’s fast and memory efficient
• MASKRCNN
• BN-Inception
• Fast Point R-CNN
• Seresnext
• UNet and Deeplabv3
• Faster RCNN
• SENet154
• ResNet152
• NASNet-A-Large
• EfficientNetB4
• ResNet101
• GAPNet
• PNASNet-5-Large
• Densenet121
• AC-GAN
• XceptionNet (96), XceptionNet (299), Inception v3 (139), InceptionResNet v2 (299), DenseNet121 (224)
• AlbuNet (resnet34) from ternausnets
• SpaceNet
• Resnet50 from selim_sef SpaceNet 4
• SCSEUnet (seresnext50) from selim_sef SpaceNet 4
• A custom Unet and Linknet architecture
• FPNetResNet50 (5 folds)
• FPNetResNet101 (5 folds)
• FPNetResNet101 (7 folds with different seeds)
• PANetDilatedResNet34 (4 folds)
• PANetResNet50 (4 folds)
• EMANetResNet101 (2 folds)
• RetinaNet
• Deformable R-FCN
• Deformable Relation Networks

2、Hardware Setups

• Use of the AWS GPU instance p2.xlarge with a NVIDIA K80 GPU
• Pascal Titan-X GPU
• Use of 8 TITAN X GPUs
• 6 GPUs: 21080Ti + 41080
• Server with 8×NVIDIA Tesla P40, 256 GB RAM and 28 CPU cores
• Intel Core i7 5930k, 2×1080, 64 GB of RAM, 2x512GB SSD, 3TB HDD
• GCP 1x P100, 8x CPU, 15 GB RAM, SSD or 2x P100, 16x CPU, 30 GB RAM
• NVIDIA Tesla P100 GPU with 16GB of RAM
• Intel Core i7 5930k, 2×1080, 64 GB of RAM, 2x512GB SSD, 3TB HDD
• 980Ti GPU, 2600k CPU, and 14GB RAM